katalon-labskatalon-labs/true-skills/skills/test-maintenance/SKILL.md

test-maintenance

Skill10 starsChanged 5 months agoMITView on GitHub

Maintain and evolve a Katalon True Platform/TestOps regression suite as the application changes. Use when you need to detect which tests broke or became flaky from stability and result history, diagnose whether a case needs repair vs regeneration, repair test assets (update, move, reorganize cases), refresh coverage after application or requirement changes, and feed the resulting gap list back into planning. Closes the lifecycle loop. For classifying a specific run's failures use analyze-failures; for authoring brand-new cases use create-test-cases. Written for the automation tester repairing a suite that has started lying, and the test lead who has to keep it trustworthy.

In this repository59 other files it gives its agents.
- AGENTS.mdAGENTS.md
- .github/copilot-instructions.mdCopilot instructions
- .cursor/rules/analyze-failures.mdcCursor rule
- .cursor/rules/create-test-cases.mdcCursor rule
- .cursor/rules/execute-test.mdcCursor rule
- .cursor/rules/exploratory-charter.mdcCursor rule
- .cursor/rules/get-started.mdcCursor rule
- .cursor/rules/platform-setup.mdcCursor rule
- .cursor/rules/playwright-execute.mdcCursor rule
- .cursor/rules/release-analyze.mdcCursor rule
- .cursor/rules/test-case-to-katalon-studio.mdcCursor rule
- .cursor/rules/test-case-to-playwright.mdcCursor rule
- .cursor/rules/test-data.mdcCursor rule
- .cursor/rules/test-estimation.mdcCursor rule
- .cursor/rules/test-maintenance.mdcCursor rule
- .cursor/rules/test-management.mdcCursor rule
- .cursor/rules/test-plan.mdcCursor rule
- .cursor/rules/test-reporting.mdcCursor rule
- .cursor/rules/test-review.mdcCursor rule
- .cursor/rules/true-platform-testing.mdcCursor rule
- .cursor/rules/upload-report.mdcCursor rule
- llms.txtllms.txt
- analyze-failuresSkill
- create-test-casesSkill
- execute-testSkill
- exploratory-charterSkill
- get-startedSkill
- platform-setupSkill
- playwright-executeSkill
- release-analyzeSkill
- test-case-to-katalon-studioSkill
- test-case-to-playwrightSkill
- test-dataSkill
- test-estimationSkill
- test-maintenanceSkill
- test-managementSkill
- test-planSkill
- test-reportingSkill
- test-reviewSkill
- true-platform-testingSkill
- upload-reportSkill
- analyze-failuresSkill
- create-test-casesSkill
- execute-testSkill
- exploratory-charterSkill
- get-startedSkill
- platform-setupSkill
- playwright-executeSkill
- release-analyzeSkill
- test-case-to-katalon-studioSkill
- test-case-to-playwrightSkill
- test-dataSkill
- test-estimationSkill
- test-managementSkill
- test-planSkill
- test-reportingSkill
- test-reviewSkill
- true-platform-testingSkill
- upload-reportSkill

The file

We split this skill into the rules it gives an agent

2 rules. About 0 are common, 1 is distinctive, and 1 is very specific to this project.

≈0
Common · estimate
Many files like this have a rule of this kind.
≈1
Distinctive · estimate
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.

Common
Distinctive
Very specific

Distinctive:
Use this skill for the maintenance stage: keep the regression suite healthy over time and feed learning back to the plan.
line 8
Very specific:
Read references/maintenance-loop.md before repairing.
line 54

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.

Files

SKILL.md

Katalon Test Maintenance

Use this skill for the maintenance stage: keep the regression suite healthy over time and feed learning back to the plan. It consumes failure/stability signals and produces repairs plus a refreshed coverage gap list.

Availability Boundary

Available via MCP: change/impact signals (fetch_test_stability_data, find_test_results, read_execution), and case-level repair (update_test_case, move_test_case, duplicate_test_case).
Not directly available: Self-healing tests, Time Capsule, Studio Tracer, object refactoring, and StudioAssist Agent-mode edits are Studio-desktop features. TrueTest regeneration from live journeys is a TrueTest surface. Narrate these; MCP repairs are case-level content edits, not object/script self-healing.

Maintenance Workflow

+---------------------+     +----------------------+     +----------------------+
| Detect change impact| --> | Diagnose: repair vs  | --> | Repair case assets   |
| stability + history |     | regenerate vs retire |     | update/move/dup      |
+---------------------+     +----------------------+     +----------------------+
                                                                   |
                                                                   v
                                                         +----------------------+
                                                         | Validate + feed back |
                                                         | to plan (gap list)   |
                                                         +----------------------+

Steps and tool rules

Detect impact. fetch_test_stability_data for flakiness trend; find_test_results + read_execution for what changed across recent runs. Build the list of broken / flaky / degraded cases.
Diagnose each. Decide per case:
- Repair — the case is salvageable: fix data, steps, or expected result via update_test_case; reorganize via move_test_case; branch a variant via duplicate_test_case.
- Regenerate — behavior changed enough that the case should be re-derived (TrueTest regeneration / re-author) -> hand to create-test-cases; narrate the TrueTest boundary.
- Retire — behavior removed: unlink/retire (prefer update/flag over unreliable delete_test_case); get approval.
Repair (MCP-legal edits only). Pass all intended updates in one update_test_case call. Do not claim to have self-healed locators/objects — that is Studio.
Validate. Re-run through execute-test and confirm via find_test_results / read_execution.
Feed back. Emit the refreshed coverage gap list to test-plan so maintenance closes the loop instead of dead-ending.

Prompt recipes

Which regression cases became flaky this month, and which should we repair vs regenerate?
Repair the login suite after the new auth flow: update the affected cases and tell me what still needs re-authoring.
After this release's UI change, refresh coverage and give me the gap list for next sprint.

Hand-offs

Classify a specific failed run first -> analyze-failures.
Re-author changed behavior -> create-test-cases.
Confirm repairs -> execute-test then test-review.
Refreshed gaps -> test-plan.

Read references/maintenance-loop.md before repairing. Consult the orchestrator's references/unavailable-capabilities.md for the Studio/TrueTest boundary.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

It worked
Partly
It didn't

What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

Check your setup, free