agentleFS
Sign inStart free
katalon-labskatalon-labs/true-skills/.github/skills/test-review/SKILL.md

test-review

Skill10 starsChanged 5 months agoMITView on GitHub

Review Katalon True Platform/TestOps test quality and coverage before tests enter the delivery pipeline. Use when you need to check whether a suite is ready to run, review requirement and configuration coverage, assess test-case quality and flakiness/stability, spot weak or unreliable cases, and produce a review verdict (Approve / Approve with fixes / Reject for pipeline) with the specific cases to fix. This is a pre-execution quality gate, not a ship decision; for the release ship/no-ship call use release-analyze. Written for the test lead gating what is allowed into the pipeline, one suite at a time.

16 other projects use a version of this.

Adopted by

Most recently taken up by

Used with

Projects with it also use maintain-verification-skill, create-verification-skill and 6 more.

Other files people use alongside this one.

Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
  • jaydubya818Skill

    maintain-verification-skill

    Use when a project's verification driver or feature map may be stale.

    Also in 3 projects that use this

  • jaydubya818Skill

    create-verification-skill

    Use when a repository needs a repeatable driver that proves real UI, CLI, or service behavior.

    Also in 3 projects that use this

  • MicrosoftSkill

    blast-radius

    Assesses network dependency and shared-infrastructure exposure. Use when the user asks which services, customers, SLA policies, routes, or redundant links are affected by a network fault.

    Also in 3 projects that use this

  • asterkinSkill

    adr

    Create Architecture Decision Records (ADRs) following project standards. Guides template usage, process, and best practices for documenting architecturally significant decisions.

    Also in 3 projects that use this

  • AnthropicSkill

    architecture

    Create or evaluate an architecture decision record (ADR). Use when choosing between technologies (e.g., Kafka vs SQS), documenting a design decision with trade-offs and consequences, reviewing a system design proposal, or designing a new component from requirements and constraints.

    Also in 3 projects that use this

  • GitHubSkill

    refactor

    Surgical code refactoring to improve maintainability without changing behavior. Covers extracting functions, renaming variables, breaking down god functions, improving type safety, eliminating code smells, and applying design patterns. Less drastic than repo-rebuilder; use for gradual improvements.

    Also in 3 projects that use this

The file

We split this skill into the rules it gives an agent

2 rules. 0 are common, 1 is distinctive, and 1 is very specific to this project.

  • 0

    Common

    Most of the 11 edited versions we read keep these.

  • 1

    Distinctive

    This author's own choices. Worth a look.

  • 1

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Use this skill for the review stage: inspect coverage, quality, and reliability so weak tests do not enter the pipeline.

    line 8
  2. Very specific:

    Read references/review-rubric.md before issuing a verdict.

    line 55

“Common” is measured: kept by at least 6 in 10 of the 11 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Katalon Test Review

Use this skill for the review stage: inspect coverage, quality, and reliability so weak tests do not enter the pipeline. The output is a verdict with named weak cases, never a metric dump.

Availability Boundary

  • Available via MCP: coverage review (fetch_requirement_data, find_test_cases_by_requirement, fetch_test_configuration_data), quality review (fetch_test_case_data), reliability review (fetch_test_stability_data, find_test_results), and environment readiness (read_auts).
  • Not directly available: code/object review, local debug, StudioAssist Ask — these are Studio-desktop operations. For code-lane review, defer to test-case-to-playwright / playwright-execute. Use Browser/Playwright only for AUT sanity checks when asked.

Review Workflow

+---------------------+     +----------------------+     +----------------------+
| Coverage review     | --> | Quality review       | --> | Reliability review   |
| reqs + config       |     | case design signals  |     | flakiness/stability  |
+---------------------+     +----------------------+     +----------------------+
                                                                   |
                                                                   v
                                                         +----------------------+
                                                         | Verdict + weak cases |
                                                         +----------------------+

Steps and tool rules

  1. Coverage review. fetch_requirement_data + find_test_cases_by_requirement for requirement coverage; fetch_test_configuration_data for browser/platform coverage. Flag orphan requirements and under-covered configurations.
  2. Quality review. fetch_test_case_data for design signals; read representative cases with read_test_case when a signal is ambiguous. Flag non-atomic cases (many assertions), missing negative/boundary variants, and vague expected results.
  3. Reliability review. fetch_test_stability_data for flakiness; find_test_results for recent pass/fail history. Flag probabilistically flaky cases that will erode pipeline trust.
  4. Environment readiness. read_auts to confirm an executable AUT exists for the suite.
  5. Verdict. One of Approve / Approve with fixes / Reject for pipeline, followed by the specific cases to fix and why. State the risk if approving with known gaps.

Verdict rubric

  • Approve: coverage meets the plan, no flaky cases in the critical path, expected results are observable.
  • Approve with fixes: ship-able but list the exact cases needing a fix (flaky, non-atomic, weak expected result) and the owner action.
  • Reject for pipeline: orphan critical requirements, or flaky cases in the smoke/regression core — fix before the suite runs.

Prompt recipes

  • Review the regression suite for release 3.2: is it ready for the pipeline? Give a verdict and list weak cases.
  • Check requirement and configuration coverage for project X and flag anything under-covered.
  • Which cases in the smoke suite are flaky enough to reject before we wire them into CI?

Hand-offs

  • Fixes needed -> create-test-cases (redesign) or test-maintenance (repair/flaky).
  • Approved -> execute-test.
  • Ship decision after execution -> release-analyze.

Read references/review-rubric.md before issuing a verdict. Consult the orchestrator's references/unavailable-capabilities.md for boundaries.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

Check your setup, free