tk-qa-verify-web
Skill448 starsChanged 12 days agoApache-2.0View on GitHub
Ported from mobile_private/.agents/skills/tk-qa-verify. Same phases, Playwright instead of Maestro, the PR's Cloudflare preview instead of a simulator. Read references/setup.md once per machine. Read references/lessons.md before you write a spec. Write every plan and report in STE (../ste-writing/SKILL.md). 1. Exercised, not read. A case counts only when the UI was driven and the new state was read back (UI assert, page.evaluate on storage, a captured request). The diff tells you what to test, never that it works. No interaction → NOT…
- Reads credentials
In this repository3 other files it gives its agents.
- CLAUDE.mdCLAUDE.md
- ste-writingSkill
- ui-component-testsSkill
The file
We split this skill into the rules it gives an agent
8 rules. About 0 are common, 3 are distinctive, and 5 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈3
Distinctive · estimate
This author's own choices. Worth a look.
5
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 17
Read references/setup.md once per machine.
- Very specific: line 34
Visual is human: Do not judge colour, spacing, animation or "looks like Figma".
- Very specific: line 37
Seeds never reach you: Do not read tests/playwright/.env or qa/.auth/, do not print env.
- Distinctive: line 44
No app code edits: If the app is wrong, the test stays red and the report says so.
- Very specific: line 119
In the spec: call await requireFunds() at the start of a money test (it checks A and B.
- Very specific: line 151
Run: from tests/playwright, QA_TICKET=<T> QA_BASE_URL=<preview> yarn qa --grep <T>, then node qa/summary.mjs <T>.
- Distinctive: line 155
Keep a ledger: case → route → interaction → read back → result → video path.
- Distinctive: line 170
Every bug a human finds after a PASS becomes two things: a case in the ticket's cases file and a line in references/lessons.md that says …
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
TK QA Verify — web
Ported from mobile_private/.agents/skills/tk-qa-verify. Same phases, Playwright instead of
Maestro, the PR's Cloudflare preview instead of a simulator.
Read references/setup.md once per machine. Read references/lessons.md before you write a spec. Write every plan and report in STE (../ste-writing/SKILL.md).
Non-negotiables
- Exercised, not read. A case counts only when the UI was driven and the new state was read
back (UI assert,
page.evaluateon storage, a captured request). The diff tells you what to test, never that it works. No interaction → NOT COVERED. - Cover the risk, not the diff. Happy path, negative (empty, error, cancel, insufficient
funds), persistence (
page.reload()and check again), adjacent screens from references/feasibility.md. - Both wallets. A = multichain, B = legacy TON. If the ticket does not name one, test both. If
it names one, add a smoke case that proves the other is not broken. For lists, order or defaults,
add a fresh empty wallet case. Wallet C (optional,
walletC) exists only for empty-battery cases: its battery stays empty forever, andsignPlanned()refuses anybattery-chargefrom or to C. If C is not configured, its cases skip and go to "Not covered". - Visual is human. Do not judge colour, spacing, animation or "looks like Figma". Record it,
attach the screenshot, mark it
human-visual. The human checks visuals before this skill runs (PR labelvisual-ok). - Seeds never reach you. Do not read
tests/playwright/.envorqa/.auth/, do not print env. A hook blocks it. Wallets come only from thewalletA/walletBfixtures. Never type a seed in a spec. Never record or screenshot a recovery phrase screen. - Money only from the approved plan. See "Money". Quote and confirm screens are free; the signing tap is not.
- Every page error and crash is a finding. The fixtures attach
errors-wallet-*. Report them under Stability, separate from the ticket verdict. - No app code edits. If the app is wrong, the test stays red and the report says so.
- Stuck 6 minutes → say so. Post what is stuck, what you tried, 2–3 options. Move on to other cases meanwhile.
Case format
tests/playwright/qa/cases/<TICKET>.json, committed with the PR. Written from the spec only.
{
"ticket": "TK-1234",
"cases": [
{
"id": "C1",
"type": "smoke | positive | negative | edge | adjacent",
"title": "Send screen shows the fee for USDT on TRON",
"wallet": "A | B | both | C (empty battery) | new",
"precondition": "Wallet A holds USDT on TRON",
"steps": ["Open Send", "Pick USDT (TRON)", "Enter 1"],
"expected": "Fee row shows a value in TRX before Confirm",
"verify": "assert | human-visual | manual",
"money": null
}
]
}money is null or
{ "kind": "transfer|swap|deposit|withdraw|battery-charge|other", "chain": "ton", "asset": "TON", "amount": "0.05", "from": "A", "to": "B|A|self|<address>" }.
Plan file
<QA home>/out/<TICKET>/plan.json, local, gitignored. QA home is tests/playwright/qa of the
main checkout (shared by all worktrees).
{
"ticket": "TK-1234",
"verdict": "Automatable | Automatable with mocks | Partial | Manual only",
"estimate": "S | M | L",
"cases": [{ "id": "C1", "class": "ui", "how": "assert Fee row text", "needs": "nothing" }],
"transactions": [
{
"id": "tx1",
"kind": "transfer",
"chain": "ton",
"asset": "TON",
"amount": "0.05",
"from": "A",
"to": "B"
},
{
"id": "tx2",
"kind": "transfer",
"chain": "ton",
"asset": "TON",
"amount": "0.05",
"from": "B",
"to": "A"
}
]
}approvedAt is set only after a human approves the plan in chat. Never set it yourself on your own
initiative. Approval of a plan authorizes exactly its listed transactions, nothing else.
Money
- Round trips first. A→B then B→A, swap X→Y then Y→X, deposit then withdraw. Only fees leave the QA wallets. Non-TON chains: send to self (wallet B is TON-only).
- Smallest amount that proves the criterion. Check provider minimums before planning (swaps, bridges, perps). If a minimum is above the cap, the case goes to "not covered" with the number.
- Cheap chains only unless the ticket is about the chain. BTC and Ethereum L1 need a line in the plan with the expected fee.
- In the spec: call
await requireFunds()at the start of a money test (it checks A and B; pass['C']only when a C case needs TON). Do every signing tap insidesignPlanned(plan, 'tx1', async () => { … }). It refuses an unapproved plan, an unlisted id, and outflow overQA_MONTHLY_CAPS, and it appends toout/spend.log. - Report every signed transaction under "Spent" with the id, asset and amount.
Phases
A — Feasibility
- Ticket: Linear MCP, read-only — description, acceptance criteria, labels, comments.
- PR:
gh pr view,gh pr diff. Confirm that the preview URL belongs to the PR's head commit. - Classify every case with references/feasibility.md. Verdict + estimate.
B — Plan (stop for approval)
Order: smoke → positive → negative → edge → adjacent (from the changed files). Each case:
precondition, steps, expected, how it is read back, what it needs (mock, funds, a fresh
wallet). List the transactions with amounts. Write plan.json and plan.md
(references/report-template.md § Plan). Stop. A human approves,
trims or extends.
C — Execute
- Specs live in
tests/playwright/qa/specs/<ticket>.spec.ts. Importtest,expect,loadPlan,signPlanned,requireFundsfrom../fixtures. UsewalletA/walletBpages. Test titles start with<TICKET> <case id>. - Selectors: role + accessible name,
data-testid, ids. Never coordinates. If a control has no stable selector, use text and propose adata-testidfollow-up in the report. - Mocks for backend states:
page.route()inside the spec. Record a real response first, then edit it. Say which cases are mocked. - Run: from
tests/playwright,QA_TICKET=<T> QA_BASE_URL=<preview> yarn qa --grep <T>, thennode qa/summary.mjs <T>. Read the summary, not the raw JSON. - A failure is a spec problem (fix the spec, run again) or an app problem (keep it red, record a finding). Max 3 runs in total. Never retry until green.
- Keep a ledger: case → route → interaction → read back → result → video path.
D — Report
Template: references/report-template.md § Report. Sections in
order, none optional: Verdict, Environment, What was checked, Not covered, Findings in the ticket,
Adjacent findings, Stability, Spent, Next. Save it to <QA home>/out/<TICKET>/REPORT.md.
E — Linear (ask first)
Show the user the findings and the comment text. Post with save_comment only after a clear yes.
Page crashes and errors outside the ticket's code go to a separate issue, if the user names one.
Lessons
Every bug a human finds after a PASS becomes two things: a case in the ticket's cases file and a line in references/lessons.md that says what the agent missed and how to catch it next time.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

