resilience-prober
Skill1 starsChanged yesterdayMITView on GitHub
Check how a running web app behaves when what it depends on goes wrong - slow or failing APIs, rate limits, expired sessions, hung requests, dropped connections, going offline, malformed payloads, one dependency down, flapping endpoints and failed script chunks. Each fault is injected in the browser only (Playwright routing), on a chosen step of a real user flow, after a clean baseline, with the expected outcome written before the run. Records what the user sees and loses - crashes, blank screens, endless spinners, silent failures, misleading empty states, lost input, no way to recover, retry storms - with screenshots, request logs and a standalone repro script. Never calls an app resilient.
- Installs packages
In this repository33 other files it gives its agents.
- CLAUDE.mdCLAUDE.md
- api-contract-testerSkill
- bug-reproducerSkill
- ci-failure-triagerSkill
- ci-suite-runnerSkill
- code-security-auditorSkill
- cross-browser-checkerSkill
- db-migration-reviewerSkill
- dead-test-finderSkill
- exploratory-charter-writerSkill
- flaky-test-hunterSkill
- hypercare-anomaly-watcherSkill
- i18n-checkerSkill
- journey-mapperSkill
- journey-to-testsSkill
- mobile-app-auditorSkill
- mutation-testerSkill
- performance-budget-auditorSkill
- post-deploy-validatorSkill
- privacy-flow-auditorSkill
- qa-orchestratorSkill
- release-readiness-gateSkill
- requirements-traceability-auditorSkill
- risk-based-test-plannerSkill
- test-case-designerSkill
- test-data-auditorSkill
- test-impact-selectorSkill
- test-suite-evaluatorSkill
- ticket-readiness-reviewerSkill
- ticket-to-tddSkill
- ui-crash-hunterSkill
- visual-regression-reviewerSkill
- wcag-auditorSkill
The file
We split this skill into the rules it gives an agent
2 rules. About 0 are common, 1 is distinctive, and 1 is very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈1
Distinctive · estimate
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 82
Run plan_scenarios.py --plan-only first: it prints the scenarios and an estimate (runs, page loads, requests at requests_per_load) against …
- Distinctive: line 173
Fallback to a full page load: When a route payload fails (a 500, a truncated or HTML body), Next aborts it and loads the page as a …
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Resilience prober
Mission
Find out whether a user still gets somewhere sensible when a dependency fails. Drive real flows, break one thing at a time at the network layer, and record the user-visible effect against an outcome written down beforehand. Merge scenarios with one root cause, rate them, and hand over a repro and a fix for each.
The UI crash hunter starts from the code and asks what makes the UI crash; this skill starts from the running app. It reuses the crash hunter's tiers (B1 app-fatal, B2 screen-fatal). The code security auditor sends attacker payloads to the server; this skill never sends anything unusual to the server. Native apps belong to the mobile app auditor.
Non-negotiable rules
- Faults are injected in the browser only. Playwright fulfils, delays or aborts the request, or takes the context offline. A write a scenario targets is answered in the browser, never forwarded.
- The baseline must be clean. Each flow runs twice without faults first; a failed or unstable
baseline makes that flow's scenarios
untested. - Expected behaviour is written before the run (
plan_scenarios.py), with the budgets: an error message within 10 s, no spinner past 30 s, input kept, recovery without a reload. - Every finding comes from a run: the flow, step, fault, effect, a screenshot and the request log.
Reading code gives a
suspectednote, never a finding. - A degradation needs a user-visible effect. A console error alone is a note.
- Retries are measured: request counts and gaps from the run log.
- "Nothing broke" is not "resilient". The report lists the flows, endpoints and faults that ran,
separately the planned faults that didn't, and what the browser can't reach (
unreachable). Never "production ready" or "recovers gracefully from every failure". - Synthetic data only in typed input and injected bodies; input is recorded as length and SHA-256.
- Safe targets. Loopback needs nothing; other hosts need
authorised_by. The site's host is registered with the browser harness and every other host is blocked. Writes are blocked unlessallow_form_submissionis true (default false; the report records which). Set arequest_budgetfor anything shared: one page load of a modern app is 50-80 requests, and the default plan is 2 baselines per flow plus--runsper scenario. - A fault that never fired tested nothing. A scenario whose fault matched no request is
not_triggered, has no effects and is not counted as run. - A broken journey without faults is the first result. A failed baseline step is screenshotted and observed, and the headline names the flow.
Inputs
| Input | Default | Notes |
flows | required | flows.json: site, app root, flows with named steps (actions, a done selector, input, write), critical_path. A step may add done_not (selectors or text= that must be absent for the step to count as done) and optional_endpoints (globs for background widgets the step does not need) |
reload_after_recovery | true | After recovery, reload and run the flow to the step again (reads under a failing or static-asset fault; writes are never sent again), recording requests to the endpoint and the Cache Storage entries; false turns it off |
scenarios | required | Explicit {flow, step, endpoint, fault, method?, headers?, resource_type?, from?} or {"auto": {"endpoints": {...}, "faults": [...]}} |
faults | data/faults.json standard set | Latency, server errors, rate limits, auth expiry, timeouts, connection, offline, payload, partial, flapping, static assets |
budgets | 10 s message, 30 s spinner | data/indicators.json |
indicators | data/indicators.json | The product's own wording and selectors, in flows.json (global or per flow): spinner_selectors (e.g. .animate-pulse skeletons), loading_words ("loading availability"), empty_state_words ("no results"), boundary_fallback_words and boundary_selectors (the app's error boundary, e.g. "this route hit turbulence"), error_selectors, error_words. Lists are added to the defaults; run.json records the merged set |
authorised_by, auth, allow_form_submission | writes blocked | allow_form_submission defaults to false here; targets, allowed_hosts, host_map and a harness block pass through to browser_harness.py; auth: {"storage_state": file} runs signed in, with a session a person captured by signing in by hand (python scripts/browser_harness.py capture-session --url <app> --out <file outside the repo>; cookies, localStorage, IndexedDB and sessionStorage); the run stays read-only and records only that a session was used, never the file or its contents |
request_budget / --max-requests | none | Requests the browser may send to the site in the whole run |
setup, init_script | none | Actions after the first load (dismiss a cookie banner), JavaScript before every page (seed localStorage) |
prefetch | allow | block aborts prefetch requests (Next-Router-Prefetch, Sec-Purpose: prefetch), so a navigation fetches and a fault can reach it. It changes a Next.js app: see below |
service_workers | block | allow (in flows.json or on one flow) lets the app's service worker run: the first load waits for it and reloads once so it controls the page, as for a returning visitor; offline then reaches the worker, and a request it answers from its cache counts as reached by the fault. Route faults may miss requests the worker answers |
nav_timeout_ms, step_timeout_ms, run_timeout_s | 30000, 10000, from the budgets | Page loads and history steps; a step before the target; the watchdog's limit for one baseline or scenario run (--run-timeout) |
baseline_retries, baseline_retry_pause_s | 1, 5 | A failed baseline run is tried again after a pause (a loaded machine can miss a step's budget once); a flow that passed only on a retry is recorded retried |
requests_per_load | 60 | For the planner's request estimate |
unreachable | none | [{flow?, step?, what, reason}]: data the browser cannot reach (fetched by a server component, cached at the edge); copied to the plan and the report |
devices | desktop Chromium | Add a mobile profile for mobile web |
har | none | Real traffic, to weight reach by each endpoint's share of requests |
Workflow
run_resilience.py needs Playwright (pip install -r requirements.txt, python -m playwright install
chromium; ask first). Everything else is stdlib.
python scripts/plan_scenarios.py flows.json --out <run>/scenarios.json [--plan-only] [--runs 2] [--repo <app source>]
python scripts/run_resilience.py flows.json <run>/scenarios.json --out <run> --runs 2 [--resume]
python scripts/classify.py <run>/run.json <run>/scenarios.json flows.json --out <run>/candidates.json
python scripts/build_report.py --run <run>/run.json --scenarios <run>/scenarios.json --candidates <run>/candidates.json \
--flows flows.json --promote K1,K2 --repros <run>/repros --out <run>/resilience_report.jsonRun plan_scenarios.py --plan-only first: it prints the scenarios and an estimate (runs, page loads,
requests at requests_per_load) against request_budget, with over_budget when the plan won't fit. Look
at the baselines before spending the scenarios: a flow whose baseline fails makes its scenarios untested.
With --repo the planner reads a Next.js app's source (next_app.py) and lists the service workers it
ships with their fetch strategies (service_workers in the plan); it warns when the run would block them.
The runner never blocks: every read of the page has a deadline (Playwright's page.evaluate has none, and
a page whose navigation timed out never answers it), and a watchdog runs the work in a child process with a
time limit per baseline or scenario run. A run past its limit is stopped with its browser, listed in
<run>/hung.json, recorded errored (a baseline run failed) with the watchdog's reason, and the run goes
on after it. run.json is written after every flow's baselines and every scenario (run_status: partial
until the end); --resume continues a stopped run, and build_report.py and the validator refuse a partial
one.
- Recon (optional, with source): the data layer, retry and timeout settings, error boundaries
(the UI crash hunter's
recon.py). - Plan. Scenarios with their expected outcome, written before anything runs. Choose faults on every critical step's first API call, and partial and static-asset faults once per flow.
- Run. Baselines twice, then each scenario twice (reproduced or only observed). Faults on a flow's
first step are in place before the page loads. Recovery uses the app's own retry control (an
accessible name matching retry, try again, reload) or the step's actions again, never a reload; a
step that only types (type-ahead) is recovered by re-sending the field's value (
refill_input). A step whose only action is a navigation cannot be repeated without reloading, so its recovery isreload(not tried) and arecovery_not_triednote, neverrecovered(so is aback,forwardorreloadstep: steps may use browser history,{"back": true}). Then the page is reloaded with the fault gone and the flow run to the step again: a step that makes no request to the endpoint after the reload, when the server never answered it well in the run, replays a failure the app stored (persisted_failure: Cache Storage, IndexedDB, localStorage). A reload that never loaded is recorded with its error and areload_not_checkednote, never read as a stored failure. A failed baseline run is tried once more after a pause; a flow that passed only then isretried(confidence medium at most). - Classify. Effects per run, merged by root cause (one effect at one step and endpoint across
faults; no recovery across a step's endpoints; retry storms per endpoint), rated by
RUBRIC.md. - Triage.
build_report.pyfills everything the run decides (scope, metadata, baseline, scenarios, coverage, unreachable, ratings, evidence, headline counts and confidence, provable limitations) and writes a guarded repro script per promoted candidate. Replace everyTODO: promote, keep for review, or dismiss with a checkable reason (anything left inneeds_reviewkeeps the headline confidence at medium at best; emptying it is what lets it reach high) (an optional background widget left empty, an optimistic update that rolls back with a message). For each finding write the title, the actual behaviour against the expected, the user impact, a fix (a timeout, an error state, keeping input, honouring Retry-After, a chunk retry), a verification step, and check the repro script. - Validate and render.
python scripts/validate_resilience.py <run>/resilience_report.json --run <run>/run.json --scenarios <run>/scenarios.json --candidates <run>/candidates.json
python scripts/render_report.py <run>/resilience_report.json [--brief]A step's done must prove the data arrived, not only that the UI moved on. An app that shows its
"ready" text whether or not the download succeeded makes every server-error scenario look passed. A
failing fault on a read whose step still completed, with no error shown, is a misleading_state
(shown_as_done) unless the step lists the endpoint in optional_endpoints; give such steps a
done_not (the app's error text) or a done only real data can produce. When the run cannot judge
something the agent can document (stand-in data for a binary download, a canvas screen with no text),
add confidence_caps: [{"cap": "medium" | "low", "reason": ...}]: it lowers the headline confidence,
never raises it.
A step whose baseline already shows under 20 visible characters (a canvas, a video) cannot be judged
blank or crashed by its text: such runs give a baseline_blank note to check against the screenshots,
and a screen with a visible video, canvas or large image (media_visible) is never blank_screen. A page
error counts toward a crash only when it is new against the baseline and the app root emptied where the
baseline's root had text. A static-asset fault that leaves the step incomplete is a chunk_failure even
when the static HTML stays.
Offline (and any fault) on a step that becomes a full page load can end on the browser's own error page
(chrome-error://): the run records browser_error_page with the URL and screenshots, and classify makes it
a browser_error_page note (no app code ran there), never errored. Offline counts only failed requests
the scenario targets (its pattern, method, headers and resource type), never every request that failed in
the context, and a not_triggered reason says whether matching requests came before the fault, only after
it, or never (a client cache). A failure that only hit images, fonts or media is optional for the step by
default (the step's data didn't depend on it), and visible broken images are counted (broken_images note).
Edge cases: service workers are blocked unless service_workers is allow (they would answer before
routing); with allow, plan offline scenarios on navigations to see what a returning visitor gets (the
worker's cached page is a served_by_service_worker note, the browser's offline page a
browser_error_page note); GraphQL needs
faults matched on the operation name; WebSockets can be closed but not faked; third-party scripts are
a natural partial failure; optimistic UI is only misleading if it doesn't roll back or warn. A fault on
the page's own HTML document (a navigation, a server-rendered page) shows the catalogue's injected body,
not the app: it becomes a document_fault note, never a blank screen or crash; aim faults at the
requests the page makes instead. Type-ahead queries for different terms are not retries (a storm counts
one URL and query, cache-busters dropped).
Next.js and other server-rendering frameworks:
- Prefetch. Links in view are prefetched at page load (
?_rsc=,Next-Router-Prefetch: 1); a fault installed for a later step misses them and the scenario isnot_triggered. The faithful choice is"from": "load"with"headers": {"Next-Router-Prefetch": "1"}(the fault hits the prefetch, as a real visitor's would fail)."prefetch": "block"makes the navigation fetch ("headers": {"RSC": "1"}) but changes the app: Next logs "Failed to fetch RSC payload ... Falling back to browser navigation" for every link and turns client navigations into full page loads, so aloading.tsxskeleton never shows and a hang looks like "nothing happens". classify notes it (prefetch_changed_app) andbuild_report.pyputs it in limitations. - Fallback to a full page load. When a route payload fails (a 500, a truncated or HTML body), Next aborts
it and loads the page as a document, which the fault didn't touch: the right page shows. The run records
the page's document loads (
documents), and a failed request followed by a 2xx document load of the same path is afull_page_fallbacknote, notmisleading_state. A step answered from the router's client cache makes no request, so the fault doesn't trigger (not_triggered, with the reason). - Matching. The document
/featuresand its payload/features?_rsc=xshare a path; tell them apart withheadersorresource_type(document,fetch). - Server components fetch on the server; browser faults reach only the payload carrying the result.
List that data in
unreachableso the report says it wasn't testable. - Error boundaries (
error.tsx) catch the error, so there is no page error: put the app's boundary wording or selector inindicators(boundary_fallback_words,boundary_selectors), or a boundary render is classed as a silent failure instead of a crash (Fatal). - Skeletons (
.animate-pulse) and text loaders go inspinner_selectors/loading_words, empty states ("No results") inempty_state_words, or endless loading and misleading empty states can't fire. - Watch window. Server-rendered pages show their main content at once; the runner keeps watching for the scenario's whole window (10 s, 30 s while loading shows) for failing faults, so a widget that never stops loading is still caught.
Output fields
scope (faults that ran, faults_untested), metadata (browser, requests sent, request budget, whether
writes were allowed, blocked requests, runs per scenario, service workers), baseline (per flow: status,
retried when a run passed only when tried again, and for a broken flow the failed step, what it showed
and screenshots), scenarios[] (with expected outcome,
status ran | untested | errored | not_triggered, reason, effects, screenshots), findings[] (effect,
impact, likelihood, reach, score, priority, confidence, floors, endpoints, faults, scenarios, expected,
actual, user impact, evidence, repro, fix, verification), notes[] (console_only, document_fault, recovery_not_tried,
baseline_blank, browser_error_page, full_page_fallback, served_by_service_worker, broken_images,
observation_unavailable, reload_not_checked, prefetch_changed_app), confidence_caps[] (optional: {cap, reason}), coverage (per
fault group: ran, untested, errored, not_triggered), unreachable[], headline, needs_review[],
triage[], limitations[]. run.json keeps each matching request with its query (sensitive values
redacted), whether the injector answered or held it, and its response time.
What the validator enforces
run.json is complete (not partial). No TODO is left. Scenarios match the plan and the run, their expected outcome predates the run, and
nothing ran for a failed baseline; a scenario whose fault never fired is not_triggered. The baseline
section equals run.json, its screenshots exist, and the headline names every broken flow. scope.faults
holds only faults that ran, faults_untested the rest; metadata and unreachable equal the run and
the plan. Findings match their candidates, cite runs that triggered the fault, come from scenarios
that ran with screenshots and request logs, and have repro scripts. A P0 is reproduced. Retry storms
carry counts and timings; lost input carries lengths and hashes only. Console-only items are never
findings. Coverage, the headline and confidence (low when under half the scenarios ran, a critical
baseline failed, or scenarios ran once; medium at most when a baseline passed only on a retry; lowered by any confidence_caps) are recomputed, and no summary, limitation or finding claims
resilience, production readiness or graceful recovery from every failure.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

