Fix a failing or flaky test

A test that fails only sometimes has a cause: a race, a wait on the wrong thing, a ceiling a slow runner outran, or a difference between systems. Find which before changing anything. Never retry, skip or quarantine a test, and never add a fixed delay (CLAUDE.md, "Tests").

1. Collect the failures

• List the CI workflow's failed runs (ci.yml; package.yml runs it too before a release). Cloud sessions use the GitHub tools; locally, where gh isn't installed, GitHub's REST API: curl -s "https://api.github.com/repos/dotBunny/Deventry/actions/workflows/ci.yml/runs?status=failure&per_page=20", and a run's jobs at .../actions/runs/<id>/jobs. Logs and artifacts need a sign-in, so ask the user for them.
• A failed leg uploads test-results-<os>: its .trx files and any *.received.txt, kept for 7 days. Download and unzip each into a folder of its own and summarize it with node .github/scripts/test-summary.mjs <folder>.
• Older runs have only their logs. Ask for the failed jobs' logs alone, save them rather than reading them whole, and search them for [FAIL] and the Error Message: after it.
• Group the failures by test, with the system, the message and how long the test took (its UnitTestResult's duration).

2. Rule out what isn't a flake

• git log the test and the code it covers since the failing commit: a later commit may already have fixed it.
• A test failing on all three systems at once is rarely a flake: a stale snapshot (the artifact has its *.received.txt) or a real regression. Fix it as one.
• A RealCli test that fails only inside a Claude Code session isn't CI's failure: there the real claude is the session's own, not the version compat/surface.yaml names (lastTested), and the session's CLAUDE_* variables pass through to it as user configuration.

3. Tell which kind it is

About fifty failures have been fixed since September, and nearly all were one of these. Each kind's rule is in CLAUDE.md, "Tests" and "Rules that keep the code testable".

• A wait for the wrong state, half of them: the assertion fails straight after a wait that passed. The wait was for something that comes first (stored before applied, Idle before the turn), stands in for the thing checked, or was still true from the step before. Read the code between the wait and the assertion, and what sets each.
• Work on the wrong thread: Collection was modified, a list or event caught half filled, a test's own command resuming outside the harness's UI lock. Look for work the test started off the UI thread, and fakes written to from sessions' threads.
• A race in the app, a fifth of them: a missing key or a wrong value with nothing slow about the run. Look for state set after the work that reads it can start, a check and its record in two steps, an older result landing after a newer one, a lock held while starting a callback.
• One system's difference: it fails on one system every time, or nearly, and passes on the others. Check the list under "Write each test for all three systems".
• A runner that stalled: a ceiling ran out. Each UnitTestResult has a startTime and a duration: sort a run's results by startTime and compare what ran alongside the failure with the same tests in another run. When unrelated tests in the same minutes took 10 to 100 times as long too, the runner stalled; when only the tests that write to disk (SQLite, git, the library, atomic writes) slowed and the ones that only compute didn't, it was the disk. The fix is the ceiling: the wait uses Waiting.Timeout.

4. Reproduce it

• Build Release, then run the one test in a loop from its xunit v3 executable, which starts faster than dotnet test: tests/<Project>/bin/Release/net10.0/<Project> -method "<Namespace.Class.Method>" -noLogo, counting the runs whose summary says Failed: 0.
• Load the machine as a busy runner is, with a busy loop per core (sh -c 'while :; do :; done') started and stopped by a script of its own. Don't stop them with pkill -f on a pattern your own shell's command line also contains.
• A race that won't show can be proven by widening its window: a temporary Thread.Sleep where the gap is, with #pragma warning disable RS0030 around it. Show the old code failing with it and the fix passing with the same sleep, then take the sleep out.

5. Fix the cause

In this order of preference:

1. The order of the code. Set what asynchronous work reads before anything can start that work, rather than counting on the UI thread to queue it (CLAUDE.md, "Rules that keep the code testable"). Read the app's code before the test's: a race the tests see can be one people hit on a slow machine, as a fifth of these were.
2. What the test waits for. The state the next step reads, not one that comes before it or is already true.
3. The ceiling. Real work is bounded by Waiting.Timeout, not a shorter time picked for the case.

A bug people could have run into, such as an error that hides the real one, gets its entry under Fixed in CHANGELOG.md.

6. Prove it

• The new or fixed test fails without the fix and passes with it.
• The free suite passes several times in a row under load: dotnet test Deventry.slnx -c Release --no-build --filter "Category!=Live&Category!=RealCli".
• The pull request says, for each failure, what it was, how it was reproduced and what shows it fixed.