run-tests
Skill71 starsChanged 36 days agoMITView on GitHub
Select and run this repository's checks after a change or while diagnosing test failures. Covers owning seams, contract gates, strict offline isolation and restricted collection; excludes live analysis batches.
- Reads credentials
50 other projects use a version of this.
Adopted by
Most recently taken up by
- tatsoft-llc/FrameworX-Projectstheir own version, first seen Nov 15, 2026
- wisterx-spec/agent-railstheir own version, first seen Nov 15, 2026
- utopia-rise/fmod-gdextensiontheir own version, first seen Nov 9, 2026
- Pubx-ai/skillstheir own version, first seen Nov 8, 2026
- coralogix/cx-clitheir own version, first seen Nov 8, 2026
- DataDog/dd-trace-pytheir own version, first seen Nov 1, 2026
Used with
Projects with it also use prepare-release, create-pr and 3 more.
Other files people use alongside this one.
Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
ShopifySkillprepare-release
Prepare the changeset-release/main branch for release by syncing version.ts files and running pnpm install
Also in 3 projects that use this
SentrySkillcreate-pr
Create pull requests following Sentry conventions. Use when opening PRs, writing PR descriptions, or preparing changes for review. Follows Sentry's code review guidelines.
Also in 4 projects that use this
SentrySkillcommit
Create commit messages following Sentry conventions. Use when committing code changes, writing commit messages, or formatting git history. Follows conventional commits with Sentry-specific issue references.
Also in 4 projects that use this
MicrosoftSkillrelease
Use ONLY when the user explicitly asks to cut/publish a Huabu desktop release or tag a version. Performs an outward action — bumps the version in apps/desktop/package.json, commits + annotated-tags vX.Y.Z, and pushes the branch + tag. That tag push auto-builds the macOS/Windows installers and publishes a GitHub Release. Provides `status`, `bump`, `tag-push`, and a one-command `run` (bump → tag-push).
Also in 5 projects that use this
GoogleSkillcode-review
Adversarial systems, concurrency, memory-model, and architecture code review protocol. Use this skill whenever asked to review code, PRs, diffs, or branches in Filament.
Also in 4 projects that use this
Other versions50 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 50
- netbox-community/netbox★ 22ksame name
- yagop/node-telegram-bot-api★ 9.2ksame name
- gitextensions/gitextensions★ 8.6ksame name
- ethereum/consensus-specs★ 4ksame name
- dotnet/sdk★ 3.3ksame name
- ComplianceAsCode/content★ 2.8ksame name
- idaholab/moose★ 2.4ksame name
- CliMA/Oceananigans.jl★ 1.5ksame name
- serenedb/serenedb★ 1.1ksame name
- microsoft/testfx★ 1.1ksame name
40 more not listed
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository11 other files it gives its agents.
- AGENTS.mdAGENTS.md
- add-agentSkill
- analyze-tickerSkill
- diagnose-runSkill
- maintain-agent-metadataSkill
- run-pipelineSkill
- run-testsSkill
- add-agentSkill
- analyze-tickerSkill
- diagnose-runSkill
- validation-batchSkill
The file
We split this skill into the rules it gives an agent
5 rules. 0 are common, 2 are distinctive, and 3 are very specific to this project.
0
Common
Most of the 45 edited versions we read keep these.
2
Distinctive
This author's own choices. Worth a look.
3
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 30
Use the actual adapter's effective request options for timeout tests, with fake clocks and transports.
- Very specific: line 69
Every non-integration test is offline under tests/conftest.py.
- Very specific: line 74
Only integration is skipped by default.
- Very specific: line 78
Keep configured addopts and plugins.
- Distinctive: line 97
If rootdir collection fails on ancestor guidance outside the allowed surface, retry the same selected tests with:
“Common” is measured: kept by at least 6 in 10 of the 45 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Run repository checks
Inspect the actual diff and owning modules. Run the narrowest meaningful checks while iterating, then the required shared gates. Do not copy suite counts or timing catalogs into this skill: they become stale as the tree grows.
Select the owning checks
| Changed surface | Checks |
| Agent nodes, graph routing or output contracts | Owning tests/agents/ files and compiled-graph tests |
| LLM dispatch, budget admission or retries | tests/llm_runtime/test_timeouts.py, tests/llm_runtime/test_migrated_caller_budgets.py, tests/llm_runtime/test_run_deadline.py, tests/test_timeout_literals.py |
| Tool execution, inspection or MCP scopes | tests/tooling/test_tool_timeout_scopes.py, tests/tooling/test_tool_reason_codes.py, tests/tools/test_inspection_outcome_tools.py, tests/mcp/test_mcp_timeouts.py, tests/mcp/test_budget.py |
| Blocking work and late results | tests/test_blocking_io.py, tests/test_to_thread_timeout_consistency.py, tests/agents/test_runtime_hard_timeout.py |
| Memory probes or availability | tests/memory/test_embedding_probe_admission.py, tests/memory/test_embedding_probe_offline.py |
| Article workflow, publication and writer fallbacks | tests/advanced/test_article_workflow_deadline.py, tests/advanced/test_article_publication.py, tests/advanced/test_article_writer.py |
| Main orchestration, persistence or baseline capture | tests/eval/, tests/test_main_cli.py |
| Logging policy | tests/test_logging_consistency.py |
| Screening behavior | tests/scripts/test_find_gems.py |
| Test isolation or fixtures | tests/test_offline_boundary.py, tests/test_isolation_fixtures.py, tests/config/test_env_file_policy.py, tests/test_conftest_recollection.py |
Use the actual adapter's effective request options for timeout tests, with fake
clocks and transports. Cover spent allowance, tier downgrade, failover, sync/async,
outer cancellation and concurrent calls sharing one model. Model fields alone do
not prove a request was sized correctly. Use stalled_worker(label, prefix=) from
tests/helpers/stalled_workers.py for releasable stalled workers; it releases and
joins them before patch teardown, including when an assertion fails. An orphan
surviving the fake can reach a provider.
Plant the regression a new architectural guard claims to catch, prove it fails, then restore the valid case. Prefer AST scans for code patterns and read the owning constant or registry rather than a test-owned copy.
Required gates
After Python changes:
poetry run ruff check src/ tests/ scripts/
poetry run ruff format --check src/ tests/ scripts/
poetry run mypy src/| Target | Purpose |
make test-ci | Deterministic credential-free suite; excludes slow, integration, LLM and API markers |
make test-prompts | Static parity and contract round-trip after prompt or parser changes |
make replay | Deterministic replay over tracked frozen fixtures |
make eval-semantic | Paid semantic judge; requires authorization and prerequisites |
make check-all | Format, lint, types, documentation and metadata gates |
make docs-guards and make claude-metadata-guards | Claude metadata or public-documentation changes |
make pre-commit | Shared static gates plus deterministic tests |
Broaden testing when the change crosses shared configuration, construction or contracts, or when a focused result leaves a specific concern unresolved. A green directory alone does not prove shared fixtures still work. Report exact commands, outcomes and integration surfaces left unverified.
Strict offline isolation
Every non-integration test is offline under tests/conftest.py. The autouse
boundary refuses sockets, DNS and libcurl, records intercepted network attempts and
fails teardown even when application code catches those refusals. Assert request
counts at faked SDK seams as well; SDK-method refusals do not enter that ledger.
Inject a fake at the data-source seam.
Only integration is skipped by default; slow is filtered by the command.
--run-integration opts in to live tests and needs authorization for external or
paid work. Do not launch provider probes as a diagnostic shortcut.
Keep configured addopts and plugins; never clear them with -o addopts="".
pytest-socket and its --allow-unix-socket addopt are removed; the repository
boundary permits AF_UNIX and socketpair locally. PYTEST_DISABLE_DOTENV must have
the exact string true, as configured in pyproject.toml. Only .env.example
contents may be inspected by agents.
Dummy keys can construct real SDKs and trigger embedding health probes. Supply
tests/helpers/fake_embeddings.py when a unit test needs availability; keep the
probe and wire fakes hermetic. Redirect persistent stores to session temporary
paths and preserve root fixtures that reset shared clients, limiters and bindings.
Standalone audit probes must install isolation before application/provider imports:
set the dotenv switch to literal true, disable LANGSMITH_TRACING,
LANGCHAIN_TRACING and LANGCHAIN_TRACING_V2, and refuse network/DNS using
tests/helpers/offline_network.py. Prefer pytest's existing boundary; assert no
attempts and retain refusal until every worker has finished.
Restricted collection and stalls
If rootdir collection fails on ancestor guidance outside the allowed surface, retry the same selected tests with:
poetry run pytest <paths> -q --rootdir=tests --confcutdir=testsThis preserves tests/conftest.py and isolation. Report the actual retry outcome;
it does not authorize reading the denied file. Do not pipe a long run through
tail: it hides progress until exit.
For a confirmed macOS cloud-placeholder stall, let the public-input preflight hydrate
files. If it fails, run poetry run python scripts/check_macos_cloud_files.py --hydrate
and retry once. It excludes ignored local configuration and private environment files.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

