agentleFS
rgoerwitrgoerwit/ai-investment-agent/.claude/skills/run-tests/SKILL.md

run-tests

Skill71 starsChanged 36 days agoMITView on GitHub

Select and run this repository's checks after a change or while diagnosing test failures. Covers owning seams, contract gates, strict offline isolation and restricted collection; excludes live analysis batches.

  • Reads credentials

50 other projects use a version of this.

Adopted by

Most recently taken up by

Used with

Projects with it also use prepare-release, create-pr and 3 more.

Other files people use alongside this one.

Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
  • ShopifySkill

    prepare-release

    Prepare the changeset-release/main branch for release by syncing version.ts files and running pnpm install

    Also in 3 projects that use this

  • SentrySkill

    create-pr

    Create pull requests following Sentry conventions. Use when opening PRs, writing PR descriptions, or preparing changes for review. Follows Sentry's code review guidelines.

    Also in 4 projects that use this

  • SentrySkill

    commit

    Create commit messages following Sentry conventions. Use when committing code changes, writing commit messages, or formatting git history. Follows conventional commits with Sentry-specific issue references.

    Also in 4 projects that use this

  • MicrosoftSkill

    release

    Use ONLY when the user explicitly asks to cut/publish a Huabu desktop release or tag a version. Performs an outward action — bumps the version in apps/desktop/package.json, commits + annotated-tags vX.Y.Z, and pushes the branch + tag. That tag push auto-builds the macOS/Windows installers and publishes a GitHub Release. Provides `status`, `bump`, `tag-push`, and a one-command `run` (bump → tag-push).

    Also in 5 projects that use this

  • GoogleSkill

    code-review

    Adversarial systems, concurrency, memory-model, and architecture code review protocol. Use this skill whenever asked to review code, PRs, diffs, or branches in Filament.

    Also in 4 projects that use this

The file

We split this skill into the rules it gives an agent

5 rules. 0 are common, 2 are distinctive, and 3 are very specific to this project.

  • 0

    Common

    Most of the 45 edited versions we read keep these.

  • 2

    Distinctive

    This author's own choices. Worth a look.

  • 3

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Use the actual adapter's effective request options for timeout tests, with fake clocks and transports.

    line 30
  2. Very specific:

    Every non-integration test is offline under tests/conftest.py.

    line 69
  3. Very specific:

    Only integration is skipped by default.

    line 74
  4. Very specific:

    Keep configured addopts and plugins.

    line 78
  5. Distinctive:

    If rootdir collection fails on ancestor guidance outside the allowed surface, retry the same selected tests with:

    line 97

“Common” is measured: kept by at least 6 in 10 of the 45 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Run repository checks

Inspect the actual diff and owning modules. Run the narrowest meaningful checks while iterating, then the required shared gates. Do not copy suite counts or timing catalogs into this skill: they become stale as the tree grows.

Select the owning checks

Changed surfaceChecks
Agent nodes, graph routing or output contractsOwning tests/agents/ files and compiled-graph tests
LLM dispatch, budget admission or retriestests/llm_runtime/test_timeouts.py, tests/llm_runtime/test_migrated_caller_budgets.py, tests/llm_runtime/test_run_deadline.py, tests/test_timeout_literals.py
Tool execution, inspection or MCP scopestests/tooling/test_tool_timeout_scopes.py, tests/tooling/test_tool_reason_codes.py, tests/tools/test_inspection_outcome_tools.py, tests/mcp/test_mcp_timeouts.py, tests/mcp/test_budget.py
Blocking work and late resultstests/test_blocking_io.py, tests/test_to_thread_timeout_consistency.py, tests/agents/test_runtime_hard_timeout.py
Memory probes or availabilitytests/memory/test_embedding_probe_admission.py, tests/memory/test_embedding_probe_offline.py
Article workflow, publication and writer fallbackstests/advanced/test_article_workflow_deadline.py, tests/advanced/test_article_publication.py, tests/advanced/test_article_writer.py
Main orchestration, persistence or baseline capturetests/eval/, tests/test_main_cli.py
Logging policytests/test_logging_consistency.py
Screening behaviortests/scripts/test_find_gems.py
Test isolation or fixturestests/test_offline_boundary.py, tests/test_isolation_fixtures.py, tests/config/test_env_file_policy.py, tests/test_conftest_recollection.py

Use the actual adapter's effective request options for timeout tests, with fake clocks and transports. Cover spent allowance, tier downgrade, failover, sync/async, outer cancellation and concurrent calls sharing one model. Model fields alone do not prove a request was sized correctly. Use stalled_worker(label, prefix=) from tests/helpers/stalled_workers.py for releasable stalled workers; it releases and joins them before patch teardown, including when an assertion fails. An orphan surviving the fake can reach a provider.

Plant the regression a new architectural guard claims to catch, prove it fails, then restore the valid case. Prefer AST scans for code patterns and read the owning constant or registry rather than a test-owned copy.

Required gates

After Python changes:

poetry run ruff check src/ tests/ scripts/
poetry run ruff format --check src/ tests/ scripts/
poetry run mypy src/
TargetPurpose
make test-ciDeterministic credential-free suite; excludes slow, integration, LLM and API markers
make test-promptsStatic parity and contract round-trip after prompt or parser changes
make replayDeterministic replay over tracked frozen fixtures
make eval-semanticPaid semantic judge; requires authorization and prerequisites
make check-allFormat, lint, types, documentation and metadata gates
make docs-guards and make claude-metadata-guardsClaude metadata or public-documentation changes
make pre-commitShared static gates plus deterministic tests

Broaden testing when the change crosses shared configuration, construction or contracts, or when a focused result leaves a specific concern unresolved. A green directory alone does not prove shared fixtures still work. Report exact commands, outcomes and integration surfaces left unverified.

Strict offline isolation

Every non-integration test is offline under tests/conftest.py. The autouse boundary refuses sockets, DNS and libcurl, records intercepted network attempts and fails teardown even when application code catches those refusals. Assert request counts at faked SDK seams as well; SDK-method refusals do not enter that ledger. Inject a fake at the data-source seam. Only integration is skipped by default; slow is filtered by the command. --run-integration opts in to live tests and needs authorization for external or paid work. Do not launch provider probes as a diagnostic shortcut.

Keep configured addopts and plugins; never clear them with -o addopts="". pytest-socket and its --allow-unix-socket addopt are removed; the repository boundary permits AF_UNIX and socketpair locally. PYTEST_DISABLE_DOTENV must have the exact string true, as configured in pyproject.toml. Only .env.example contents may be inspected by agents.

Dummy keys can construct real SDKs and trigger embedding health probes. Supply tests/helpers/fake_embeddings.py when a unit test needs availability; keep the probe and wire fakes hermetic. Redirect persistent stores to session temporary paths and preserve root fixtures that reset shared clients, limiters and bindings.

Standalone audit probes must install isolation before application/provider imports: set the dotenv switch to literal true, disable LANGSMITH_TRACING, LANGCHAIN_TRACING and LANGCHAIN_TRACING_V2, and refuse network/DNS using tests/helpers/offline_network.py. Prefer pytest's existing boundary; assert no attempts and retain refusal until every worker has finished.

Restricted collection and stalls

If rootdir collection fails on ancestor guidance outside the allowed surface, retry the same selected tests with:

poetry run pytest <paths> -q --rootdir=tests --confcutdir=tests

This preserves tests/conftest.py and isolation. Report the actual retry outcome; it does not authorize reading the denied file. Do not pipe a long run through tail: it hides progress until exit.

For a confirmed macOS cloud-placeholder stall, let the public-input preflight hydrate files. If it fails, run poetry run python scripts/check_macos_cloud_files.py --hydrate and retry once. It excludes ignored local configuration and private environment files.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.