agentleFS
rgoerwitrgoerwit/ai-investment-agent/.agents/skills/run-tests/SKILL.md

run-tests

Skill71 starsChanged 36 days agoMITView on GitHub

Select and run this repository's relevant static checks and tests after a change or when diagnosing CI. Use for test selection, validation, prompt contracts, or failing quality gates in this repository.

  • Reads credentials

50 other projects use a version of this.

Adopted by

Most recently taken up by

Used with

Projects with it also use prepare-release, create-pr and 3 more.

Other files people use alongside this one.

Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
  • ShopifySkill

    prepare-release

    Prepare the changeset-release/main branch for release by syncing version.ts files and running pnpm install

    Also in 3 projects that use this

  • SentrySkill

    create-pr

    Create pull requests following Sentry conventions. Use when opening PRs, writing PR descriptions, or preparing changes for review. Follows Sentry's code review guidelines.

    Also in 4 projects that use this

  • SentrySkill

    commit

    Create commit messages following Sentry conventions. Use when committing code changes, writing commit messages, or formatting git history. Follows conventional commits with Sentry-specific issue references.

    Also in 4 projects that use this

  • MicrosoftSkill

    release

    Use ONLY when the user explicitly asks to cut/publish a Huabu desktop release or tag a version. Performs an outward action — bumps the version in apps/desktop/package.json, commits + annotated-tags vX.Y.Z, and pushes the branch + tag. That tag push auto-builds the macOS/Windows installers and publishes a GitHub Release. Provides `status`, `bump`, `tag-push`, and a one-command `run` (bump → tag-push).

    Also in 5 projects that use this

  • GoogleSkill

    code-review

    Adversarial systems, concurrency, memory-model, and architecture code review protocol. Use this skill whenever asked to review code, PRs, diffs, or branches in Filament.

    Also in 4 projects that use this

The file

We split this skill into the rules it gives an agent

6 rules. 0 are common, 3 are distinctive, and 3 are very specific to this project.

  • 0

    Common

    Most of the 45 edited versions we read keep these.

  • 3

    Distinctive

    This author's own choices. Worth a look.

  • 3

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Run the narrowest tests that exercise the changed behavior, including a happy path, an edge case, and an error or fallback path where …

    line 12
  2. Distinctive:

    Run formatting and linting for touched Python files.

    line 14
  3. Distinctive:

    Add the specialized contract gate when the touched surface requires it.

    line 16
  4. Very specific:

    Use make check-all for the complete static surface and make test-ci for the deterministic credential-free suite.

    line 43
  5. Very specific:

    Default pytest skips integration.

    line 47
  6. Very specific:

    If pytest collection specifically fails while accessing ancestor guidance, retry the same tests with --rootdir=tests --confcutdir=tests.

    line 78

“Common” is measured: kept by at least 6 in 10 of the 45 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Run repository checks

Base selection on the actual diff and the owning modules. Do not assume a user-level skill or local configuration exists.

  1. Inspect git status --short and the relevant diff.
  2. Run the narrowest tests that exercise the changed behavior, including a happy path, an edge case, and an error or fallback path where applicable.
  3. Run formatting and linting for touched Python files. Before handoff, run the repository-wide static gates when the change affects shared infrastructure.
  4. Add the specialized contract gate when the touched surface requires it.
  5. Report exact commands, outcomes, and any credentialed or integration surface that remains unverified.

Required mappings

  • Prompt JSON, prompt parsers, labels, enums, block markers, or verdict headers: make test-prompts.
  • Main orchestration, persistence, or baseline capture: poetry run pytest tests/eval/ tests/test_main_cli.py -v.
  • Blocking work or async timeouts: poetry run pytest tests/test_to_thread_timeout_consistency.py tests/agents/test_runtime_hard_timeout.py -v.
  • LLM budget resolution, transport admission or retry ownership: poetry run pytest tests/llm_runtime/test_timeouts.py tests/llm_runtime/test_migrated_caller_budgets.py tests/llm_runtime/test_run_deadline.py tests/test_timeout_literals.py -v.
  • Tool/inspection scope or MCP deadline changes: poetry run pytest tests/tooling/test_tool_timeout_scopes.py tests/tooling/test_tool_reason_codes.py tests/tools/test_inspection_outcome_tools.py tests/mcp/test_mcp_timeouts.py tests/mcp/test_budget.py -v.
  • Article workflow, publication or writer fallback changes: poetry run pytest tests/advanced/test_article_workflow_deadline.py tests/advanced/test_article_publication.py tests/advanced/test_article_writer.py -v.
  • Blocking worker lifecycle or late-result handling: poetry run pytest tests/test_blocking_io.py tests/test_tavily_utils.py tests/data/test_gap_fill.py -v.
  • Logging and exception safety: poetry run pytest tests/test_logging_consistency.py -v.
  • MCP behavior: poetry run pytest tests/mcp/ -v.
  • Screening behavior: poetry run pytest tests/scripts/test_find_gems.py -v.
  • Codex metadata or public documentation: make docs-guards and make agent-metadata-guards.

Use make check-all for the complete static surface and make test-ci for the deterministic credential-free suite. Do not launch paid, networked, integration, or destructive tests without the request and prerequisites that authorize them.

Default pytest skips integration; --run-integration is an explicit opt-in, not permission to spend or access external services. Non-integration tests are strictly offline under tests/conftest.py. Preserve configured addopts and plugins, including the exact string true for PYTEST_DISABLE_DOTENV; there is no pytest-socket or --allow-unix-socket addopt. Dummy credentials can still trigger real SDK probes: use fake providers for unit availability tests. Use stalled_worker(label, prefix=) from tests/helpers/stalled_workers.py for releasable stalled workers. It releases and joins them before monkeypatch teardown, including when an assertion fails, so late work cannot escape the boundary.

For a standalone audit probe, install isolation before importing application or provider modules: set PYTEST_DISABLE_DOTENV to the exact string true, disable LANGSMITH_TRACING, LANGCHAIN_TRACING and LANGCHAIN_TRACING_V2, and refuse network/DNS through tests/helpers/offline_network.py. Prefer the existing pytest boundary. Dummy credentials alone are insufficient; assert no attempts and retain refusal until all probe workers finish.

The offline ledger records intercepted socket/DNS/libcurl attempts; assert request counts at faked SDK seams too, because SDK-method refusals are not ledger entries.

Budget tests must exercise the selected adapter's effective request options, not only mocked factory arguments or model fields. Cover quick/full mode, sync/async, minimum-allowance rejection, tier downgrade, model failover, outer cancellation, and concurrent calls sharing a model. Use fake clocks/transports for request budgets and releasable stalled workers for caller bounds; test interpreter exit in a subprocess with an external test timeout. Plant the defect a new guard claims to reject, then prove the valid centralized-policy case passes. Polling, cooldowns and protocol constants are not request deadlines.

Restricted collection

If pytest collection specifically fails while accessing ancestor guidance, retry the same tests with --rootdir=tests --confcutdir=tests. This retains tests/conftest.py and its credential-free isolation. It does not authorize access to private configuration or suppress failures inside tests. Record the exact remaining limitation instead of treating every sandbox collection failure as an unrun suite.

Detect macOS collection stalls

A focused file should normally start promptly. The full suite can take much longer, with continuously advancing collection; that alone is not a hang. If a focused probe is repeatedly slow or collection stops advancing, let pytest's macOS preflight hydrate public inputs automatically. If it cannot, immediately run poetry run python scripts/check_macos_cloud_files.py --hydrate, then rerun the exact test once. The checker excludes ignored local configuration and every .env* file except .env.example, even if accidentally tracked.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.