agentleFS
ianshankianshank/Agents/experiments/AGENTS.md

Agents / experiments

AGENTS.md1 starsChanged 58 days agoApache-2.0View on GitHub

Temporary subtrees outside the main quality bar. Each owns its gate; none is in check-all. An experiment consumes the monorepo's packages to answer one empirical question, then gets promoted or deleted. README.md states the policy and how to add one. This file is what an agent needs when working inside an experiment: what isolation means here, and which gate actually runs. - Nothing here is in make check-all. The root gate will stay green while an experiment is broken, so…

The file

Files
AGENTS.md

AGENTS.md — experiments

Temporary subtrees outside the main quality bar. Each owns its gate; none is in check-all.

An experiment consumes the monorepo's packages to answer one empirical question, then gets promoted or deleted. README.md states the policy and how to add one. This file is what an agent needs when working inside an experiment: what isolation means here, and which gate actually runs.

Map

PathRole
backend-validation/Claimed-versus-observed evidence for the eval-backend decision; own pyproject.toml, own gate, ships unsigned
backend-validation/PROBES.yaml, RUBRIC.mdLive probes, gated behind an explicit human sign-off that stops the phase chain
trace-analytics/Offline SQL sketches over fixture JSONL using the standard library only
trace-analytics/expected.jsonThe golden file check_offline.py --update rewrites

Diagram

flowchart TD
  accTitle: how an experiment relates to the repository gate
  accDescr: The root check-all target covers the packages only; each experiment owns a separate make check that the root never invokes, and a promoted experiment leaves as a package or an ADR.

  ROOT["make check-all<br/>root plus five packages"]
  PKG["the packages"]

  subgraph EXP["experiments/ (you are here)"]
    BV["backend-validation<br/>make -C ... check"]
    TA["trace-analytics<br/>make -C ... check"]
    SIGN["PROBES.yaml sign-off<br/>stops until a human signs"]
  end

  OUT["promoted to a package or an ADR,<br/>or deleted"]

  ROOT --> PKG
  PKG --> BV
  PKG --> TA
  BV --> SIGN
  BV --> OUT
  TA --> OUT

  classDef here fill:#e8f0fe,stroke:#1a73e8,stroke-width:2px
  class EXP here

Rules that bite here

  • Nothing here is in make check-all. The root gate will stay green while an experiment is broken, so run the experiment's own gate yourself. That isolation is the point, not an oversight.
  • The dependency arrow points one way. An experiment may depend on the harness and the packages; the root and the packages must never depend on an experiment. An experiment that something imports is a package, and should be promoted rather than imported.
  • Zero writes outside the subtree. backend-validation has a PR-scoped isolation check for exactly this, and trace-analytics states it as an invariant.
  • trace-analytics deliberately has no pyproject.toml. A manifest there would pull in dependency and coverage obligations the experiment does not have. Do not add one to make tooling happy.
  • Keep README.md's experiment table in sync. Both backend-validation and trace-analytics must appear there when present on disk; add a row when you introduce an experiment rather than leaving the index to drift.
  • Live probes are sign-off gated. backend-validation's preflight phase stops with a distinct exit code until a human sign-off file exists. Do not route around it.

Verify

make -C experiments/trace-analytics check
make -C experiments/backend-validation check

Subagents

Task in this directoryAgentWhy
Confirm nothing in the packages imports an experimentexplorerRead-only Grep; the one-way dependency rule has no CI check at the root
Run one experiment's own gate and isolate the failuretest-runnerHas Bash; each experiment has a different make check and its own floor
Review an experiment before promoting it into a packagenarrow-criticPromotion imports unsigned code into the protected quality bar

See also

DocRead it when
README.mdYou are adding an experiment and need the policy and the exit criteria
backend-validation/README.mdYou are running the backend validation phases or its sign-off chain
trace-analytics/README.mdYou are touching the SQL sketches or regenerating the golden file
../docs/CHARTER.mdAn experiment is edging toward a declared non-goal and needs escalation
../docs/AGENTS.mdThe experiment has served its purpose and its result belongs in an ADR

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.