agentleFS
TidalPaladinTidalPaladin/skills/autoresearch/SKILL.md

autoresearch

Skill1 starsChanged 24 days agoApache-2.0View on GitHub

Run a bounded empirical research study with reproducible results and recoverable state.

100 other projects use a version of this.

Adopted by

Most recently taken up by

Used with

Projects with it also use aeon-update, auto-workflow and 10 more.

Other files people use alongside this one.

Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
  • aaronjmarsSkill

    aeon-update

    Pull framework updates from the upstream Aeon repo into this instance - 3-way merges canon's new commits into a PR, never clobbering operator config.

    Also in 4 projects that use this

  • aaronjmarsSkill

    auto-workflow

    Two-mode aeon.yml workflow builder - analyze inspects URLs and emits a tiered, signal-verified skill-enablement plan plus an aeon.yml diff; enable flips slugs to enabled:true and opens a PR.

    Also in 4 projects that use this

  • +10 more from aaronjmars

The file

Show its rules and text

We split this skill into the rules it gives an agent

23 rules. 0 are common, 19 are distinctive, and 4 are very specific to this project.

  • 0

    Common

    Most of the 96 edited versions we read keep these.

  • 19

    Distinctive

    This author's own choices. Worth a look.

  • 4

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Run controlled experiments with a fixed protocol and verifiable provenance.

    line 8
  2. Distinctive:

    Use the repository adapter for domain commands, data, metrics, and hardware.

    line 9
  3. Distinctive:

    Do not start unrelated research merely because a task involves experiments.

    line 10
  4. Distinctive:

    Read and inspect within the repository and declared experiment environment.

    line 16
  5. Distinctive:

    Use that workflow only for study branches, task-scoped staging, commits, and pushes.

    line 20
  6. Distinctive:

    Do not create a pull request.

    line 21
  7. Distinctive:

    Do not delete artifacts, change protected branches, alter production systems, or exceed recorded resource limits without explicit …

    line 22
  8. Distinctive:

    Ask when a decision exceeds the accepted study design, resource limits, retention policy, or scope.

    line 23
  9. Distinctive:

    Do not treat invocation as authorization for destructive retention, external tracker writes, or other publication methods.

    line 24
  10. Very specific:

    Call get_goal before the first study launch.

    line 28
  11. Very specific:

    If no unfinished goal exists, call create_goal with a bounded study objective.

    line 29
  12. Distinctive:

    If the unfinished goal does not cover the study, explain the mismatch.

    line 32
  13. Very specific:

    If create_goal reports an unfinished goal, call get_goal again and reuse that goal when compatible.

    line 33
  14. Distinctive:

    If goal tools are unavailable, stop before launch.

    line 34
  15. Distinctive:

    Do not set a goal token budget unless the user explicitly supplied one.

    line 36
  16. Distinctive:

    Use a repository research skill or domain adapter for the experiment mechanics.

    line 38
  17. Distinctive:

    Before launch, and for monitoring, recovery, retention, or adapter work, read run lifecycle.

    line 47
  18. Distinctive:

    Keep training, validation, and test roles distinct.

    line 61
  19. Distinctive:

    Use deterministic selection and record every random seed.

    line 63
  20. Distinctive:

    Prefer common-horizon comparisons.

    line 67
  21. Distinctive:

    After a run becomes terminal:

    line 72
  22. Distinctive:

    If no candidate qualifies, record that result and stop unless the trial budget authorizes another experiment.

    line 82
  23. Very specific:

    Call update_goal with status=complete only when the goal objective is complete.

    line 88

“Common” is measured: kept by at least 6 in 10 of the 96 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Autoresearch

Run controlled experiments with a fixed protocol and verifiable provenance. Use the repository adapter for domain commands, data, metrics, and hardware. Do not start unrelated research merely because a task involves experiments.

Operating Contract

Preserve the user's scope and authority:

  • Read and inspect within the repository and declared experiment environment.
  • Modify code, configuration, state, logs, and managed artifacts only within the study scope.
  • Explicit $autoresearch invocation authorizes this skill to use $git-github-workflow no pr.
  • Explicit invocation also authorizes goal inspection, creation, and truthful terminal updates for the current study.
  • Use that workflow only for study branches, task-scoped staging, commits, and pushes.
  • Do not create a pull request.
  • Do not delete artifacts, change protected branches, alter production systems, or exceed recorded resource limits without explicit authorization.
  • Ask when a decision exceeds the accepted study design, resource limits, retention policy, or scope.
  • Do not treat invocation as authorization for destructive retention, external tracker writes, or other publication methods.

Allow planning and recovery without an active persistent goal. Before launch:

  1. Call get_goal before the first study launch. Track whether this skill creates or reuses the goal.
  2. If no unfinished goal exists, call create_goal with a bounded study objective.
  3. Include the outcome, study constraints, and verification evidence in the objective.
  4. Reuse a compatible unfinished goal. Do not replace an existing goal.
  5. If the unfinished goal does not cover the study, explain the mismatch. Ask the user to edit or clear it.
  6. If create_goal reports an unfinished goal, call get_goal again and reuse that goal when compatible.
  7. If goal tools are unavailable, stop before launch. Tell the user how to start the equivalent goal.

Do not set a goal token budget unless the user explicitly supplied one. Goal authority does not expand other permissions or approval limits.

Use a repository research skill or domain adapter for the experiment mechanics. This includes commands, data, metrics, hardware, checkpoints, and events. If none exists, define and validate a minimal repository adapter before launch. Stop when required mechanics remain undefined.

When an issue arises, keep the goal active and continue with the highest-value authorized attempt. Use a retry, checkpoint, preregistered fallback, component isolation, or independent work when the protocol permits it. Do not pause or block only because the preferred approach failed. Ask the user only when an essential choice, authority, or fact is missing. Mark a goal blocked only after the same condition occurs for three consecutive goal turns. Meaningful authorized work must also be unavailable. Do not confuse that condition with an owned $notify-wake wait.

Read for the Current Phase

  • For study design, local records, or external tracking, read study protocol.
  • Before launch, and for monitoring, recovery, retention, or adapter work, read run lifecycle.
  • For qualifying wake delivery, use $notify-wake. It owns transport, authority capture, delivery, reconciliation, and owned goal waits.

Experimental Discipline

Use this order:

  1. Recover existing study state and inspect prior artifacts.
  2. State one concrete question and one falsifiable hypothesis.
  3. Define the baseline before interpreting variants.
  4. Change one mechanism at a time unless the study tests an interaction.
  5. Fix the evaluation protocol before inspecting outcomes.
  6. Keep training, validation, and test roles distinct.
  7. Pair seeds, initialization, subsets, and conditions when the comparison requires it.
  8. Use deterministic selection and record every random seed.
  9. Record failed, crashed, timed-out, cancelled, censored, and incomplete runs.
  10. Record a dated protocol amendment before changing a hypothesis or metric after observing results.

Prefer common-horizon comparisons. Report endpoint quality and the cost or time needed to reach meaningful targets. Report effect sizes, uncertainty, paired differences, and limitations. Do not claim statistical significance from a sample size that cannot support it.

Analysis and Promotion

After a run becomes terminal:

  1. Validate artifacts, provenance, and notification state.
  2. Compute the predefined metrics and convergence measures.
  3. Compare against the baseline at the predefined horizons and thresholds.
  4. Promote no more candidates than the study specification permits.
  5. Replicate only candidates that meet the recorded promotion rule.
  6. Report means, dispersion, paired differences, censored runs, costs, and limitations.
  7. Mark a candidate confirmed only after it meets the replication rule.

If no candidate qualifies, record that result and stop unless the trial budget authorizes another experiment.

Completion and Handoff

Complete a study only when all permitted runs are terminal or censored. Require complete evaluation, replication, provenance, logs, retention, and authorized publication. Stored artifacts must reproduce the comparisons.

Call update_goal with status=complete only when the goal objective is complete. Complete a goal that this skill created after all required work finishes. Leave a reused broader goal active unless its complete objective is also achieved. Keep the existing three-turn rule for status=blocked.

Report identifiers, terminal-state counts, record locations, metrics, uncertainty, resources, provenance, artifact retention, notification status, limitations, and next actions.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.