agentleFS
wheeloswheelos/apollo-lite/.agents/skills/agent-evaluation/SKILL.md

agent-evaluation

Skill177 starsChanged 7 days agoApache-2.0View on GitHub

Evaluate changes to agent instructions, skills, routing, and tools using frozen cases and reviewed evidence. Use for agent regression, prompt comparisons, recovery checks, and capability acceptance.

23 other projects use a version of this.

Adopted by

Most recently taken up by

Used with

Projects with it also use tool-design, context-engineering and 5 more.

Other files people use alongside this one.

Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
  • mjunaidcaSkill

    tool-design

    Design tools that agents can use effectively. Use when creating new tools for agents, debugging tool-related failures, or optimizing existing tool sets.

    Also in 3 projects that use this

  • mrgoonieSkill

    context-engineering

    Master context engineering for AI agent systems. Use when designing agent architectures, debugging context failures, optimizing token usage, implementing memory systems, building multi-agent coordination, evaluating agent performance, or developing LLM-powered pipelines. Covers context fundamentals, degradation patterns, optimization techniques (compaction, masking, caching), compression strategies, memory architectures, multi-agent patterns, LLM-as-Judge evaluation, tool design, and project development.

    Also in 4 projects that use this

  • xenitV1Skill

    clean-code

    Pragmatic coding standards - concise, direct, no over-engineering, no unnecessary comments

    Also in 3 projects that use this

  • OpenAISkill

    test-driven-development

    Use when implementing any feature or bugfix, before writing implementation code

    Also in 3 projects that use this

  • BlockSkill

    frontend-design

    Create distinctive, production-grade frontend interfaces with high design quality. Use this skill when the user asks to build web components, pages, or applications. Generates creative, polished code that avoids generic AI aesthetics.

    Also in 3 projects that use this

  • GoogleSkill

    code-review

    Adversarial systems, concurrency, memory-model, and architecture code review protocol. Use this skill whenever asked to review code, PRs, diffs, or branches in Filament.

    Also in 3 projects that use this

The file

We split this skill into the rules it gives an agent

4 rules. 0 are common, 4 are distinctive, and 0 are very specific to this project.

  • 0

    Common

    Most of the 19 edited versions we read keep these.

  • 4

    Distinctive

    This author's own choices. Worth a look.

  • 0

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Read the evaluation procedure and select relevant cases from the case set.

    line 10
  2. Distinctive:

    Run only explicitly authorized tasks.

    line 16
  3. Distinctive:

    Use the evaluation command to check report structure and aggregate gate results.

    line 21
  4. Distinctive:

    Keep tuning and holdout cases distinct.

    line 29

“Common” is measured: kept by at least 6 in 10 of the 19 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Agent Evaluation

Procedure

  1. Read the evaluation procedure and select relevant cases from the case set.
  2. Freeze task inputs, code, context, skill/tool versions, model configuration, environment, and approvals for each comparison. Use separate worktrees or the existing isolated harness; never revert someone else's work.
  3. Run only explicitly authorized tasks. Preserve actual transcripts, tool results, diffs, and evidence outside durable knowledge. No automatic road/HIL operations, publication, paid runs, or agent spawning.
  4. Have a reviewer judge each case gate from the evidence. Record false or not-run for missing proof; an agent's own completion claim is insufficient.
  5. Use the evaluation command to check report structure and aggregate gate results. Compare like-for-like runs; do not present schema validation as an agent benchmark or averages as permission to ignore hard-gate failures.

Acceptance and failure handling

Report case coverage, hard-gate failures, correctness, unnecessary changes, invalid operations, recovery, cost, and human intervention when measured. Keep tuning and holdout cases distinct. Do not weaken gates after seeing a failure; version intentional criterion changes and rerun comparable baselines.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.