agentleFS
youcislayoucisla/Agent-Foundry/skills/core/feedback-loop/SKILL.md

feedback-loop

Skill1 starsChanged 23 days agoMITView on GitHub

'After shipping, set up the loop: instrument → measure → review weekly

9 other projects use a version of this.

Adopted by

Most recently taken up by

The file

We split this skill into the rules it gives an agent

7 rules. 0 are common, 7 are distinctive, and 0 are very specific to this project.

  • 0

    Common

    Most of the 6 edited versions we read keep these.

  • 7

    Distinctive

    This author's own choices. Worth a look.

  • 0

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Every step is required. Skipping any one breaks the loop.

    line 25
  2. Distinctive:

    Before launch, decide what to measure.

    line 29
  3. Distinctive:

    Add tracking BEFORE launch.

    line 42
  4. Distinctive:

    Don't stare at a dashboard.

    line 46
  5. Distinctive:

    If you can't name the number you expect, you didn't define success.

    line 52
  6. Distinctive:

    Don't skip weeks. Skipping = loop is dead.

    line 73
  7. Distinctive:

    Never pool scores across model revisions, runtime versions, or config fingerprints.

    line 100

“Common” is measured: kept by at least 6 in 10 of the 6 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Feedback Loop

Shipped is not done. The value comes from the loop after launch.

Procedure

The Loop

Instrument → Measure → Review → Adjust → Repeat (weekly)

Every step is required. Skipping any one breaks the loop.

1 — Instrument

Before launch, decide what to measure. Three layers:

LayerTells youExamples
BusinessDid the change move the needle?Activation, retention, revenue
ProductDid users do the thing?Button clicks, feature adoption
SystemDid the system work?Errors, latency, uptime

For a new feature:

  • 1 business metric (the one that justifies the feature)
  • 2-3 product metrics (the funnel steps)
  • 3-5 system metrics (so you know it works)

Add tracking BEFORE launch. Not after.

2 — Measure

Don't stare at a dashboard. Measure by collecting the number once.

Two formats:

  • Daily metric sheet (automated): email/Slack at 9am with yesterday's key numbers
  • Weekly review doc (one page): trends and anomalies

If you can't name the number you expect, you didn't define success.

3 — Review Weekly

30 minutes. Same time every week. Same template:

Week of [date]

[Big metric]: X (vs. target Y, vs. last week Z)
- Why did it move?
- What surprised me?

[Secondary metric 1]: ...
[Secondary metric 2]: ...

Actions:
1. [One thing we'll do this week]
2. [One thing we'll stop doing]

Don't skip weeks. Skipping = loop is dead.

Anti-patterns

  • Launching without a pre-declared success metric.
  • Averaging per-row ratios instead of summing.
  • Pooling results across model or runtime versions.
  • Reviewing only when someone remembers.

References

  • details

Verification Checklist

  • Business, product, and system metrics are defined before launch.
  • The expected direction is stated.
  • Ratio aggregation uses sums.
  • Review has an owner, time, and template.
  • Results are segmented by config and version.

Aggregation Rules

  • A ratio is sum(numerator) / sum(denominator). Never average per-row means.
  • Combine only axes whose denominator is greater than zero. One missing axis must not null the others.
  • Weight feedback by feedback rows, not by all rows in the group.
  • Calibrate judge scores against human feedback per skill; report the bins and sample counts.
  • Never pool scores across model revisions, runtime versions, or config fingerprints.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.