agentleFS
mthinesmthines/agent-skills/skills/quality/confidence/SKILL.md

confidence

Skill20 starsChanged 13 days agoMITView on GitHub

Rates confidence that work fully solves the stated requirement, in plan, code (default), or analysis mode; plan mode adds deterministic rule checks, and a failed rule caps the gate at 89% regardless of LLM score. Use before autonomous execution, after implementation, or during investigation; triggers on "confidence check", "validate plan", "rate confidence", "quality gate", "/confidence".

3 other projects use a version of this.

Adopted by

Most recently taken up by

The file

We split this skill into the rules it gives an agent

7 rules. About 0 are common, 2 are distinctive, and 5 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈2

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 5

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Check the arguments: $ARGUMENTS

    line 33
  2. Very specific:

    If no argument is provided, default to code.

    line 42
  3. Very specific:

    If arguments contain "fix" (e.g., code fix, plan fix, analysis fix), run in Fix Mode: after the review, apply fixes for every concern found …

    line 46
  4. Very specific:

    Run all 11 rules in rules/plan-rule-checks.md — read it in every plan-mode run.

    line 63
  5. Very specific:

    Use when the graded artifact is an analysis — a root-cause write-up, a refactor rationale, a /create-skill diagnose proposal, a holistic …

    line 83
  6. Distinctive:

    Do not inflate scores: a low score with clear reasoning beats a false 95%, and a failed rule check is always surfaced, even when the LLM …

    line 124
  7. Distinctive:

    When used as a quality gate before autonomous execution:

    line 136

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Confidence Assessment

Rate your confidence that the current work fully solves the stated requirement. Plan mode is gated on two signals — LLM dimensional scoring and deterministic rule checks; rationale: references/multi-signal-rationale.md.

Contents: Mode Detection · Assessment Dimensions · Output Format · Score Thresholds · Iteration Protocol

Mode Detection

Check the arguments: $ARGUMENTS

ArgumentDefaultValidatesWhen to use
planImplementation plan completenessAfter Phase 1 planning, before autonomous execution
codeyesCode implementation correctnessAfter writing code, before PR
analysisAnalysis accuracy (root cause, refactor rationale, or skill gap)During investigation, before proposing a fix, refactor, or skill-source diff
bug-analysisDeprecated alias for analysis — behaves identicallyBackwards-compatible; emit a one-line deprecation note in the report header

If no argument is provided, default to code.

Alias handling. bug-analysis resolves to analysis with identical dimensions, weights, thresholds, and Fix Mode behaviour. When invoked with the alias, prepend a single line to the report header: > Note: \bug-analysis` is a deprecated alias for `analysis`. Update the caller when convenient.`

If arguments contain "fix" (e.g., code fix, plan fix, analysis fix), run in Fix Mode: after the review, apply fixes for every concern found — read rules/fix-mode.md when in Fix Mode.

Assessment Dimensions

For plan mode

Both signals must pass for the gate to clear.

Step 1 — LLM dimensional scoring

DimensionWeightWhat to evaluate
Completeness40%Are ALL Phase 0 requirements captured? Are the Core sections populated (and the Extended sections present where the task needs them)? Could a new session execute from this plan alone? Do not penalize an Extended section legitimately omitted because its Include when trigger does not apply (e.g. no Existing Code Survey on a modification-only plan). Score down an Existing Code Survey whose "Searched for" column is vague ("looked around") or whose BUILD NEW verdicts show no searches — rule #10 checks presence, this dimension checks search quality.
Feasibility30%Is the technical approach sound? Are patterns consistent with the codebase? Are risks identified where applicable?
No ambiguity30%Are implementation steps specific enough to execute without interpretation? Are edge cases addressed where applicable?

Step 2 — Deterministic rule checks (run via Bash). Run all 11 rules in rules/plan-rule-checks.md — read it in every plan-mode run. A single failed rule caps the gate at 89% regardless of LLM score; surface the failed rule and either fix the plan or escalate to the user.

Step 3 — Combined gate

overall_score = min(weighted_LLM_score, max_allowed_by_rule_checks)
where max_allowed_by_rule_checks = 100% if all rules pass, else 89%

For code mode

DimensionWeightWhat to evaluate
Correctness40%Does the logic actually address the problem as described?
Completeness30%Are all cases, edge cases, and requirements covered?
No regressions30%Could this break existing behavior or introduce side effects?

For analysis mode

Use when the graded artifact is an analysis — a root-cause write-up, a refactor rationale, a /create-skill diagnose proposal, a holistic re-analysis after a stuck loop, or any reasoning artifact that precedes a proposed change.

DimensionWeightWhat to evaluate
Evidence strength40%Is the analysis backed by concrete evidence (logs, traces, code paths, file:line references, reproduced behaviour)?
Root cause certainty30%Is this the underlying cause or just a symptom? How deep did the investigation go? For refactor / diagnose analyses, read "root cause" as "the actual structural reason," not literal bug aetiology.
Outcome confidence30%Will the proposed action (fix, refactor, skill-source diff) resolve the situation without introducing new problems?

Output Format

Output in this exact format:

## Confidence: X%

### LLM dimensional scoring

| Dimension | Score | Notes |
|-----------|-------|-------|
| <dim 1>   | X%    | ...   |
| <dim 2>   | X%    | ...   |
| <dim 3>   | X%    | ...   |

### Deterministic rule checks (plan mode only — omit for code/analysis)

| # | Rule                | Status      | Evidence                  |
|---|---------------------|-------------|---------------------------|
| 1 | <rule description>  | ✓ pass / ✗ fail | <command output snippet> |
| ... |                   |             |                           |

### Combined gate

- Weighted LLM score: X%
- Rule checks: <N pass> / <total> (cap: 100% if all pass, else 89%)
- **Final: X%**

The weighted LLM score is the weighted average over the dimension weights above. For plan mode, Final is min(weighted_LLM_score, rule_check_cap). For code and analysis modes (including the deprecated bug-analysis alias), omit the rule-check section and Final = weighted_LLM_score.

Do not inflate scores: a low score with clear reasoning beats a false 95%, and a failed rule check is always surfaced, even when the LLM score is high.

Score Thresholds

ScoreAction
90-100%Proceed — work is ready
70-89%List specific concerns and what would raise confidence. If in Fix Mode, apply fixes.
Below 70%Recommend concrete next steps to validate or fix. Do NOT proceed with autonomous execution.

Iteration Protocol (plan mode)

When used as a quality gate before autonomous execution:

If confidence is below 90%, do up to 2 iterations of additional research, analysis, and evidence collection to raise the score. After each iteration, re-run the confidence assessment. If still below 90% after 2 iterations, present findings to the user and ask whether to proceed or refine further.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.