agentleFS
tolomtolom/prompt-xray-skills/skills/08-prompt-debugger/SKILL.md

prompt-debugger

Skill1 starsChanged 5 months agoMITView on GitHub

Diagnose poor LLM outputs by comparing the prompt, input, expected output, and actual output. Use this skill when a prompt is already being used but the model response is wrong, vague, off-format, too generic, overconfident, or not useful.

2 other projects use a version of this.

Adopted by

Most recently taken up by

The file

Show its rules and text

We split this skill into the rules it gives an agent

14 rules. About 0 are common, 14 are distinctive, and 0 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈14

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 0

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Use this skill to diagnose why a prompt produced a poor result.

    line 10
  2. Distinctive:

    Do not randomly rewrite the prompt without analyzing the failure.

    line 14
  3. Distinctive:

    Use this skill when the user says:

    line 18
  4. Distinctive:

    Ask for or infer:

    line 30
  5. Distinctive:

    If actual output is not available, run a structural diagnosis of the prompt instead.

    line 40
  6. Distinctive:

    Write the difference in plain language.

    line 63
  7. Distinctive:

    Check whether the input provided enough information.

    line 86
  8. Distinctive:

    If the prompt asks the model to infer unavailable facts, mark this as a context problem, not only a prompt problem.

    line 88
  9. Distinctive:

    If the issue is formatting, check whether the prompt defines:

    line 92
  10. Distinctive:

    Prefer small targeted changes:

    line 103
  11. Distinctive:

    add a missing rule;

    line 105
  12. Distinctive:

    add a test case;

    line 110
  13. Distinctive:

    Do not rewrite everything unless necessary.

    line 113
  14. Distinctive:

    Add one test case that would verify the fix.

    line 121

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Prompt Debugger Skill

Purpose

Use this skill to diagnose why a prompt produced a poor result.

The goal is to identify the likely failure cause before rewriting the prompt.

Do not randomly rewrite the prompt without analyzing the failure.

When to use

Use this skill when the user says:

  • the prompt does not work;
  • the model output is bad;
  • the output is too generic;
  • the model ignores the format;
  • the assistant gives wrong or unsupported details;
  • the prompt works sometimes but not consistently;
  • the output is not useful for the workflow.

Required input

Ask for or infer:

  • current prompt;
  • user input;
  • actual model output;
  • expected output;
  • target model/platform;
  • known constraints;
  • whether the output is for a human or software.

If actual output is not available, run a structural diagnosis of the prompt instead.

Diagnosis procedure

1. Identify the observed failure

Classify the issue:

  • unclear task;
  • missing context;
  • weak role;
  • conflicting instructions;
  • poor output format;
  • insufficient examples;
  • unsupported claims;
  • too much freedom;
  • wrong level of detail;
  • bad input data;
  • model limitation;
  • tool/source limitation.

2. Compare expected vs actual behavior

Write the difference in plain language.

Example:

Expected: concise JSON extraction.
Actual: long Markdown explanation with extra fields.

3. Identify prompt-level causes

Look for:

  • vague verbs;
  • missing constraints;
  • missing source-of-truth policy;
  • missing output contract;
  • missing examples;
  • no incomplete-data behavior;
  • too many goals in one instruction.

4. Identify context-level causes

Check whether the input provided enough information.

If the prompt asks the model to infer unavailable facts, mark this as a context problem, not only a prompt problem.

5. Identify output-contract causes

If the issue is formatting, check whether the prompt defines:

  • exact sections;
  • JSON keys;
  • field types;
  • required vs optional fields;
  • examples;
  • no-extra-text rules.

6. Recommend the smallest fix

Prefer small targeted changes:

  • add a missing rule;
  • add output format;
  • add incomplete-context behavior;
  • add examples;
  • split the task;
  • add a test case;
  • define source of truth.

Do not rewrite everything unless necessary.

7. Create a revised prompt

Rewrite the prompt only after the diagnosis.

8. Create a confirmation test

Add one test case that would verify the fix.

Output format

Return the diagnosis as:

# Prompt Debug Report

## Observed Problem

- Expected behavior:
- Actual behavior:
- Main failure type:

## Likely Causes

### 1. Cause name

- Evidence:
- Why it matters:
- Fix:

## Prompt-Level Issues

- ...

## Context-Level Issues

- ...

## Output-Format Issues

- ...

## Smallest Useful Fix

...

## Revised Prompt

```text
<revised prompt>

Confirmation Test

Input

...

Expected behavior

...

Follow-Up Improvements

  • ...

## Quality checklist

A good debug report should:

- separate prompt issues from context issues;
- identify the actual failure type;
- recommend the smallest fix;
- include a revised prompt;
- include a confirmation test.

## Boundaries

This skill diagnoses prompt behavior based on available examples. It does not guarantee that the revised prompt will work across all models without testing.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.