agentleFS
tolomtolom/prompt-xray-skills/skills/07-prompt-test-generator/SKILL.md

prompt-test-generator

Skill1 starsChanged 5 months agoMITView on GitHub

Generate practical quality-check cases for LLM prompts, including typical inputs, edge inputs, ambiguous requests, incomplete context, expected behavior, and review criteria. Use this skill when a prompt needs to be validated before reuse or production.

The file

We split this skill into the rules it gives an agent

10 rules. About 0 are common, 10 are distinctive, and 0 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈10

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 0

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Use this skill to create a practical quality-check suite for an LLM prompt.

    line 10
  2. Distinctive:

    Use this skill when the user asks to:

    line 18
  3. Distinctive:

    check whether a prompt is reliable.

    line 22
  4. Distinctive:

    If some inputs are missing, infer likely cases and mark assumptions.

    line 38
  5. Distinctive:

    Create common inputs the prompt should handle well.

    line 70
  6. Distinctive:

    Create uncommon but realistic inputs:

    line 74
  7. Distinctive:

    Create inputs where important details are missing.

    line 84
  8. Distinctive:

    Create inputs that can be interpreted in more than one way.

    line 90
  9. Distinctive:

    Create inputs that might cause the model to ignore the requested output format.

    line 96
  10. Distinctive:

    Avoid vague criteria such as "good answer".

    line 104

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Prompt Test Generator Skill

Purpose

Use this skill to create a practical quality-check suite for an LLM prompt.

The goal is to see whether the prompt behaves consistently under realistic inputs.

A prompt that cannot be checked is not production-ready.

When to use

Use this skill when the user asks to:

  • test a prompt;
  • create prompt evals;
  • check whether a prompt is reliable;
  • generate edge cases;
  • validate a system prompt;
  • compare prompt versions;
  • prepare a prompt for production.

Required input

The user should provide:

  • the prompt to check;
  • intended use case;
  • expected output format;
  • target audience;
  • known quality concerns if available.

If some inputs are missing, infer likely cases and mark assumptions.

Procedure

1. Identify prompt type

Classify the prompt as one of:

  • content generation;
  • extraction;
  • classification;
  • RAG answer;
  • system prompt;
  • agent instruction;
  • coding-agent prompt;
  • sales/support prompt;
  • product-spec prompt.

2. Define success behavior

Extract what a successful output should do.

Define:

  • required output structure;
  • correctness criteria;
  • tone/format criteria;
  • grounding criteria;
  • incomplete-information behavior.

3. Generate typical cases

Create common inputs the prompt should handle well.

4. Generate edge cases

Create uncommon but realistic inputs:

  • very short input;
  • long input;
  • mixed language;
  • incomplete but usable data;
  • multiple requests in one message.

5. Generate incomplete-context cases

Create inputs where important details are missing.

Expected behavior should be explicit uncertainty, assumptions, partial output, or a clarifying question.

6. Generate ambiguous cases

Create inputs that can be interpreted in more than one way.

Expected behavior should usually be clarification or clearly marked assumptions.

7. Generate format-stability cases

Create inputs that might cause the model to ignore the requested output format.

Expected behavior should preserve the required format.

8. Define review criteria

For each case, define concrete review criteria.

Avoid vague criteria such as "good answer".

Output format

Return the quality-check suite as:

# Prompt Quality-Check Suite

## Prompt Type

- Type:
- Intended use:
- Main quality risk:

## Success Criteria

- [ ] Output follows the required format.
- [ ] Answer is grounded in provided context.
- [ ] Missing information is handled explicitly.
- [ ] Ambiguity is handled explicitly.
- [ ] Constraints are followed.

## Cases

### Case 1: Typical input

**Input**

```text
...

Expected behavior

...

Review criteria

  • ...

Case 2: Edge input

Input

...

Expected behavior

...

Review criteria

  • ...

Case 3: Incomplete context

Input

...

Expected behavior

...

Review criteria

  • ...

Case 4: Ambiguous request

Input

...

Expected behavior

...

Review criteria

  • ...

Case 5: Format stability

Input

...

Expected behavior

...

Review criteria

  • ...

Coverage Notes

  • Covered risks:
  • Uncovered risks:
  • Recommended next checks:

## Quality checklist

A good prompt quality-check suite should include:

- typical cases;
- edge cases;
- incomplete-context cases;
- ambiguous cases;
- format-stability cases;
- concrete review criteria.

## Boundaries

This skill creates quality-check cases. It does not execute model evaluations automatically.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.