Prompt Engineer

Craft and optimize LLM prompts for maximum effectiveness, consistency, and cost efficiency.

Core Workflow

1. Requirements Analysis

Before writing or editing any prompt:

1. Identify the use case — what task the prompt must accomplish
2. Define success criteria — accuracy target, format requirements, tone
3. Understand constraints — token budget, latency ceiling, cost limit, model choice
4. Review existing prompts and their failure modes (if any)
5. Determine safety requirements — input validation, output filtering, injection defense

2. Prompt Design

Select the appropriate pattern based on task complexity. See references/patterns.md for detailed guidance on each.

Design principles:

• Instruction clarity — state the task, constraints, and output format explicitly
• Minimal tokens — every token must earn its place; compress without losing meaning
• Modular structure — separate system prompt, context, instructions, and examples
• Variable placeholders — use {{variable}} for dynamic content injection
• Error recovery — include fallback instructions for ambiguous or invalid input

3. Optimization

Iterate on prompts to reduce cost and improve quality:

• Token reduction — remove redundant phrasing, compress examples, use abbreviations the model understands
• Context compression — summarize long context; only include what the model needs for the current step
• Output constraints — specify format (JSON, markdown, list) to reduce parsing overhead
• Caching — identify static prompt sections that can be cached across calls
• Batch processing — group similar requests when possible

4. Evaluation & Testing

See references/evaluation.md for detailed frameworks.

Minimum evaluation protocol:

1. Create a test set — 20+ examples covering happy path, edge cases, and adversarial inputs
2. Define metrics — accuracy, format compliance, latency, token usage, cost per call
3. Run baseline — measure current prompt performance
4. A/B test variants — change one variable at a time, measure statistical significance
5. Regression test — ensure new versions don't break existing passing cases

Target thresholds (adjust per use case):

• Accuracy: >90%
• Latency: <2s
• Token efficiency: track input/output ratio

5. Production Deployment

See references/production.md for safety mechanisms, multi-model strategies, and team workflows.

Production checklist:

• Prompt versioned in source control
• Safety filters enabled (input validation + output filtering)
• Monitoring active (accuracy, latency, cost, error rate)
• Fallback strategy defined (retry logic, model fallback chain)
• Documentation complete (purpose, variables, examples, known limitations)
• Cost tracking per query/endpoint

Quick Reference — Prompt Template Structure

[System Prompt]
  Role definition + behavioral constraints + output format

[Context Section]
  {{dynamic_context}} — injected per request

[Instructions]
  Step-by-step task description

[Examples] (if few-shot)
  Input → Output pairs (3-5 diverse examples)

[User Input]
  {{user_query}}

[Output Constraints]
  Format specification + validation rules

Reference Files

• references/patterns.md — Detailed prompt patterns (zero-shot, few-shot, CoT, ToT, ReAct, constitutional AI, role-based)
• references/evaluation.md — Evaluation frameworks, A/B testing methodology, metrics, statistical analysis
• references/production.md — Safety mechanisms, multi-model strategies, production systems, monitoring, team workflows