Prompt Engineering Canon

Operating procedure for writing effective LLM prompts, distilled from Google's Prompt Engineering whitepaper (Lee Boonstra, Feb 2025). Gives you: a technique-selection decision framework (zero-shot through ReAct), exact sampling-config starting values (temperature / top-K / top-P), and the whitepaper's full best-practices list compressed into rules and a checklist. Written against Gemini in Vertex AI, but the techniques are model-agnostic (the paper names Gemini, GPT, Claude, Gemma, LLaMA).

When to use

Use this skill when you are:

• Writing or rewriting a prompt for any LLM task: classification, extraction, summarization, Q&A, translation, code generation/explanation/translation/debugging, reasoning.
• Choosing sampling configuration (temperature, top-K, top-P, token limit) via an API or Vertex AI Studio.
• Debugging bad LLM output: wrong answers on reasoning tasks, inconsistent classifications, malformed JSON, repetition loops.
• Deciding whether a task needs examples, reasoning steps, multiple samples, or external tools.

Do NOT reach for the heavier techniques by default: chain-of-thought costs extra output tokens (money + latency), and self-consistency multiplies that by the number of samples. Start simple (zero-shot), escalate only when the simple prompt fails.

Decision framework

Configure sampling first

1. Set config before tuning prompt wording — temperature, top-K, and top-P interact, and extreme values cancel each other: temperature 0 makes top-K/top-P irrelevant; top-K=1 makes temperature/top-P irrelevant; top-P=0 (or near it) makes temperature/top-K irrelevant. Where all three exist (as in Vertex AI Studio), tokens passing both top-K and top-P become candidates, then temperature samples among them.
2. Use the whitepaper's starting points, then iterate:
   • Coherent, mildly creative default: temperature 0.2, top-P 0.95, top-K 30.
   • Especially creative: temperature 0.9, top-P 0.99, top-K 40.
   • Less creative: temperature 0.1, top-P 0.9, top-K 20.
   • Task with a single correct answer (e.g., a math problem): temperature 0.
3. Temperature 0 is greedy decoding — mostly deterministic, but ties between equally probable tokens may still break randomly, so identical outputs are not guaranteed.
4. Cap output length deliberately. A token limit does not make the model more succinct — it just truncates. If you need short output, also engineer the prompt for it (e.g., "Explain quantum physics in a tweet length message."). Length restriction is especially important for ReAct, which otherwise keeps emitting useless tokens after the answer you want.
5. If output degenerates into repeated filler (the "repetition loop bug"), suspect the sampling config — it occurs at both low temperature (overly deterministic path revisits itself) and high temperature (random walk returns to a prior state). Fix by re-balancing temperature and top-K/top-P, not by rewording alone.
6. Remember: more freedom (higher temperature, top-K, top-P, output tokens) raises the risk of less relevant text.

Pick the technique

7. Start zero-shot: a task description plus the input, no examples. Simplest and cheapest.
8. When zero-shot fails or you need a specific output structure/pattern, add examples (one-shot / few-shot). Rule of thumb: at least 3–5 examples for few-shot; more for complex tasks, fewer if input length is constrained. Examples must be relevant, diverse, high quality, and well written — one small mistake can confuse the model. Include edge cases if inputs vary.
9. Separate the three framing layers and use each for its job:
   • System prompt: the model's overarching purpose and hard output requirements (format, safety). Adding a line like 'You should be respectful in your answer.' is a legitimate safety control.
   • Contextual prompt: task-specific, dynamic background information for the current input.
   • Role prompt: a character/identity that sets tone, style, and voice (e.g., travel guide, book editor). Styles the paper finds effective: Confrontational, Descriptive, Direct, Formal, Humorous, Influential, Informal, Inspirational, Persuasive.
10. For knowledge-heavy or bias-prone tasks, use step-back prompting: first ask a general question about the underlying principles, then feed that answer in as context for the specific task. This activates relevant background knowledge and mitigates bias from specific details.
11. For reasoning tasks (math, multi-step logic, anything you could solve by "talking through it"), use Chain of Thought: append "Let's think step by step." or provide worked single-shot/few-shot reasoning examples. CoT is low-effort, needs no finetuning, gives interpretable intermediate steps, and drifts less across model versions. Cost: more output tokens.
12. CoT hard rules: set temperature to 0 (reasoning tasks have one correct answer; CoT is based on greedy decoding); the final answer must come AFTER the reasoning (the reasoning tokens change the prediction of the answer); make the final answer mechanically extractable, separate from the reasoning.
13. When CoT answers are inconsistent and accuracy matters more than cost, use self-consistency: (1) send the same CoT prompt multiple times at high temperature to get diverse reasoning paths, (2) extract the answer from each response, (3) take the majority vote. You get a pseudo-probability of correctness; expect high cost.
14. For complex problems needing exploration of alternatives (not one linear chain), consider Tree of Thoughts (ToT): the model branches over multiple reasoning paths maintained as a tree.
15. When the task needs external information or actions (search, code interpreter, APIs), use ReAct: a thought-action loop where the model reasons, plans an action, executes it, observes the result, and re-plans until solved. In practice you must resend previous prompts/responses each turn and trim the extra generated content.
16. To optimize prompts at scale, use Automatic Prompt Engineering (APE): prompt a model to generate candidate prompt variants, score them with a metric such as BLEU or ROUGE, keep the best, tweak, repeat.
17. For code tasks (writing, explaining, translating between languages, debugging/reviewing), the same text-prompting techniques apply — but always read and test generated code before trusting it.

Shape the output

18. For non-creative tasks — extracting, selecting, parsing, ordering, ranking, categorizing — request structured output (JSON or XML). JSON benefits per the paper: consistent style, focus on the wanted data, less chance of hallucination, relationship-aware, typed, sortable. Forcing a structure limits hallucinations.
19. Budget for JSON's cost: it needs significantly more tokens than plain text and truncation at the token limit produces invalid JSON. Use the json-repair library (PyPI) to fix truncated/malformed JSON output.
20. Structure inputs too: provide a JSON Schema defining expected fields and types so the model knows what to attend to; date/timestamp fields with formats make it "time-aware."

Write the prompt text

21. Design with simplicity. "if it's already confusing for you it will likely be also confusing for the model" (Boonstra, Prompt Engineering whitepaper). Cut unnecessary information; lead with an action verb: Act, Analyze, Categorize, Classify, Contrast, Compare, Create, Describe, Define, Evaluate, Extract, Find, Generate, Identify, List, Measure, Organize, Parse, Pick, Predict, Provide, Rank, Recommend, Return, Retrieve, Rewrite, Select, Show, Sort, Summarize, Translate, Write.
22. Be specific about the desired output (length, style, content). "Generate a 3 paragraph blog post about the top 5 video game consoles..." beats "Generate a blog post about video game consoles."
23. Prefer instructions over constraints: tell the model what TO do, not what NOT to do. Positive instructions communicate the goal directly; constraint lists leave the model guessing and can clash with each other. Reserve constraints for safety, strict formats, or hard requirements.
24. Use variables (e.g., {city}) instead of hardcoding values, so prompts are reusable and integrable into applications.
25. For few-shot classification, mix up the response classes across your examples — a fixed order overfits the model to the sequence rather than the class features. Rule of thumb: start with 6 few-shot examples and test accuracy from there.
26. Experiment with input formats: the same goal phrased as a question, a statement, or an instruction yields different outputs. Try more than one.
27. Adapt prompts on model updates — architecture, data, and capability changes shift prompt behavior; retest existing prompts on new versions.

Operationalize

28. Document every prompt attempt in full detail, in a structured table with fields: Name (name + version), Goal, Model (name + version), Temperature (value between 0–1), Token Limit, Top-K, Top-P, Prompt, Output. Also track the iteration/version, a result field (OK / NOT OK / SOMETIMES OK), and feedback. For RAG systems, also capture the query, chunk settings, and chunk output that determined the inserted content.
29. In the codebase, save prompts in separate files from code for maintainability, and rely on automated tests and evaluation procedures to measure how well a prompt generalizes.
30. Prompt engineering is iterative: craft, test, analyze, document, refine. When the model or config changes, re-run previous prompts. Having multiple prompt engineers attempt the same task surfaces performance variance.

Checklist

• Config set for the task: temp 0.2 / top-P 0.95 / top-K 30 default; temp 0.9 / .99 / 40 creative; temp 0.1 / .9 / 20 conservative; temp 0 for single-correct-answer tasks
• Token limit set intentionally; prompt also asks for the target length if short output is needed
• Simplest sufficient technique chosen: zero-shot → few-shot (3–5+ examples) → step-back / CoT → self-consistency / ToT / ReAct
• Few-shot examples: relevant, diverse, well written, edge cases included, classes mixed (start at 6 for classification)
• System / contextual / role concerns separated and each stated explicitly
• Prompt leads with an action verb; specific about output length, style, and content
• Positive instructions used; constraints only for safety or strict format
• CoT: temperature 0, reasoning before the final answer, answer extractable
• Structured (JSON/XML) output requested for extract/parse/rank/categorize tasks; json-repair handling in place for truncation
• Variables used instead of hardcoded values
• Attempt logged: name+version, goal, model, temperature, token limit, top-K, top-P, full prompt, output, OK/NOT OK/SOMETIMES OK
• Prompts stored separately from code; automated evals in place before shipping

Anti-patterns

• Relying on a token limit to make output succinct — it only truncates; truncated JSON is unusable without repair.
• Running CoT or self-consistency at high temperature for the final answer (CoT requires temperature 0), or asking for the answer before the reasoning.
• Trusting unit-of-one results: identical prompts can differ across models, sampling settings, model versions, and even identical reruns (probability ties break randomly).
• Piling on negative constraints ("do not mention X, do not list Y") instead of positive instructions — constraints leave the model guessing and can conflict.
• Few-shot classification examples grouped by class — the model memorizes the order instead of the class features.
• Sloppy examples: a single small mistake in a few-shot example can confuse the model into wrong output.
• Vague prompts with extraneous backstory instead of a concise instruction — ambiguous prompts produce ambiguous, inaccurate responses.
• Shipping LLM-generated code unread and untested — the model repeats training data and cannot be assumed correct.
• Skipping documentation of attempts — you lose the record needed to debug regressions, re-test on new model versions, and recall what worked.
• Ignoring extreme-setting interactions, e.g., tuning top-K/top-P while temperature is 0 (they are irrelevant there).

Source

• Prompt Engineering (whitepaper, Lee Boonstra; Google/Kaggle) — https://www.kaggle.com/whitepaper-prompt-engineering — February 2025

Distilled from the official document(s) above on 2026-08-12. If this skill and the source disagree, trust the source.