llm-caching
Skill0 starsChanged 4 months agoMITView on GitHub
Prompt-caching design rules for code that calls the Claude API. Use when writing or reviewing any code that builds LLM prompts, or when API costs or cache hit rates need improving. Do NOT use for performance work unrelated to LLM API calls.
In this repository5 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- knowledge-baseSkill
- new-endpointSkill
- security-auditSkill
The file
We split this skill into the rules it gives an agent
2 rules. About 0 are common, 0 are distinctive, and 2 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈0
Distinctive · estimate
This author's own choices. Worth a look.
2
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 30
Deterministic tools: Never add/remove/reorder tools mid-conversation (position 0 → invalidates everything).
- Very specific: line 45
Read usage.cache_read_input_tokens - zero across repeated requests = silent invalidator.
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Prompt caching for LLM API code
Facts verified against platform.claude.com/docs prompt-caching (2026-07-21). Re-check pricing and per-model minimums there before big decisions.
The invariant
Caching is an exact prefix match. Render order: tools → system → messages.
One changed byte invalidates everything after it. Design order: stable content first,
volatile content last.
Economics
- Cache read: 0.1× base input. Write: 1.25× (5-min TTL) or 2× (1-hour TTL).
- Break-even: 2 requests (5m), 3+ requests (1h). Use 1h when gaps between calls exceed 5 min (agentic steps, slow conversations); 1h blocks must come BEFORE 5m blocks.
- Minimum cacheable prefix (below it, markers are silently ignored): Fable 5 = 512 tokens; Opus 4.8 / Sonnet 5 / Sonnet 4.6 = 1024; Opus 4.7 = 2048; Opus 4.6/4.5 & Haiku 4.5 = 4096.
- Cache scope: workspace-level on the Claude API (org-level on Bedrock/Vertex).
Design rules
- Freeze the system prompt. No
Date.now(), no UUIDs, no user names interpolated - dynamic context goes at the END ofmessages, never insystem. - Deterministic tools. Never add/remove/reorder tools mid-conversation (position 0 →
invalidates everything). Serialize sorted (
sort_keys=True; watch Go/Swift map ordering). - Breakpoint on the last STABLE block, not the last block. Max 4 breakpoints.
Multi-turn: top-level automatic
cache_controlis the simplest correct option. - Cheap changes:
tool_choice, thinking config, and images invalidate only the messages tier - the tools+system cache survives them. - Long agentic turns: cache lookback is max 20 content blocks. If one turn adds more than 20 blocks (tool_use/tool_result pairs), add an intermediate breakpoint every ~15.
- Parallel fan-out: N simultaneous identical requests all pay full price. Send one, await the FIRST streamed token, then fire the rest.
- Pre-warm latency-sensitive apps with a
max_tokens: 0request at startup - writes the cache, bills no output. Skip if traffic is continuous or the prefix is tiny.
Verify (never assume it works)
- Read
usage.cache_read_input_tokens- zero across repeated requests = silent invalidator. Diff the rendered prompt bytes of two requests to find it. - Total prompt =
input_tokens + cache_creation_input_tokens + cache_read_input_tokens- judge cost by the sum, not byinput_tokensalone. - Record measured hit rates and cost deltas in MEMORY.md → "Verified facts".
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

