agentleFS
conectlensconectlens/lenserfight/.agents/skills/ai-cost-manager-reducer/SKILL.md

ai-cost-manager-reducer

Skill18 starsChanged 6 months agoMITView on GitHub

Audit AI-powered features, API calls, agent workflows, prompt chains, tool calls, embeddings, vector search, retries, streaming flows, background jobs, cron tasks, caching layers, logs, and model selection logic to detect token waste, repeated AI calls, and inefficient prompt chains. Produces severity-ranked findings with safe optimization strategies that preserve reasoning quality and chain reliability. Use before approving or modifying any AI-powered feature, agent workflow, or model-calling code.

The file

Show its rules and text

We split this skill into the rules it gives an agent

16 rules. About 0 are common, 12 are distinctive, and 4 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈12

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 4

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Verify these metrics are tracked or identify where they should be added:

    line 142
  2. Distinctive:

    Use bullet points over paragraphs for instructions.

    line 178
  3. Very specific:

    Add a model_tier: 'fast' | 'standard' | 'premium' config to each call site.

    line 184
  4. Distinctive:

    Use embedding similarity to match near-identical prompts to cached responses.

    line 192
  5. Distinctive:

    Check deterministic conditions before calling AI (e.g., if content is empty, skip summarization).

    line 201
  6. Distinctive:

    Add tool result staleness check before re-calling the same tool in an agent loop.

    line 202
  7. Very specific:

    Add if (cache.has(key)) return cache.get(key) before every AI call.

    line 203
  8. Very specific:

    Always set max_tokens proportional to expected output size.

    line 206
  9. Very specific:

    Use response_format: { type: 'json_object' } or Zod schemas to prevent verbose free-text.

    line 207
  10. Distinctive:

    Do not recommend removing a reasoning step unless you can show the output is unused or redundant.

    line 222
  11. Distinctive:

    Do not recommend switching models unless you verify the cheaper model produces equivalent quality on this task.

    line 223
  12. Distinctive:

    Do not recommend aggressive caching for calls where staleness would break product correctness (e.g., real-time scoring).

    line 224
  13. Distinctive:

    If a call site cannot be safely optimized, say so and explain the constraint.

    line 226
  14. Distinctive:

    Prefer observable, incremental changes (add a cache layer, tune max_tokens) over restructuring the entire chain.

    line 227
  15. Distinctive:

    Check whether our embedding pipeline is regenerating vectors that are already stored.

    line 236
  16. Distinctive:

    Before we launch this new AI feature, review it for cost explosion scenarios under load.

    line 237

≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

AI Cost Manager & Reducer

Audit as an efficiency engineer, not a summarizer. Every finding must name a concrete waste pattern, its cost risk, and a safe optimization strategy that preserves reasoning quality and chain correctness.

Quick Start

  1. Identify all AI call sites: LLM API calls, embedding generation, vector search, tool dispatch, agent loops.
  2. Trace each call's purpose, frequency, input size, output size, and model selection.
  3. Detect waste patterns (see catalogue below).
  4. Rank findings by severity and estimated cost impact.
  5. Propose safe optimizations with validation steps.

Never recommend removing a reasoning step unless you have verified the step is truly redundant or its output is unused downstream.


Audit Scope

AI Call Sites to Inspect

  • LLM completions: openai, @anthropic-ai/sdk, Supabase Edge Functions calling AI APIs, server actions, background workers, cron jobs
  • Embedding generation: embeddings.create(), embed(), pgvector insert paths, semantic search pipelines
  • Vector search: cosine similarity queries, <=> operator, HNSW/IVFFlat index queries
  • Tool/function calls: agent tool dispatch loops, multi-step orchestration, MCP tool invocations
  • Streaming flows: streamed completions feeding downstream parsers or UI
  • Retry logic: exponential backoff, retry-on-failure wrappers, queue consumers
  • Caching layers: Redis/KV caches keyed on prompts, memoization wrappers, CDN-cached AI responses
  • Classification / extraction calls: sentiment, category, entity extraction, intent detection
  • Summarization pipelines: document chunking, recursive summarization, context compression
  • Model selection logic: environment switches, tier routing, fallback chains

Waste Pattern Catalogue

Token Waste

IDPatternSignal
TW-1Oversized system prompt injected on every callSystem prompt > 500 tokens with mostly static content
TW-2Full conversation history resent without truncationContext grows unboundedly per session
TW-3Redundant context injectionSame DB rows / user profile fetched and embedded in prompt on every request
TW-4Verbose output without max_tokens capResponses regularly exceed task requirements
TW-5Unstructured output where JSON/schema would halve tokensFree-text response parsed by regex downstream
TW-6Recursive summarization without memoizationSame document re-summarized on every call

Repeated / Duplicate Calls

IDPatternSignal
RC-1Identical prompt sent multiple times per requestNo cache; same user query triggers 2+ completions
RC-2Duplicate embedding generationSame text embedded on insert and on query with no stored cache
RC-3Classification called once per item in a loopPer-row intent/sentiment when batch classification is available
RC-4Agent re-reads same tool output in next iterationTool result not stored in agent state
RC-5Background job re-processes unchanged recordsNo updated_at / version gate before AI call

Model Overuse

IDPatternSignal
MO-1GPT-4/Claude Opus used for simple classification or formattingTask needs < 200 tokens and has deterministic answer
MO-2Expensive model used in high-frequency cronCron calls Opus/GPT-4 every minute on potentially unchanged data
MO-3No model routing between tiersSame model regardless of task complexity or latency budget
MO-4Embedding model overqualified for the retrieval taskAda-3 on short single-word lookups where text-embedding-3-small suffices

Retry & Loop Storms

IDPatternSignal
RL-1Unbounded agent loopwhile (true) or recursive agent with no step cap
RL-2Retry on every error including non-retryable (400s)Retry wrapper doesn't skip validation errors
RL-3Retry storm on quota hit without jitterSimultaneous retries after 429 with no backoff spread
RL-4Tool call in loop without early exitAgent calls search tool repeatedly even after finding answer

Caching Gaps

IDPatternSignal
CG-1No prompt-level cacheIdentical prompts hit the API on every call
CG-2No embedding cachepgvector rows recalculated instead of reused
CG-3No semantic cacheSimilar (not identical) prompts bypass cache
CG-4Short or missing TTL on AI responsesCache evicts results before they can be reused
CG-5Streaming response not cachedStreamed output never stored; replays cost full tokens

Chain Inefficiency

IDPatternSignal
CI-1Sequential calls that could be parallelizedStep B doesn't need step A's output but waits for it
CI-2Multi-step chain where one call could replace twoClassify then reformat is one structured-output call
CI-3Intermediate results discarded and regeneratedSummarization result thrown away; re-run next request
CI-4Unnecessary tool callsAgent calls search when result is already in context

Review Workflow

1. Map the AI Call Graph

For each AI call site, record:

File → function → purpose → model → avg input tokens → avg output tokens → call frequency → cached?

Flag any call where: frequency × token-cost > threshold, or model tier doesn't match task complexity.

2. Check Caching Coverage

  • Is there a cache keyed on the normalized prompt or embedding input?
  • What is the TTL? Is it appropriate for data freshness requirements?
  • Is the cache hit rate observable?
  • For embeddings: are vectors stored in pgvector and reused, or regenerated on every query?

3. Evaluate Model Routing

For each LLM call:

  • Is the task classification/formatting/extraction with a deterministic answer? → Route to smallest capable model.
  • Is the task reasoning/generation/planning with variable depth? → Justify the model tier.
  • Is there a fallback to a cheaper model on timeout or quota?

4. Inspect Agent Loops

  • Does the loop have an explicit step cap (maxIterations, maxSteps)?
  • Is there an early-exit condition checked before each tool call?
  • Are tool results stored and checked before re-calling the same tool?
  • Are all tool calls necessary, or can some be replaced with deterministic code?

5. Assess Prompt Size

  • What is the static vs. dynamic portion of the prompt? Can static parts use prompt caching (Anthropic prefix caching, OpenAI system message caching)?
  • Is context truncated or summarized before injection?
  • Is the conversation history bounded?

6. Measure Observability

Verify these metrics are tracked or identify where they should be added:

  • Cost per request / per feature
  • Token usage breakdown (input vs. output)
  • Cache hit rate
  • AI call count per workflow
  • Model tier distribution
  • Retry rate

Output Format

Each finding must include:

ID: <waste-pattern-id>
Severity: critical | high | medium | low
Affected: <file:line or function or workflow name>
Current cost risk: <what happens at scale — e.g., "1,000 battle evaluations/day × 4,000 tokens = $X/day">
Failure / explosion scenario: <what breaks or costs explode under load>
Optimization strategy: <concrete change — model swap, caching, dedup, batching, prompt compression, etc.>
Preserves reasoning: <yes/no + explanation of why chain still works>
Validation steps:
  1. <unit test or integration check>
  2. <metric to observe before/after>
  3. <rollback condition>

Order findings by severity. Group by call site when multiple findings share a file.


Optimization Strategies Reference

Prompt Compression

  • Strip redundant preamble; move static instructions to prefix-cached system prompt.
  • Use bullet points over paragraphs for instructions.
  • Replace examples with a reference to a cached few-shot prompt ID.

Cheaper Model Routing

  • Classification, extraction, formatting → gpt-4o-mini, claude-haiku-4-5, or gemini-flash.
  • Reasoning, planning, long-form generation → justify Sonnet/GPT-4o; Opus only for highest complexity.
  • Add a model_tier: 'fast' | 'standard' | 'premium' config to each call site.

Call Deduplication / Memoization

  • Hash the normalized prompt; store result in Redis/KV with appropriate TTL.
  • For embeddings: store vector in pgvector at insert time; never recompute at query time.
  • For classifications: store result in a classifications table keyed on (content_hash, classifier_version).

Semantic Caching

  • Use embedding similarity to match near-identical prompts to cached responses.
  • Apply only to read-heavy, low-variance query patterns (e.g., FAQ answering, tag suggestion).

Batching

  • Replace per-item loops with embeddings.create({ input: [...] }) batch calls.
  • Batch classification requests using structured multi-item prompts.
  • Group background job records into single API calls with array inputs.

Early Exits

  • Check deterministic conditions before calling AI (e.g., if content is empty, skip summarization).
  • Add tool result staleness check before re-calling the same tool in an agent loop.
  • Add if (cache.has(key)) return cache.get(key) before every AI call.

Stricter Output Control

  • Always set max_tokens proportional to expected output size.
  • Use response_format: { type: 'json_object' } or Zod schemas to prevent verbose free-text.
  • Set temperature: 0 for deterministic classification/extraction tasks.

Reusable Intermediate Results

  • Store summarization outputs in the DB; invalidate only on source update.
  • Cache agent scratchpad state between steps; serialize to Redis on each tool return.

Fallback Strategy

  • On 429 / timeout: route to cheaper model before retrying same tier.
  • On repeated tool failure: exit loop with partial result rather than infinite retry.

Constraints

  • Do not recommend removing a reasoning step unless you can show the output is unused or redundant.
  • Do not recommend switching models unless you verify the cheaper model produces equivalent quality on this task.
  • Do not recommend aggressive caching for calls where staleness would break product correctness (e.g., real-time scoring).
  • Mark every recommendation with "Preserves reasoning: yes/no" and explain.
  • If a call site cannot be safely optimized, say so and explain the constraint.
  • Prefer observable, incremental changes (add a cache layer, tune max_tokens) over restructuring the entire chain.

Example Triggers

  • Review our AI battle evaluation pipeline for token waste and model overuse.
  • Our monthly AI bill doubled; find the hotspots before we scale to 10k users.
  • Audit agent workflows in libs/domains/execution/src/ for unbounded loops and retry storms.
  • Check whether our embedding pipeline is regenerating vectors that are already stored.
  • Before we launch this new AI feature, review it for cost explosion scenarios under load.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.