AI Token Improvement Plan Engineer

When to Use

• Audit where tokens are spent (by feature, model, tenant, turn type)
• Prioritize cost-saving initiatives with estimated savings and risk
• Draft a phased improvement plan with metrics and rollback criteria
• Define eval gates so cost cuts do not regress quality or safety
• Brief leadership on trade-offs (model tier, context size, agent depth)

When NOT to Use

• Implement context packing or compression code → ai-context-engineer
• Rewrite one prompt or agent tool schema → prompt-engineer
• Build RAG ingest, chunking, or agent runtime → ai-engineer
• Weekly cost review ritual and release governance → ai-lead-ops
• Memory store architecture → ai-memory-developer
• Adversarial safety testing → ai-redteam
• AI policy and regulatory mapping → ai-risk-governance

Related skills

Core Workflows

1. Scope and baseline

Define:

• Surface area — chat, copilot, batch, agents, embeddings-only
• Time window — 7/30 days; exclude anomalies (launches, incidents)
• Segmentation — model, feature flag, tenant tier, environment

Collect baseline metrics (see references/token_audit.md).

Deliverable: current state table with top 5 cost drivers (≥70% of spend if possible).

2. Categorize spend

Bucket each driver:

Tag fixed per request vs scales with users/sessions.

3. Initiative backlog

For each idea record:

• Description and owner team
• Estimated savings (% tokens or $/month) with assumptions
• Effort (S/M/L) and dependencies
• Quality risk (low/med/high) and required evals
• Measurement — metric that proves success

Prioritize with impact × confidence ÷ effort; never ship without eval plan.

See references/improvement_backlog.md and references/optimization_playbook.md.

4. Quality and safety guardrails

Every initiative must list:

• Golden-set evals (task accuracy, format)
• Safety set (if user-facing)
• Latency check (p95)
• Rollback trigger (e.g., CSAT drop, eval regression >X%)

See references/measurement_and_kpis.md.

5. Phased plan

Typical phases:

Each phase: goals, initiatives, owners, exit criteria.

See references/rollout_plan.md.

6. Executive summary

## Situation — spend and growth rate
## Target — $ or tokens/session goal by date
## Top drivers — ranked list
## Plan — 3–5 initiatives per phase
## Risks — quality, latency, eng capacity
## Asks — budget, headcount, eval time

Hand ongoing ops to ai-lead-ops after plan approval.

When to load references

• Baseline and attribution → references/token_audit.md
• Backlog template and scoring → references/improvement_backlog.md
• Technique catalog → references/optimization_playbook.md
• KPIs and eval gates → references/measurement_and_kpis.md
• Phasing and rollback → references/rollout_plan.md