agentleFS
verivus-ossverivus-oss/llm-cli-gateway/.agents/skills/least-cost-routing/SKILL.md

least-cost-routing

Skill16 starsChanged 2 months agoMITView on GitHub

Route a model-agnostic request to the cheapest capable (provider, model) via route_request, subject to a quality-tier floor and a hard budget cap. Use when you do not care WHICH model runs a task, only that it is cheap enough and good enough.

The file

We split this skill into the rules it gives an agent

7 rules. About 0 are common, 0 are distinctive, and 7 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈0

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 7

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    When the user explicitly authorizes full provider permissions and native MCP access for that mandatory review, use multi-llm-review's …

    line 30
  2. Very specific:

    Use route_request when the task is model-agnostic (summarize, answer, classify, rewrite) and you want the cheapest model that clears a …

    line 41
  3. Very specific:

    Use claude_request / codex_request / etc.

    line 43
  4. Very specific:

    Every routed response carries a routing block in structuredContent.routing, and a one-line [routing] chosen=...

    line 83
  5. Very specific:

    No eligible candidate: routing.error = "NoEligibleCandidate" with the per-candidate rejection reasons (auth / breaker / capacity / …

    line 97
  6. Very specific:

    When routing is enabled, three read-only surfaces expose how it is behaving (all economics-only, no prompts/secrets/principal):

    line 108
  7. Very specific:

    When set, the reviewer target list is filled via the LCR selector: cheapest picks the single cheapest eligible provider.

    line 133

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Least-cost routing (route_request)

route_request (sync) and route_request_async (async) pick the cheapest eligible (provider, model) candidate that meets your constraints, then dispatch it through the normal provider path. They are dormant by default: registered only when [least_cost].enabled = true in ~/.llm-cli-gateway/config.toml and Personal Agent Config Kit is disabled. If the tools are absent, routing is off or Kit is active; use a specific provider tool only when that tool is valid for the active boundary.

Never use this for a mandatory exhaustive review

Least-cost routing is for a model-agnostic, cost-constrained task. It is not a review-completion mechanism. Do not use route_request, route_request_async, select:"cheapest", select:"cheapest_per_tier", maxCostUsd, a budget waiver, or output/token caps to restrict a required cross-provider review. A complete review must use the local stdio gateway MCP surface and its explicit required reviewer roster until every reviewer returns an evidence-backed unconditional approval. If a reviewer cannot run, repair/retry it or report the review incomplete/blocked; do not replace it with the cheapest candidate.

When the user explicitly authorizes full provider permissions and native MCP access for that mandatory review, use multi-llm-review's full-access protocol: build and launch the current target checkout's node dist/index.js --transport=stdio server, not a global gateway; reapply each provider-native grant per fresh job; provide the corrective-program report and exact diff/file identity; and set no caller caps. A user-required 90-second progress cadence also applies. Neither cost routing nor an LCR budget setting can implement or shorten that protocol.

When to use this vs a named provider tool

  • Use route_request when the task is model-agnostic (summarize, answer, classify, rewrite) and you want the cheapest model that clears a quality floor.
  • Use claude_request / codex_request / etc. when you need a SPECIFIC model or provider, a provider-specific flag, session resume, or a worktree. In phase_1 route_request is fresh one-shot only: it accepts a registered workspace for CLI candidates, but no sessionId or worktree.
  • Discover configured API and CLI candidates with list_models() and provider capabilities before forming an explicit candidate list. Do not assume a canonical CLI provider is installed, authenticated, priced, or eligible.

CLI candidates selected by route_request do not inherit the gateway process repository. The unscoped execution boundary gives each child a fresh private neutral cwd. For repository-dependent work, pass a registered workspace and restrict candidates to CLI providers authorized for that workspace. A routed HTTP/API provider does not receive workspace files, and workspace does not turn it into a filesystem-capable provider. Use an explicitly targeted named provider tool when the provider itself must be fixed.

Inputs

  • prompt (or promptParts): the request. Exactly one.
  • minTier: economy | standard | frontier (default standard). A hard floor, never downgraded to save money.
  • maxCostUsd: per-request budget cap. Over-budget fails closed.
  • expectedOutputTokens / maxOutputTokens: tune the ranking and the conservative budget bound.
  • requiredCapabilities: { images?, attachments?, toolCalling?, jsonSchema?, outputFormat?, effort? }. A candidate missing a required capability is excluded.
  • candidates: an explicit (provider, model)[] to restrict the pool (also whitelists otherwise-untiered / maintain-only candidates like cursor/devin).
  • workspace: registered gateway workspace alias used as the cwd when the selected candidate is a CLI provider. It is not sent to HTTP/API providers.
  • allowUnpriced + budgetWaiver: BOTH are required to admit an unpriced (source: "unknown") candidate. An unpriced candidate always ranks strictly last and cannot win over any priced candidate.
  • fallback: a (provider, model) used ONLY when the eligible pool is empty; bypasses cost ranking but still passes eligibility and (unless budgetWaiver) the budget gate.

Reading the result

Every routed response carries a routing block in structuredContent.routing, and a one-line [routing] chosen=... est=$... (basis, confidence) considered=N reroutes=M banner prepended to the text. Fields: chosen, tier, estCostUsd, costBasis (provider-reported | derived-from-tokens | pre-flight-estimate), confidence, nearTie, estInputTokens, estOutputTokens, priceAsOf, priceSource, consideredCount, rejected: [{candidate, reason}], reroutes, and (on failure) error.

estCostUsd is always an ESTIMATE labelled with its inputs, never a billed cost.

Failure semantics (fail closed)

  • Over budget: routing.error = "BudgetExceeded", isError: true. Raise maxCostUsd rather than expecting a silent downgrade.
  • No eligible candidate: routing.error = "NoEligibleCandidate" with the per-candidate rejection reasons (auth / breaker / capacity / capability / tier / price / budget). Loosen minTier, add candidates, or set allowUnpriced + budgetWaiver.
  • For synchronous routing, a transient dispatch failure can re-select over the remaining pool up to [least_cost].max_reroutes. Non-transient failures drop the candidate and continue. route_request_async makes one selection and does not perform reroutes after its job is admitted.

Observability (phase_2)

When routing is enabled, three read-only surfaces expose how it is behaving (all economics-only, no prompts/secrets/principal):

  • routing://decisions (MCP resource): the recent routed decisions, each with provider/model, estCostUsd, costBasis, confidence, reason, considered, reroutes, at.
  • routing://priors (MCP resource): the learned per-(provider,model) output-token priors (median/p90/samples), the per-(content-type,family) input calibration k + sample count + confidence, and priceAsOf. Priors are anonymized model-level economics; priors_scope (global | principal | off) scopes the learning (or disables it).
  • llm_process_health gains a leastCost block: per-provider telemetry tier (T1..T4) and per-candidate eligibility (priced? / authed? / breaker? / tier?).
  • doctor --json gains a least_cost block: pricing asOf + staleness, per-provider telemetry tier, per-candidate eligibility, untiered models, and per-(content-type,family) calibration quality (k/samples/confidence).

API-provider models are priced from a published catalog (prefer_catalog_price picks catalog over the CLI table when both resolve).

Cheapest-reviewer selection (phase_3)

validate_with_models, red_team_review, consensus_check, second_opinion, and ask_model accept an OPT-IN select: "cheapest" | "cheapest_per_tier". When omitted, the explicit models/model list is used verbatim (default unchanged). When set, the reviewer target list is filled via the LCR selector: cheapest picks the single cheapest eligible provider; cheapest_per_tier picks the cheapest provider in each quality tier (economy/standard/frontier). It fails closed (no jobs started) when [least_cost].enabled is false or nothing is eligible.

These opt-in selectors are appropriate only where the user deliberately wants a cost-constrained sampling/review task. They are forbidden for a mandatory complete review under this repository's no-limit review contract.

Personal Agent Config Kit

route_request, route_request_async, validation tools, and least-cost selection are not registered while Personal Agent Config Kit mode is enabled, even if [least_cost].enabled = true. Kit is local-only, supports Claude/Codex/Mistral only, and requires durable SQLite/PostgreSQL admission. Do not try to route around that boundary; either use a normal non-Kit gateway only with the user's explicit approval of the changed boundary, or report the requested routing unavailable.

Config (operator, ~/.llm-cli-gateway/config.toml)

[least_cost]
enabled = true                 # nothing routes until this is true
min_tier = "standard"
max_cost_usd = 0.50
allow_unpriced = false
max_reroutes = 2

[least_cost.tiers]             # (provider:family) -> tier; ships with defaults
"claude:claude-opus" = "frontier"

[least_cost.candidates]
allow = []                     # empty = all eligible
deny = []

See docs/least-cost-routing-contract.md for the frozen decisions and invariants.

max_reroutes limits only the router's transient dispatch retries for an ordinary cost-routed request. It is never an approval/review-round limit.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.