mecatl model router config builder

Prerequisites

• The operator has a mecatl deployment and an API key for at least one provider (OpenRouter, Anthropic, or OpenAI). The config targets ONE provider — provider is fixed per session, so every alias must resolve to a model id on the same provider.
• Web search is available for live benchmark/pricing lookup.

Workflow

Step 1 — Elicit preferences (BEFORE any search)

Ask one concise question at a time in this order; wait for the answer before the next question. Use ordinary conversational text as the default, with the recommended option stated plainly. Do not render banners, checkmark summaries, or a widget/protocol syntax.

Interpret natural-language answers rather than requiring exact option labels. If an answer is ambiguous, ask a brief follow-up. The operator may revise an earlier answer conversationally at any point (for example, “Actually, use Anthropic”); confirm the changed answer and revisit any dependent choice if necessary.

If the current client explicitly provides a native question UI, it may present the same question and choices there. Never assume it exists or expose its internal protocol in chat.

Q1 — Provider

Ask: “Which provider should this config use? I recommend OpenRouter because one key can reach multiple vendors. OpenRouter, Anthropic direct, OpenAI direct, or another provider?”

Q2 — Priority axis

Ask: “What matters most: balanced cost and capability (recommended), cost-tiered (prefer cheaper models when they can do the job, saving money but possibly trading away capability), or capability-first (prefer the strongest fit, with potentially higher cost)?”

Q3 — Open vs proprietary

Ask: “Are hosted proprietary models acceptable (recommended), or do you require MIT/Apache open-weight models?”

Q4 — Multimodal

Ask: “How often do you need image input: not at all (recommended), rarely, or commonly?”

Q5 — Target ceiling (optional)

Ask: “Is there a target coding ceiling or model you want to match? You can name one, or say there is no specific target.”

Q6 — Existing config (auto-discovered, not asked blank)

Do NOT ask. Silently run cat ~/.config/mecatl/settings.yaml.

• Found with models: block — show it fenced, then ask: “I found an existing models: config. Should I use it as a base (recommended) or start fresh?”
• Not found or no models: block — skip silently, proceed to Step 2.

Record all answers. These determine which models are even candidates.

Step 2 — Search the latest benchmarks & pricing

For each tier the operator needs (heavy/coder/quick, + optional image/specialty), search for:

• SWE-bench Verified (the primary coding benchmark — prefer independent Vals.ai scores over vendor self-reports; note both).
• SWE-bench Pro (harder real-world repos) and Terminal-Bench (agentic multi-step shell — closest proxy to a harness loop) where available.
• LiveCodeBench / HumanEval (pure code-gen) as secondary signals.
• Pricing (input + output per 1M tokens) on the operator's chosen provider — verify the exact OpenRouter/Anthropic/OpenAI model id.
• Modality (text-only vs multimodal) — critical if vision is in scope.
• License (MIT/Apache open vs proprietary) if the operator cares.
• Context window (1M+ is common in 2026; note anything below).

Cross-check at least two sources per model (vendor model card + independent leaderboard like Vals.ai / llm-stats / LLMReference). Flag vendor-reported vs independently-verified scores — they diverge by 2–4 pts regularly.

Present a shortlist table (model | key benchmarks | price | modality | license) for each tier, then recommend one per tier with rationale tied to the operator's stated preferences.

Step 3 — Map to the config schema

Read references/config-format.md for the exact schema and rules. Then build the config:

1. Aliases — the spine. Define heavy/coder/quick (minimum) pointing at concrete provider model ids. Add image (or another specialty alias) only if the operator needs multimodal. Every alias must be on the chosen provider.
2. default — the session model. Usually the heavy alias (the strongest reasoning model), unless the operator wants a cheaper default.
3. Slots — route the four housekeeping calls (compaction/ask-reviewer/ guardrail/router) to the quick alias (they're one-turn, tool-less calls that need instruction-following + JSON discipline, not deep coding). Route plan to heavy (the opusplan pattern — plan-mode turns swap to a strong reasoning model).
4. Router categories — 3 categories minimum (large/medium/small mapped to heavy/coder/quick). Add a 4th ONLY for a genuinely distinct model class (e.g. image for multimodal) — do NOT add a 5th; more categories degrade classifier accuracy and widen the steering surface. Write thorough description: fields — the classifier reads them literally to route.
5. default-category — usually medium (the bulk of delegation work).

Step 4 — Deliver the config

Emit the complete models: YAML block, ready to drop into ~/.config/mecatl/settings.yaml. Include inline # → <concrete-id> comments on each category's model: line so the operator can see the resolution at a glance.

After the config, include:

• A changes-from-previous summary (if revising) — one line per changed alias/ slot/category, with the rationale.
• A verification note — tell the operator to check the mecated log for model slot ACTIVE / subagent model router ACTIVE lines on startup (a missing line = that binding failed to resolve and degraded to the session model).
• An honest gaps note — call out anything the chosen fleet can't do (e.g. "nothing here reaches Opus 4.8's 88.6% SWE-bench Verified; that ceiling requires Anthropic"). Don't oversell.

Guidelines

• Provider-fixed invariant: never mix providers across aliases. If the operator wants Anthropic models, ALL aliases are anthropic/* ids. OpenRouter is the common choice because it aggregates many vendors behind one provider.
• Cost-tiered default: when the operator says "cost-tiered" or doesn't specify, prefer open/cheap models (DeepSeek V4-Flash/Pro, GLM-5.2, Gemini Flash) and put the expensive frontier model only where it's load-bearing (heavy/ plan/large). Housekeeping slots always go on the cheapest credible model.
• Capability-first default: when the operator says "capability-first", put the strongest available model (Opus 4.8 / GPT-5.5 / Gemini 3 Pro) at heavy/ default/plan and a strong mid (Sonnet 4.6 / Gemini 3.5 Flash) at coder, accepting the higher spend. Still route housekeeping to a cheaper tier — there's no value in running compaction summaries on Opus.
• Vision: if the operator needs images and the heavy/coder models are text-only, add an image alias pointing at a multimodal model (Gemini 3.5 Flash is the strongest cheap multimodal coder in mid-2026). If vision is rare, make image a category the router can pick OR an alias the operator invokes via per-call model: image (more reliable — the classifier can't detect attached files, only prompt text that mentions them).
• Category descriptions: write them thoroughly — they're the classifier's only signal. Describe the task profile in concrete terms ("screenshots, UI mockups, design specs, diagrams" not just "visual tasks").
• 3–4 categories max: difficulty tiers (large/medium/small) + at most one distinct-class category (image/specialty). More categories hurt classifier accuracy and widen the untrusted-prompt steering surface.
• Cite sources: when recommending a model, note whether its benchmark is vendor-self-reported or independently verified (Vals.ai etc.) — the gap matters.

Error handling

See also

• references/config-format.md — the complete models: YAML schema, key rules, and a worked OpenRouter example.

Interaction example

Ask normally, one question at a time. For example:

1. “Which provider should this config use? I recommend OpenRouter, but Anthropic direct, OpenAI direct, or another provider also work.” → OpenRouter
2. “What matters most: balanced cost and capability, cost-tiered, or capability-first?” → capability-first
3. “Are hosted proprietary models acceptable, or do you require open weights?” → proprietary
4. “How often do you need image input: not at all, rarely, or commonly?” → rarely
5. “Is there a target coding ceiling or model you want to match?” → Opus 4.8 territory

The operator can correct an answer naturally, such as “Actually, use Anthropic instead,” before the search begins. Then silently inspect ~/.config/mecatl/settings.yaml; if it has a models: block, ask whether to use it as a base or start fresh. Otherwise, begin Step 2 and search for models that fit the recorded preferences.