decision-models
Skill0 starsChanged 34 days agoBSD-3-ClauseView on GitHub
Build software with Clef and Jev decision models (`cloudflare.clef`, `typesafe.jev`) through the Dartmouth Chat API (`POST https://chat.dartmouth.edu/api/v1/systemone` for System One payloads; `POST /api/v1/decisions` for the OpenAI Decisions schema). Decision models read `state` and typed `questions` (`noul`, `choice`, `score`) and return a calibrated probability for every allowed option without generating free-form text. Use when a feature needs a fast semantic judgment (routing, triage, tool or handler selection, extraction by selection, verification, moderation, reranking) or when an LLM prompt-and-parse step should become a structured decision. Also ships `decide`, a FastMCP 4 server that lets coding agents (Claude Code, Codex, goose, OpenCode, Cursor) call a decision model as a tool during a session, for one input or packed batches.
- Reads credentials
3 other projects use a version of this.
Adopted by
Most recently taken up by
- timmo001/skillstheir own version, first seen Nov 7, 2026
- Haple/system-onetheir own version, first seen Oct 20, 2026
- josemarcilio/skillstheir own version, first seen Oct 13, 2026
Other versions3 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 3
- timmo001/skills★ 2same name
- josemarcilio/skills★ 1same name
- Haple/system-one★ 0same name
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository6 other files it gives its agents.
- AGENTS.mdAGENTS.md
- dartmouth-brandingSkill
- dartmouth-chatSkill
- langchain-dartmouthSkill
- presentation-designSkill
- research-container-deploymentSkill
The file
We split this skill into the rules it gives an agent
8 rules. 0 are common, 2 are distinctive, and 6 are very specific to this project.
0
Common
Most of the 3 edited versions we read keep these.
2
Distinctive
This author's own choices. Worth a look.
6
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 64
Don't call decide for everyday single-item choices you already have in context (such as picking which file to read or which tool to run …
- Very specific: line 83
Treat warnings as a signal to rewrite the question or ask the user.
- Very specific: line 85
If decide isn't configured, don't fake it.
- Very specific: line 90
When designing a new decision workflow or checking primitives and patterns, consult TypeSafe's live documentation index …
- Very specific: line 122
Check GET https://chat.dartmouth.edu/api/v1/models for the deployed IDs.
- Distinctive: line 220
Start from what the app should show, select, change, or hand off, and work backward to the judgments needed.
- Distinctive: line 232
Verify and escalate: Check claims or fields against evidence.
- Very specific: line 268
Pack many small items into one request: When many items need the same questions, put them under one state ({"items": {"a": ..., "b": ...}}) …
“Common” is measured: kept by at least 6 in 10 of the 3 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Build with Clef and Jev
Clef and Jev (System One) are decision models, not LLMs. You send state and typed questions;
the model reads them once and scores every allowed option of every question together.
Output is probabilities only, so an answer can never be an option you didn't
define and there is no JSON to repair.
Code owns the workflow; the decision model supplies judgment where code needs understanding.
A coding agent keeps its own LLM. Clef or Jev is called either by the code the
agent writes, or by the agent itself through the decide MCP tool (below).
In Python, call the API through DartmouthDecisions from the langchain_dartmouth
package rather than hand-rolling HTTP — see the
langchain-dartmouth skill.
For a drop-in AGENTS.md template guiding coding agents on when to pick a
decision model vs an LLM vs plain code, see
references/agents-md-template.md.
Call from a coding agent: the decide MCP
scripts/decide_mcp.py is a single-file FastMCP 4
stdio server that talks to the Dartmouth Chat API (https://chat.dartmouth.edu/api).
It exposes decide for one input, decide_batch for the same questions over
many items or to rerank candidates, and decision_models, plus a
decide://guide resource and a design_questions prompt. It validates questions
before sending and returns warnings for unsure answers. Setup for each agent,
environment variables, and tests are in references/decide-mcp.md.
Don't call decide for everyday single-item choices you already have in context
(such as picking which file to read or which tool to run next)—serialising the
tool call costs more tokens and time than deciding directly. Reach for decide /
decide_batch in three situations:
- Many-item triage, filtering, or reranking (
decide_batch): scoring or filtering dozens of search hits, TODOs, issues, log lines, or candidate records against one rubric in a packed batch (itemininstructions), instead of looping or emitting a long LLM table. - Interactive question design & calibration (
decide/decide_batch): when writing decision questions for application code, test them on sample inputs in-session to verify clean separation (nonoul ≈ 0.5orlabel_masswarnings) before committing them. - Independent rubric / blast-radius checks (
decide): scoring a diff, migration, or log against a fixed checklist (touches auth/credentials? irreversible data change?) with a decorrelated model.
Rules when calling decide:
- Batch every independent question about one state into one call, put full
meaning in
instructions, add anotherexit, and keep math and dates out. - Treat
warningsas a signal to rewrite the question or ask the user. Never act irreversibly on them. - If
decideisn't configured, don't fake it. Fall back to your own reasoning and say so.
Read the live docs
When designing a new decision workflow or checking primitives and patterns, consult
TypeSafe's live documentation index (https://docs.typesafe.ai/llms.txt)—every
concept and cookbook applies to both Jev and Clef:
- Mintlify serves Markdown by appending
.mdto any page path (e.g.https://docs.typesafe.ai/concepts/how-to-build-with-system-one.md,https://docs.typesafe.ai/concepts/state.md,https://docs.typesafe.ai/primitives.md,https://docs.typesafe.ai/confidence.md). - Inspect the closest cookbook (
function_calling.md,pre_parsed_value_extraction_cookbook.md,rerank_typesafe.md,hierarchical_classification.md,citation_check.md,sde_cascade.md) for decomposition patterns. - Always call through the Dartmouth Chat API endpoints below rather than calling
api.typesafe.aidirectly.
Endpoints and access
- Base URL:
https://chat.dartmouth.edu/api(Dartmouth Chat, which forwards decision requests to Dartmouth's LiteLLM gateway). Paths below are relative to the host. - API key: your personal Dartmouth Chat key: chat.dartmouth.edu › Profile › Settings › Account › API Key. It shares your account's credit budget (
GET /api/v1/credits/balance). Keep it inDCHAT_KEYand send it asAuthorization: Bearer $DCHAT_KEY. - Images: Clef takes an
imageslist (data URLs, at most 2) and Jev takes namedassets(PNG/JPEG, 280–1568 px per side; also Box questions). Formats, limits and examples are in references/images.md.
| Use case | Endpoint |
System One schema (state + questions object), Clef or Jev | POST /api/v1/systemone |
OpenAI Decisions schema (input + questions array) | POST /api/v1/decisions |
| List models | GET /api/v1/models (decision models have "mode": "evaluation") |
Use
/api/v1/..../api/systemonereturns 405, and/api/chat/completionsrejects decision models. AGETto an unknown path returns the web app's HTML with status 200, so check for a JSON body. Responses can addid,objectand per-questiondiagnostics(label_mass,argmax_is_label): treatargmax_is_label: false, or alabel_masswell below 1, as an unreliable answer. Request the model ID (e.g.cloudflare.clef); the response'smodelis the served name (clef,jev-1.14.0).
OpenAI Decisions format (/api/v1/decisions): questions is a list; each item has name, type (predicate for yes/no, choice, score) and instructions. Choice options are choices: [{"value": "billing", "description": "..."}], and score levels are levels: [{"label": "No impact"}, ...], lowest first. answers comes back as a list: {"type": "predicate", "name": ..., "probability": 0.98}, and choice/score answers carry probabilities as a list of {value, probability}. Prefer /api/v1/systemone unless your client already speaks OpenAI Decisions.
Pick the model: Clef vs Jev
Check GET https://chat.dartmouth.edu/api/v1/models for the deployed IDs. As of 2026-10-29 the API serves cloudflare.clef and typesafe.jev; clef-flash is not deployed there.
| Model | Origin & Hosting | Modalities & Limits | Best for |
cloudflare.clef (clef) | Cloudflare open-weight (Apache 2.0, Qwen3.8-27B + joint schema head), hosted on Dartmouth vLLM | Text, JSON & images (16,384 ctx; choice 2..26 options; ≤2 images) | Default. Fast, private classification, routing, triage, tool selection (BANKING77, BFCL), and scoring. |
typesafe.jev (jev-1.14.0) | TypeSafe proprietary System One flagship model, cloud-hosted API | Text, JSON & PNG/JPEG images; Box questions (choice 2..255 options) | Leads on reasoning-heavy decision sets (GPQA, MMLU-Pro, BBH, When2Call). Returns {"type": "noul", "noul": 0.95} on noul (without the extra probabilities/confidence fields vLLM adds). |
Because TypeSafe created the System One API for Jev and Cloudflare built Clef to speak that exact same schema, switching or A/B testing between cloudflare.clef and typesafe.jev on /api/v1/systemone is just changing "model".
Request contract
Send only model, state, and questions, plus images (Clef) or assets (Jev) when you attach images. Never add LLM chat fields (messages, temperature, max_tokens, system, stream, or top-level instructions).
model(string): deployed model ID (e.g.cloudflare.clef).state(required, string, JSON object, or JSON array): what the questions are about.- Context limit is 16,384 tokens. Questions are counted first; if they don't fit, the request fails with
400. Otherwise, longstateis silently truncated to fit. Chunk or pre-select long text in code.
- Context limit is 16,384 tokens. Questions are counted first; if they don't fit, the request fails with
questions(required, object):1..64questions mapping<question_id>-> question definition. Question IDs must be non-empty strings without:or newlines.noul(yes/no):instructions(string) and optionalcriteria: {"true": "...", "false": "..."}saying what yes and no mean.- Returns
{"type": "noul", "noul": 0.98, "probabilities": {"yes": 0.98, "no": 0.02}, "confidence": 0.98}(noulisP(yes)).
- Returns
choice(pick one):instructions(string) andcriteriamapping option ID -> description (stringornull),2..26options.- Returns
{"type": "choice", "choice": "<winner>", "probabilities": {"<option>": 0.96, ...}, "confidence": 0.96}.
- Returns
score(ordered scale):instructions(string) andcriteriaas an ordered array of2..10level descriptions, lowest first (indexed from0).- Returns
{"type": "score", "score": 1.96, "legend": {"0": "...", ...}, "probabilities": {"0": 0.01, ...}, "confidence": 0.96}.scoreis the probability-weighted expected index (can be fractional).
- Returns
Example: HTTP POST /api/v1/systemone
Python quick start: in a Python LangChain app, skip the raw HTTP and use
DartmouthDecisions (typed Noul/Choice/Score questions over the same
endpoint, authenticated with DARTMOUTH_CHAT_API_KEY) from the
langchain_dartmouth package —
see the langchain-dartmouth skill and its
decision-models cookbook recipe.
The curl below shows the underlying request contract:
curl -sS "https://chat.dartmouth.edu/api/v1/systemone" \
-H "Authorization: Bearer $DCHAT_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "cloudflare.clef",
"state": {
"ticket": "The checkout crashes every time I pay. I cannot place an order."
},
"questions": {
"is_defect": {
"type": "noul",
"instructions": "Is the customer reporting a software defect?"
},
"team": {
"type": "choice",
"instructions": "Which team should handle this ticket?",
"criteria": {
"billing": "payments, charges, and checkout failures",
"shipping": "deliveries and tracking",
"other": null
}
},
"severity": {
"type": "score",
"instructions": "Rate the impact of this issue.",
"criteria": ["No impact", "Minor inconvenience", "Cannot complete a purchase"]
}
}
}'Response:
{
"model": "clef",
"answers": {
"is_defect": {
"type": "noul",
"noul": 0.98,
"probabilities": {"yes": 0.98, "no": 0.02},
"confidence": 0.98
},
"team": {
"type": "choice",
"choice": "billing",
"confidence": 0.96,
"probabilities": {"billing": 0.96, "shipping": 0.03, "other": 0.01}
},
"severity": {
"type": "score",
"score": 1.96,
"confidence": 0.96,
"legend": {"0": "No impact", "1": "Minor inconvenience", "2": "Cannot complete a purchase"},
"probabilities": {"0": 0.01, "1": 0.02, "2": 0.97}
}
},
"usage": {"input_tokens": 184, "output_tokens": 0}
}Find the useful shape
Start from what the app should show, select, change, or hand off, and work backward to the judgments needed. Rules, arithmetic, counting, date math, exact lookups, and execution stay in code.
- Route and fill arguments. Pick a handler plus its typed parameters in one request; ask branch-specific questions speculatively, consume only the relevant ones.
- Select, don't generate. Find candidates in code (regex, parser, retrieval), have Clef pick one, then copy or normalize it. It cannot pick an omitted value.
- Judge evidence. Rerank retrieved chunks; hierarchical classification.
- Score once, reuse. Store per-dimension scores; let code change weights and thresholds without rerunning inference. Scores make good ML features.
- Verify and escalate. Check claims or fields against evidence; send low-confidence or failing cases to a human or a reasoning LLM.
- Gate an agent. Decide next tool, whether a step is done, or whether an action needs approval, while an LLM does the actual work.
Design the questions
Question IDs are NOT used by the model to understand the question; put the full
meaning in instructions. Reference parts of state with backticked paths
like ticket.messages[0].text. Prefer named JSON fields when state has several
parts.
- One narrow, coherent judgment per question. Say what counts ("names the
browser and the action that triggers it", not "has repro steps"). Use strings
for simple questions, or structured objects/arrays in
instructionsorcriteriawhen definitions, contrasts, exclusions, or examples clarify meaning. - State each speculative premise explicitly: because questions in one request run in parallel and cannot see each other's answers, write branch-specific questions so they make sense on their own (e.g. "Assuming the customer is asking for a refund, which item is it for?").
- Every
choicethat might not fit gets an exit:other: nullor"not stated"(or pair it with a separatenoulpresence check when presence is independently useful). - Score levels describe concrete situations and stand alone.
- Multi-label: one
noulper label, not achoice. - No arithmetic, counts, or dates inside questions.
- A
noulnear0.5means equal probability for yes and no, not "medium intensity" (use ascorefor intensity/degree). Anoulnear0.5or lowconfidenceusually means a vague question—rewrite it before blaming the model.
Compose and verify
- Send all independent questions about one state in ONE request (speculative fan-out; they run in parallel in a single pass and can't see each other). Never loop over questions. Make a second request only when an answer decides what to fetch next.
- Pack many small items into one request. When many items need the same
questions, put them under one
state({"items": {"a": ..., "b": ...}}) with per-item question ids (a.sensitive,b.sensitive) up to 64 questions per request. One integration went from more than 60 s (one call per entity, timing out) to about 0.2 s (21 entities × 3 questions per call). Keep packed state well under the context limit, and spot-check packed answers against single-item calls. - Rerank in one pass, and never let a threshold return nothing. Score every
candidate on one
scorerubric in a single request, sort, then filter. Vague queries can push every candidate into the same middle bucket, so if the filter would drop everything, fall back to the top-k by score. - Size timeouts for batches and degrade gracefully. Size HTTP timeouts for packed batches rather than a single call, and fall back to rules or regex when the decision endpoint is slow or unreachable instead of failing the feature.
- Give each component one role. Exact lookups and joins go in code or the database, retrieval goes to embeddings, judgments over the retrieved candidates go to the decision model, and only written output goes to an LLM.
- Confidence measures distribution concentration, not permission to act. Gate irreversible actions on thresholds tuned on labeled data, with human or LLM fallback. Note that several acceptable options can spread probability—low confidence does not invalidate a harmless preference choice. Ignore uncertainty on branches you don't use.
- Keep policy explicit in code: weighted scores suit compensating trade-offs,
whereas an "any serious violation" rule needs separate
noulconditions. - Keep all questions, model choice, and thresholds in one reviewable file.
- Log the
modelfield, your config version, andusage.input_tokensper call. - Typed output guarantees the interface, not truth. Build a 20–100 example labeled set early; rerun it on every question edit or model change.
- Keep API tokens server-side.
- For failures, inspect exact state (was it truncated at 16K?), questions, candidates, answers, and composition before blaming the model.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

