optimize-classifier
Skill168 starsChanged 8 months agoMITView on GitHub
Analyze your Model Matchmaker override patterns and tune the local classifier to match your preferences. Use after collecting 50+ recommendations.
In this repository2 other files it gives its agents.
- context-monitorSkill
- share-analyticsSkill
The file
We split this skill into the rules it gives an agent
12 rules. About 0 are common, 11 are distinctive, and 1 is very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈11
Distinctive · estimate
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 13
Analyzes: when you overrode the advisor's recommendations
- Very specific: line 26
Read the NDJSON log file at ~/.cursor/hooks/model-matchmaker.ndjson.
- Distinctive: line 28
Check if there's enough data:
- Distinctive: line 31
If not enough data, tell the user: "You need more usage data.
- Distinctive: line 158
If user says "yes" or approves specific keywords:
- Distinctive: line 162
Add the new keywords to the list with a comment explaining they were auto-added
- Distinctive: line 163
Write the file back using the StrReplace tool
- Distinctive: line 168
Only suggest keywords with 3+ override occurrences (confidence threshold)
- Distinctive: line 169
Don't add generic words that could cause false positives ("the", "make", "update")
- Distinctive: line 171
If the user's overrides are inconsistent (sometimes want Opus, sometimes want Sonnet for the same keyword), note that and ask for …
- Distinctive: line 188
After your first 50-100 prompts (to establish baseline patterns)
- Distinctive: line 190
After a major project shift (switching from backend to frontend work, for example)
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Optimize My Classifier
This skill helps you personalize your Model Matchmaker classifier based on your actual usage patterns. After you've collected 50+ recommendations, this skill analyzes when you disagreed with the advisor and tunes your local classifier to match your preferences.
What This Skill Does
- Reads your local Model Matchmaker logs (
~/.cursor/hooks/model-matchmaker.ndjson) - Analyzes when you overrode the advisor's recommendations
- Finds patterns in the prompts where you disagreed (common words, task types)
- Suggests keyword additions to your
model-advisor.shfile - Updates your classifier (with your approval) so future recommendations match your preferences
Privacy: Everything happens locally. No data leaves your machine. You review and approve every change.
Instructions for the AI
You are helping the user optimize their Model Matchmaker classifier based on their personal override patterns. Follow these steps:
Step 1: Read and Validate Log File
Read the NDJSON log file at ~/.cursor/hooks/model-matchmaker.ndjson.
Check if there's enough data:
- Need at least 50
recommendationevents total - Need at least 5
OVERRIDEevents to find patterns - If not enough data, tell the user: "You need more usage data. Come back after 50+ prompts with at least a few overrides."
Step 2: Analyze Override Patterns
Filter to action: "OVERRIDE" events and group by model direction:
Group A: User preferred Opus over recommended Haiku/Sonnet
- These are prompts where the classifier said "use a cheaper model" but you said "no, I need Opus"
- Extract common words from
prompt_snippetfields in this group
Group B: User preferred Sonnet over recommended Opus
- These are prompts where the classifier said "use Opus" but you said "no, Sonnet is fine"
- Extract common words from
prompt_snippetfields in this group
Group C: User preferred Sonnet over recommended Haiku
- Extract common words from
prompt_snippetfields in this group
Group D: User preferred Haiku over recommended Sonnet/Opus
- Extract common words from
prompt_snippetfields in this group
Step 3: Extract Keyword Candidates
For each group, find words that appear in 3+ override prompts (minimum frequency threshold).
Exclude common stop words:
- the, a, an, is, are, was, were, be, been, being, have, has, had, do, does, did
- in, on, at, to, for, of, with, from, by, about, as, into, through, during
- this, that, these, those, I, you, we, they, it, he, she, me, my, your
Focus on action verbs and technical terms:
- Examples: debug, investigate, refactor, optimize, analyze, build, create, fix, update, configure
Step 4: Generate Proposed Changes
For each keyword group, map to the correct section of model-advisor.sh:
For Group A keywords (user prefers Opus):
→ Add to opus_keywords list around line 46
For Group B/C keywords (user prefers Sonnet):
→ Add to sonnet_patterns list around line 63
For Group D keywords (user prefers Haiku):
→ Add to haiku_patterns list around line 53
Step 5: Calculate Impact
For each proposed keyword:
- Count how many past overrides would become correct recommendations if this keyword were added
- Show confidence: "3 overrides → 'debug' should trigger Opus"
Step 6: Present Recommendations
Output in this format:
# Classifier Optimization Report
## Your Usage Summary
- Total recommendations: N
- Overrides: M (X.X%)
- Ready for optimization: [Yes/No - need 5+ overrides]
## Patterns Found
### You prefer Opus for:
**Keyword: "debug"**
- Frequency: 5 overrides
- Current behavior: Recommends Sonnet
- Proposed: Add "debug" to opus_keywords
- Impact: 5 past overrides would become correct recommendations
**Keyword: "investigate"**
- Frequency: 3 overrides
- Current behavior: Recommends Sonnet
- Proposed: Add "investigate" to opus_keywords
- Impact: 3 past overrides would become correct recommendations
[Repeat for other keywords]
### You prefer Sonnet over Opus for:
[Same structure]
### You prefer Haiku for:
[Same structure]
## Proposed Changes to ~/.cursor/hooks/model-advisor.sh
**Add to opus_keywords (line 46):**
```python
opus_keywords = [
"architect", "architecture", "evaluate", "tradeoff", "trade-off",
"strategy", "strategic", "compare approaches", "why does", "deep dive",
"redesign", "across the codebase", "investor", "multi-system",
"complex refactor", "analyze", "analysis", "plan mode", "rethink",
"high-stakes", "critical decision",
# NEW - Added based on your override patterns:
"debug", "investigate" # 8 overrides support this
]Add to sonnet_patterns (line 63): [Show specific regex additions if any]
Next Steps
Would you like me to apply these changes to your classifier?
- Yes - I'll update ~/.cursor/hooks/model-advisor.sh with these keywords
- Some of them - Tell me which keywords to add
- No - Just show me the analysis, don't change anything
If you approve, I'll:
- Read your current model-advisor.sh
- Add the new keywords to the appropriate lists
- Write the updated file back
- Confirm the changes
You can test immediately by restarting Cursor or starting a new composer session.
### Step 7: Apply Changes (If User Approves)
If user says "yes" or approves specific keywords:
1. Read `~/.cursor/hooks/model-advisor.sh`
2. Locate the appropriate keyword list (opus_keywords, sonnet_patterns, or haiku_patterns)
3. Add the new keywords to the list with a comment explaining they were auto-added
4. Write the file back using the StrReplace tool
5. Confirm: "Updated! Your classifier now recommends [model] for prompts containing [keywords]. Changes take effect in your next Cursor session."
### Important Notes
- Only suggest keywords with 3+ override occurrences (confidence threshold)
- Don't add generic words that could cause false positives ("the", "make", "update")
- Show the user exactly what will change before changing it
- If the user's overrides are inconsistent (sometimes want Opus, sometimes want Sonnet for the same keyword), note that and ask for clarification
---
## How to Use This Skill
As a user, invoke this skill by saying something like:
- "Optimize my Model Matchmaker classifier"
- "Tune my classifier based on my overrides"
- "Analyze my Model Matchmaker usage and improve it"
- "Update my classifier to match my preferences"
The AI will analyze your logs, find patterns, and propose keyword additions. You review and approve before any changes are made.
## When to Run This
- After your first 50-100 prompts (to establish baseline patterns)
- Monthly or quarterly as your workflow evolves
- After a major project shift (switching from backend to frontend work, for example)
- Whenever you notice you're overriding the advisor frequentlyDoes it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

