solo-model-shrink
Skill18 starsChanged 4 months agoMITView on GitHub
Take a trained neural model to devices — ONNX export, int8 quantization, Core ML conversion, on-device benchmarking, download-on-demand delivery. Use when user says "сожми модель", "quantize the model", "convert to Core ML / ONNX", "model is too big for the app", "run the model on iPhone/Android/web", or an ML feature needs to ship inside a mobile/web app. Do NOT use for training or fine-tuning models, or for LLM API integration (that's app plumbing, not model porting).
In this repository62 other files it gives its agents.
- CLAUDE.mdCLAUDE.md
- solo-android-releaseSkill
- solo-apple-app-iconSkill
- solo-auditSkill
- solo-boardSkill
- solo-buildSkill
- solo-community-outreachSkill
- solo-content-genSkill
- solo-creator-growthSkill
- solo-customer-finderSkill
- solo-deep-researchSkill
- solo-deploySkill
- solo-diagnoseSkill
- solo-domainSkill
- solo-factorySkill
- solo-github-outreachSkill
- solo-gmail-cliSkill
- solo-grillSkill
- solo-hackathonSkill
- solo-humanizeSkill
- solo-i18nSkill
- solo-index-youtubeSkill
- solo-initSkill
- solo-ios-devSkill
- solo-ios-releaseSkill
- solo-kaggle-gpuSkill
- solo-kindleSkill
- solo-knowledgeSkill
- solo-landing-genSkill
- solo-launchSkill
- solo-legalSkill
- solo-memory-auditSkill
- solo-metrics-trackSkill
- solo-mirrorSkill
- solo-music-reels-cutSkill
- solo-open-datasetSkill
- solo-paperSkill
- solo-pdf-printSkill
- solo-pipelineSkill
- solo-planSkill
- solo-redditSkill
- solo-researchSkill
- solo-retroSkill
- solo-reviewSkill
- solo-scaffoldSkill
- solo-secretsSkill
- solo-seo-auditSkill
- solo-seo-cliSkill
- solo-setupSkill
- solo-sgrSkill
- solo-skill-auditSkill
- solo-skill-shelfSkill
- solo-stem-splitSkill
- solo-streamSkill
- solo-supervoxSkill
- solo-swiftui-design-systemSkill
- solo-transcribeSkill
- solo-validateSkill
- solo-video-editSkill
- solo-video-promoSkill
- The first 60 shown.
The file
We split this skill into the rules it gives an agent
3 rules. About 0 are common, 2 are distinctive, and 1 is very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈2
Distinctive · estimate
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 16
Every step below was paid for once.
- Distinctive: line 42
After EVERY replacement: run the model, assert max abs diff ≈ 0 vs the original.
- Very specific: line 45
Identify failing ops by graph census, not by reading code: When the converter names a node (blocks/0/partial/76), dump ts.inlined_graph, …
≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
model-shrink — ship a trained model to phones and browsers
A PyTorch checkpoint becomes: an eval, an ONNX int8 file (web/Android), a Core ML package (Apple), an on-device number, and a download-on-demand delivery. Every step below was paid for once; the order is the method.
Workflow
- Eval before anything. Ground truth + one scalar metric (F-measure, accuracy — whatever the task has) + a script that runs any model variant against it. Every later decision — quantize? fp16? distill? — is this script's output, never a guess. Keep 5–10 held-out samples as a smoke set.
- Export ONNX at a fixed shape (
torch.onnx.export, one export per batch size — see Gotchas). Verify against the original: max abs diff and the eval score.onnxruntimeCPU is the portable baseline: web (onnxruntime-webWASM), Android (ORT mobile), desktop. - Quantize weights to int8 (
onnxruntime.quantization/ct.optimize.coreml.linear_quantize_weights). Weights-only int8 is usually free — measured twice on a 20M-param transformer: F 0.872→0.871 (ONNX) and 0.864→0.863 (Core ML), 4x smaller. Re-run the eval anyway; "usually" is not "always". - Apple: convert to native Core ML (
coremltools), never the ORT Core ML execution provider (see Don't). Conversion of transformer-ish models fails on Python-int shape math; the fixes that work:- Replace einops layers/
rearrangewith explicitpermute/reshape. - Bake shapes as constants: capture
tuple(int(s) for s in x.shape)on a module attribute during a warm eager pass, use those ints inforward— the trace then contains zeroaten::size/aten::Intops. Fixed export shape makes this sound. - Derive dims from weights (
linear.out_features), not from tensors. - After EVERY replacement: run the model, assert max abs diff ≈ 0 vs the original. A stack of "obviously equivalent" rewrites drifts; a stack of asserted ones doesn't.
- Replace einops layers/
- Identify failing ops by graph census, not by reading code. When the
converter names a node (
blocks/0/partial/76), dumpts.inlined_graph, filter nodes by scope and kind (aten::Int,aten::size,prim::NumToTensor), and match. The class you think is in the call path may be a similarly-named neighbour that is never called — a patch that changes nothing (diff 0.000e+00 because the code never ran) looks exactly like a patch that is perfectly equivalent. - Benchmark on the target device, with the model's native window size and
deterministic non-zero input. Compare compute units (
cpuOnly, GPU, ANE) per model — a rotary-attention transformer measured 2x FASTER on iPhone CPU than on the ANE, and the ANE's first load cost 43s of one-time compilation. Dev-machine numbers do not transfer; a busy dev machine's numbers don't even reproduce. - Deliver as a download, not in the bundle. Tens of MB belong in a
FaceModelStore-style on-demand store: download →
MLModel.compileModel→ Application Support (excluded from backups) → validate by loading before declaring installed. An.mlpackageis a directory: host it as its files under one base URL (Manifest.json, Data/com.apple.CoreML/model.mlmodel, Data/com.apple.CoreML/weights/weight.bin) and pin that list with a test. - Leave a bench door open: a debug HTTP endpoint in the app that loads the installed model and times N predicts turns every future variant into a one-curl measurement on a real device over Wi-Fi.
Gotchas
- ONNX tracing bakes batch size and lies silently. A model traced at
batch 1 accepts
(4, N, D)and returns garbage shaped like an answer (four sequences fused into one attention pass) — wrong result, not an error. One export per batch size, or stay at batch 1. - Version pairs are load-bearing: coremltools supports a narrow torch range (e.g. torch 2.7.x + coremltools 9.0), and torchaudio must match torch to the minor or imports die with ABI errors. Pin the pair in a dedicated venv and write it in the doc.
- Unsubstituted build variables pass URL parsing.
$(MODEL_URL)left in Info.plist is non-empty andURL(string:)accepts it — the download then fails at runtime with a nonsense error. Reject any value containing$(. - Batching probably doesn't help on-device inference (measured 0.91–1.01x) and an unshuffled benchmark invents speedups from machine-load drift — interleave variants within each round.
- fp16 conversion may fail where int8 succeeds (Cast nodes in attention); int8 is smaller anyway. Don't fight fp16 for its own sake.
Don't
- Don't use ONNX Runtime's Core ML execution provider. Measured: 11x slower than ORT CPU on the same model, plus a memory leak that grew to 41 GB over repeated runs. Native coremltools conversion of the same model ran 7x faster than ORT CPU. The EP's failure mode — graph split across unsupported ops, tensors ping-ponging between CPU and ANE — is structural.
- Don't start with distillation. A converted+quantized original at 20 MB delivered on demand ships today; a distilled student is weeks of teacher infrastructure. Distill only after conversion is proven impossible or the size budget is single-digit MB.
- Don't tune performance from a laptop. Simulators lack hardware codecs, ANEs differ per chip, and a loaded machine turns 14s into 29s with no code change. One number from the target device beats twenty from the Mac.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

