modelopt-quant-recipe-search
Skill5.6k starsChanged todayApache-2.0View on GitHub
Use when the user asks to find, search for, or optimize the best quantization recipe for a model, including direct requests like "find the best quantization recipe and generate a PTQ checkpoint." Guides the multi-candidate loop: choose compute-vs-memory success metrics, select ModelOpt recipe baselines, design AutoQuant/manual recipe deltas, interpret sensitivity, and decide next candidates. Do NOT use for a single known PTQ recipe run (use modelopt-ptq), serving (use modelopt-deployment), creating/running evals (use modelopt-evaluation or modelopt-launching-evals), monitoring jobs (use modelopt-monitor), MLflow browsing (use modelopt-accessing-mlflow), or comparing completed baseline-vs-candidate scores only (use modelopt-compare-results).
1 other project uses a version of this.
Adopted by
Steady: 0 new projects took it in the last 4 weeks, none the 4 weeks before.thin data
Most recently taken up by
- NVIDIA/Model-Optimizerword for word, first seen Nov 15, 2026
Who's talking about NVIDIA/TensorRT-Model-Optimizer
- NVIDIA/Model-Optimizer“Replace ONNX simplification package from onnxsim to onnxslim”GitHub · Oct 28, 2025 · 1 reactions
- NVIDIA/Model-Optimizer“Fix FLOPs calculation”GitHub · Sep 29, 2025 · 1 reactions
- NVIDIA/Model-Optimizer“[Autocast] Fix edge case casting input directly to output”GitHub · Sep 9, 2025 · 1 reactions
- NVIDIA/Model-Optimizer“[Autocast] Fix edge case with cast producing network output”GitHub · Sep 8, 2025 · 1 reactions
- NVIDIA/Model-Optimizer“"Unknown quantization method" when loading nvfp4 checkpoint with vLLM”GitHub · Aug 19, 2025 · 1 reactions
The biggest 5 of 45 mentions linking to the repository that our crawl found. Each line is their own headline.
See all 45
Talked about on GitHub (45 times).
On GitHub · 45
- Oct 28, 2025Replace ONNX simplification package from onnxsim to onnxslim — NVIDIA/Model-Optimizer1 reaction · 23 comments
- Sep 29, 2025Fix FLOPs calculation — NVIDIA/Model-Optimizer1 reaction · 11 comments
- Sep 9, 2025[Autocast] Fix edge case casting input directly to output — NVIDIA/Model-Optimizer1 reaction · 10 comments
- Sep 8, 2025[Autocast] Fix edge case with cast producing network output — NVIDIA/Model-Optimizer1 reaction · 6 comments
- Aug 19, 2025"Unknown quantization method" when loading nvfp4 checkpoint with vLLM — NVIDIA/Model-Optimizer1 reaction · 3 comments
40 more on GitHub
- Mar 30, 2025[Quantization] Add TRT-ModelOpt as a Backend — huggingface/diffusers1 reaction · 50 comments
- Mar 11, 2025✨[Feature] Supporting Blackwell FP4 — pytorch/TensorRT1 reaction
- Sep 16, 2026推論最適化ツール・OSS — warabinbin/papers0 reactions
- Aug 10, 2026fold_weight crashes with AttributeError: 'NoneType' object has no attribute 'data' on Megatron models with tied word embeddings — NVIDIA/Model-Optimizer0 reactions · 1 comment
- Dec 5, 2025Kimi-k2 calib+export — NVIDIA/Model-Optimizer0 reactions · 1 comment
- Dec 4, 2025Quantization — HashCitrine/sample-to-make-llm-service0 reactions
- Dec 4, 2025Diffusion Rename [TRT MO -> MO] — NVIDIA/Model-Optimizer0 reactions · 2 comments
- Dec 3, 2025[5704162] Create a copy to avoid leaking ProcessGroup into state dict — NVIDIA/Model-Optimizer0 reactions · 6 comments
- Dec 2, 2025fix: Fix auto quantize multi-gpu tests — NVIDIA/Model-Optimizer0 reactions · 3 comments
- Dec 2, 2025qwen3_8b eagle3 model train and vllm ckpt export — NVIDIA/Model-Optimizer0 reactions · 1 comment
- Nov 27, 2025Add all example e2e tests for github PR merge / nightly — NVIDIA/Model-Optimizer0 reactions · 1 comment
- Nov 26, 2025[Bug Fix]: Correct block_size in NVFP4_MLP_WEIGHT_ONLY_CFG to support vLLM inference — NVIDIA/Model-Optimizer0 reactions · 3 comments
- Nov 26, 2025fix: relax sympy version — inisis/OnnxSlim0 reactions · 1 comment
- Nov 24, 2025[None][doc] Deprecate --export_fmt flag when using ModelOpt — NVIDIA/TensorRT-LLM0 reactions · 4 comments
- Nov 24, 2025[Help wanted] CNN QAT: How to correctly export ONNX and use DeviceModel for inference — NVIDIA/Model-Optimizer0 reactions · 1 comment
- Nov 22, 2025fix: issues related to TensorRT-Model-Optimizer — inisis/OnnxSlim0 reactions · 2 comments
- Nov 20, 2025[2/N] Added KDLoss based AutoQuantize — NVIDIA/Model-Optimizer0 reactions · 1 comment
- Nov 20, 2025[1/N] Refactored AutoQuantizeSearcher to _AutoQuantizeBaseSearcher & AutoQuantizeGradientSearcher; seperated quant modules and score modules — NVIDIA/Model-Optimizer0 reactions · 11 comments
- Nov 17, 2025failed to quant the Qwen3-VL-30B-A3B — NVIDIA/Model-Optimizer0 reactions · 1 comment
- Nov 13, 2025Support Wan2.2 t2v diffusers quantization — NVIDIA/Model-Optimizer0 reactions · 3 comments
- Nov 3, 2025PyTorch geometric quantization support — NVIDIA/Model-Optimizer0 reactions · 1 comment
- Nov 3, 2025Failed to save quantized model if some tensors offload to CPU — NVIDIA/Model-Optimizer0 reactions · 2 comments
- Oct 28, 2025[OMNIML-2917] export layer config using actual prefix instead of hard… — NVIDIA/Model-Optimizer0 reactions · 9 comments
- Oct 27, 2025Add convert_llama3_config_to_decilm_config + unit test — NVIDIA/Model-Optimizer0 reactions · 2 comments
- Sep 25, 2025CodeRabbit Generated Unit Tests: Add GPU tests for Megatron quantization; review/refactor unit tests — NVIDIA/Model-Optimizer0 reactions · 2 comments
- Sep 25, 2025📝 Add docstrings to `kaix/kvcache_mcore` — NVIDIA/Model-Optimizer0 reactions · 2 comments
- Sep 15, 2025FP8 Block quantize onnx export support — NVIDIA/Model-Optimizer0 reactions · 3 comments
- Sep 15, 2025add end_process in deepseek ptq — NVIDIA/Model-Optimizer0 reactions · 10 comments
- Jul 13, 2025feat: Simplify and Improve Whisper Example — NVIDIA/TensorRT-LLM0 reactions · 3 comments
- Jul 3, 2025[future request] Support more quant for torch backend — NVIDIA/TensorRT-LLM0 reactions · 2 comments
- Jul 3, 2025🐛 [Bug] examples/dynamo/torch_compile_gpt2.py run failed — pytorch/TensorRT0 reactions · 2 comments
- Jun 15, 2025INT8 to FP8 scale conversion — NVIDIA/Model-Optimizer0 reactions · 6 comments
- Jun 13, 2025Fix speculative decoding example — NVIDIA/Model-Optimizer0 reactions · 6 comments
- May 21, 2025`get_modelike_from_algo_cfg` doesn't accept `QuantizeAlgorithmConfig`, but it's typing says it should — NVIDIA/Model-Optimizer0 reactions · 3 comments
- May 17, 2025Decode latency of FP4 LLM is higher than expected — NVIDIA/TensorRT-LLM0 reactions · 1 comment
- May 7, 2025Error in torch_compile.ipynb — Project-MONAI/tutorials0 reactions · 1 comment
- May 6, 2025当我使用增量训练的时候,只改变修改一个feature的zch_size的大小,其他所有配置都不改变,会出错。 — alibaba/TorchEasyRec0 reactions
- Apr 30, 2025INT8 quantization for HiFi-GAN vocoder -- performance issue — NVIDIA/TensorRT0 reactions · 7 comments
- Mar 24, 2025bert model split into many layers after int8 quantization — NVIDIA/TensorRT0 reactions · 13 comments
- Sep 8, 2024urlopen error retrieval incomplete in modules/autoencoder_mednist.ipynb — Project-MONAI/tutorials0 reactions
Other versions1 project uses a version of this.
Word for word · 1
- NVIDIA/Model-Optimizer★ 5.6k100% same
Edited · 0
None found.
Wrote their own version · 0
None found.
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository33 other files it gives its agents.
- AGENTS.mdAGENTS.md
- tools/debugger/CLAUDE.mdCLAUDE.md
- tools/launcher/CLAUDE.mdCLAUDE.md
- accessing-mlflowSkill
- benchmark-model-kernelsSkill
- commonSkill
- compare-resultsSkill
- day0-releaseSkill
- debugSkill
- deploymentSkill
- evaluationSkill
- launching-evalsSkill
- modelopt-accessing-mlflowSkill
- modelopt-benchmark-model-kernelsSkill
- modelopt-compare-resultsSkill
- modelopt-day0-releaseSkill
- modelopt-debugSkill
- modelopt-deploymentSkill
- modelopt-evaluationSkill
- modelopt-launching-evalsSkill
- modelopt-monitorSkill
- modelopt-ptqSkill
- modelopt-qadSkill
- modelopt-release-cherry-pickSkill
- modelopt-run-agentxSkill
- modelopt-speculative-decodingSkill
- monitorSkill
- ptqSkill
- qadSkill
- quant-recipe-searchSkill
- release-cherry-pickSkill
- run-agentxSkill
- speculative-decodingSkill
The file
Show its rules and text
We split this skill into the rules it gives an agent
29 rules. About 0 are common, 18 are distinctive, and 11 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈18
Distinctive · estimate
This author's own choices. Worth a look.
11
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 17
Use this skill when quantization is an iterative recipe search, not a one-off PTQ run.
- Distinctive: line 23
Treat a direct request such as "find the best quantization recipe and generate a PTQ checkpoint for this model" as enough to start.
- Very specific: line 29
Use modelopt-ptq to produce and validate checkpoints.
- Very specific: line 30
Use modelopt-deployment to serve checkpoints and debug serving-specific flags.
- Very specific: line 31
Use modelopt-evaluation to create NEL configs and submit evals.
- Very specific: line 32
Use modelopt-launching-evals to run, resume, debug, and analyze NEL runs.
- Very specific: line 33
Use modelopt-monitor for active job tracking.
- Very specific: line 34
Use modelopt-accessing-mlflow for MLflow artifact lookup.
- Very specific: line 35
Use modelopt-compare-results for validated baseline-vs-candidate deltas and score-field comparability.
- Distinctive: line 37
Do not duplicate those workflows here.
- Distinctive: line 53
If any of these are missing, ask for them.
- Distinctive: line 56
Default success rule: maximize the chosen performance objective while keeping each benchmark within 1 percentage point of the matching …
- Distinctive: line 62
Keep the search space explicit.
- Very specific: line 81
Do not collapse the search to one dimension such as numeric format only.
- Distinctive: line 87
Read result tables, recipe logs, AutoQuant states, sensitivity reports, and experiment notes before proposing new work.
- Very specific: line 89
Ask modelopt-monitor, modelopt-launching-evals, or modelopt-compare-results to recover active job state and completed metrics when needed.
- Distinctive: line 99
Always include BF16/FP16 and a near-lossless FP8/W8A8 baseline unless FP8 itself is the target.
- Distinctive: line 103
Add an AutoQuant candidate in the requested primary family when AutoQuant is available.
- Distinctive: line 106
Add at least one manual or sensitivity-guided candidate so AutoQuant can be compared against controlled ablations and there is a fallback …
- Distinctive: line 109
When sensitivity or model behavior implicates boundary layers, add a controlled first-layer, last-layer, or combined BF16 exclusion …
- Distinctive: line 111
Do not preserve boundary layers without testing the trade-off.
- Distinctive: line 117
Use AutoQuant for broad candidate generation and sensitivity reports.
- Distinctive: line 129
Ensure positional exclusions preserve complete fused runtime groups, then evaluate the candidate against the same BF16 baseline and …
- Very specific: line 132
If the checkpoint is valid but serving fails due to runtime support, do not reject the recipe immediately.
- Distinctive: line 138
Run cheap screen evals for every candidate that passes the gates.
- Very specific: line 158
When the user has authorized QAD, a candidate is a recipe plus the QAD settings from the modelopt-qad skill's recipe loop, which ends each …
- Distinctive: line 163
After a pass, add one module group to the passing recipe, in order of savings on the success metric.
- Distinctive: line 170
After a clear miss, shrink the added group to a slice.
- Distinctive: line 174
Stop when every group has been added or dropped, or the QAD budget is spent.
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Quant Recipe Search
Use this skill when quantization is an iterative recipe search, not a one-off PTQ run. The skill owns strategy: define success, choose the search space, sequence candidates, and decide the next iteration. It delegates checkpoint generation, serving, evaluation, monitoring, and metric comparison to the existing execution skills.
Treat a direct request such as "find the best quantization recipe and generate a PTQ checkpoint for this model" as enough to start. Recover local state first, then ask only for missing decisions that change the search.
Skill Boundaries
- Use
modelopt-ptqto produce and validate checkpoints. - Use
modelopt-deploymentto serve checkpoints and debug serving-specific flags. - Use
modelopt-evaluationto create NEL configs and submit evals. - Use
modelopt-launching-evalsto run, resume, debug, and analyze NEL runs. - Use
modelopt-monitorfor active job tracking. - Use
modelopt-accessing-mlflowfor MLflow artifact lookup. - Use
modelopt-compare-resultsfor validated baseline-vs-candidate deltas and score-field comparability.
Do not duplicate those workflows here. This skill should leave the user with a clear recipe portfolio, success metric, experiment sequence, and next decision.
Problem
The task is to find the best recipe for a user-defined target, not merely to produce a quantized checkpoint. A generated PTQ checkpoint is only a candidate. It becomes a recommended recipe only after evaluation and comparison against the matching baseline.
Required inputs before planning candidates:
- Optimization goal: compute/throughput, memory/latency, or a custom metric.
- Primary quantization family: for example NVFP4, W4A16 NVFP4, FP8/W8A8, INT4/AWQ, or a custom mixed set.
- Benchmark set or baseline results: the user-defined acceptance surface.
If any of these are missing, ask for them. Do not silently default to FP8/W8A8 or call a checkpoint "best" before evaluation.
Default success rule: maximize the chosen performance objective while keeping each benchmark within 1 percentage point of the matching BF16/FP16 baseline. Near-threshold or noisy regressions require reruns before making a decision.
Search Space
Keep the search space explicit. A candidate recipe is a tuple across these axes:
- Numeric format: FP8/W8A8, NVFP4/W4A4, W4A16 NVFP4, INT4/AWQ, or mixed formats such as NVFP4+FP8.
- Calibration/search algorithm: max calibration, MSE calibration, GPTQ, AWQ, AutoQuant scoring, and calibration dataset or sample-count variants.
- Selection method: manual/heuristic rules, sensitivity-guided manual recipes, AutoQuant selection, or a hybrid of AutoQuant plus manual overrides.
- Module family: attention, MLP, MoE experts, routers/gates, embeddings,
lm_head, adapters, vision encoders, and model-specific modules. - Layer position: first/last transformer-layer counts or explicit ordinal ranges to keep in BF16. First 3-4 and last 1-2 layers are common starting candidate ranges, not defaults.
- Runtime fusion constraints: modules fused by the inference library must
use compatible quantization. Examples: vLLM Qwen
linear_attn.in_proj_qkvzand fused MoE expert projections such as gate/up (w1/w3). - Calibration budget: dataset mix, sample count, sequence length, and batch settings.
Do not collapse the search to one dimension such as numeric format only. Read
references/recipe_iteration.md when choosing concrete axes or candidates.
Design Workflow
- Recover state
- Read result tables, recipe logs, AutoQuant states, sensitivity reports, and experiment notes before proposing new work.
- Ask
modelopt-monitor,modelopt-launching-evals, ormodelopt-compare-resultsto recover active job state and completed metrics when needed.
- Define the target
- Confirm the optimization goal, primary quantization family, benchmark set, accuracy-loss threshold, calibration budget, and cost metric.
- Include quantization metadata such as scale storage in active-cost or size estimates.
- Pick baselines and first candidates
- Always include BF16/FP16 and a near-lossless FP8/W8A8 baseline unless FP8 itself is the target.
- For ModelOpt work, start from
modelopt_recipes: model-specific recipes first, then general PTQ presets or recipe fragments. - Add an AutoQuant candidate in the requested primary family when AutoQuant is available. Expect AutoQuant to find a better trade-off than a first manual recipe, but validate that assumption with the same evals.
- Add at least one manual or sensitivity-guided candidate so AutoQuant can be compared against controlled ablations and there is a fallback if AutoQuant misses the best frontier or hits runtime constraints.
- When sensitivity or model behavior implicates boundary layers, add a controlled first-layer, last-layer, or combined BF16 exclusion candidate. Do not preserve boundary layers without testing the trade-off.
- Generate candidates
- Delegate checkpoint generation and PTQ validation to
modelopt-ptq. - Change one major axis at a time: format, calibration algorithm, module family, layer position, granularity, or calibration data.
- Use AutoQuant for broad candidate generation and sensitivity reports; use manual recipes for controlled module-family ablations and overrides.
- Resolve positional ordinals against the model's transformer block sequence. For manual recipes, add those blocks to the recipe exclusions. For AutoQuant or hybrid candidates, pass positional exclusions into the selected implementation when supported; otherwise apply a manual override to its result and record the limitation.
- Delegate checkpoint generation and PTQ validation to
- Gate before scaling
- Validate checkpoint coverage and metadata.
- Reject or rewrite recipes that mix quantization algorithms inside a fused runtime group.
- Ensure positional exclusions preserve complete fused runtime groups, then evaluate the candidate against the same BF16 baseline and acceptance criteria as every other recipe.
- If the checkpoint is valid but serving fails due to runtime support, do not
reject the recipe immediately. Delegate to
modelopt-deployment/modelopt-debugfor small patches or flags, then rerun a pipe-clean check.
Iteration Loop
- Run cheap screen evals for every candidate that passes the gates.
- Compare accuracy, verbosity/token usage, and active cost against baselines.
- Rerun noisy or near-threshold results before labeling a regression.
- Decide the next candidate:
- Accuracy drop: protect or ablate sensitive module families, try MSE/GPTQ, use AutoQuant sensitivity to choose overrides, or test first/last-layer BF16 exclusions when evidence points to boundary sensitivity.
- Poor performance/cost: quantize the next high-cost active family, adjust active-cost objective, or try a more aggressive format.
- AutoQuant underperforms manual recipes: inspect sensitivity reports, achieved bits, excluded modules, and runtime-fusion constraints; keep the manual recipe in the portfolio instead of forcing the AutoQuant result.
- Runtime incompatibility: rewrite around fused groups or isolate deployment support from checkpoint quality.
- Repeated AutoQuant recipes: inspect achieved bits and recipe hashes, then adjust constraints before launching a larger sweep.
- Promote only when
modelopt-compare-resultsshows no failed external sanity check, the candidate is comparable to the validated measured baseline, and the user-defined goal is met. An externally unverified baseline is non-blocking.
When the user has authorized QAD, a candidate is a recipe plus the QAD settings
from the modelopt-qad skill's recipe loop, which ends each recipe with a pass or a
clear miss. A PTQ recipe that already meets the threshold counts as a pass.
Step the layout from that outcome:
- After a pass, add one module group to the passing recipe, in order of
savings on the success metric; the MTP head is its own group. A group can be
a slice: its middle layers with the first and last few kept, or some of its
projections (such as GDN
in_projonly, or attentionlinear_qkvonly). Find and evaluate the new layout's best PTQ recipe, starting from the passing recipe's calibration, before its QAD. Step one layout at a time; themodelopt-qadrecipe loop runs its knob variants in parallel. - After a clear miss, shrink the added group to a slice; if that misses too, drop the group and try the next one on the last passing recipe. If no recipe has passed yet, report no QAD pass and return to the accuracy-drop steps above.
- Stop when every group has been added or dropped, or the QAD budget is spent. Return the passing recipe that best meets the success metric, with its QAD checkpoint.
Maintain a recipe portfolio table with recipe name, objective, active-cost estimate, calibration notes, checkpoint path, eval/log references, accuracy, verbosity, positional exclusions, and decision.
References
- For recipe design, search-space details, sensitivity, and active-cost
accounting, read
references/recipe_iteration.md. - For a concrete prior case study, read
references/qwen36_case_study.mdonly when Qwen3.5/Qwen3.6 details are relevant.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

