modelopt-run-agentx
Skill5.6k starsChanged todayApache-2.0View on GitHub
Run the AgentX agentic serving benchmark with the SemiAnalysis harness. Use for AgentX setup, dataset downloads, concurrency sweeps, or serving latency and throughput comparisons.
- Installs packages
1 other project uses a version of this.
Adopted by
Steady: 0 new projects took it in the last 4 weeks, none the 4 weeks before.thin data
Most recently taken up by
- NVIDIA/Model-Optimizerword for word, first seen Nov 15, 2026
Who's talking about NVIDIA/TensorRT-Model-Optimizer
- NVIDIA/Model-Optimizer“Replace ONNX simplification package from onnxsim to onnxslim”GitHub · Oct 28, 2025 · 1 reactions
- NVIDIA/Model-Optimizer“Fix FLOPs calculation”GitHub · Sep 29, 2025 · 1 reactions
- NVIDIA/Model-Optimizer“[Autocast] Fix edge case casting input directly to output”GitHub · Sep 9, 2025 · 1 reactions
- NVIDIA/Model-Optimizer“[Autocast] Fix edge case with cast producing network output”GitHub · Sep 8, 2025 · 1 reactions
- NVIDIA/Model-Optimizer“"Unknown quantization method" when loading nvfp4 checkpoint with vLLM”GitHub · Aug 19, 2025 · 1 reactions
The biggest 5 of 45 mentions linking to the repository that our crawl found. Each line is their own headline.
See all 45
Talked about on GitHub (45 times).
On GitHub · 45
- Oct 28, 2025Replace ONNX simplification package from onnxsim to onnxslim — NVIDIA/Model-Optimizer1 reaction · 23 comments
- Sep 29, 2025Fix FLOPs calculation — NVIDIA/Model-Optimizer1 reaction · 11 comments
- Sep 9, 2025[Autocast] Fix edge case casting input directly to output — NVIDIA/Model-Optimizer1 reaction · 10 comments
- Sep 8, 2025[Autocast] Fix edge case with cast producing network output — NVIDIA/Model-Optimizer1 reaction · 6 comments
- Aug 19, 2025"Unknown quantization method" when loading nvfp4 checkpoint with vLLM — NVIDIA/Model-Optimizer1 reaction · 3 comments
40 more on GitHub
- Mar 30, 2025[Quantization] Add TRT-ModelOpt as a Backend — huggingface/diffusers1 reaction · 50 comments
- Mar 11, 2025✨[Feature] Supporting Blackwell FP4 — pytorch/TensorRT1 reaction
- Sep 16, 2026推論最適化ツール・OSS — warabinbin/papers0 reactions
- Aug 10, 2026fold_weight crashes with AttributeError: 'NoneType' object has no attribute 'data' on Megatron models with tied word embeddings — NVIDIA/Model-Optimizer0 reactions · 1 comment
- Dec 5, 2025Kimi-k2 calib+export — NVIDIA/Model-Optimizer0 reactions · 1 comment
- Dec 4, 2025Quantization — HashCitrine/sample-to-make-llm-service0 reactions
- Dec 4, 2025Diffusion Rename [TRT MO -> MO] — NVIDIA/Model-Optimizer0 reactions · 2 comments
- Dec 3, 2025[5704162] Create a copy to avoid leaking ProcessGroup into state dict — NVIDIA/Model-Optimizer0 reactions · 6 comments
- Dec 2, 2025fix: Fix auto quantize multi-gpu tests — NVIDIA/Model-Optimizer0 reactions · 3 comments
- Dec 2, 2025qwen3_8b eagle3 model train and vllm ckpt export — NVIDIA/Model-Optimizer0 reactions · 1 comment
- Nov 27, 2025Add all example e2e tests for github PR merge / nightly — NVIDIA/Model-Optimizer0 reactions · 1 comment
- Nov 26, 2025[Bug Fix]: Correct block_size in NVFP4_MLP_WEIGHT_ONLY_CFG to support vLLM inference — NVIDIA/Model-Optimizer0 reactions · 3 comments
- Nov 26, 2025fix: relax sympy version — inisis/OnnxSlim0 reactions · 1 comment
- Nov 24, 2025[None][doc] Deprecate --export_fmt flag when using ModelOpt — NVIDIA/TensorRT-LLM0 reactions · 4 comments
- Nov 24, 2025[Help wanted] CNN QAT: How to correctly export ONNX and use DeviceModel for inference — NVIDIA/Model-Optimizer0 reactions · 1 comment
- Nov 22, 2025fix: issues related to TensorRT-Model-Optimizer — inisis/OnnxSlim0 reactions · 2 comments
- Nov 20, 2025[2/N] Added KDLoss based AutoQuantize — NVIDIA/Model-Optimizer0 reactions · 1 comment
- Nov 20, 2025[1/N] Refactored AutoQuantizeSearcher to _AutoQuantizeBaseSearcher & AutoQuantizeGradientSearcher; seperated quant modules and score modules — NVIDIA/Model-Optimizer0 reactions · 11 comments
- Nov 17, 2025failed to quant the Qwen3-VL-30B-A3B — NVIDIA/Model-Optimizer0 reactions · 1 comment
- Nov 13, 2025Support Wan2.2 t2v diffusers quantization — NVIDIA/Model-Optimizer0 reactions · 3 comments
- Nov 3, 2025PyTorch geometric quantization support — NVIDIA/Model-Optimizer0 reactions · 1 comment
- Nov 3, 2025Failed to save quantized model if some tensors offload to CPU — NVIDIA/Model-Optimizer0 reactions · 2 comments
- Oct 28, 2025[OMNIML-2917] export layer config using actual prefix instead of hard… — NVIDIA/Model-Optimizer0 reactions · 9 comments
- Oct 27, 2025Add convert_llama3_config_to_decilm_config + unit test — NVIDIA/Model-Optimizer0 reactions · 2 comments
- Sep 25, 2025CodeRabbit Generated Unit Tests: Add GPU tests for Megatron quantization; review/refactor unit tests — NVIDIA/Model-Optimizer0 reactions · 2 comments
- Sep 25, 2025📝 Add docstrings to `kaix/kvcache_mcore` — NVIDIA/Model-Optimizer0 reactions · 2 comments
- Sep 15, 2025FP8 Block quantize onnx export support — NVIDIA/Model-Optimizer0 reactions · 3 comments
- Sep 15, 2025add end_process in deepseek ptq — NVIDIA/Model-Optimizer0 reactions · 10 comments
- Jul 13, 2025feat: Simplify and Improve Whisper Example — NVIDIA/TensorRT-LLM0 reactions · 3 comments
- Jul 3, 2025[future request] Support more quant for torch backend — NVIDIA/TensorRT-LLM0 reactions · 2 comments
- Jul 3, 2025🐛 [Bug] examples/dynamo/torch_compile_gpt2.py run failed — pytorch/TensorRT0 reactions · 2 comments
- Jun 15, 2025INT8 to FP8 scale conversion — NVIDIA/Model-Optimizer0 reactions · 6 comments
- Jun 13, 2025Fix speculative decoding example — NVIDIA/Model-Optimizer0 reactions · 6 comments
- May 21, 2025`get_modelike_from_algo_cfg` doesn't accept `QuantizeAlgorithmConfig`, but it's typing says it should — NVIDIA/Model-Optimizer0 reactions · 3 comments
- May 17, 2025Decode latency of FP4 LLM is higher than expected — NVIDIA/TensorRT-LLM0 reactions · 1 comment
- May 7, 2025Error in torch_compile.ipynb — Project-MONAI/tutorials0 reactions · 1 comment
- May 6, 2025当我使用增量训练的时候,只改变修改一个feature的zch_size的大小,其他所有配置都不改变,会出错。 — alibaba/TorchEasyRec0 reactions
- Apr 30, 2025INT8 quantization for HiFi-GAN vocoder -- performance issue — NVIDIA/TensorRT0 reactions · 7 comments
- Mar 24, 2025bert model split into many layers after int8 quantization — NVIDIA/TensorRT0 reactions · 13 comments
- Sep 8, 2024urlopen error retrieval incomplete in modules/autoencoder_mednist.ipynb — Project-MONAI/tutorials0 reactions
Other versions1 project uses a version of this.
Word for word · 1
- NVIDIA/Model-Optimizer★ 5.6k100% same
Edited · 0
None found.
Wrote their own version · 0
None found.
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository33 other files it gives its agents.
- AGENTS.mdAGENTS.md
- tools/debugger/CLAUDE.mdCLAUDE.md
- tools/launcher/CLAUDE.mdCLAUDE.md
- accessing-mlflowSkill
- benchmark-model-kernelsSkill
- commonSkill
- compare-resultsSkill
- day0-releaseSkill
- debugSkill
- deploymentSkill
- evaluationSkill
- launching-evalsSkill
- modelopt-accessing-mlflowSkill
- modelopt-benchmark-model-kernelsSkill
- modelopt-compare-resultsSkill
- modelopt-day0-releaseSkill
- modelopt-debugSkill
- modelopt-deploymentSkill
- modelopt-evaluationSkill
- modelopt-launching-evalsSkill
- modelopt-monitorSkill
- modelopt-ptqSkill
- modelopt-qadSkill
- modelopt-quant-recipe-searchSkill
- modelopt-release-cherry-pickSkill
- modelopt-speculative-decodingSkill
- monitorSkill
- ptqSkill
- qadSkill
- quant-recipe-searchSkill
- release-cherry-pickSkill
- run-agentxSkill
- speculative-decodingSkill
The file
We split this skill into the rules it gives an agent
6 rules. About 0 are common, 3 are distinctive, and 3 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈3
Distinctive · estimate
This author's own choices. Worth a look.
3
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 18
Start each benchmark campaign with the latest harness and compatible serving-stack releases unless the user requests specific versions.
- Distinctive: line 22
Keep them fixed across sweep points, cache formats, and repeats unless they are the variable under test.
- Distinctive: line 41
Check the current tutorial for supported CLI options and scenario defaults.
- Very specific: line 45
Before benchmarking, verify that the loader passes that SHA as revision to Hugging Face load_dataset.
- Very specific: line 49
Prefer the deployed checkpoint's local tokenizer.
- Distinctive: line 91
Use the benchmarking guide for sweep isolation, performance metrics, and comparison controls.
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Run AgentX
Setup
Resolve the endpoint URL, served model name, matching tokenizer, and requested
concurrency from the task. Reuse the model checkpoint. If serving is needed, follow
deployment to download missing weights and launch it.
Enable prefix caching, streaming usage, and cached-token reporting. For vLLM, use
--enable-prefix-caching --enable-prompt-tokens-details. Verify the context limit
fits the traces with the model's tokenizer; never silently truncate requests.
Start each benchmark campaign with the latest harness and compatible serving-stack releases unless the user requests specific versions. Record and pin the resolved harness and other code revisions, dataset/model/tokenizer revisions, serving framework (e.g. vLLM), and client/server dependencies or container digests in the run artifacts. Keep them fixed across sweep points, cache formats, and repeats unless they are the variable under test. Record local patches; treat upgrades as a new comparison baseline.
Install the latest AgentX harness
in a separate client environment. Set AGENTX_PYTHON to a Python executable
satisfying the latest release's Requires-Python metadata on
PyPI.
Installing upstream aiperf alone is insufficient.
set -euo pipefail
export AGENTX_WORKDIR="${AGENTX_WORKDIR:-$PWD/agentx-work}"
"${AGENTX_PYTHON:?}" -m venv "$AGENTX_WORKDIR/venv"
source "$AGENTX_WORKDIR/venv/bin/activate"
python -m pip install --upgrade agentx-harnessSet AGENTX_DATASET to an accepted with-subagents alias from the
AgentX tutorial.
Check the current tutorial for supported CLI options and scenario defaults.
The public loader downloads and caches the traces and prompt reconstruction assets.
An alias, even a dated one, does not select an immutable Hugging Face revision.
Resolve the alias's repository to a commit SHA and record both with every run.
Before benchmarking, verify that the loader passes that SHA as revision to
Hugging Face load_dataset. Without a revision flag, set the selected Weka loader's
hf_revision to that SHA in the campaign's client installation.
Prefer the deployed checkpoint's local tokenizer. If it requires custom code, add
--tokenizer-trust-remote-code only after the user explicitly trusts its repository.
Choose concurrency
Sweep concurrency upward until KV-cache usage nears capacity, then refine around that point. Resweep when the model, hardware, or cache budget changes. Publish the full sweep and repeat promising points before claiming an advantage.
Run
In each shell, set AGENTX_WORKDIR to the setup directory and set AGENTX_DATASET,
AGENTX_URL, AGENTX_MODEL, AGENTX_TOKENIZER, AGENTX_MAX_CONTEXT_LENGTH,
and AGENTX_CONCURRENCY from the deployment and sweep. Use the server's base URL
and actual context limit. Fix AGENTX_SEED across comparisons.
First check endpoint health and repeat a long prompt to verify nonzero cached-token
usage. Keep the scenario's default trajectory window and duration. Run each
concurrency separately with a fresh artifact directory:
set -euo pipefail
source "${AGENTX_WORKDIR:?}/venv/bin/activate"
export HF_HOME="${HF_HOME:-$AGENTX_WORKDIR/hf-cache}"
export AIPERF_DATASET_MMAP_CACHE_DIR="$AGENTX_WORKDIR/dataset-cache"
mkdir -p "$AGENTX_WORKDIR/results"
AGENTX_RUN_DIR=$(mktemp -d "$AGENTX_WORKDIR/results/run-XXXXXX")
python -m pip freeze > "$AGENTX_RUN_DIR/client-packages.txt"
aiperf profile \
--scenario inferencex-agentx-mvp \
--url "${AGENTX_URL:?}" --model "${AGENTX_MODEL:?}" \
--tokenizer "${AGENTX_TOKENIZER:?}" --endpoint-type chat \
--public-dataset "${AGENTX_DATASET:?}" \
--max-context-length "${AGENTX_MAX_CONTEXT_LENGTH:?}" \
--concurrency "${AGENTX_CONCURRENCY:?}" \
--random-seed "${AGENTX_SEED:?}" \
--streaming --use-server-token-count --extra-inputs ignore_eos:true \
--artifact-dir "$AGENTX_RUN_DIR" --ui simple \
2>&1 | tee "$AGENTX_RUN_DIR/client.log"Compare and report
Use the benchmarking guide for sweep isolation, performance metrics, and comparison controls. For AgentX:
- Concurrency counts session trees. Report actual overlapping HTTP requests separately because child sessions can overlap.
- Hold KV-cache bytes fixed when comparing KV formats. Keep the dataset, seed, context filter, and warmup fixed, and start a fresh server per point.
- Report token-weighted cache hits from server-reported usage, cache usage, and preemptions. Missing usage means cache hits are unknown.
- Report
metadata.submission_validand anymetadata.submission_invalid_reasonsfrom the result export. Flag invalid or missing validity instead of treating the run as a valid comparison. - Include request errors, unfinished requests, and warmup failures. Synthetic prompts measure serving performance, not model quality; shortened smoke runs are not benchmark results.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

