trtllm-visualgen-bench
Skill15k starsChanged 3 months agoLicence not identifiedView on GitHub
Benchmark a TensorRT-LLM VisualGen model end to end, server included — pick the workload, bring up the `trtllm-serve` server it needs (resolve the checkpoint, write the config, launch, verify it came up as configured) or use one already running, then run and read the result. Use when the user asks to benchmark or serve a VisualGen model, or asks for a model's request YAML or its default generation params.
- Installs packages
Trending
More talk lately.about the whole repository
Dec 2025Jun 2026Nov 2026
Who's talking about NVIDIA/TensorRT-LLM
- WorldofAI“NVIDIA's TensorRT-LLM: Building Powerful RAG Apps! (Opensource)”YouTube · Mar 14, 2024 · 6.1k views
- NVIDIA Developer“TensorRT LLM 1.0 Livestream: New Easy-To-Use Pythonic Runtime”YouTube · Sep 26, 2025 · 4.7k views
- Fahd Mirza“TensorRT LLM Introduction”YouTube · Nov 2, 2023 · 2.9k views
- AAIF Live“How We Cut LLM Latency 70% With TensorRT in Production”YouTube · Apr 20, 2026 · 537 views
- GitHub Daily Trend AI Podcast“GitHub - NVIDIA/TensorRT-LLM: TensorRT-LLM provides users with an easy-to-use Python API to defin...”YouTube · Aug 20, 2024 · 333 views
The biggest 5 of 73 mentions linking to the repository that our crawl found. Each line is their own headline.
See all 73
Talked about on GitHub, YouTube, Bluesky, Hacker News and Stack Overflow (73 times).
On GitHub · 47
- Dec 19, 2023Custom all reduce kernels — vllm-project/vllm24 reactions · 33 comments
- Oct 20, 2023TensorRT LLM backend. — oobabooga/textgen22 reactions · 10 comments
- Nov 21, 2025[sgl-kernel][Feat][B200][1/N]Support MXFP8 Grouped GEMM in Blackwell — sgl-project/sglang11 reactions · 7 comments
- Mar 17, 2024Add TensorRT-LLM support — oobabooga/textgen10 reactions · 7 comments
- Jun 28, 2025[b200] support trt-llm allreduce fuse rms_norm_add kernel — sgl-project/sglang9 reactions · 2 comments
42 more on GitHub
- Mar 19, 2025[Feature] Support Cohere Command-A (Cohere2ForCausalLM arch) — sgl-project/sglang8 reactions · 6 comments
- Oct 21, 2023TensorRT-LLM backend support? — lm-sys/FastChat7 reactions · 5 comments
- May 18, 2026DeepSeek V4 kernel support status tracker — flashinfer-ai/flashinfer6 reactions · 9 comments
- Jan 28, 2026[Feature] Update Transformers to v5 — verl-project/verl6 reactions · 2 comments
- Jun 18, 2024Multi-Lora support — kserve/kserve6 reactions · 9 comments
- Apr 4, 2026[Feature] Distributed Weight Data Parallelism (DWDP) for Sparse MoE Models — sgl-project/sglang5 reactions · 12 comments
- Feb 28, 2026[Feature]: Eliminate redundant multimodal preprocessing for gRPC backends — smg-project/smg5 reactions · 2 comments
- May 15, 2025feat: move AR fusion kernels from trtllm — flashinfer-ai/flashinfer5 reactions · 1 comment
- Feb 14, 2024modelInstanceState: [json.exception.out_of_range.403] key 'builder_config' not found — triton-inference-server/tensorrtllm_backend5 reactions · 12 comments
- Dec 6, 2023TensorRT version is not compatible, expecting library version 9.2.0.4 got 9.2.0.5 — triton-inference-server/tensorrtllm_backend5 reactions · 14 comments
- Dec 6, 2023Serialization (Serialization assertion stdVersionRead == kSERIALIZATION_VERSION failed.Version tag does not match. Note: Current Version: 226, Serialized Engine Version: 228) — triton-inference-server/tensorrtllm_backend5 reactions · 17 comments
- Nov 14, 2025Add optimal gb200 moe comm kernels — flashinfer-ai/flashinfer4 reactions · 2 comments
- Apr 3, 2025[WIP] adapt trtllm fp blockscale kernel — sgl-project/sglang4 reactions · 2 comments
- Mar 13, 2025feat: spark tts use TensorRT-LLM + tritonserver deploy on the modal — ai-bot-pro/achatbot4 reactions · 5 comments
- Jan 23, 2024[Feature Request] medusa support — triton-inference-server/tensorrtllm_backend4 reactions · 1 comment
- Jul 17, 2025[Fix][Ready]Fix register spilling in cutlass nvfp4 gemm kernel on Blackwell — sgl-project/sglang3 reactions · 1 comment
- May 20, 2024Can you provide an example of a visual language model or multimodal model launch by triton server? — triton-inference-server/tensorrtllm_backend3 reactions · 8 comments
- Feb 24, 2024FYI: Solution to AttributeError: module 'mpmath' has no attribute 'rational' — YanWenKun/ComfyUI-Docker3 reactions · 2 comments
- Feb 2, 2024大佬有没有对比和VLLM的推理效果？ — Tlntin/Qwen-TensorRT-LLM3 reactions · 2 comments
- Oct 26, 2023[Feature] Add support for TensorRT-LLM inference engine — open-compass/opencompass3 reactions
- Jun 22, 2026[Kernel] GLM5 Router GEMM — vllm-project/vllm2 reactions
- Apr 6, 2026Second part of refactoring the routing part — flashinfer-ai/flashinfer2 reactions · 15 comments
- Mar 31, 2026[Core]: TRT-LLM Integration (co-author: @feixiangpeng) — LMCache/LMCache2 reactions
- Mar 27, 2026Add TensorRT-LLM KV connector integration — LMCache/LMCache2 reactions · 4 comments
- Mar 17, 2026[rollout] feat: enable Async RL for trtllm rollout — verl-project/verl2 reactions · 1 comment
- Mar 14, 2026[Kernel] Porting the TRTLLM minimax_allreduce_rms kernels — vllm-project/vllm2 reactions · 7 comments
- Jan 21, 2026[triton_kernels][Feature Request] Publish triton_kernels to PyPI or package it alongside Triton — triton-lang/triton2 reactions · 2 comments
- Jul 24, 2025Optimizations for TRTLLM MNNVL Allreduce — flashinfer-ai/flashinfer2 reactions · 2 comments
- Jan 5, 2025[Feature] adapt fused sigmoid gate for MoE model — sgl-project/sglang2 reactions · 20 comments
- Oct 14, 2024[RFC] LowBit Fused Attention — pytorch/ao2 reactions · 3 comments
- Mar 8, 2024Infer failed: Unable to parse 'data': Shape does not match true shape of 'data' field in generate endpoint — triton-inference-server/tensorrtllm_backend2 reactions · 2 comments
- Jan 4, 2024Pip Installation — huggingface/optimum-nvidia2 reactions · 4 comments
- Dec 12, 2023Feature Request: Whisper Tensorrt-llm backend support — m-bain/whisperX2 reactions · 2 comments
- Oct 20, 2023[0.5.0][Bug] Release build failure — triton-inference-server/tensorrtllm_backend2 reactions · 3 comments
- Nov 4, 2026test(sidecar): cover clean TensorRT-LLM liveness shutdown — ai-dynamo/dynamo1 reaction · 4 comments
- Sep 30, 2026fix(trtllm): preserve request semantics and rank-zero routing in sidecar — ai-dynamo/dynamo1 reaction · 7 comments
- Sep 1, 2026[Docs] Add Qwen2.5-VL guide for the TRT-LLM PyTorch backend, deprecate Llava1.5 guide — triton-inference-server/tutorials1 reaction · 1 comment
- Aug 10, 2026fix(trtllm): prompt_logprobs off-by-one silently corrupts every prompt score — ai-dynamo/dynamo1 reaction · 6 comments
- Jul 27, 2026fix(trtllm): make disagg node id modulus configurable — ai-dynamo/dynamo1 reaction · 5 comments
- Jul 15, 2026feat(kvbm): add on_rewind to trim slot state after specdec rewind — ai-dynamo/dynamo1 reaction · 15 comments
- Apr 18, 2026Port TRT-LLM fused qk norm rope — flashinfer-ai/flashinfer1 reaction · 6 comments
- Mar 29, 2026chore: guard regression with trtllm_bf16_routed_moe tests — flashinfer-ai/flashinfer1 reaction · 4 comments
On YouTube · 18
- Mar 14, 2024NVIDIA's TensorRT-LLM: Building Powerful RAG Apps! (Opensource) — WorldofAI6.1k views · 6 comments
- Sep 26, 2025TensorRT LLM 1.0 Livestream: New Easy-To-Use Pythonic Runtime — NVIDIA Developer4.7k views · 2 comments
- Nov 2, 2023TensorRT LLM Introduction — Fahd Mirza2.9k views · 14 comments
- Apr 20, 2026How We Cut LLM Latency 70% With TensorRT in Production — AAIF Livesays it tested it537 views · 1 comment
- Aug 20, 2024GitHub - NVIDIA/TensorRT-LLM: TensorRT-LLM provides users with an easy-to-use Python API to defin... — GitHub Daily Trend AI Podcast333 views
13 more on YouTube
- Oct 11, 2026Stop Wasting GPU Compute: NVIDIA TensorRT-LLM — RepoManiac324 views
- Oct 22, 2023NVIDIA/TensorRT-LLM - Gource visualisation — Gourcer313 views
- Jul 1, 2026NVIDIA TensorRT-LLM GitHub Tutorial: Continuous Batching, KV Cache, and GPU Optimization — Alex Hitt277 views
- Sep 2, 2026TensorRT just made the same GPU 2.7x faster — Readheights AIsays it tested it232 views
- Apr 17, 2026How We Cut LLM Latency By 70% With NVIDIA TensorRT-LLM. MLOps Community - Maher Hanafi, SVP of Eng — Maher Hanafisays it tested it225 views · 1 comment
- Sep 3, 2026TensorRT & TensorRT-LLM Explained — The Complete Guide | From Model to Production in 12 Minutes — ErrorFixer214 views
- May 12, 2024NVIDIA AI Revolutionizes Inference: TensorRT Model Optimizer for GPU Efficiency — SourceCode AI192 views · 1 comment
- Feb 15, 2025🚀 NVIDIA’s New KV Cache Optimizations in TensorRT-LLM – AI Just Got Smarter! 🚀 — AINewsMediaNetwork113 views
- Sep 6, 2026vLLM vs TensorRT LLM: Benchmarking TTFT, Throughput & GPU Memory — Cloud Explainerssays it tested it110 views
- Apr 19, 2026Supercharge Your AI Models with TensorRT-LLM — Github Signals88 views
- Apr 20, 2026How to Stop Wasting $60K on Idle GPUs — 1 Line Fix — Neuralscale Engineering83 views
- Apr 28, 2026How We Cut LLM GPU Costs from $60K to $6K — Inference Optimization Guide — Neuralscale Engineering47 views
- Sep 10, 2026Random Deletion Made This AI 43% Faster — CloudAICodesays it tested it28 views
On Bluesky · 4
- Jan 28, 2026I'm looking at Nvidia's TensorRT-LLM. With this I can re-compile models for my gfx card. According to the magic leprechaun that lives in my… — @gostack.bsky.social1 likes and repost
- Mar 25, 2025📦 NVIDIA / TensorRT-LLM ⭐ 9,809 (+17) 🗒 C++ TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models… — @github-trending.bsky.social1 likes and repost
- Feb 17, 2024📦 NVIDIA / TensorRT-LLM ⭐ 5,005 (+55) 🗒 C++ TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models… — @github-trending.bsky.social1 likes and repost
- Oct 21, 2023🔥 Hot Repo！ 🔥 (100+ new stars) 📦 NVIDIA / TensorRT-LLM ⭐ 1,042 (+167) 🗒 C++ TensorRT-LLM provides users with an easy-to-use Python API… — @github-trending.bsky.social0 likes and reposts · 1 reply
On Hacker News · 3
- Mar 11, 2025TensorRT-LLM runtime now open-source4 points · 1 comment
- Apr 19, 2026TensorRT LLM2 points
- Oct 20, 2023Nvidia Releases TensorRT-LLM2 points · 1 comment
On Stack Overflow · 1
- Dec 31, 2024Why ucx/libucx can not be found by cmake?-1 votes · 2 answers
In this repository51 other files it gives its agents.
- agent-flow/AGENTS.mdAGENTS.md
- AGENTS.mdAGENTS.md
- .codex/AGENTS.mdAGENTS.md
- cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/AGENTS.mdAGENTS.md
- jenkins/AGENTS.mdAGENTS.md
- tensorrt_llm/runtime/kv_cache_manager_v2/AGENTS.mdAGENTS.md
- tests/AGENTS.mdAGENTS.md
- tests/integration/defs/visual_gen/AGENTS.mdAGENTS.md
- tests/unittest/_torch/visual_gen/AGENTS.mdAGENTS.md
- agent-flow/CLAUDE.mdCLAUDE.md
- CLAUDE.mdCLAUDE.md
- tensorrt_llm/runtime/kv_cache_manager_v2/CLAUDE.mdCLAUDE.md
- tests/CLAUDE.mdCLAUDE.md
- tests/integration/defs/visual_gen/CLAUDE.mdCLAUDE.md
- tests/unittest/_torch/visual_gen/CLAUDE.mdCLAUDE.md
- trtllm-modeling-v2-entrySkill
- trtllm-modeling-v2-featuresSkill
- perf-analyzeSkill
- perf-optimizeSkill
- exec-env-checkSkill
- exec-local-compileSkill
- exec-local-dockerSkill
- exec-local-slurmSkill
- exec-remote-slurmSkill
- exec-slurm-compileSkill
- kernel-cute-writingSkill
- kernel-tileir-optimizationSkill
- kernel-triton-writingSkill
- perf-analysisSkill
- perf-host-analysisSkill
- perf-host-optimizationSkill
- perf-instability-cookbookSkill
- perf-nsight-compute-analysisSkill
- perf-nsight-systemsSkill
- perf-optimization-casebookSkill
- perf-optimizationSkill
- perf-regression-cookbookSkill
- perf-torch-cuda-graphsSkill
- perf-torch-sync-freeSkill
- perf-workload-profilingSkill
- trtllm-case-executorSkill
- trtllm-codebase-explorationSkill
- trtllm-code-contributionSkill
- trtllm-flashinfer-upgradeSkill
- trtllm-model-onboard-multimodalSkill
- trtllm-moe-developSkill
- trtllm-serve-config-guideSkill
- trtllm-test-script-builderSkill
- trtllm-test-specialistSkill
- trtllm-visualgen-lpips-golden-refreshSkill
- visual-gen-component-testSkill
The file
We split this skill into the rules it gives an agent
1 rule. About 0 are common, 0 are distinctive, and 1 is very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈0
Distinctive · estimate
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.
The licence could not be identified — read it at the source. We count its rules but don't quote them. Read them on GitHub.
≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.