NVIDIAKnown publisherNVIDIA/TensorRT-LLM/tests/CLAUDE.md

TensorRT-LLM / tests

CLAUDE.md15k starsChanged 3 months agoLicence not identifiedView on GitHub

1 other project uses a version of this.

Dec 19, 2023Custom all reduce kernels — vllm-project/vllm24 reactions · 33 comments
Oct 20, 2023TensorRT LLM backend. — oobabooga/textgen22 reactions · 10 comments
Nov 21, 2025[sgl-kernel][Feat][B200][1/N]Support MXFP8 Grouped GEMM in Blackwell — sgl-project/sglang11 reactions · 7 comments
Mar 17, 2024Add TensorRT-LLM support — oobabooga/textgen10 reactions · 7 comments
Jun 28, 2025[b200] support trt-llm allreduce fuse rms_norm_add kernel — sgl-project/sglang9 reactions · 2 comments

42 more on GitHub

Mar 19, 2025[Feature] Support Cohere Command-A (Cohere2ForCausalLM arch) — sgl-project/sglang8 reactions · 6 comments
Oct 21, 2023TensorRT-LLM backend support? — lm-sys/FastChat7 reactions · 5 comments
May 18, 2026DeepSeek V4 kernel support status tracker — flashinfer-ai/flashinfer6 reactions · 9 comments
Jan 28, 2026[Feature] Update Transformers to v5 — verl-project/verl6 reactions · 2 comments
Jun 18, 2024Multi-Lora support — kserve/kserve6 reactions · 9 comments
Apr 4, 2026[Feature] Distributed Weight Data Parallelism (DWDP) for Sparse MoE Models — sgl-project/sglang5 reactions · 12 comments
Feb 28, 2026[Feature]: Eliminate redundant multimodal preprocessing for gRPC backends — smg-project/smg5 reactions · 2 comments
May 15, 2025feat: move AR fusion kernels from trtllm — flashinfer-ai/flashinfer5 reactions · 1 comment
Feb 14, 2024modelInstanceState: [json.exception.out_of_range.403] key 'builder_config' not found — triton-inference-server/tensorrtllm_backend5 reactions · 12 comments
Dec 6, 2023TensorRT version is not compatible, expecting library version 9.2.0.4 got 9.2.0.5 — triton-inference-server/tensorrtllm_backend5 reactions · 14 comments
Dec 6, 2023Serialization (Serialization assertion stdVersionRead == kSERIALIZATION_VERSION failed.Version tag does not match. Note: Current Version: 226, Serialized Engine Version: 228) — triton-inference-server/tensorrtllm_backend5 reactions · 17 comments
Nov 14, 2025Add optimal gb200 moe comm kernels — flashinfer-ai/flashinfer4 reactions · 2 comments
Apr 3, 2025[WIP] adapt trtllm fp blockscale kernel — sgl-project/sglang4 reactions · 2 comments
Mar 13, 2025feat: spark tts use TensorRT-LLM + tritonserver deploy on the modal — ai-bot-pro/achatbot4 reactions · 5 comments
Jan 23, 2024[Feature Request] medusa support — triton-inference-server/tensorrtllm_backend4 reactions · 1 comment
Jul 17, 2025[Fix][Ready]Fix register spilling in cutlass nvfp4 gemm kernel on Blackwell — sgl-project/sglang3 reactions · 1 comment
May 20, 2024Can you provide an example of a visual language model or multimodal model launch by triton server? — triton-inference-server/tensorrtllm_backend3 reactions · 8 comments
Feb 24, 2024FYI: Solution to AttributeError: module 'mpmath' has no attribute 'rational' — YanWenKun/ComfyUI-Docker3 reactions · 2 comments
Feb 2, 2024大佬有没有对比和VLLM的推理效果？ — Tlntin/Qwen-TensorRT-LLM3 reactions · 2 comments
Oct 26, 2023[Feature] Add support for TensorRT-LLM inference engine — open-compass/opencompass3 reactions
Jun 22, 2026[Kernel] GLM5 Router GEMM — vllm-project/vllm2 reactions
Apr 6, 2026Second part of refactoring the routing part — flashinfer-ai/flashinfer2 reactions · 15 comments
Mar 31, 2026[Core]: TRT-LLM Integration (co-author: @feixiangpeng) — LMCache/LMCache2 reactions
Mar 27, 2026Add TensorRT-LLM KV connector integration — LMCache/LMCache2 reactions · 4 comments
Mar 17, 2026[rollout] feat: enable Async RL for trtllm rollout — verl-project/verl2 reactions · 1 comment
Mar 14, 2026[Kernel] Porting the TRTLLM minimax_allreduce_rms kernels — vllm-project/vllm2 reactions · 7 comments
Jan 21, 2026[triton_kernels][Feature Request] Publish triton_kernels to PyPI or package it alongside Triton — triton-lang/triton2 reactions · 2 comments
Jul 24, 2025Optimizations for TRTLLM MNNVL Allreduce — flashinfer-ai/flashinfer2 reactions · 2 comments
Jan 5, 2025[Feature] adapt fused sigmoid gate for MoE model — sgl-project/sglang2 reactions · 20 comments
Oct 14, 2024[RFC] LowBit Fused Attention — pytorch/ao2 reactions · 3 comments
Mar 8, 2024Infer failed: Unable to parse 'data': Shape does not match true shape of 'data' field in generate endpoint — triton-inference-server/tensorrtllm_backend2 reactions · 2 comments
Jan 4, 2024Pip Installation — huggingface/optimum-nvidia2 reactions · 4 comments
Dec 12, 2023Feature Request: Whisper Tensorrt-llm backend support — m-bain/whisperX2 reactions · 2 comments
Oct 20, 2023[0.5.0][Bug] Release build failure — triton-inference-server/tensorrtllm_backend2 reactions · 3 comments
Nov 4, 2026test(sidecar): cover clean TensorRT-LLM liveness shutdown — ai-dynamo/dynamo1 reaction · 4 comments
Sep 30, 2026fix(trtllm): preserve request semantics and rank-zero routing in sidecar — ai-dynamo/dynamo1 reaction · 7 comments
Sep 1, 2026[Docs] Add Qwen2.5-VL guide for the TRT-LLM PyTorch backend, deprecate Llava1.5 guide — triton-inference-server/tutorials1 reaction · 1 comment
Aug 10, 2026fix(trtllm): prompt_logprobs off-by-one silently corrupts every prompt score — ai-dynamo/dynamo1 reaction · 6 comments
Jul 27, 2026fix(trtllm): make disagg node id modulus configurable — ai-dynamo/dynamo1 reaction · 5 comments
Jul 15, 2026feat(kvbm): add on_rewind to trim slot state after specdec rewind — ai-dynamo/dynamo1 reaction · 15 comments
Apr 18, 2026Port TRT-LLM fused qk norm rope — flashinfer-ai/flashinfer1 reaction · 6 comments
Mar 29, 2026chore: guard regression with trtllm_bf16_routed_moe tests — flashinfer-ai/flashinfer1 reaction · 4 comments

On YouTube · 18

Mar 14, 2024NVIDIA's TensorRT-LLM: Building Powerful RAG Apps! (Opensource) — WorldofAI6.1k views · 6 comments
Sep 26, 2025TensorRT LLM 1.0 Livestream: New Easy-To-Use Pythonic Runtime — NVIDIA Developer4.7k views · 2 comments
Nov 2, 2023TensorRT LLM Introduction — Fahd Mirza2.9k views · 14 comments
Apr 20, 2026How We Cut LLM Latency 70% With TensorRT in Production — AAIF Livesays it tested it537 views · 1 comment
Aug 20, 2024GitHub - NVIDIA/TensorRT-LLM: TensorRT-LLM provides users with an easy-to-use Python API to defin... — GitHub Daily Trend AI Podcast333 views

13 more on YouTube

Oct 11, 2026Stop Wasting GPU Compute: NVIDIA TensorRT-LLM — RepoManiac324 views
Oct 22, 2023NVIDIA/TensorRT-LLM - Gource visualisation — Gourcer313 views
Jul 1, 2026NVIDIA TensorRT-LLM GitHub Tutorial: Continuous Batching, KV Cache, and GPU Optimization — Alex Hitt277 views
Sep 2, 2026TensorRT just made the same GPU 2.7x faster — Readheights AIsays it tested it232 views
Apr 17, 2026How We Cut LLM Latency By 70% With NVIDIA TensorRT-LLM. MLOps Community - Maher Hanafi, SVP of Eng — Maher Hanafisays it tested it225 views · 1 comment
Sep 3, 2026TensorRT & TensorRT-LLM Explained — The Complete Guide | From Model to Production in 12 Minutes — ErrorFixer214 views
May 12, 2024NVIDIA AI Revolutionizes Inference: TensorRT Model Optimizer for GPU Efficiency — SourceCode AI192 views · 1 comment
Feb 15, 2025🚀 NVIDIA’s New KV Cache Optimizations in TensorRT-LLM – AI Just Got Smarter! 🚀 — AINewsMediaNetwork113 views
Sep 6, 2026vLLM vs TensorRT LLM: Benchmarking TTFT, Throughput & GPU Memory — Cloud Explainerssays it tested it110 views
Apr 19, 2026Supercharge Your AI Models with TensorRT-LLM — Github Signals88 views
Apr 20, 2026How to Stop Wasting $60K on Idle GPUs — 1 Line Fix — Neuralscale Engineering83 views
Apr 28, 2026How We Cut LLM GPU Costs from $60K to $6K — Inference Optimization Guide — Neuralscale Engineering47 views
Sep 10, 2026Random Deletion Made This AI 43% Faster — CloudAICodesays it tested it28 views

On Bluesky · 4

Jan 28, 2026I'm looking at Nvidia's TensorRT-LLM. With this I can re-compile models for my gfx card. According to the magic leprechaun that lives in my… — @gostack.bsky.social1 likes and repost
Mar 25, 2025📦 NVIDIA / TensorRT-LLM ⭐ 9,809 (+17) 🗒 C++ TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models… — @github-trending.bsky.social1 likes and repost
Feb 17, 2024📦 NVIDIA / TensorRT-LLM ⭐ 5,005 (+55) 🗒 C++ TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models… — @github-trending.bsky.social1 likes and repost
Oct 21, 2023🔥 Hot Repo！ 🔥 (100+ new stars) 📦 NVIDIA / TensorRT-LLM ⭐ 1,042 (+167) 🗒 C++ TensorRT-LLM provides users with an easy-to-use Python API… — @github-trending.bsky.social0 likes and reposts · 1 reply

On Hacker News · 3

Mar 11, 2025TensorRT-LLM runtime now open-source4 points · 1 comment
Apr 19, 2026TensorRT LLM2 points
Oct 20, 2023Nvidia Releases TensorRT-LLM2 points · 1 comment

On Stack Overflow · 1

Dec 31, 2024Why ucx/libucx can not be found by cmake?-1 votes · 2 answers

Other versions1 project uses a version of this.
Word for word · 1
flagos-ai/FlagGems★ 1.2k100% same
Edited · 0
None found.
Wrote their own version · 0
None found.
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository51 other files it gives its agents.
- agent-flow/AGENTS.mdAGENTS.md
- AGENTS.mdAGENTS.md
- .codex/AGENTS.mdAGENTS.md
- cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/AGENTS.mdAGENTS.md
- jenkins/AGENTS.mdAGENTS.md
- tensorrt_llm/runtime/kv_cache_manager_v2/AGENTS.mdAGENTS.md
- tests/AGENTS.mdAGENTS.md
- tests/integration/defs/visual_gen/AGENTS.mdAGENTS.md
- tests/unittest/_torch/visual_gen/AGENTS.mdAGENTS.md
- agent-flow/CLAUDE.mdCLAUDE.md
- CLAUDE.mdCLAUDE.md
- tensorrt_llm/runtime/kv_cache_manager_v2/CLAUDE.mdCLAUDE.md
- tests/integration/defs/visual_gen/CLAUDE.mdCLAUDE.md
- tests/unittest/_torch/visual_gen/CLAUDE.mdCLAUDE.md
- trtllm-modeling-v2-entrySkill
- trtllm-modeling-v2-featuresSkill
- perf-analyzeSkill
- perf-optimizeSkill
- exec-env-checkSkill
- exec-local-compileSkill
- exec-local-dockerSkill
- exec-local-slurmSkill
- exec-remote-slurmSkill
- exec-slurm-compileSkill
- kernel-cute-writingSkill
- kernel-tileir-optimizationSkill
- kernel-triton-writingSkill
- perf-analysisSkill
- perf-host-analysisSkill
- perf-host-optimizationSkill
- perf-instability-cookbookSkill
- perf-nsight-compute-analysisSkill
- perf-nsight-systemsSkill
- perf-optimization-casebookSkill
- perf-optimizationSkill
- perf-regression-cookbookSkill
- perf-torch-cuda-graphsSkill
- perf-torch-sync-freeSkill
- perf-workload-profilingSkill
- trtllm-case-executorSkill
- trtllm-codebase-explorationSkill
- trtllm-code-contributionSkill
- trtllm-flashinfer-upgradeSkill
- trtllm-model-onboard-multimodalSkill
- trtllm-moe-developSkill
- trtllm-serve-config-guideSkill
- trtllm-test-script-builderSkill
- trtllm-test-specialistSkill
- trtllm-visualgen-benchSkill
- trtllm-visualgen-lpips-golden-refreshSkill
- visual-gen-component-testSkill

The file

We found no rules to list in this file: it reads as prose or reference rather than instructions.

Create a free account to mount

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

It worked
Partly
It didn't

What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

Check your setup, free

TensorRT-LLM / tests

Trending

Most recently taken up by

Who's talking about NVIDIA/TensorRT-LLM

On GitHub · 47