NVIDIAKnown publisherNVIDIA/Megatron-LM/tests/unit_tests/distributed/mfsdp_v2/AGENTS.md

Megatron-LM / mfsdp_v2

AGENTS.md18k starsChanged 22 days agoLicence not identifiedView on GitHub

- Read the test files' module docstrings to choose a focused destination by the behavior being tested. Extend an existing test or parametrization when it covers the same behavior. -…

Sep 2, 2026[recipe] feat: preview DeepSeek-V4 exact 0-diff true-on-policy RL on Megatron and vLLM — verl-project/verl14 reactions · 2 comments
Nov 12, 2025[algo] feat: support router replay — verl-project/verl11 reactions · 11 comments
May 20, 2025[RFC] validation and evaluation in torchtitan — pytorch/torchtitan9 reactions · 5 comments
Jan 13, 2023Will Megatron-DeepSpeed rebase from https://github.com/NVIDIA/Megatron-LM in the future? — deepspeedai/Megatron-DeepSpeed7 reactions · 6 comments
Aug 29, 2025Potential Issue with Qwen2VisionModel RoPE — NVIDIA-NeMo/Speech6 reactions · 16 comments

36 more on GitHub

Dec 26, 2025[PyTorch]Add Casting-Free FP8-Flow-MoE Blockwise Optimizations — NVIDIA/TransformerEngine5 reactions · 11 comments
Oct 24, 2024[RFC] MOE design in Torchtune — meta-pytorch/torchtune5 reactions · 8 comments
Oct 25, 2021libai 设计文档 — Oneflow-Inc/libai5 reactions · 14 comments
Sep 22, 2026📣 [Announcement] Megatron Bridge will be consolidated with Megatron LM — NVIDIA-NeMo/Megatron-Bridge4 reactions · 3 comments
Mar 7, 2025feat: add training parallelism: DP,MP(TP,SP,PP,EP) — ai-bot-pro/achatbot4 reactions · 3 comments
Dec 18, 2023Supervised Fine-tuning for HugginFace pretrained weight. — deepspeedai/Megatron-DeepSpeed4 reactions · 7 comments
Feb 20, 2021[WIP] Add Megatron-11B — huggingface/transformers4 reactions · 8 comments
Dec 15, 2025Introduce Megatron-style parallel state management — deepspeedai/DeepSpeed3 reactions · 10 comments
Dec 6, 2025[Draft] Update Megatron patch to work for Megatron v0.15.0 — THUDM/slime3 reactions · 2 comments
Aug 30, 2024Memory Leak when using nn.Module hooks and thunder.jit — Lightning-AI/lightning-thunder3 reactions · 2 comments
Aug 4, 2022Fused Softmax Kernels — huggingface/transformers3 reactions · 8 comments
Sep 2, 2021syncing with the upstream? — deepspeedai/Megatron-DeepSpeed3 reactions
Sep 1, 2026nightly test fail after mbridge bump in #3906 — NVIDIA-NeMo/RL2 reactions
Jun 8, 2026[chore] move megatron-core from dev latest to main latest, bump megatron-bridge to latest — NovaSky-AI/SkyRL2 reactions
May 8, 2026[NeMo FW 26.06 Release] MBridge v0.5.0 Roadmap — NVIDIA-NeMo/Megatron-Bridge2 reactions
Apr 2, 2026Nemotron-CLIMB data curation recipe — NVIDIA-NeMo/Curator2 reactions · 4 comments
Mar 31, 2026[megatron] feat: support cp for bshd format — verl-project/verl2 reactions
Mar 9, 2026megatron compatibility issues with linear_attn — alibaba/ROLL2 reactions
Feb 6, 2026Fix on TE to support Mcore Vision Encoder CUDA Graph — NVIDIA/TransformerEngine2 reactions · 2 comments
Dec 4, 2025build: make mamba and nv-grouped-gemm optional dependencies — NVIDIA-NeMo/Megatron-Bridge2 reactions · 2 comments
Aug 29, 2025[PyTorch] fix cross entropy vanishing gradients — NVIDIA/TransformerEngine2 reactions · 4 comments
Jun 12, 2025[RFC] To resolve CPU memory peak in checkpointing process: migration to megatron dist_ckpt? — verl-project/verl2 reactions · 2 comments
Jul 12, 2024dataloader hangs with fork and pin memory=True — pytorch/pytorch2 reactions · 2 comments
Sep 26, 2021Add same_shape case for BiasDropout — microsoft/onnxruntime2 reactions · 6 comments
Oct 21, 2026feat(jt): 为纯 AdamW 接入路由偏置更新 — hf1s/hyper-parallel1 reaction · 1 comment
Oct 8, 2026Guard zero-gradient blocks in the default FP8 block-scaling recipe — NVIDIA/TransformerEngine1 reaction · 4 comments
Oct 5, 2026add megatron core inference as a new inference backend — verl-project/verl1 reaction · 1 comment
Sep 29, 2026Deterministic MoE Router Kernel By Removing Global Atomic Ops — NVIDIA/TransformerEngine1 reaction · 8 comments
Aug 10, 2026Optionally pad QK/V for better kernel selection — NVIDIA/TransformerEngine1 reaction · 11 comments
Jul 31, 2026[PTQ] Store FP32 global scaling factors (absmax or scale_inv) for all quantized activations and weights. — NVIDIA/TransformerEngine1 reaction · 21 comments
May 14, 2026[megatron, ckpt] feat: add Qwen3.5 Megatron-FSDP SFT example — verl-project/verl1 reaction
Mar 22, 2026Overlap-param-gather Muon — nmntz/Megatron-LM1 reaction · 1 comment
Feb 4, 2026[megatron][perf] Integrate megatron dynamic context parallelism — NovaSky-AI/SkyRL1 reaction · 1 comment
Mar 13, 2025[REQUEST] support Megatron-LM — SwanHubX/SwanLab1 reaction · 4 comments
Jul 9, 2024Provide shutdown() method for ProcessGroupGloo distributed backend — pytorch/pytorch1 reaction
Nov 26, 2021[cpp extension] making it possible to parallelize the building process — pytorch/pytorch1 reaction · 1 comment

On YouTube · 18

Aug 13, 2026Nemotron 3.5 Lightning: Why 46 of 52 Layers Aren't Transformers — Cloud Codes74k views · 36 comments
Oct 9, 2026Open Models: How to Own Your AI | Kari Ann Briski, NVIDIA — SyntaxGTM (Tom Granot)29k views · 3 comments
Oct 31, 2023Training LLMs at Scale - Deepak Narayanan | Stanford MLSys #83 — Stanford MLSys Seminars18k views · 7 comments
Nov 12, 2025From Data to Deployment: Building European AI at Scale — NVIDIA Developer2k views
Oct 23, 2024Nvidia's new AI can build LLMs cheaper! Upcycling MoE explained by Ethan He — Tech Cindy1k views · 1 comment

13 more on YouTube

Nov 12, 2025Livestream: From Data to Deployment Building European AI at Scale — NVIDIA Developer785 views · 3 comments
Oct 29, 2024[Paper Club] Upcycling Large Language Models into Mixture of Experts — Latent Space613 views · 4 comments
Jul 1, 2026NVIDIA Megatron-LM GitHub Guide: Billion-Parameter Transformer Training — Alex Hitt247 views
Mar 10, 2026Scalable MoE Training with NVIDIA Megatron Core — Vinh Nguyen241 views
Mar 18, 2026Slime Framework Review 2026: RL Scaling for LLMs and VLMs — Blueprint Bytes170 views
Apr 30, 2026🔥 nvidia just open-sourced Megatron LM and devs are #Shorts — Nightly Bits148 views · 1 comment
Mar 10, 2026[Podcast] Scalable MoE Training with NVIDIA Megatron Core — Vinh Nguyen124 views
Apr 10, 2026🔥 3 things devs don't know about Megatron-LM #Shorts — Nightly Bits120 views · 1 comment
Nov 30, 2023NVIDIA/Megatron-LM - Gource visualisation — Gourcer89 views
Mar 28, 2026Building Pipeline Parallelism from First Principles for Interviews: Step-by-Step Deep Dive — CloudStation202025 views
Oct 15, 2026HAMi: Heterogeneous GPU Sharing on Kubernetes, Megatron LM | GitHub Curated Picks | 2026-10-14 — Insight AI14 views
Jul 1, 2026NVIDIA/Megatron-LM — 20 stars today on GitHub — DevTrends Daily14 views · 1 comment
Sep 12, 2026System Design Primer, ArmorPaint & Megatron Core | GitHub Picks | 2026-09-11 — Insight AI10 views

On Stack Overflow · 1

Oct 6, 2016Keras uses way too much GPU memory when calling train_on_batch, fit, etc22 votes

Where files are headingNew AGENTS.md files have more rules about keeping secrets out of the code, and fewer about code style.
Of the AGENTS.md files we read, the 31,542 first seen in the last six months, against the older ones: the kind of rule that gained most share, and the kind that lost most. About files like this in general, not this one, and an estimate, since a kind is a pattern in the words.
In this repository18 other files it gives its agents.
- AGENTS.mdAGENTS.md
- megatron/core/models/hybrid/AGENTS.mdAGENTS.md
- batch-and-test-mergequeue-prsSkill
- mcore-build-and-dependencySkill
- mcore-bump-base-imageSkill
- mcore-cicdSkill
- mcore-create-issueSkill
- mcore-determinism-debugSkill
- mcore-linting-and-formattingSkill
- mcore-migrate-gpt-to-hybridSkill
- mcore-onboard-gb200-1node-testsSkill
- mcore-run-on-slurmSkill
- mcore-split-prSkill
- mcore-testingSkill
- nightly-syncSkill
- pr-reviewSkill
- respond-to-issueSkill
- update-golden-valuesSkill

The file

We split this AGENTS.md into the rules it gives an agent

2 rules. About 0 are common, 1 is distinctive, and 1 is very specific to this project.

≈0
Common · estimate
Many files like this have a rule of this kind.
≈1
Distinctive · estimate
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.

The licence could not be identified — read it at the source. We count its rules but don't quote them. Read them on GitHub.

≈ “Common” is an estimate: at least 1 in 5 of the 40,246 AGENTS.md files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

It worked
Partly
It didn't

What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

Check your setup, free