nemotron
Skill8 starsChanged 8 months agoMITView on GitHub
NVIDIA Nemotron — open LLM family with hybrid Mamba-Transformer MoE architecture, 1M context, NIM deployment, synthetic data generation
- Reads credentials
In this repository23 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- validate-skillsSkill
- alchemi-toolkit-opsSkill
- autoresearch-setupSkill
- bionemoSkill
- biosimspaceSkill
- cuda-quantumSkill
- cutileSkill
- daskSkill
- flashinferSkill
- materials-projectSkill
- mpiSkill
- omniverse-simreadySkill
- physicsnemoSkill
- polarsSkill
- prime-verifiersSkill
- qiskitSkill
- raySkill
- rdkitSkill
- slurmSkill
- tinkerSkill
- tritonSkill
The file
We split this skill into the rules it gives an agent
3 rules. About 0 are common, 3 are distinctive, and 0 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈3
Distinctive · estimate
This author's own choices. Worth a look.
0
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 394
NIM Deployment: Use pre-built containers for production
- Distinctive: line 395
Synthetic Data: Start with small batches, scale up
- Distinctive: line 399
Async Generation: Use for large-scale data creation
≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
NVIDIA Nemotron Skill
Family of open LLMs with hybrid Mamba-Transformer MoE architecture, 1M-token context windows, optimized for agentic AI with NIM deployment and synthetic data generation.
Official Sources:
Model Family
Reasoning Models
# Nano 30B A3B - Cost-efficient agentic tasks
# - 3.2B active params (31.6B total)
# - 1M token context
# - 4x faster than Nemotron 2 Nano
# Super 49B - Multi-agent reasoning
# - ~49B params
# - High accuracy, efficient deep research
# Ultra 253B - Enterprise workflows
# - ~253B params
# - Maximum accuracy for complex scenariosSpecialized Models
# Vision Language 12B
# - Document intelligence
# - Video understanding
# RAG Models
# - Extraction, embedding, reranking
# Safety Models
# - Jailbreak detection
# - Content safety (multilingual)
# Speech Models
# - ASR, TTS, neural MTQuick Start with NIM
# Get NGC API key from https://build.nvidia.com
# Authenticate Docker
echo "$NGC_API_KEY" | docker login nvcr.io --username '$oauthtoken' --password-stdin
# Deploy Nemotron Nano
docker run -it --rm --gpus all \
-e NGC_API_KEY=$NGC_API_KEY \
-p 8000:8000 \
nvcr.io/nim/nvidia/nemotron-3-nano-30b-a3b:1.0.0
# Wait for startup
# INFO: Application startup complete
# INFO: Uvicorn running on http://0.0.0.0:8000API Usage
from openai import OpenAI
# Connect to NIM
client = OpenAI(
base_url="http://localhost:8000/v1",
api_key="not-used"
)
# Chat completion
response = client.chat.completions.create(
model="nvidia/nemotron-3-nano-30b-a3b",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Explain quantum computing."}
],
temperature=0.7,
max_tokens=500
)
print(response.choices[0].message.content)
# Streaming
for chunk in client.chat.completions.create(
model="nvidia/nemotron-3-nano-30b-a3b",
messages=[{"role": "user", "content": "Count to 10"}],
stream=True
):
print(chunk.choices[0].delta.content or "", end="")NeMo Framework Training
from nemo.collections import llm
from nemo import lightning as nl
import nemo_run as run
# Pretraining recipe
pretrain = llm.nemotron3_8b.pretrain_recipe(
name="nemotron3_8b_pretraining",
dir="/workspace/checkpoints",
num_nodes=2,
num_gpus_per_node=8
)
# Custom data (replace MockDataModule)
pretrain.data = YourDataModule()
# Execute locally
executor = run.LocalExecutor()
run.run(pretrain, executor=executor)
# Or run directly
run.run(pretrain, direct=True)Synthetic Data Generation
from nemo_curator.synthetic import NemotronGenerator, OpenAIClient
# Connect to API
client = OpenAIClient(
base_url="https://integrate.api.nvidia.com/v1",
api_key="your_api_key",
model_name="nvidia/nemotron-3-nano-30b-a3b"
)
generator = NemotronGenerator(client)
# Generate open Q&A pipeline
qa_data = generator.run_open_qa_pipeline(
n_macro_topics=5,
n_subtopics=10,
n_openlines=3,
output_file="qa_dataset.jsonl"
)
# Generate math problems
math_data = generator.run_math_pipeline(
n_macro_topics=3,
n_subtopics=5,
school_level="university",
output_file="math_dataset.jsonl"
)
# Generate Python coding problems
code_data = generator.run_python_pipeline(
n_macro_topics=4,
n_subtopics=6,
output_file="code_dataset.jsonl"
)Available Model Sizes
# Nemotron 3 (NeMo 2.0)
llm.nemotron3_4b.pretrain_recipe()
llm.nemotron3_8b.pretrain_recipe()
# Nemotron 4 (NeMo 2.0)
llm.nemotron4_15b.pretrain_recipe() # 16k context
llm.nemotron4_15b_16k.pretrain_recipe()
llm.nemotron4_15b_64k.pretrain_recipe()
llm.nemotron4_22b.pretrain_recipe() # 16k context
llm.nemotron4_22b_16k.pretrain_recipe()
llm.nemotron4_22b_64k.pretrain_recipe()
llm.nemotron4_340b.pretrain_recipe()Deployment Options
NIM Microservices
# Pre-optimized from API Catalog
docker run -it --rm --gpus all \
-e NGC_API_KEY=$NGC_API_KEY \
-p 8000:8000 \
nvcr.io/nim/nvidia/nemotron-3-nano-30b-a3b:1.0.0
# Custom model with multi-LLM NIM
docker run -it --rm --gpus all \
-e NGC_API_KEY=$NGC_API_KEY \
-e NIM_MODEL_NAME=custom-model \
-v /path/to/model:/model \
-p 8000:8000 \
nvcr.io/nim/meta/llama-3-multi-llm:1.0.0Open Frameworks
# vLLM
from vllm import LLM, SamplingParams
llm = LLM(model="nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16")
outputs = llm.generate(["Explain AI"], SamplingParams(temperature=0.7))
# Hugging Face Transformers
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
"nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16",
torch_dtype=torch.bfloat16,
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16")
# Ollama
# ollama pull nemotron-3-nano
# ollama run nemotron-3-nanoArchitecture Highlights
Hybrid Mamba-Transformer MoE
Layer Pattern:
[Mamba-2, MoE] pairs + selective self-attention
Benefits:
- Mamba: Efficient long-sequence modeling (1M tokens)
- Transformer: Precision reasoning (code, math)
- MoE: Scalable compute (31.6B total, 3.2B active)Advanced Features (Super/Ultra)
# Latent MoE
# - 4x more experts at same cost
# - Reduced communication overhead
# Multi-Token Prediction (MTP)
# - Predicts multiple tokens simultaneously
# - ~2.4% accuracy improvement
# - Speculative decoding speedups
# NVFP4 Training
# - 4-bit floating-point format
# - Cost-accuracy optimization
# - 25 trillion token pretrainingSynthetic Data Pipelines
Open Q&A
# Generate topics → subtopics → questions → revisions
data = generator.run_open_qa_pipeline(
n_macro_topics=10,
n_subtopics=20,
n_openlines=5,
output_file="openqa.jsonl"
)Writing Tasks
# Generate and revise writing prompts
data = generator.run_writing_pipeline(
n_macro_topics=5,
n_subtopics=10,
output_file="writing.jsonl"
)Math Problems
# School-level math generation
data = generator.run_math_pipeline(
n_macro_topics=4,
n_subtopics=8,
school_level="high school", # beginner, middle school, high school, university
output_file="math.jsonl"
)Coding Problems
# Python-focused problem generation
data = generator.run_python_pipeline(
n_macro_topics=6,
n_subtopics=12,
output_file="python.jsonl"
)Dialogue Generation
# Multi-turn conversations
dialogue = generator.generate_dialogue(
persona="expert programmer",
n_user_turns=3,
topic="async Python"
)
# Two-turn prompts
two_turn = generator.generate_two_turn_prompt(
topic="machine learning",
n_samples=100
)Async Generation
from nemo_curator.synthetic import AsyncNemotronGenerator
# Concurrent requests with rate limiting
async_gen = AsyncNemotronGenerator(
client,
max_concurrent_requests=10
)
data = async_gen.run_open_qa_pipeline(
n_macro_topics=100,
n_subtopics=200,
output_file="large_dataset.jsonl"
)Integration with NeMo Curator
from nemo_curator import DocumentDataset
from nemo_curator.synthetic import NemotronGenerator
# Generate synthetic data
generator = NemotronGenerator(client)
df = generator.run_open_qa_pipeline(n_macro_topics=5)
# Convert to DocumentDataset
dataset = DocumentDataset.from_pandas(df)
# Apply filtering
from nemo_curator.filters import WordCountFilter
filtered = dataset.filter(WordCountFilter(min_words=10))
# Deduplicate
from nemo_curator.modules import ExactDuplicates
deduped = ExactDuplicates().deduplicate(filtered)
# Export
deduped.to_pandas().to_json("curated_data.jsonl", orient="records", lines=True)Hardware Requirements
Nano 30B A3B:
- GPU: 1x A100 80GB or H100
- RAM: 64GB+
- VRAM: ~60GB
Super 49B:
- GPU: 2x A100 80GB or 1x H100
- RAM: 128GB+
Ultra 253B:
- GPU: 4x A100 80GB or 2x H100
- RAM: 256GB+Best Practices
- NIM Deployment: Use pre-built containers for production
- Synthetic Data: Start with small batches, scale up
- Training: Replace MockDataModule with real data
- Context Window: Leverage 1M tokens for long documents
- API Compatibility: OpenAI-compatible endpoints
- Async Generation: Use for large-scale data creation
- Model Selection: Nano for cost, Ultra for accuracy
Common Patterns
# Deploy + Generate + Train pipeline
# 1. Deploy NIM for inference
# 2. Generate synthetic data
# 3. Train custom model with NeMo
# Agentic AI workflow
# - Use Nemotron for reasoning
# - 1M context for long-running memory
# - Tool calling via function API
# Document intelligence
# - VL 12B for multimodal understanding
# - RAG models for extraction/embedding
# - 1M context for full documentsReferences
- Model Family - Nano, Super, Ultra, VL, RAG, Safety, Speech models
- NIM Deployment - Containers, API, inference, scaling
- Synthetic Data - NemotronGenerator, pipelines, customization
- NeMo Training - Pretraining recipes, configurations, distributed training
- Architecture - Hybrid Mamba-Transformer MoE, training techniques
- API Reference - Endpoints, parameters, examples
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

