agentleFS
dtunaidtunai/agent-skills-for-compute/skills/nemotron/SKILL.md

nemotron

Skill8 starsChanged 8 months agoMITView on GitHub

NVIDIA Nemotron — open LLM family with hybrid Mamba-Transformer MoE architecture, 1M context, NIM deployment, synthetic data generation

  • Reads credentials

The file

We split this skill into the rules it gives an agent

3 rules. About 0 are common, 3 are distinctive, and 0 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈3

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 0

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    NIM Deployment: Use pre-built containers for production

    line 394
  2. Distinctive:

    Synthetic Data: Start with small batches, scale up

    line 395
  3. Distinctive:

    Async Generation: Use for large-scale data creation

    line 399

≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

NVIDIA Nemotron Skill

Family of open LLMs with hybrid Mamba-Transformer MoE architecture, 1M-token context windows, optimized for agentic AI with NIM deployment and synthetic data generation.

Official Sources:

Model Family

Reasoning Models

# Nano 30B A3B - Cost-efficient agentic tasks
# - 3.2B active params (31.6B total)
# - 1M token context
# - 4x faster than Nemotron 2 Nano

# Super 49B - Multi-agent reasoning
# - ~49B params
# - High accuracy, efficient deep research

# Ultra 253B - Enterprise workflows
# - ~253B params
# - Maximum accuracy for complex scenarios

Specialized Models

# Vision Language 12B
# - Document intelligence
# - Video understanding

# RAG Models
# - Extraction, embedding, reranking

# Safety Models
# - Jailbreak detection
# - Content safety (multilingual)

# Speech Models
# - ASR, TTS, neural MT

Quick Start with NIM

# Get NGC API key from https://build.nvidia.com

# Authenticate Docker
echo "$NGC_API_KEY" | docker login nvcr.io --username '$oauthtoken' --password-stdin

# Deploy Nemotron Nano
docker run -it --rm --gpus all \
  -e NGC_API_KEY=$NGC_API_KEY \
  -p 8000:8000 \
  nvcr.io/nim/nvidia/nemotron-3-nano-30b-a3b:1.0.0

# Wait for startup
# INFO: Application startup complete
# INFO: Uvicorn running on http://0.0.0.0:8000

API Usage

from openai import OpenAI

# Connect to NIM
client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="not-used"
)

# Chat completion
response = client.chat.completions.create(
    model="nvidia/nemotron-3-nano-30b-a3b",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain quantum computing."}
    ],
    temperature=0.7,
    max_tokens=500
)

print(response.choices[0].message.content)

# Streaming
for chunk in client.chat.completions.create(
    model="nvidia/nemotron-3-nano-30b-a3b",
    messages=[{"role": "user", "content": "Count to 10"}],
    stream=True
):
    print(chunk.choices[0].delta.content or "", end="")

NeMo Framework Training

from nemo.collections import llm
from nemo import lightning as nl
import nemo_run as run

# Pretraining recipe
pretrain = llm.nemotron3_8b.pretrain_recipe(
    name="nemotron3_8b_pretraining",
    dir="/workspace/checkpoints",
    num_nodes=2,
    num_gpus_per_node=8
)

# Custom data (replace MockDataModule)
pretrain.data = YourDataModule()

# Execute locally
executor = run.LocalExecutor()
run.run(pretrain, executor=executor)

# Or run directly
run.run(pretrain, direct=True)

Synthetic Data Generation

from nemo_curator.synthetic import NemotronGenerator, OpenAIClient

# Connect to API
client = OpenAIClient(
    base_url="https://integrate.api.nvidia.com/v1",
    api_key="your_api_key",
    model_name="nvidia/nemotron-3-nano-30b-a3b"
)

generator = NemotronGenerator(client)

# Generate open Q&A pipeline
qa_data = generator.run_open_qa_pipeline(
    n_macro_topics=5,
    n_subtopics=10,
    n_openlines=3,
    output_file="qa_dataset.jsonl"
)

# Generate math problems
math_data = generator.run_math_pipeline(
    n_macro_topics=3,
    n_subtopics=5,
    school_level="university",
    output_file="math_dataset.jsonl"
)

# Generate Python coding problems
code_data = generator.run_python_pipeline(
    n_macro_topics=4,
    n_subtopics=6,
    output_file="code_dataset.jsonl"
)

Available Model Sizes

# Nemotron 3 (NeMo 2.0)
llm.nemotron3_4b.pretrain_recipe()
llm.nemotron3_8b.pretrain_recipe()

# Nemotron 4 (NeMo 2.0)
llm.nemotron4_15b.pretrain_recipe()      # 16k context
llm.nemotron4_15b_16k.pretrain_recipe()
llm.nemotron4_15b_64k.pretrain_recipe()
llm.nemotron4_22b.pretrain_recipe()      # 16k context
llm.nemotron4_22b_16k.pretrain_recipe()
llm.nemotron4_22b_64k.pretrain_recipe()
llm.nemotron4_340b.pretrain_recipe()

Deployment Options

NIM Microservices

# Pre-optimized from API Catalog
docker run -it --rm --gpus all \
  -e NGC_API_KEY=$NGC_API_KEY \
  -p 8000:8000 \
  nvcr.io/nim/nvidia/nemotron-3-nano-30b-a3b:1.0.0

# Custom model with multi-LLM NIM
docker run -it --rm --gpus all \
  -e NGC_API_KEY=$NGC_API_KEY \
  -e NIM_MODEL_NAME=custom-model \
  -v /path/to/model:/model \
  -p 8000:8000 \
  nvcr.io/nim/meta/llama-3-multi-llm:1.0.0

Open Frameworks

# vLLM
from vllm import LLM, SamplingParams

llm = LLM(model="nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16")
outputs = llm.generate(["Explain AI"], SamplingParams(temperature=0.7))

# Hugging Face Transformers
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16",
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16")

# Ollama
# ollama pull nemotron-3-nano
# ollama run nemotron-3-nano

Architecture Highlights

Hybrid Mamba-Transformer MoE

Layer Pattern:
[Mamba-2, MoE] pairs + selective self-attention

Benefits:
- Mamba: Efficient long-sequence modeling (1M tokens)
- Transformer: Precision reasoning (code, math)
- MoE: Scalable compute (31.6B total, 3.2B active)

Advanced Features (Super/Ultra)

# Latent MoE
# - 4x more experts at same cost
# - Reduced communication overhead

# Multi-Token Prediction (MTP)
# - Predicts multiple tokens simultaneously
# - ~2.4% accuracy improvement
# - Speculative decoding speedups

# NVFP4 Training
# - 4-bit floating-point format
# - Cost-accuracy optimization
# - 25 trillion token pretraining

Synthetic Data Pipelines

Open Q&A

# Generate topics → subtopics → questions → revisions
data = generator.run_open_qa_pipeline(
    n_macro_topics=10,
    n_subtopics=20,
    n_openlines=5,
    output_file="openqa.jsonl"
)

Writing Tasks

# Generate and revise writing prompts
data = generator.run_writing_pipeline(
    n_macro_topics=5,
    n_subtopics=10,
    output_file="writing.jsonl"
)

Math Problems

# School-level math generation
data = generator.run_math_pipeline(
    n_macro_topics=4,
    n_subtopics=8,
    school_level="high school",  # beginner, middle school, high school, university
    output_file="math.jsonl"
)

Coding Problems

# Python-focused problem generation
data = generator.run_python_pipeline(
    n_macro_topics=6,
    n_subtopics=12,
    output_file="python.jsonl"
)

Dialogue Generation

# Multi-turn conversations
dialogue = generator.generate_dialogue(
    persona="expert programmer",
    n_user_turns=3,
    topic="async Python"
)

# Two-turn prompts
two_turn = generator.generate_two_turn_prompt(
    topic="machine learning",
    n_samples=100
)

Async Generation

from nemo_curator.synthetic import AsyncNemotronGenerator

# Concurrent requests with rate limiting
async_gen = AsyncNemotronGenerator(
    client,
    max_concurrent_requests=10
)

data = async_gen.run_open_qa_pipeline(
    n_macro_topics=100,
    n_subtopics=200,
    output_file="large_dataset.jsonl"
)

Integration with NeMo Curator

from nemo_curator import DocumentDataset
from nemo_curator.synthetic import NemotronGenerator

# Generate synthetic data
generator = NemotronGenerator(client)
df = generator.run_open_qa_pipeline(n_macro_topics=5)

# Convert to DocumentDataset
dataset = DocumentDataset.from_pandas(df)

# Apply filtering
from nemo_curator.filters import WordCountFilter
filtered = dataset.filter(WordCountFilter(min_words=10))

# Deduplicate
from nemo_curator.modules import ExactDuplicates
deduped = ExactDuplicates().deduplicate(filtered)

# Export
deduped.to_pandas().to_json("curated_data.jsonl", orient="records", lines=True)

Hardware Requirements

Nano 30B A3B:
- GPU: 1x A100 80GB or H100
- RAM: 64GB+
- VRAM: ~60GB

Super 49B:
- GPU: 2x A100 80GB or 1x H100
- RAM: 128GB+

Ultra 253B:
- GPU: 4x A100 80GB or 2x H100
- RAM: 256GB+

Best Practices

  1. NIM Deployment: Use pre-built containers for production
  2. Synthetic Data: Start with small batches, scale up
  3. Training: Replace MockDataModule with real data
  4. Context Window: Leverage 1M tokens for long documents
  5. API Compatibility: OpenAI-compatible endpoints
  6. Async Generation: Use for large-scale data creation
  7. Model Selection: Nano for cost, Ultra for accuracy

Common Patterns

# Deploy + Generate + Train pipeline
# 1. Deploy NIM for inference
# 2. Generate synthetic data
# 3. Train custom model with NeMo

# Agentic AI workflow
# - Use Nemotron for reasoning
# - 1M context for long-running memory
# - Tool calling via function API

# Document intelligence
# - VL 12B for multimodal understanding
# - RAG models for extraction/embedding
# - 1M context for full documents

References

  • Model Family - Nano, Super, Ultra, VL, RAG, Safety, Speech models
  • NIM Deployment - Containers, API, inference, scaling
  • Synthetic Data - NemotronGenerator, pipelines, customization
  • NeMo Training - Pretraining recipes, configurations, distributed training
  • Architecture - Hybrid Mamba-Transformer MoE, training techniques
  • API Reference - Endpoints, parameters, examples

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.