NVIDIA documented Nemotron-4 340B Base, Instruct, and Reward. The reward model could score candidate answers, supporting the use of generated examples to train smaller models.
NVIDIA Nemotron
Also known as: Nemotron 4 · Nemotron 3 · Llama Nemotron
Plain English
Nemotron is NVIDIA’s family of language and multimodal models for reasoning, synthetic data generation, and tool-using applications. Releases include base, instruction-tuned, and reward models.
Technical Overview
Earlier Nemotron releases include dense generators and reward models used for synthetic training data. Nemotron 3 combines Mamba recurrence, Transformer attention, and sparse experts to reduce the amount of computation needed per token. Larger variants use techniques such as latent expert projections, NVFP4 pretraining, and native speculative decoding.
Timeline (4)
2024
Paper2025
PaperNVIDIA described Nano V2 VL, a vision-language model for document understanding and long video. It extends the hybrid Nemotron backbone with a visual encoder for those inputs.
- Release
NVIDIA introduced the Nemotron 3 family and released Nano. The hybrid sparse model activates only a subset of its parameters per token, lowering the compute required for generation.
2026
ReleaseNVIDIA announced Nemotron 3 Super with 120B total and 12B active parameters. Its latent expert layers process smaller projected representations, and its multi-token prediction layers can propose tokens for speculative decoding.
See also (6)
Open full graphNVIDIA Nemotron