NVIDIA · mixture of experts · NVIDIA Nemotron Open Model License
What runs Nemotron 3 Super 120B-A12B
Hybrid Mamba/MoE model with 120B total and 12B active parameters and multi-token prediction. Sized for 128 GB machines and DGX Spark-class boxes.
- Parameters
- 120B, 12B active
- 4-bit file
- 81.0 GB
- Q4_K_M
- Needs at 8k context
- 82.6 GB
- context memory estimated
- Longest context
- 262,144 tokens
Context memory: 8,192 bytes per token for the full-attention layers only. LatentMoE Mamba-2 + attention hybrid: only 8 of 88 layers are attention (2 x 8 x 2 KV heads x 128 x 2 = 8,192 bytes/token); Mamba layers hold a fixed-size state. Card gives no per-token figure. How context memory works.
Measured on this hardware
Machines that hold the 4-bit file with 8,192 tokens of context and have published runs on the same chip or cards, fastest first. Median of single-user runs with plain decoding.
| Machine | Fit | Price | Generation |
|---|---|---|---|
| Apple Mac Studio (M5 Max, 40-core GPU)Apple unified memory | Fitsassumed limitfrom 128 GB | ~$3,099–$5,099 | 58 tok/smedian of 1 run |
| Apple MacBook Pro 14-inch (M5 Max, 40-core GPU)Apple unified memory | Fitsassumed limitfrom 128 GB | ~$4,699–$6,699 | 58 tok/smedian of 1 run |
| Apple MacBook Pro 16-inch (M5 Max, 40-core GPU)Apple unified memory | Fitsassumed limitfrom 128 GB | ~$4,999–$6,999 | 58 tok/smedian of 1 run |
| NVIDIA DGX SparkNVIDIA GB10 | Fitsassumed limit | ~$4,699 | 17 tok/smedian of 2 runs |
| Dell Pro Precision with GB10NVIDIA GB10 | Fitsassumed limit | ~$5,523 | 17 tok/smedian of 2 runs |
| Framework Desktop (Ryzen AI Max+ 395)AMD unified memory | Fitsfrom 128 GB | ~$1,959–$3,449 | 15 tok/smedian of 2 runs |
| GMKtec EVO-X2 (Ryzen AI Max+ 395)AMD unified memory | Fitsfrom 128 GB | ~$2,600–$4,000 | 15 tok/smedian of 2 runs |
| Beelink GTR9 Pro (Ryzen AI Max+ 395)AMD unified memory | Fits | ~$4,699 | 15 tok/smedian of 2 runs |
| Minisforum MS-S1 MAX (Ryzen AI Max+ 395)AMD unified memory | Fits | ~$4,899 | 15 tok/smedian of 2 runs |
| Corsair AI Workstation 300 (Ryzen AI Max+ 395)AMD unified memory | Fits | ~$4,800 | 15 tok/smedian of 2 runs |
| HP Z2 Mini G1a (Ryzen AI Max+ PRO 395)AMD unified memory | Fits | ~$4,999 | 15 tok/smedian of 2 runs |
Estimated, not measured
These fit but nobody has published a run we could match. The figure is memory bandwidth divided by the bytes read per token, scaled by the share of that ceiling measured runs reach. How estimates work.
| Machine | Fit | Price | Generation |
|---|---|---|---|
| Dell Pro Precision with GB300NVIDIA workstation | Fits | ~$159,283 | ~385–491 tok/sestimate |
| Dell Pro Max Tower T2 with NVIDIA RTX PRO 6000 BlackwellNVIDIA workstation | Fits | ~$13,909 | ~51–75 tok/sestimate |
| 1x RTX PRO 6000 BlackwellSingle GPU | Fits | ~$20,833 | ~51–75 tok/sestimate |
| Apple Mac Studio (M5 Ultra, 64-core GPU)Apple unified memory | Fitsassumed limitfrom 256 GB | ~$5,499–$9,499 | ~46–82 tok/sestimate |
| Apple Mac Studio (M5 Ultra, 80-core GPU)Apple unified memory | Fitsassumed limitfrom 256 GB | ~$6,799–$10,799 | ~46–82 tok/sestimate |
| 4x Instinct MI50 32GB (used) on EPYC 7003Multi-GPU | Fits | ~$1,998 + unpriced parts | ~29–43 tok/sestimate |
| 4x RTX 3090 (used) on EPYC 7003, open frameMulti-GPU | Fits | ~$2,178 + unpriced parts | ~27–39 tok/sestimate |
| RTX 5090 + EPYC 9355P with 768 GB DDR5 (MoE offload)Hybrid GPU + RAM | GPU + RAM offloadassumed limit | ~$3,763 + unpriced parts | ~23–34 tok/sestimate, part in RAM |
| tiny corp tinybox red flashGPU desktop (prebuilt) | Fits | ~$22,000 | ~18–27 tok/sestimate |
| Framework Desktop (Ryzen AI Max+ PRO 495)AMD unified memory | Fitsassumed limit | ~$6,799 | ~15–19 tok/sestimate |
| RTX 3090 (used) + 128 GB DDR5 on AM5 (MoE offload)Hybrid GPU + RAM | GPU + RAM offloadassumed limit | ~$4,833 + unpriced parts | ~3.9–5.7 tok/sestimate, part in RAM |
Fits, no speed figure
Too large for
Apple Mac Studio (M5 Max, 32-core GPU), Apple Mac mini (M5 Pro, 16-core GPU), Apple Mac mini (M5 Pro, 20-core GPU), Apple MacBook Pro 14-inch (M5 Max, 32-core GPU), Apple MacBook Pro 16-inch (M5 Max, 32-core GPU), Framework Desktop (Ryzen AI Max 385), 1x RTX 3090 (used), 2x RTX 3090 (used), 1x RTX 5090, 2x RTX 5090, 2x Radeon RX 7900 XTX, 2x Intel Arc Pro B60. A smaller quant may fit: try the calculator.
Files
| Quant | Format | Size | Repo |
|---|---|---|---|
| IQ2_XXS | GGUF | 46.7 GB | bartowski/nvidia_Nemotron-3-Super-120B-A12B-GGUF |
| IQ2_M | GGUF | 50.3 GB | bartowski/nvidia_Nemotron-3-Super-120B-A12B-GGUF |
| Q2_K | GGUF | 51.1 GB | bartowski/nvidia_Nemotron-3-Super-120B-A12B-GGUF |
| Q2_K_L | GGUF | 51.5 GB | bartowski/nvidia_Nemotron-3-Super-120B-A12B-GGUF |
| IQ3_XXS | GGUF | 57.4 GB | bartowski/nvidia_Nemotron-3-Super-120B-A12B-GGUF |
| Q3_K_M | GGUF | 60.2 GB | bartowski/nvidia_Nemotron-3-Super-120B-A12B-GGUF |
| IQ3_M | GGUF | 61.8 GB | bartowski/nvidia_Nemotron-3-Super-120B-A12B-GGUF |
| 4bit | MLX | 63.3 GB | mlx-community/NVIDIA-Nemotron-3-Super-120B-A12B-4bit |
| Q4_K_M | GGUF | 81.0 GB | bartowski/nvidia_Nemotron-3-Super-120B-A12B-GGUF |
| Q5_K_M | GGUF | 90.0 GB | bartowski/nvidia_Nemotron-3-Super-120B-A12B-GGUF |
| Q6_K | GGUF | 105.8 GB | bartowski/nvidia_Nemotron-3-Super-120B-A12B-GGUF |
| 8bit | MLX | 119.5 GB | mlx-community/NVIDIA-Nemotron-3-Super-120B-A12B-8bit |
| Q8_0 | GGUF | 119.6 GB | bartowski/nvidia_Nemotron-3-Super-120B-A12B-GGUF |
Sources
- huggingface.co: model card: params, context, license, attention layout, description
- huggingface.co: downloads (last 30 days), license tag, safetensors param count
- huggingface.co: layer count, KV heads and head size
- huggingface.co: GGUF file sizes (shards summed)
- huggingface.co: MLX 4bit safetensors sizes
- huggingface.co: MLX 8bit safetensors sizes