← All GPUs
NVIDIA · Workstation · Blackwell (GB202)
NVIDIA RTX PRO 6000 Blackwell Workstation Edition
600 W double-flow-through cooler dumps heat into the case; the Max-Q edition is the one meant for stacking.
VRAM
96 GB
GDDR7 ECC
Memory bandwidth
1,792 GB/s
FP16 compute
not published
Price
~$16,000
new, maker or suggested price
What fits at 4-bit
The 4-bit file plus 8,192 tokens of context and 1.5 GB of overhead, against the VRAM of one card. Speeds are measured medians where someone has published a run at that card count; a blank speed means no run, not a slow one. Machine pages carry estimates for the gaps.
| Model | Needs | 1 card (96 GB) |
|---|---|---|
| Qwen3.5 4B4B · Q4_K_M | 4.3 GB | fits |
| Llama 3.1 8B Instruct8B · Q4_K_M | 7.1 GB | 144 tok/smedian of 2 runs |
| Qwen3 8B8.2B · Q4_K_M | 7.3 GB | 174 tok/smedian of 1 run |
| Gemma 4 E4B IT8B · Q4_K_M | 6.5 GB | fits |
| Granite 4.2 8B8.8B · Q4_K_M | 7.9 GB | fits |
| Qwen3.5 9B9B · Q4_K_M | 7.0 GB | fits |
| Gemma 4 12B IT11.95B · Q4_K_M | 8.2 GB | fits |
| Ministral 3 14B Reasoning (2512)13.9B · Q4_K_M | 10.4 GB | fits |
| Phi-414.7B · Q4_K_M | 11.3 GB | fits |
| gpt-oss-20b21B, 3.6B active · MXFP4 (native) | 12.5 GB | 287 tok/smedian of 3 runs |
| Devstral Small 2 24B (2512)24B · Q4_K_M | 16.1 GB | fits |
| Qwen3.6 27B27B · Q4_K_M | 17.7 GB | fits |
| Gemma 4 26B-A4B IT25.2B, 3.8B active · Q4_K_M | 17.4 GB | 197 tok/smedian of 1 run |
| Qwen3.8 27B27B · Q4_K_M | 18.2 GB | fits |
| Granite 4.2 30B29.3B · Q4_K_M | 20.3 GB | fits |
| Gemma 4 31B IT30.7B · Q4_K_M | 18.9 GB | 62 tok/smedian of 1 run |
| Qwen3 Coder 30B-A3B Instruct30.5B, 3.3B active · Q4_K_M | 19.5 GB | fits |
| Ornith 1.5 35B-A3B35B, 3B active · Q4_K_M | 21.9 GB | fits |
| Qwen3.6 35B-A3B35B, 3B active · Q4_K_M | 22.4 GB | fits |
| Nemotron 3.5 Lightning 30B-A3B30B, 3B active · Q4_K_M | 25.3 GB | fits |
| Llama 3.3 70B Instruct70B · Q4_K_M | 43.6 GB | 32 tok/smedian of 1 run |
| Qwen3 Coder Next80B, 3B active · Q4_K_M | 46.9 GB | 91 tok/smedian of 1 run |
| gpt-oss-120b117B, 5.1B active · MXFP4 (native) | 60.2 GB | 202 tok/smedian of 4 runs |
| Llama 4 Scout 17B-16E Instruct109B, 17B active · Q4_K_M | 62.4 GB | fits |
| Mistral Small 4 119B (2603)119B, 6.5B active · Q4_K_M | 69.3 GB | fits |
| GLM-4.5-Air106B, 12B active · Q4_K_M | 70.9 GB | 101 tok/smedian of 1 run |
| Mistral Medium 3.5 128B128B · Q4_K_M | 74.0 GB | fits |
| Qwen3.5 122B-A10B122B, 10B active · Q4_K_M | 73.0 GB | 98 tok/smedian of 1 run |
| Nemotron 3 Super 120B-A12B120B, 12B active · Q4_K_M | 82.6 GB | fits |
Machines built on it
Measured speeds
One card
| Model | Quant | Backend | Generation | Prompt | Context | Source |
|---|---|---|---|---|---|---|
| Gemma 4 26B-A4B IT | Q4_K | llama.cpp (build not stated) | 197 tok/s | 9,437 tok/s4,096-token prompt | 4,096 | review2026-03 |
| Gemma 4 31B IT | Q4_K | llama.cpp (build not stated) | 62 tok/s | 3,750 tok/s4,096-token prompt | 4,096 | review2026-03 |
| GLM-4.5-Air | Q4_K | llama.cpp (build not stated) | 101 tok/s | 2,795 tok/s4,096-token prompt | 4,096 | review2026-03 |
| gpt-oss-120b | MXFP4 | llama.cpp (build not stated) | 210 tok/s | 4,579 tok/s4,096-token prompt | 4,096 | review2026-03 |
| gpt-oss-120b | MXFP4 | SGLang (lmsysorg/sglang:spark image) | 208 tok/s | 949 tok/s2,048-token prompt | 0 | review2025-10 |
| gpt-oss-120b | MXFP4 MoE | llama.cpp b6205 CUDA | 196 tok/s | 5,518 tok/s2,048-token prompt | 0 | llama.cpp benchmark discussion2025-08 |
| gpt-oss-120b | MXFP4 | Ollama (version not stated) | 153 tok/s | 3,409 tok/s | 0 | review2025-10 |
| gpt-oss-120b | Q8_0 | llama.cpp b8288 | 221 tok/s | 6,513 tok/s4,096-token prompt | 4,096 | review2026-03 |
| gpt-oss-20b | MXFP4 | SGLang (lmsysorg/sglang:spark image) | 300 tok/s | 2,481 tok/s2,048-token prompt | 0 | review2025-10 |
| gpt-oss-20b | MXFP4 MoE | llama.cpp b6205 CUDA | 287 tok/s | 11,522 tok/s2,048-token prompt | 0 | llama.cpp benchmark discussion2025-08 |
| gpt-oss-20b | MXFP4 | Ollama (version not stated) | 215 tok/s | 10,108 tok/s | 0 | review2025-10 |
| Llama 3.1 8B Instruct | Q4_K_M | Ollama (version not stated) | 202 tok/s | 38,864 tok/s | 0 | review2025-10 |
| Llama 3.1 8B Instruct | Q4_K_M | LocalScore (llamafile; version not stated on leaderboard) | 86 tok/s | 6,346 tok/s | 0 | LocalScore2026-10 |
| Llama 3.1 8B Instruct | Q8_0 | llama.cpp b7278 | 165 tok/s | 14,040 tok/s512-token prompt | 0 | owner report2025-12 |
| Llama 3.1 8B Instruct | Q8_0 | llama.cpp b7278 | 156 tok/s | 10,093 tok/s512-token prompt | 0 | owner report2025-12 |
| Llama 3.1 8B Instruct | FP8 | SGLang (lmsysorg/sglang:spark image) | 144 tok/s | 38,744 tok/s2,048-token prompt | 0 | review2025-10 |
| Llama 3.1 8B Instruct | Q8_0 | Ollama (version not stated) | 144 tok/s | 40,037 tok/s | 0 | review2025-10 |
| Llama 3.3 70B Instruct | Q4_K | llama.cpp (build not stated) | 32 tok/s | 1,689 tok/s4,096-token prompt | 4,096 | review2026-03 |
| Llama 3.3 70B Instruct | Q8_0 | llama.cpp b7278 | 21 tok/s | 1,735 tok/s512-token prompt | 0 | owner report2025-12 |
| Llama 3.3 70B Instruct | Q8_0 | llama.cpp b7278 | 19 tok/s | 907 tok/s512-token prompt | 0 | owner report2025-12 |
| Qwen3 8B | Q4_K | llama.cpp (build not stated) | 174 tok/s | 10,964 tok/s4,096-token prompt | 4,096 | review2026-03 |
| Qwen3 Coder Next | INT4 experts / HQQ4 attention / k4v4 KV | Krasis (MoE runtime; version not stated in table) | 91 tok/s | 11,211 tok/s | 0 | independent benchmark2026-10 |
| Qwen3.5 122B-A10B | Q4_K | llama.cpp b8288 | 98 tok/s | 3,055 tok/s4,096-token prompt | 4,096 | review2026-03 |
Speculative decoding, several users, or long context
Kept apart because each measures a different workload from one person chatting near the start of a conversation.
| Model | Quant | Backend | Generation | Prompt | Context | Measured on | Source |
|---|---|---|---|---|---|---|---|
| Gemma 4 26B-A4B IT | Q4_K | llama.cpp (build not stated) | 173 tok/s | 8,454 tok/s16,384-token prompt | 16,384 | RTX PRO 6000 Blackwell Workstation Edition | review2026-03 |
| Gemma 4 26B-A4B IT | Q4_K | llama.cpp (build not stated) | 170 tok/s | 7,108 tok/s32,768-token prompt | 32,768 | RTX PRO 6000 Blackwell Workstation Edition | review2026-03 |
| Gemma 4 31B IT | Q4_K | llama.cpp (build not stated) | 60 tok/s | 3,062 tok/s16,384-token prompt | 16,384 | RTX PRO 6000 Blackwell Workstation Edition | review2026-03 |
| Gemma 4 31B IT | Q4_K | llama.cpp (build not stated) | 56 tok/s | 2,087 tok/s32,768-token prompt | 32,768 | RTX PRO 6000 Blackwell Workstation Edition | review2026-03 |
| GLM-4.5-Air | Q4_K | llama.cpp (build not stated) | 83 tok/s | 2,107 tok/s16,384-token prompt | 16,384 | RTX PRO 6000 Blackwell Workstation Edition | review2026-03 |
| GLM-4.5-Air | Q4_K | llama.cpp (build not stated) | 64 tok/s | 1,450 tok/s32,768-token prompt | 32,768 | RTX PRO 6000 Blackwell Workstation Edition | review2026-03 |
| gpt-oss-120b | MXFP4 | llama.cpp (build not stated) | 182 tok/s | 4,061 tok/s16,384-token prompt | 16,384 | RTX PRO 6000 Blackwell Workstation Edition | review2026-03 |
| gpt-oss-120b | MXFP4 | llama.cpp (build not stated) | 162 tok/s | 3,368 tok/s32,768-token prompt | 32,768 | RTX PRO 6000 Blackwell Workstation Edition | review2026-03 |
| gpt-oss-120b | Q8_0 | llama.cpp b8288 | 193 tok/s | 5,722 tok/s16,384-token prompt | 16,384 | RTX PRO 6000 Blackwell Workstation Edition | review2026-03 |
| gpt-oss-120b | Q8_0 | llama.cpp b8288 | 179 tok/s | 5,000 tok/s32,768-token prompt | 32,768 | RTX PRO 6000 Blackwell Workstation Edition | review2026-03 |
| Llama 3.3 70B Instruct | Q4_K | llama.cpp (build not stated) | 28 tok/s | 1,355 tok/s16,384-token prompt | 16,384 | RTX PRO 6000 Blackwell Workstation Edition | review2026-03 |
| Llama 3.3 70B Instruct | Q4_K | llama.cpp (build not stated) | 26 tok/s | 1,008 tok/s32,768-token prompt | 32,768 | RTX PRO 6000 Blackwell Workstation Edition | review2026-03 |
| Qwen3 8B | Q4_K | llama.cpp (build not stated) | 141 tok/s | 7,588 tok/s16,384-token prompt | 16,384 | RTX PRO 6000 Blackwell Workstation Edition | review2026-03 |
| Qwen3 8B | Q4_K | llama.cpp (build not stated) | 111 tok/s | 5,301 tok/s32,768-token prompt | 32,768 | RTX PRO 6000 Blackwell Workstation Edition | review2026-03 |
| Qwen3.5 122B-A10B | Q4_K | llama.cpp b8288 | 94 tok/s | 2,836 tok/s16,384-token prompt | 16,384 | RTX PRO 6000 Blackwell Workstation Edition | review2026-03 |
| Qwen3.5 122B-A10B | Q4_K | llama.cpp b8288 | 91 tok/s | 2,583 tok/s32,768-token prompt | 32,768 | RTX PRO 6000 Blackwell Workstation Edition | review2026-03 |
Specifications
- Memory
- 96 GB GDDR7 ECC
- FP16 basis
- NVIDIA publishes 4,000 AI TOPS (FP4 with sparsity) and 126 TFLOPS FP32; no dense FP16 tensor figure.
- Board power
- 600 W
- Slots
- 2
- Length
- 305 mm
- PCIe
- Gen 5 x16
- NVLink
- No
- Cooling
- Open-air
- Software
- CUDA, Vulkan
- Released
- 2025
- Availability
- sold new
- Price basis
- NVIDIA US Marketplace listing reported at $16,000 (out of stock) in Aug 2026, up from $13,250 in June 2026 and $8,565 at the March 2025 launch listing; NVIDIA's store itself could not be loaded from our tooling.
- Maker page
- nvidia.com
Sources
- nvidia.com: datasheet: 96 GB GDDR7 ECC 512-bit, 1792 GB/s, PCIe 5.0 x16, 600 W, 5.4 x 12 in dual slot, double flow-through, 4000 AI TOPS FP4 sparse
- letsdatascience.com: $16,000 marketplace listing Aug 2026
- tomshardware.com: $13,250 June 2026; $8,565 launch price