← All GPUs
NVIDIA · Consumer · Ampere (GA102)
NVIDIA GeForce RTX 3090
Last GeForce with an NVLink connector; Founders Edition is 3-slot, 313 mm.
VRAM
24 GB
GDDR6X
Memory bandwidth
936 GB/s
FP16 compute
71.2 TFLOPS
dense tensor
Price
$1,499
launch price; sold used now, used price not tracked yet
What fits at 4-bit
The 4-bit file plus 8,192 tokens of context and 1.5 GB of overhead, against the VRAM of 1, 2, 4 and 6 cards. Speeds are measured medians where someone has published a run at that card count; a blank speed means no run, not a slow one. Machine pages carry estimates for the gaps.
| Model | Needs | 1 card (24 GB) | 2 cards (48 GB) | 4 cards (96 GB) | 6 cards (144 GB) |
|---|---|---|---|---|---|
| Qwen3.5 4B4B · Q4_K_M | 4.3 GB | fits | fits | fits | fits |
| Llama 3.1 8B Instruct8B · Q4_K_M | 7.1 GB | 122 tok/smedian of 2 runs | fits | fits | fits |
| Qwen3 8B8.2B · Q4_K_M | 7.3 GB | 115 tok/smedian of 1 run | fits | fits | fits |
| Gemma 4 E4B IT8B · Q4_K_M | 6.5 GB | fits | fits | fits | fits |
| Granite 4.2 8B8.8B · Q4_K_M | 7.9 GB | fits | fits | fits | fits |
| Qwen3.5 9B9B · Q4_K_M | 7.0 GB | fits | fits | fits | fits |
| Gemma 4 12B IT11.95B · Q4_K_M | 8.2 GB | fits | fits | fits | fits |
| Ministral 3 14B Reasoning (2512)13.9B · Q4_K_M | 10.4 GB | fits | fits | fits | fits |
| Phi-414.7B · Q4_K_M | 11.3 GB | fits | fits | fits | fits |
| gpt-oss-20b21B, 3.6B active · MXFP4 (native) | 12.5 GB | 162 tok/smedian of 3 runs | fits | fits | fits |
| Devstral Small 2 24B (2512)24B · Q4_K_M | 16.1 GB | fits | fits | fits | fits |
| Qwen3.6 27B27B · Q4_K_M | 17.7 GB | fits | fits | fits | fits |
| Gemma 4 26B-A4B IT25.2B, 3.8B active · Q4_K_M | 17.4 GB | 119 tok/smedian of 1 run | fits | fits | fits |
| Qwen3.8 27B27B · Q4_K_M | 18.2 GB | fits | fits | fits | fits |
| Granite 4.2 30B29.3B · Q4_K_M | 20.3 GB | fits | fits | fits | fits |
| Gemma 4 31B IT30.7B · Q4_K_M | 18.9 GB | 35 tok/smedian of 1 run | fits | fits | fits |
| Qwen3 Coder 30B-A3B Instruct30.5B, 3.3B active · Q4_K_M | 19.5 GB | fits | fits | fits | fits |
| Ornith 1.5 35B-A3B35B, 3B active · Q4_K_M | 21.9 GB | fits | fits | fits | fits |
| Qwen3.6 35B-A3B35B, 3B active · Q4_K_M | 22.4 GB | fits | fits | fits | fits |
| Nemotron 3.5 Lightning 30B-A3B30B, 3B active · Q4_K_M | 25.3 GB | too large | fits | fits | fits |
| Llama 3.3 70B Instruct70B · Q4_K_M | 43.6 GB | too large | 20 tok/smedian of 1 run | fits | fits |
| Qwen3 Coder Next80B, 3B active · Q4_K_M | 46.9 GB | too large | fits | fits | fits |
| gpt-oss-120b117B, 5.1B active · MXFP4 (native) | 60.2 GB | too large | too large | fits | fits |
| Llama 4 Scout 17B-16E Instruct109B, 17B active · Q4_K_M | 62.4 GB | too large | too large | fits | fits |
| Mistral Small 4 119B (2603)119B, 6.5B active · Q4_K_M | 69.3 GB | too large | too large | fits | fits |
| GLM-4.5-Air106B, 12B active · Q4_K_M | 70.9 GB | too large | too large | fits | fits |
| Mistral Medium 3.5 128B128B · Q4_K_M | 74.0 GB | too large | too large | fits | fits |
| Qwen3.5 122B-A10B122B, 10B active · Q4_K_M | 73.0 GB | too large | too large | fits | fits |
| Nemotron 3 Super 120B-A12B120B, 12B active · Q4_K_M | 82.6 GB | too large | too large | fits | fits |
| Qwen3.8 Flash-Next125B, 6B active · Q4_K_M | 113.1 GB | too large | too large | too large | fits |
| MiniMax M2.7228.7B · Q4_K_M | 132.7 GB | too large | too large | too large | fits |
Machines built on it
- 1x RTX 3090 (used)~$2,944 + unpriced parts
- 2x RTX 3090 (used)~$3,639 + unpriced parts
- 4x RTX 3090 (used) on EPYC 7003, open frame~$2,178 + unpriced parts
- RTX 3090 (used) + 128 GB DDR5 on AM5 (MoE offload)~$4,833 + unpriced parts
Measured speeds
One card
| Model | Quant | Backend | Generation | Prompt | Context | Source |
|---|---|---|---|---|---|---|
| Gemma 4 26B-A4B IT | Q4_K | llama.cpp (build not stated) | 119 tok/s | 3,626 tok/s4,096-token prompt | 4,096 | review2026-03 |
| Gemma 4 31B IT | Q4_K | llama.cpp (build not stated) | 35 tok/s | 1,156 tok/s4,096-token prompt | 4,096 | review2026-03 |
| gpt-oss-20b | MXFP4 MoE | llama.cpp b6250 CUDA | 163 tok/s | 5,838 tok/s2,048-token prompt | 0 | llama.cpp benchmark discussion2025-08 |
| gpt-oss-20b | MXFP4 MoE | llama.cpp b6210 CUDA | 162 tok/s | 5,171 tok/s2,048-token prompt | 0 | llama.cpp benchmark discussion2025-08 |
| gpt-oss-20b | MXFP4 | llama.cpp (build not stated) | 148 tok/s | 4,400 tok/s4,096-token prompt | 4,096 | review2026-03 |
| gpt-oss-20b | MXFP4 MoE | llama.cpp b6250 CUDA | – | 57 tok/s2,048-token prompt | 0 | llama.cpp benchmark discussion2025-08 |
| Llama 3.1 8B Instruct | Q4_K_M | llama.cpp b11741 | 147 tok/s | – | 0 | review2026-10 |
| Llama 3.1 8B Instruct | Q4_K_M | LocalScore (llamafile; version not stated) | 96 tok/s | 3,550 tok/s | 0 | LocalScore2026-10 |
| Qwen3 8B | Q4_K | llama.cpp (build not stated) | 115 tok/s | 4,050 tok/s4,096-token prompt | 4,096 | review2026-03 |
2 cards
| Model | Quant | Backend | Generation | Prompt | Context | Source |
|---|---|---|---|---|---|---|
| Llama 3.3 70B Instruct | Q4_K_M | llama.cpp b11741 | 20 tok/s | 577 tok/s512-token prompt | 0 | review2026-10 |
Speculative decoding, several users, or long context
Kept apart because each measures a different workload from one person chatting near the start of a conversation.
| Model | Quant | Backend | Generation | Prompt | Context | Measured on | Source |
|---|---|---|---|---|---|---|---|
| Gemma 4 26B-A4B IT | Q4_K | llama.cpp (build not stated) | 115 tok/s | 3,069 tok/s16,384-token prompt | 16,384 | RTX 3090 | review2026-03 |
| Gemma 4 26B-A4B IT | Q4_K | llama.cpp (build not stated) | 108 tok/s | 2,453 tok/s32,768-token prompt | 32,768 | RTX 3090 | review2026-03 |
| Gemma 4 31B IT | Q4_K | llama.cpp (build not stated) | 34 tok/s | 913 tok/s16,384-token prompt | 16,384 | RTX 3090 | review2026-03 |
| Gemma 4 31B IT | Q4_K | llama.cpp (build not stated) | 31 tok/s | 724 tok/s32,768-token prompt | 32,768 | RTX 3090 | review2026-03 |
| Gemma 4 31B IT | AutoRound INT4 (int4_per_token_head KV) | vLLM v0.28.0 (patched) | 79 tok/s | – | 0 | RTX 3090 | independent benchmark2026-09 |
| Gemma 4 31B IT | AutoRound INT4 (int4_per_token_head KV) | vLLM v0.28.0 (patched) | 35 tok/s | – | 0 | RTX 3090 | independent benchmark2026-09 |
| gpt-oss-20b | MXFP4 | llama.cpp (build not stated) | 129 tok/s | 3,244 tok/s16,384-token prompt | 16,384 | RTX 3090 | review2026-03 |
| gpt-oss-20b | MXFP4 | llama.cpp (build not stated) | 113 tok/s | 2,547 tok/s32,768-token prompt | 32,768 | RTX 3090 | review2026-03 |
| Llama 3.3 70B Instruct | Q4_K_M | llama.cpp b11741 | 32 tok/s | 763 tok/s512-token prompt | 0 | 2× RTX 3090 | review2026-10 |
| Llama 3.3 70B Instruct | Q4_K_M | llama.cpp b11741 | 30 tok/s | 656 tok/s | 8,192 | 2× RTX 3090 | review2026-10 |
| Llama 3.3 70B Instruct | AWQ (casperhansen) | vLLM 0.31.0 | 31 tok/s | – | 0 | 2× RTX 3090 | review2026-10 |
| Qwen3 8B | Q4_K | llama.cpp (build not stated) | 88 tok/s | 2,573 tok/s16,384-token prompt | 16,384 | RTX 3090 | review2026-03 |
| Qwen3 8B | Q4_K | llama.cpp (build not stated) | 68 tok/s | 1,715 tok/s32,768-token prompt | 32,768 | RTX 3090 | review2026-03 |
| Qwen3.6 35B-A3B | Q5_K - Medium | llama.cpp b10809 CUDA | 156 tok/s | 4,059 tok/s512-token prompt | 0 | 4× RTX 3090 | llama.cpp benchmark discussion2026-09 |
| Qwen3.8 27B | AWQ4 (abliterated body, INT8 embed/head) | vLLM + DFlash2 + HyperQwen patches + LMCache | 153 tok/s | – | 18,649 | 2× RTX 3090 | independent benchmark2026-10 |
| Qwen3.8 27B | UD-Q4_K_M | llama.cpp b11382 (Windows CUDA 13) | 56 tok/s | – | 0 | RTX 3090 | independent benchmark2026-10 |
| Qwen3.8 27B | UD-Q4_K_M | llama.cpp b11382 (Windows CUDA 13)4 requests at once | 38 tok/s | 1,073 tok/s12,000-token prompt | 0 | RTX 3090 | independent benchmark2026-10 |
| Qwen3.8 27B | Q5_K - Medium | llama.cpp b10809 CUDA | 77 tok/s | 1,716 tok/s512-token prompt | 0 | 4× RTX 3090 | llama.cpp benchmark discussion2026-09 |
| Qwen3.8 27B | INT8 W8A16 | vLLM (club-3090 vllm-qwen38-27b-dual container) | 85 tok/s | 1,892 tok/s10,000-token prompt | 0 | 2× RTX 3090 | independent benchmark2026-08 |
| Qwen3.8 Flash-Next | UD-Q4_K_XL | llama.cpp fork (lukolszewski/llama.cpp-multigpu on df03399b8, CUDA) | 42 tok/s | 2,127 tok/s50,000-token prompt | 50,000 | 6× RTX 3090 | llama.cpp benchmark discussion2026-10 |
| Qwen3.8 Flash-Next | UD-Q4_K_XL | llama.cpp fork (lukolszewski/llama.cpp-multigpu on df03399b8, CUDA) | 34 tok/s | 2,111 tok/s250,000-token prompt | 250,000 | 6× RTX 3090 | llama.cpp benchmark discussion2026-10 |
| Qwen3.8 Flash-Next | UD-Q4_K_XL | llama.cpp fork (lukolszewski/llama.cpp-multigpu on df03399b8, CUDA)5 requests at once | 27 tok/s | – | 250,000 | 6× RTX 3090 | llama.cpp benchmark discussion2026-10 |
| Qwen3.8 Flash-Next | UD-Q4_K_XL | llama.cpp (upstream commit df03399b8, CUDA) | 26 tok/s | 568 tok/s50,000-token prompt | 50,000 | 6× RTX 3090 | llama.cpp benchmark discussion2026-10 |
| Qwen3.8 Flash-Next | UD-Q4_K_XL | llama.cpp (upstream commit df03399b8, CUDA) | 10 tok/s | 263 tok/s250,000-token prompt | 250,000 | 6× RTX 3090 | llama.cpp benchmark discussion2026-10 |
Specifications
- Memory
- 24 GB GDDR6X
- FP16 basis
- Dense FP16/BF16 tensor with FP32 accumulate (NVIDIA whitepaper). FP16 tensor with FP16 accumulate is 142.3 dense; 142.4/284.6 are the sparse figures.
- Board power
- 350 W
- Slots
- 3
- Length
- 313 mm
- PCIe
- Gen 4 x16
- NVLink
- Yes
- Cooling
- Open-air
- Software
- CUDA, Vulkan
- Released
- 2020
- Availability
- used only
- Price basis
- Not sold new by NVIDIA.
- Maker page
- nvidia.com
Sources
- nvidia.com: 24 GB GDDR6X, 384-bit, 350 W, FE 313 mm 3-slot, PCIe Gen 4, NVLink (SLI-ready) Yes
- images.nvidia.com: NVIDIA RTX Blackwell whitepaper appendix tables: memory, bandwidth, TGP, PCIe gen, dense/sparse FP16 tensor TFLOPS
- nvidia.com: launch price from $1,499