NVIDIA · Consumer · Ada Lovelace (AD102)

NVIDIA GeForce RTX 4090

No NVLink; FE is 3-slot (61 mm) and 304 mm long.

VRAM

24 GB

GDDR6X

Memory bandwidth

1,008 GB/s

FP16 compute

165.2 TFLOPS

dense tensor

Price

$1,599

launch price; sold used now, used price not tracked yet

What fits at 4-bit

The 4-bit file plus 8,192 tokens of context and 1.5 GB of overhead, against the VRAM of 1 and 6 cards. Speeds are measured medians where someone has published a run at that card count; a blank speed means no run, not a slow one. Machine pages carry estimates for the gaps.

Model	Needs	1 card (24 GB)	6 cards (144 GB)
Qwen3.5 4B4B · Q4_K_M	4.3 GB	fits	fits
Llama 3.1 8B Instruct8B · Q4_K_M	7.1 GB	105 tok/smedian of 2 runs	fits
Qwen3 8B8.2B · Q4_K_M	7.3 GB	141 tok/smedian of 1 run	fits
Gemma 4 E4B IT8B · Q4_K_M	6.5 GB	fits	fits
Granite 4.2 8B8.8B · Q4_K_M	7.9 GB	fits	fits
Qwen3.5 9B9B · Q4_K_M	7.0 GB	fits	fits
Gemma 4 12B IT11.95B · Q4_K_M	8.2 GB	fits	fits
Ministral 3 14B Reasoning (2512)13.9B · Q4_K_M	10.4 GB	fits	fits
Phi-414.7B · Q4_K_M	11.3 GB	fits	fits
gpt-oss-20b21B, 3.6B active · MXFP4 (native)	12.5 GB	238 tok/smedian of 4 runs	fits
Devstral Small 2 24B (2512)24B · Q4_K_M	16.1 GB	fits	fits
Qwen3.6 27B27B · Q4_K_M	17.7 GB	fits	fits
Gemma 4 26B-A4B IT25.2B, 3.8B active · Q4_K_M	17.4 GB	fits	fits
Qwen3.8 27B27B · Q4_K_M	18.2 GB	fits	fits
Granite 4.2 30B29.3B · Q4_K_M	20.3 GB	fits	fits
Gemma 4 31B IT30.7B · Q4_K_M	18.9 GB	fits	fits
Qwen3 Coder 30B-A3B Instruct30.5B, 3.3B active · Q4_K_M	19.5 GB	fits	fits
Ornith 1.5 35B-A3B35B, 3B active · Q4_K_M	21.9 GB	fits	fits
Qwen3.6 35B-A3B35B, 3B active · Q4_K_M	22.4 GB	fits	fits
Nemotron 3.5 Lightning 30B-A3B30B, 3B active · Q4_K_M	25.3 GB	too large	fits
Llama 3.3 70B Instruct70B · Q4_K_M	43.6 GB	too large	fits
Qwen3 Coder Next80B, 3B active · Q4_K_M	46.9 GB	too large	fits
gpt-oss-120b117B, 5.1B active · MXFP4 (native)	60.2 GB	too large	fits
Llama 4 Scout 17B-16E Instruct109B, 17B active · Q4_K_M	62.4 GB	too large	fits
Mistral Small 4 119B (2603)119B, 6.5B active · Q4_K_M	69.3 GB	too large	fits
GLM-4.5-Air106B, 12B active · Q4_K_M	70.9 GB	too large	fits
Mistral Medium 3.5 128B128B · Q4_K_M	74.0 GB	too large	fits
Qwen3.5 122B-A10B122B, 10B active · Q4_K_M	73.0 GB	too large	fits
Nemotron 3 Super 120B-A12B120B, 12B active · Q4_K_M	82.6 GB	too large	fits
Qwen3.8 Flash-Next125B, 6B active · Q4_K_M	113.1 GB	too large	fits
MiniMax M2.7228.7B · Q4_K_M	132.7 GB	too large	fits

Measured speeds

One card

Single-user runs on 1 card
Model	Quant	Backend	Generation	Prompt	Context	Source
gpt-oss-20b	MXFP4 MoE	llama.cpp b8244 CUDA	258 tok/s	–	0	llama.cpp benchmark discussion2026-03
gpt-oss-20b	MXFP4 MoE	llama.cpp b8244 CUDA	255 tok/s	10,366 tok/s2,048-token prompt	0	llama.cpp benchmark discussion2026-03
gpt-oss-20b	MXFP4 MoE	llama.cpp b6210 CUDA	222 tok/s	8,022 tok/s2,048-token prompt	0	llama.cpp benchmark discussion2025-08
gpt-oss-20b	MXFP4	llama.cpp (build not stated)	191 tok/s	8,369 tok/s4,096-token prompt	4,096	review2026-03
Llama 3.1 8B Instruct	Q4_K_M	LocalScore (llamafile; version not stated on leaderboard)	120 tok/s	7,595 tok/s	0	LocalScore2026-10
Llama 3.1 8B Instruct	Q4_K_M	LocalScore (llamafile; version not stated on leaderboard)	91 tok/s	6,697 tok/s	0	LocalScore2026-10
Qwen3 8B	Q4_K	llama.cpp (build not stated)	141 tok/s	9,251 tok/s4,096-token prompt	4,096	review2026-03

Speculative decoding, several users, or long context

Kept apart because each measures a different workload from one person chatting near the start of a conversation.

Other runs on this card
Model	Quant	Backend	Generation	Prompt	Context	Measured on	Source
gpt-oss-20b	MXFP4	llama.cpp (build not stated)	163 tok/s	5,856 tok/s16,384-token prompt	16,384	RTX 4090	review2026-03
gpt-oss-20b	MXFP4	llama.cpp (build not stated)	141 tok/s	3,706 tok/s32,768-token prompt	32,768	RTX 4090	review2026-03
Qwen3 8B	Q4_K	llama.cpp (build not stated)	108 tok/s	5,531 tok/s16,384-token prompt	16,384	RTX 4090	review2026-03
Qwen3 8B	Q4_K	llama.cpp (build not stated)	82 tok/s	3,560 tok/s32,768-token prompt	32,768	RTX 4090	review2026-03
Qwen3.8 Flash-Next	UD-Q4_K_XL	llama.cpp fork (lukolszewski/llama.cpp-multigpu on df03399b8, CUDA)	58 tok/s	7,050 tok/s50,000-token prompt	50,000	6× RTX 4090	llama.cpp benchmark discussion2026-10
Qwen3.8 Flash-Next	UD-Q4_K_XL	llama.cpp fork (lukolszewski/llama.cpp-multigpu on df03399b8, CUDA)	49 tok/s	7,403 tok/s250,000-token prompt	250,000	6× RTX 4090	llama.cpp benchmark discussion2026-10
Qwen3.8 Flash-Next	UD-Q4_K_XL	llama.cpp (upstream commit df03399b8, CUDA)	46 tok/s	1,728 tok/s50,000-token prompt	50,000	6× RTX 4090	llama.cpp benchmark discussion2026-10
Qwen3.8 Flash-Next	UD-Q4_K_XL	llama.cpp fork (lukolszewski/llama.cpp-multigpu on df03399b8, CUDA)5 requests at once	31 tok/s	–	250,000	6× RTX 4090	llama.cpp benchmark discussion2026-10
Qwen3.8 Flash-Next	UD-Q4_K_XL	llama.cpp (upstream commit df03399b8, CUDA)	21 tok/s	744 tok/s250,000-token prompt	250,000	6× RTX 4090	llama.cpp benchmark discussion2026-10

Specifications

Memory: 24 GB GDDR6X
FP16 basis: Dense FP16/BF16 tensor with FP32 accumulate; FP16 accumulate 330.3 dense, 660.6 sparse (NVIDIA whitepapers).
Board power: 450 W
Slots: 3
Length: 304 mm
PCIe: Gen 4 x16
NVLink: No
Cooling: Open-air
Software: CUDA, Vulkan
Released: 2022
Availability: used only
Price basis: Production ended in October 2024 per trade reports; not sold new by NVIDIA.
Maker page: nvidia.com

Sources

nvidia.com: 24 GB GDDR6X, 450 W TGP, FE 304 mm 3-slot, PCIe Gen 4, NVLink No
images.nvidia.com: NVIDIA RTX Blackwell whitepaper appendix tables: memory, bandwidth, TGP, PCIe gen, dense/sparse FP16 tensor TFLOPS
nvidia.com: launch price from $1,599
tweaktown.com: production end report