← All machines

Single GPU · DIY build

1x RTX 3090 (used)

The cheapest way to 24 GB of CUDA memory: one used RTX 3090 on a current AM5 desktop. It runs 24-32B dense models at 4-bit and small MoE models entirely on the GPU.

Memory for models

24 GB

VRAM

Memory bandwidth

936 GB/s

Parts

~$2,944 + unpriced parts

used GPU price not included

Largest at 4-bit

Qwen3.6 35B-A3B

22.4 GB at 8,192 tokens of context

gpt-oss-20b, 4-bit

162 tok/smedian of 3 runs

Parts

PartQtyEach
GPUNVIDIA GeForce RTX 3090 (used)1not priced
CPUAMD Ryzen 7 9700X (8-core, 65 W)1~$359
MotherboardASUS TUF GAMING X870E-PLUS WIFI71~$325
RAMCorsair Vengeance 64GB (2x32GB) DDR5-6000 CL301~$1,275
PSUCorsair RM1000x (1000 W)1~$210
CaseCorsair 7000D AIRFLOW full tower (450 mm GPU clearance, 8 slots)1~$290
StorageCorsair MP600 CORE XT 2TB PCIe 4.0 NVMe1~$485
Priced parts only$2,944used GPU price not included

New parts are priced at the maker’s own store or list price. Used cards have no price here yet, and a total with a missing GPU would understate the build, so none is shown.

What it runs at 4-bit

Every model in the catalog at its 4-bit file with 8,192 tokens of context. 19 of 41 fit. Speeds are the median of measured runs on the same GPUs where any exist; otherwise an estimate from memory bandwidth. Try other quants and context lengths.

ModelNeedsFitGeneration speed
Kimi K32.8T, 104B active · UD-Q4_K_XL1406.8 GBcontext memory estimatedDoes not fit
DeepSeek V4 Pro1.6T, 49B active · Q4_K_M890.5 GBcontext memory not countedDoes not fit
Kimi K2.61T, 32B active · UD-Q4_K_XL545.7 GBcontext memory estimatedDoes not fit
DeepSeek V4.1 Flash552B, 16B active · Q4 (DwarfStar)484.5 GBDoes not fit
GLM-5.3753.3B · UD-Q4_K_XL437.4 GBcontext memory estimatedDoes not fit
DeepSeek V3.2671B, 37B active · Q4_K_M379.6 GBcontext memory estimatedDoes not fit
MiniMax M3428B, 23B active · Q4_K_M244.8 GBcontext memory not countedDoes not fit
Qwen3.5 397B-A17B397B, 17B active · Q4_K_M229.1 GBcontext memory estimatedDoes not fit
GLM-5.3 Flash320B, 18B active · UD-Q4_K_XL187.6 GBcontext memory estimatedDoes not fit
Hy3295B, 21B active · Q4_K_M173.6 GBDoes not fit
MiniMax M2.7228.7B · Q4_K_M132.7 GBDoes not fit
Qwen3.8 Flash-Next125B, 6B active · Q4_K_M113.1 GBcontext memory estimatedDoes not fit
Nemotron 3 Super 120B-A12B120B, 12B active · Q4_K_M82.6 GBcontext memory estimatedDoes not fit
Qwen3.5 122B-A10B122B, 10B active · Q4_K_M73.0 GBcontext memory estimatedDoes not fit
Mistral Medium 3.5 128B128B · Q4_K_M74.0 GBDoes not fit
GLM-4.5-Air106B, 12B active · Q4_K_M70.9 GBDoes not fit
Mistral Small 4 119B (2603)119B, 6.5B active · Q4_K_M69.3 GBcontext memory estimatedDoes not fit
Llama 4 Scout 17B-16E Instruct109B, 17B active · Q4_K_M62.4 GBcontext memory not countedDoes not fit
gpt-oss-120b117B, 5.1B active · MXFP4 (native)60.2 GBcontext memory estimatedDoes not fit
Qwen3 Coder Next80B, 3B active · Q4_K_M46.9 GBcontext memory estimatedDoes not fit
Llama 3.3 70B Instruct70B · Q4_K_M43.6 GBDoes not fit
Nemotron 3.5 Lightning 30B-A3B30B, 3B active · Q4_K_M25.3 GBcontext memory estimatedDoes not fit
Qwen3.6 35B-A3B35B, 3B active · Q4_K_M22.4 GBcontext memory estimatedFits~122–178 tok/sestimate
Ornith 1.5 35B-A3B35B, 3B active · Q4_K_M21.9 GBcontext memory estimatedFits~125–183 tok/sestimate
Qwen3 Coder 30B-A3B Instruct30.5B, 3.3B active · Q4_K_M19.5 GBFits~116–169 tok/sestimate
Gemma 4 31B IT30.7B · Q4_K_M18.9 GBcontext memory estimatedFits35 tok/smedian of 1 run
Granite 4.2 30B29.3B · Q4_K_M20.3 GBFits~25–40 tok/sestimate
Qwen3.8 27B27B · Q4_K_M18.2 GBcontext memory estimatedFits~26–41 tok/sestimate
Gemma 4 26B-A4B IT25.2B, 3.8B active · Q4_K_M17.4 GBcontext memory estimatedFits119 tok/smedian of 1 run
Qwen3.6 27B27B · Q4_K_M17.7 GBcontext memory estimatedFits~27–43 tok/sestimate
Devstral Small 2 24B (2512)24B · Q4_K_M16.1 GBFits~32–50 tok/sestimate
gpt-oss-20b21B, 3.6B active · MXFP4 (native)12.5 GBcontext memory estimatedFits162 tok/smedian of 3 runs
Phi-414.7B · Q4_K_M11.3 GBFits~52–81 tok/sestimate
Ministral 3 14B Reasoning (2512)13.9B · Q4_K_M10.4 GBFits~56–87 tok/sestimate
Gemma 4 12B IT11.95B · Q4_K_M8.2 GBcontext memory estimatedFits~64–101 tok/sestimate
Qwen3.5 9B9B · Q4_K_M7.0 GBcontext memory estimatedFits~81–127 tok/sestimate
Granite 4.2 8B8.8B · Q4_K_M7.9 GBFits~83–130 tok/sestimate
Gemma 4 E4B IT8B · Q4_K_M6.5 GBcontext memory not countedFits~151–237 tok/sestimate
Qwen3 8B8.2B · Q4_K_M7.3 GBFits115 tok/smedian of 1 run
Llama 3.1 8B Instruct8B · Q4_K_M7.1 GBFits122 tok/smedian of 2 runs
Qwen3.5 4B4B · Q4_K_M4.3 GBcontext memory estimatedFits~167–263 tok/sestimate

Measured speeds

9 published runs with one user, plain decoding and at most 4,096 tokens already in context, on RTX 3090. Runs on RTX 3090 (used) + 128 GB DDR5 on AM5 (MoE offload) with the same GPUs count here too, and say so.

Plain single-user runs
ModelQuantBackendGenerationPromptContextMeasured onSource
Gemma 4 26B-A4B ITQ4_Kllama.cpp (build not stated)119 tok/s3,626 tok/s4,096-token prompt4,096RTX 3090review2026-03
Gemma 4 31B ITQ4_Kllama.cpp (build not stated)35 tok/s1,156 tok/s4,096-token prompt4,096RTX 3090review2026-03
gpt-oss-20bMXFP4 MoEllama.cpp b6250 CUDA163 tok/s5,838 tok/s2,048-token prompt0RTX 3090llama.cpp benchmark discussion2025-08
gpt-oss-20bMXFP4 MoEllama.cpp b6210 CUDA162 tok/s5,171 tok/s2,048-token prompt0RTX 3090llama.cpp benchmark discussion2025-08
gpt-oss-20bMXFP4llama.cpp (build not stated)148 tok/s4,400 tok/s4,096-token prompt4,096RTX 3090review2026-03
gpt-oss-20bMXFP4 MoEllama.cpp b6250 CUDA–57 tok/s2,048-token prompt0RTX 3090llama.cpp benchmark discussion2025-08
Llama 3.1 8B InstructQ4_K_Mllama.cpp b11741147 tok/s–0RTX 3090review2026-10
Llama 3.1 8B InstructQ4_K_MLocalScore (llamafile; version not stated)96 tok/s3,550 tok/s0RTX 3090LocalScore2026-10
Qwen3 8BQ4_Kllama.cpp (build not stated)115 tok/s4,050 tok/s4,096-token prompt4,096RTX 3090review2026-03

With speculative decoding

A small draft model or extra prediction heads guess several tokens ahead. Generation is faster on text the guesses get right, so these are kept out of the medians above.

Runs with speculative decoding
ModelQuantBackendGenerationPromptContextMeasured onSource
Gemma 4 31B ITAutoRound INT4 (int4_per_token_head KV)vLLM v0.28.0 (patched)79 tok/s–0RTX 3090independent benchmark2026-09
Gemma 4 31B ITAutoRound INT4 (int4_per_token_head KV)vLLM v0.28.0 (patched)35 tok/s–0RTX 3090independent benchmark2026-09
Qwen3.8 27BUD-Q4_K_Mllama.cpp b11382 (Windows CUDA 13)56 tok/s–0RTX 3090independent benchmark2026-10

Many users at once

Total throughput with several requests in flight. One person chatting sees far less than this.

Runs with several requests at once
ModelQuantBackendGenerationPromptContextMeasured onSource
Qwen3.8 27BUD-Q4_K_Mllama.cpp b11382 (Windows CUDA 13)4 requests at once38 tok/s1,073 tok/s12,000-token prompt0RTX 3090independent benchmark2026-10

Deep into a long context

Measured with more than 4,096 tokens already in context, where every new token also reads the context memory.

Runs at long context
ModelQuantBackendGenerationPromptContextMeasured onSource
Gemma 4 26B-A4B ITQ4_Kllama.cpp (build not stated)115 tok/s3,069 tok/s16,384-token prompt16,384RTX 3090review2026-03
Gemma 4 26B-A4B ITQ4_Kllama.cpp (build not stated)108 tok/s2,453 tok/s32,768-token prompt32,768RTX 3090review2026-03
Gemma 4 31B ITQ4_Kllama.cpp (build not stated)34 tok/s913 tok/s16,384-token prompt16,384RTX 3090review2026-03
Gemma 4 31B ITQ4_Kllama.cpp (build not stated)31 tok/s724 tok/s32,768-token prompt32,768RTX 3090review2026-03
gpt-oss-20bMXFP4llama.cpp (build not stated)129 tok/s3,244 tok/s16,384-token prompt16,384RTX 3090review2026-03
gpt-oss-20bMXFP4llama.cpp (build not stated)113 tok/s2,547 tok/s32,768-token prompt32,768RTX 3090review2026-03
Qwen3 8BQ4_Kllama.cpp (build not stated)88 tok/s2,573 tok/s16,384-token prompt16,384RTX 3090review2026-03
Qwen3 8BQ4_Kllama.cpp (build not stated)68 tok/s1,715 tok/s32,768-token prompt32,768RTX 3090review2026-03

Specifications

GPUs
1 × NVIDIA GeForce RTX 3090
VRAM
24 GB
System RAM
64 GB
RAM bandwidth
2 channels x 6000 MT/s x 8 bytes = 96.0 GB/s (dual-channel AM5)
Power supply
1000 W
Estimated load
RTX 3090 350 W + Ryzen 7 9700X 65 W + 100 W = 515 W
Software
CUDA, Vulkan

Before you build it

  • DDR5 prices are at shortage highs: Corsair lists a 64 GB (2x32) DDR5-6000 kit at $1,274.99, so RAM can cost more than a used GPU.
  • NVIDIA's own requirement for an RTX 3090 system is a 750 W PSU; the Founders Edition is 3 slots and 313 mm long.

Sources

  • shop-us-en.amd.com: price of AMD Ryzen 7 9700X (8-core, 65 W)
  • eshop.asus.com: price of ASUS TUF GAMING X870E-PLUS WIFI7
  • corsair.com: price of Corsair Vengeance 64GB (2x32GB) DDR5-6000 CL30
  • corsair.com: price of Corsair RM1000x (1000 W)
  • corsair.com: price of Corsair 7000D AIRFLOW full tower (450 mm GPU clearance, 8 slots)
  • corsair.com: price of Corsair MP600 CORE XT 2TB PCIe 4.0 NVMe
  • nvidia.com: RTX 3090 350 W, 3-slot 313 mm FE, 750 W system PSU

Compared with

Similar machines