← All machines

Hybrid GPU + RAM · DIY build

RTX 3090 (used) + 128 GB DDR5 on AM5 (MoE offload)

Budget MoE offload: a used 3090 holds attention and KV cache while 128 GB of desktop DDR5 holds the experts, enough for ~100B-class MoE models at 4-bit.

Memory for models

24 GB

VRAM

Memory bandwidth

936 GB/s

Parts

~$4,833 + unpriced parts

used GPU price not included

Largest at 4-bit

Qwen3.6 35B-A3B

22.4 GB at 8,192 tokens of context

gpt-oss-20b, 4-bit

162 tok/smedian of 3 runs

Parts

PartQtyEach
GPUNVIDIA GeForce RTX 3090 (used)1not priced
CPUAMD Ryzen 9 9950X (16-core, 170 W)1~$649
MotherboardASUS TUF GAMING X870E-PLUS WIFI71~$325
RAMCorsair Vengeance 128GB (2x64GB) DDR5-6400 CL42 + HX1200i 1200 W PSU bundle1~$3,084
PSUCorsair HX1200i (1200 W), included in the RAM bundle above1~$0
CaseCorsair 7000D AIRFLOW full tower (450 mm GPU clearance, 8 slots)1~$290
StorageCorsair MP600 CORE XT 2TB PCIe 4.0 NVMe1~$485
Priced parts only$4,833used GPU price not included

New parts are priced at the maker’s own store or list price. Used cards have no price here yet, and a total with a missing GPU would understate the build, so none is shown.

What it runs at 4-bit

Every model in the catalog at its 4-bit file with 8,192 tokens of context. 31 of 41 fit. Speeds are the median of measured runs on the same GPUs where any exist; otherwise an estimate from memory bandwidth. Try other quants and context lengths.

ModelNeedsFitGeneration speed
Kimi K32.8T, 104B active · UD-Q4_K_XL1406.8 GBcontext memory estimatedDoes not fit
DeepSeek V4 Pro1.6T, 49B active · Q4_K_M890.5 GBcontext memory not countedDoes not fit
Kimi K2.61T, 32B active · UD-Q4_K_XL545.7 GBcontext memory estimatedDoes not fit
DeepSeek V4.1 Flash552B, 16B active · Q4 (DwarfStar)484.5 GBDoes not fit
GLM-5.3753.3B · UD-Q4_K_XL437.4 GBcontext memory estimatedDoes not fit
DeepSeek V3.2671B, 37B active · Q4_K_M379.6 GBcontext memory estimatedDoes not fit
MiniMax M3428B, 23B active · Q4_K_M244.8 GBcontext memory not countedDoes not fit
Qwen3.5 397B-A17B397B, 17B active · Q4_K_M229.1 GBcontext memory estimatedDoes not fit
GLM-5.3 Flash320B, 18B active · UD-Q4_K_XL187.6 GBcontext memory estimatedDoes not fit
Hy3295B, 21B active · Q4_K_M173.6 GBDoes not fit
MiniMax M2.7228.7B · Q4_K_M132.7 GBGPU + RAM offloadassumed limit–
Qwen3.8 Flash-Next125B, 6B active · Q4_K_M113.1 GBcontext memory estimatedGPU + RAM offloadassumed limit~5.4–7.9 tok/sestimate, part in RAM
Nemotron 3 Super 120B-A12B120B, 12B active · Q4_K_M82.6 GBcontext memory estimatedGPU + RAM offloadassumed limit~3.9–5.7 tok/sestimate, part in RAM
Qwen3.5 122B-A10B122B, 10B active · Q4_K_M73.0 GBcontext memory estimatedGPU + RAM offloadassumed limit~5.6–8.2 tok/sestimate, part in RAM
Mistral Medium 3.5 128B128B · Q4_K_M74.0 GBGPU + RAM offloadassumed limit~0.9–1.4 tok/sestimate, part in RAM
GLM-4.5-Air106B, 12B active · Q4_K_M70.9 GBGPU + RAM offloadassumed limit~4.3–6.2 tok/sestimate, part in RAM
Mistral Small 4 119B (2603)119B, 6.5B active · Q4_K_M69.3 GBcontext memory estimatedGPU + RAM offloadassumed limit~9.1–13 tok/sestimate, part in RAM
Llama 4 Scout 17B-16E Instruct109B, 17B active · Q4_K_M62.4 GBcontext memory not countedGPU + RAM offloadassumed limit~3.7–5.4 tok/sestimate, part in RAM
gpt-oss-120b117B, 5.1B active · MXFP4 (native)60.2 GBcontext memory estimatedGPU + RAM offloadassumed limit~14–21 tok/sestimate, part in RAM
Qwen3 Coder Next80B, 3B active · Q4_K_M46.9 GBcontext memory estimatedGPU + RAM offloadassumed limit~25–36 tok/sestimate, part in RAM
Llama 3.3 70B Instruct70B · Q4_K_M43.6 GBGPU + RAM offloadassumed limit~2.1–3.4 tok/sestimate, part in RAM
Nemotron 3.5 Lightning 30B-A3B30B, 3B active · Q4_K_M25.3 GBcontext memory estimatedGPU + RAM offloadassumed limit~64–93 tok/sestimate, part in RAM
Qwen3.6 35B-A3B35B, 3B active · Q4_K_M22.4 GBcontext memory estimatedFits~122–178 tok/sestimate
Ornith 1.5 35B-A3B35B, 3B active · Q4_K_M21.9 GBcontext memory estimatedFits~125–183 tok/sestimate
Qwen3 Coder 30B-A3B Instruct30.5B, 3.3B active · Q4_K_M19.5 GBFits~116–169 tok/sestimate
Gemma 4 31B IT30.7B · Q4_K_M18.9 GBcontext memory estimatedFits35 tok/smedian of 1 run
Granite 4.2 30B29.3B · Q4_K_M20.3 GBFits~25–40 tok/sestimate
Qwen3.8 27B27B · Q4_K_M18.2 GBcontext memory estimatedFits~26–41 tok/sestimate
Gemma 4 26B-A4B IT25.2B, 3.8B active · Q4_K_M17.4 GBcontext memory estimatedFits119 tok/smedian of 1 run
Qwen3.6 27B27B · Q4_K_M17.7 GBcontext memory estimatedFits~27–43 tok/sestimate
Devstral Small 2 24B (2512)24B · Q4_K_M16.1 GBFits~32–50 tok/sestimate
gpt-oss-20b21B, 3.6B active · MXFP4 (native)12.5 GBcontext memory estimatedFits162 tok/smedian of 3 runs
Phi-414.7B · Q4_K_M11.3 GBFits~52–81 tok/sestimate
Ministral 3 14B Reasoning (2512)13.9B · Q4_K_M10.4 GBFits~56–87 tok/sestimate
Gemma 4 12B IT11.95B · Q4_K_M8.2 GBcontext memory estimatedFits~64–101 tok/sestimate
Qwen3.5 9B9B · Q4_K_M7.0 GBcontext memory estimatedFits~81–127 tok/sestimate
Granite 4.2 8B8.8B · Q4_K_M7.9 GBFits~83–130 tok/sestimate
Gemma 4 E4B IT8B · Q4_K_M6.5 GBcontext memory not countedFits~151–237 tok/sestimate
Qwen3 8B8.2B · Q4_K_M7.3 GBFits115 tok/smedian of 1 run
Llama 3.1 8B Instruct8B · Q4_K_M7.1 GBFits122 tok/smedian of 2 runs
Qwen3.5 4B4B · Q4_K_M4.3 GBcontext memory estimatedFits~167–263 tok/sestimate

Measured speeds

9 published runs with one user, plain decoding and at most 4,096 tokens already in context, on RTX 3090. Runs on 1x RTX 3090 (used) with the same GPUs count here too, and say so.

Plain single-user runs
ModelQuantBackendGenerationPromptContextMeasured onSource
Gemma 4 26B-A4B ITQ4_Kllama.cpp (build not stated)119 tok/s3,626 tok/s4,096-token prompt4,096RTX 3090review2026-03
Gemma 4 31B ITQ4_Kllama.cpp (build not stated)35 tok/s1,156 tok/s4,096-token prompt4,096RTX 3090review2026-03
gpt-oss-20bMXFP4 MoEllama.cpp b6250 CUDA163 tok/s5,838 tok/s2,048-token prompt0RTX 3090llama.cpp benchmark discussion2025-08
gpt-oss-20bMXFP4 MoEllama.cpp b6210 CUDA162 tok/s5,171 tok/s2,048-token prompt0RTX 3090llama.cpp benchmark discussion2025-08
gpt-oss-20bMXFP4llama.cpp (build not stated)148 tok/s4,400 tok/s4,096-token prompt4,096RTX 3090review2026-03
gpt-oss-20bMXFP4 MoEllama.cpp b6250 CUDA–57 tok/s2,048-token prompt0RTX 3090llama.cpp benchmark discussion2025-08
Llama 3.1 8B InstructQ4_K_Mllama.cpp b11741147 tok/s–0RTX 3090review2026-10
Llama 3.1 8B InstructQ4_K_MLocalScore (llamafile; version not stated)96 tok/s3,550 tok/s0RTX 3090LocalScore2026-10
Qwen3 8BQ4_Kllama.cpp (build not stated)115 tok/s4,050 tok/s4,096-token prompt4,096RTX 3090review2026-03

With speculative decoding

A small draft model or extra prediction heads guess several tokens ahead. Generation is faster on text the guesses get right, so these are kept out of the medians above.

Runs with speculative decoding
ModelQuantBackendGenerationPromptContextMeasured onSource
Gemma 4 31B ITAutoRound INT4 (int4_per_token_head KV)vLLM v0.28.0 (patched)79 tok/s–0RTX 3090independent benchmark2026-09
Gemma 4 31B ITAutoRound INT4 (int4_per_token_head KV)vLLM v0.28.0 (patched)35 tok/s–0RTX 3090independent benchmark2026-09
Qwen3.8 27BUD-Q4_K_Mllama.cpp b11382 (Windows CUDA 13)56 tok/s–0RTX 3090independent benchmark2026-10

Many users at once

Total throughput with several requests in flight. One person chatting sees far less than this.

Runs with several requests at once
ModelQuantBackendGenerationPromptContextMeasured onSource
Qwen3.8 27BUD-Q4_K_Mllama.cpp b11382 (Windows CUDA 13)4 requests at once38 tok/s1,073 tok/s12,000-token prompt0RTX 3090independent benchmark2026-10

Deep into a long context

Measured with more than 4,096 tokens already in context, where every new token also reads the context memory.

Runs at long context
ModelQuantBackendGenerationPromptContextMeasured onSource
Gemma 4 26B-A4B ITQ4_Kllama.cpp (build not stated)115 tok/s3,069 tok/s16,384-token prompt16,384RTX 3090review2026-03
Gemma 4 26B-A4B ITQ4_Kllama.cpp (build not stated)108 tok/s2,453 tok/s32,768-token prompt32,768RTX 3090review2026-03
Gemma 4 31B ITQ4_Kllama.cpp (build not stated)34 tok/s913 tok/s16,384-token prompt16,384RTX 3090review2026-03
Gemma 4 31B ITQ4_Kllama.cpp (build not stated)31 tok/s724 tok/s32,768-token prompt32,768RTX 3090review2026-03
gpt-oss-20bMXFP4llama.cpp (build not stated)129 tok/s3,244 tok/s16,384-token prompt16,384RTX 3090review2026-03
gpt-oss-20bMXFP4llama.cpp (build not stated)113 tok/s2,547 tok/s32,768-token prompt32,768RTX 3090review2026-03
Qwen3 8BQ4_Kllama.cpp (build not stated)88 tok/s2,573 tok/s16,384-token prompt16,384RTX 3090review2026-03
Qwen3 8BQ4_Kllama.cpp (build not stated)68 tok/s1,715 tok/s32,768-token prompt32,768RTX 3090review2026-03

Specifications

GPUs
1 × NVIDIA GeForce RTX 3090
VRAM
24 GB
System RAM
128 GB
RAM bandwidth
2 channels x 6400 MT/s x 8 bytes = 102.4 GB/s (dual-channel AM5)
Power supply
1200 W
Estimated load
RTX 3090 350 W + Ryzen 9 9950X 170 W + 100 W = 620 W
Software
CUDA, Vulkan

Before you build it

  • llama.cpp keeps MoE expert weights in system RAM with --cpu-moe (all layers) or --n-cpu-moe N (first N layers), leaving attention and shared weights on the GPU.
  • AM5 is dual-channel: ~102 GB/s here versus ~614 GB/s on 12-channel EPYC, so offloaded experts generate several times slower.
  • Two 64 GB DIMMs keep the 128 GB at rated speed; ASUS notes supported data rate depends on the number of modules fitted.
  • DDR5 prices are at shortage highs: Corsair lists a 64 GB (2x32) DDR5-6000 kit at $1,274.99, so RAM can cost more than a used GPU.

Sources

Compared with

Similar machines