← All machines

Multi-GPU · DIY build

2x RTX 3090 (used)

Two used 3090s give 48 GB for 70B-class models at 4-bit. The 3090 is the newest GeForce card with an NVLink connector, which only matters for tensor-parallel serving.

Memory for models

46.5 GB

VRAM across all cards

Memory bandwidth

936 GB/s

per card; 2 cards take turns on each token

Parts

~$3,639 + unpriced parts

GPU and risers prices not included

Largest at 4-bit

Llama 3.3 70B Instruct

43.6 GB at 8,192 tokens of context

gpt-oss-20b, 4-bit

~117–171 tok/sestimate

no measurement on this hardware yet

Parts

PartQtyEach
GPUNVIDIA GeForce RTX 3090 (used)2not priced
RisersRTX 3090 NVLink bridge (optional, 3- or 4-slot spacing to match the board)1not priced
CPUAMD Ryzen 9 9950X (16-core, 170 W)1~$649
MotherboardASUS ProArt X870E-CREATOR WIFI (2x PCIe 5.0 x16 slots, x16 or x8/x8)1~$550
RAMCorsair Vengeance 64GB (2x32GB) DDR5-6000 CL301~$1,275
PSUCorsair HX1500i (1500 W at 115-240 V, 1200 W at 100-115 V)1~$390
CaseCorsair 7000D AIRFLOW full tower (450 mm GPU clearance, 8 slots)1~$290
StorageCorsair MP600 CORE XT 2TB PCIe 4.0 NVMe1~$485
Priced parts only$3,639GPU and risers prices not included

New parts are priced at the maker’s own store or list price. Used cards have no price here yet, and a total with a missing GPU would understate the build, so none is shown.

What it runs at 4-bit

Every model in the catalog at its 4-bit file with 8,192 tokens of context. 21 of 41 fit. Speeds are the median of measured runs on the same GPUs where any exist; otherwise an estimate from memory bandwidth. Try other quants and context lengths.

ModelNeedsFitGeneration speed
Kimi K32.8T, 104B active · UD-Q4_K_XL1406.8 GBcontext memory estimatedDoes not fit
DeepSeek V4 Pro1.6T, 49B active · Q4_K_M890.5 GBcontext memory not countedDoes not fit
Kimi K2.61T, 32B active · UD-Q4_K_XL545.7 GBcontext memory estimatedDoes not fit
DeepSeek V4.1 Flash552B, 16B active · Q4 (DwarfStar)484.5 GBDoes not fit
GLM-5.3753.3B · UD-Q4_K_XL437.4 GBcontext memory estimatedDoes not fit
DeepSeek V3.2671B, 37B active · Q4_K_M379.6 GBcontext memory estimatedDoes not fit
MiniMax M3428B, 23B active · Q4_K_M244.8 GBcontext memory not countedDoes not fit
Qwen3.5 397B-A17B397B, 17B active · Q4_K_M229.1 GBcontext memory estimatedDoes not fit
GLM-5.3 Flash320B, 18B active · UD-Q4_K_XL187.6 GBcontext memory estimatedDoes not fit
Hy3295B, 21B active · Q4_K_M173.6 GBDoes not fit
MiniMax M2.7228.7B · Q4_K_M132.7 GBDoes not fit
Qwen3.8 Flash-Next125B, 6B active · Q4_K_M113.1 GBcontext memory estimatedDoes not fit
Nemotron 3 Super 120B-A12B120B, 12B active · Q4_K_M82.6 GBcontext memory estimatedDoes not fit
Qwen3.5 122B-A10B122B, 10B active · Q4_K_M73.0 GBcontext memory estimatedDoes not fit
Mistral Medium 3.5 128B128B · Q4_K_M74.0 GBDoes not fit
GLM-4.5-Air106B, 12B active · Q4_K_M70.9 GBDoes not fit
Mistral Small 4 119B (2603)119B, 6.5B active · Q4_K_M69.3 GBcontext memory estimatedDoes not fit
Llama 4 Scout 17B-16E Instruct109B, 17B active · Q4_K_M62.4 GBcontext memory not countedDoes not fit
gpt-oss-120b117B, 5.1B active · MXFP4 (native)60.2 GBcontext memory estimatedDoes not fit
Qwen3 Coder Next80B, 3B active · Q4_K_M46.9 GBcontext memory estimatedDoes not fit
Llama 3.3 70B Instruct70B · Q4_K_M43.6 GBFits20 tok/smedian of 1 run
Nemotron 3.5 Lightning 30B-A3B30B, 3B active · Q4_K_M25.3 GBcontext memory estimatedFits~91–133 tok/sestimate
Qwen3.6 35B-A3B35B, 3B active · Q4_K_M22.4 GBcontext memory estimatedFits~122–178 tok/sestimate
Ornith 1.5 35B-A3B35B, 3B active · Q4_K_M21.9 GBcontext memory estimatedFits~125–183 tok/sestimate
Qwen3 Coder 30B-A3B Instruct30.5B, 3.3B active · Q4_K_M19.5 GBFits~116–169 tok/sestimate
Gemma 4 31B IT30.7B · Q4_K_M18.9 GBcontext memory estimatedFits~25–39 tok/sestimate
Granite 4.2 30B29.3B · Q4_K_M20.3 GBFits~25–40 tok/sestimate
Qwen3.8 27B27B · Q4_K_M18.2 GBcontext memory estimatedFits~26–41 tok/sestimate
Gemma 4 26B-A4B IT25.2B, 3.8B active · Q4_K_M17.4 GBcontext memory estimatedFits~91–132 tok/sestimate
Qwen3.6 27B27B · Q4_K_M17.7 GBcontext memory estimatedFits~27–43 tok/sestimate
Devstral Small 2 24B (2512)24B · Q4_K_M16.1 GBFits~32–50 tok/sestimate
gpt-oss-20b21B, 3.6B active · MXFP4 (native)12.5 GBcontext memory estimatedFits~117–171 tok/sestimate
Phi-414.7B · Q4_K_M11.3 GBFits~52–81 tok/sestimate
Ministral 3 14B Reasoning (2512)13.9B · Q4_K_M10.4 GBFits~56–87 tok/sestimate
Gemma 4 12B IT11.95B · Q4_K_M8.2 GBcontext memory estimatedFits~64–101 tok/sestimate
Qwen3.5 9B9B · Q4_K_M7.0 GBcontext memory estimatedFits~81–127 tok/sestimate
Granite 4.2 8B8.8B · Q4_K_M7.9 GBFits~83–130 tok/sestimate
Gemma 4 E4B IT8B · Q4_K_M6.5 GBcontext memory not countedFits~151–237 tok/sestimate
Qwen3 8B8.2B · Q4_K_M7.3 GBFits~91–143 tok/sestimate
Llama 3.1 8B Instruct8B · Q4_K_M7.1 GBFits~93–146 tok/sestimate
Qwen3.5 4B4B · Q4_K_M4.3 GBcontext memory estimatedFits~167–263 tok/sestimate

Measured speeds

1 published run with one user, plain decoding and at most 4,096 tokens already in context, on 2× RTX 3090.

Plain single-user runs
ModelQuantBackendGenerationPromptContextMeasured onSource
Llama 3.3 70B InstructQ4_K_Mllama.cpp b1174120 tok/s577 tok/s512-token prompt0this machinereview2026-10

With speculative decoding

A small draft model or extra prediction heads guess several tokens ahead. Generation is faster on text the guesses get right, so these are kept out of the medians above.

Runs with speculative decoding
ModelQuantBackendGenerationPromptContextMeasured onSource
Qwen3.8 27BAWQ4 (abliterated body, INT8 embed/head)vLLM + DFlash2 + HyperQwen patches + LMCache153 tok/s–18,649this machineindependent benchmark2026-10
Qwen3.8 27BINT8 W8A16vLLM (club-3090 vllm-qwen38-27b-dual container)85 tok/s1,892 tok/s10,000-token prompt0this machineindependent benchmark2026-08

Cards working in parallel

Tensor parallel splits every layer across the cards so they read their share of the weights at the same time, instead of taking turns. It can beat one card’s bandwidth limit, needs fast links between cards, and is kept out of the medians above.

Runs with tensor parallel
ModelQuantBackendGenerationPromptContextMeasured onSource
Llama 3.3 70B InstructQ4_K_Mllama.cpp b1174132 tok/s763 tok/s512-token prompt0this machinereview2026-10
Llama 3.3 70B InstructQ4_K_Mllama.cpp b1174130 tok/s656 tok/s8,192this machinereview2026-10
Llama 3.3 70B InstructAWQ (casperhansen)vLLM 0.31.031 tok/s–0this machinereview2026-10

Specifications

GPUs
2 × NVIDIA GeForce RTX 3090
VRAM
48 GB
System RAM
64 GB
RAM bandwidth
2 channels x 6000 MT/s x 8 bytes = 96.0 GB/s (dual-channel AM5)
Power supply
1500 W
Estimated load
2x RTX 3090 350 W + Ryzen 9 9950X 170 W + 100 W = 970 W
Software
CUDA, Vulkan

Before you build it

  • On the ProArt X870E-CREATOR both CPU-attached slots share 16 lanes: x16 with one card, x8/x8 with two.
  • NVLink does almost nothing for llama.cpp/Ollama layer splitting (only a small activation crosses per token); a published dual-3090 vLLM tensor-parallel test gained about 50% throughput with the bridge.
  • Open-air cards in adjacent slots blow exhaust straight into the next card's intake; check slot spacing (and NVLink bridge spacing) before buying the board.
  • Corsair rates the HX1500i at 1,500 W only on 115-240 V input; at 100-115 V it is a 1,200 W supply.
  • A US 15 A / 120 V circuit is 1,800 W on paper, but the NEC 80% rule for continuous loads (3 h+) limits it to 12 A, about 1,440 W, shared with everything else on that circuit.
  • DDR5 prices are at shortage highs: Corsair lists a 64 GB (2x32) DDR5-6000 kit at $1,274.99, so RAM can cost more than a used GPU.

Sources

Compared with

Similar machines