← All machines

Multi-GPU · DIY build

4x Instinct MI50 32GB (used) on EPYC 7003

128 GB of 1 TB/s HBM2 for the price of used datacenter cards, at the cost of no fans, no official ROCm and ~1.5 kW. Built for big MoE models that need memory more than compute.

Memory for models

123.5 GB

VRAM across all cards

Memory bandwidth

1,024 GB/s

per card; 4 cards take turns on each token

Parts

~$1,998 + unpriced parts

GPU, motherboard, ram, cooling and case prices not included

Largest at 4-bit

Qwen3.8 Flash-Next

113.1 GB at 8,192 tokens of context

gpt-oss-20b, 4-bit

~128–187 tok/sestimate

no measurement on this hardware yet

Parts

PartQtyEach
GPUAMD Instinct MI50 32GB (used)4not priced
CPUAMD EPYC 7313P (16-core, 155 W, SP3)1~$913
MotherboardASRock Rack ROMED8-2T (7x PCIe 4.0 x16)1not priced
RAM8x 32GB DDR4-3200 ECC RDIMM8not priced
PSUCorsair HX1500i (1500 W at 115-240 V, 1200 W at 100-115 V)1~$390
PSUCorsair RM1000x (1000 W)1~$210
CoolingBlower fans with 3D-printed shrouds, one per MI504not priced
CaseOpen-air frame1not priced
CoolingSP3 tower cooler1not priced
StorageCorsair MP600 CORE XT 2TB PCIe 4.0 NVMe1~$485
Priced parts only$1,998GPU, motherboard, ram, cooling and case prices not included

New parts are priced at the maker’s own store or list price. Used cards have no price here yet, and a total with a missing GPU would understate the build, so none is shown.

What it runs at 4-bit

Every model in the catalog at its 4-bit file with 8,192 tokens of context. 30 of 41 fit. Speeds are the median of measured runs on the same GPUs where any exist; otherwise an estimate from memory bandwidth. Try other quants and context lengths.

ModelNeedsFitGeneration speed
Kimi K32.8T, 104B active · UD-Q4_K_XL1406.8 GBcontext memory estimatedDoes not fit
DeepSeek V4 Pro1.6T, 49B active · Q4_K_M890.5 GBcontext memory not countedDoes not fit
Kimi K2.61T, 32B active · UD-Q4_K_XL545.7 GBcontext memory estimatedDoes not fit
DeepSeek V4.1 Flash552B, 16B active · Q4 (DwarfStar)484.5 GBDoes not fit
GLM-5.3753.3B · UD-Q4_K_XL437.4 GBcontext memory estimatedDoes not fit
DeepSeek V3.2671B, 37B active · Q4_K_M379.6 GBcontext memory estimatedDoes not fit
MiniMax M3428B, 23B active · Q4_K_M244.8 GBcontext memory not countedDoes not fit
Qwen3.5 397B-A17B397B, 17B active · Q4_K_M229.1 GBcontext memory estimatedDoes not fit
GLM-5.3 Flash320B, 18B active · UD-Q4_K_XL187.6 GBcontext memory estimatedDoes not fit
Hy3295B, 21B active · Q4_K_M173.6 GBDoes not fit
MiniMax M2.7228.7B · Q4_K_M132.7 GBDoes not fit
Qwen3.8 Flash-Next125B, 6B active · Q4_K_M113.1 GBcontext memory estimatedFits49 tok/smedian of 1 run
Nemotron 3 Super 120B-A12B120B, 12B active · Q4_K_M82.6 GBcontext memory estimatedFits~29–43 tok/sestimate
Qwen3.5 122B-A10B122B, 10B active · Q4_K_M73.0 GBcontext memory estimatedFits~41–59 tok/sestimate
Mistral Medium 3.5 128B128B · Q4_K_M74.0 GBFits~6.7–11 tok/sestimate
GLM-4.5-Air106B, 12B active · Q4_K_M70.9 GBFits~31–45 tok/sestimate
Mistral Small 4 119B (2603)119B, 6.5B active · Q4_K_M69.3 GBcontext memory estimatedFits~64–94 tok/sestimate
Llama 4 Scout 17B-16E Instruct109B, 17B active · Q4_K_M62.4 GBcontext memory not countedFits~25–36 tok/sestimate
gpt-oss-120b117B, 5.1B active · MXFP4 (native)60.2 GBcontext memory estimatedFits~93–136 tok/sestimate
Qwen3 Coder Next80B, 3B active · Q4_K_M46.9 GBcontext memory estimatedFits~140–204 tok/sestimate
Llama 3.3 70B Instruct70B · Q4_K_M43.6 GBFits~12–19 tok/sestimate
Nemotron 3.5 Lightning 30B-A3B30B, 3B active · Q4_K_M25.3 GBcontext memory estimatedFits~100–146 tok/sestimate
Qwen3.6 35B-A3B35B, 3B active · Q4_K_M22.4 GBcontext memory estimatedFits~133–195 tok/sestimate
Ornith 1.5 35B-A3B35B, 3B active · Q4_K_M21.9 GBcontext memory estimatedFits~137–200 tok/sestimate
Qwen3 Coder 30B-A3B Instruct30.5B, 3.3B active · Q4_K_M19.5 GBFits~127–185 tok/sestimate
Gemma 4 31B IT30.7B · Q4_K_M18.9 GBcontext memory estimatedFits~27–43 tok/sestimate
Granite 4.2 30B29.3B · Q4_K_M20.3 GBFits~28–44 tok/sestimate
Qwen3.8 27B27B · Q4_K_M18.2 GBcontext memory estimatedFits~29–45 tok/sestimate
Gemma 4 26B-A4B IT25.2B, 3.8B active · Q4_K_M17.4 GBcontext memory estimatedFits~99–145 tok/sestimate
Qwen3.6 27B27B · Q4_K_M17.7 GBcontext memory estimatedFits~30–47 tok/sestimate
Devstral Small 2 24B (2512)24B · Q4_K_M16.1 GBFits~35–55 tok/sestimate
gpt-oss-20b21B, 3.6B active · MXFP4 (native)12.5 GBcontext memory estimatedFits~128–187 tok/sestimate
Phi-414.7B · Q4_K_M11.3 GBFits~56–89 tok/sestimate
Ministral 3 14B Reasoning (2512)13.9B · Q4_K_M10.4 GBFits~61–96 tok/sestimate
Gemma 4 12B IT11.95B · Q4_K_M8.2 GBcontext memory estimatedFits~70–111 tok/sestimate
Qwen3.5 9B9B · Q4_K_M7.0 GBcontext memory estimatedFits~88–139 tok/sestimate
Granite 4.2 8B8.8B · Q4_K_M7.9 GBFits~91–142 tok/sestimate
Gemma 4 E4B IT8B · Q4_K_M6.5 GBcontext memory not countedFits~165–259 tok/sestimate
Qwen3 8B8.2B · Q4_K_M7.3 GBFits~100–157 tok/sestimate
Llama 3.1 8B Instruct8B · Q4_K_M7.1 GBFits~102–160 tok/sestimate
Qwen3.5 4B4B · Q4_K_M4.3 GBcontext memory estimatedFits~183–288 tok/sestimate

Measured speeds

1 published run with one user, plain decoding and at most 4,096 tokens already in context, on 4× Instinct MI50 (32GB).

Plain single-user runs
ModelQuantBackendGenerationPromptContextMeasured onSource
Qwen3.8 Flash-NextUD-Q4_K_XLllama.cpp gfx906-furnace fork (ROCm 7.1)49 tok/s1,977 tok/s8,192-token prompt04× Instinct MI50 (32GB)independent benchmark2026-10

With speculative decoding

A small draft model or extra prediction heads guess several tokens ahead. Generation is faster on text the guesses get right, so these are kept out of the medians above.

Runs with speculative decoding
ModelQuantBackendGenerationPromptContextMeasured onSource
Qwen3.8 Flash-NextUD-Q4_K_XLllama-server, gfx906-furnace fork62 tok/s1,561 tok/s32,768-token prompt04× Instinct MI50 (32GB)independent benchmark2026-10

Deep into a long context

Measured with more than 4,096 tokens already in context, where every new token also reads the context memory.

Runs at long context
ModelQuantBackendGenerationPromptContextMeasured onSource
Qwen3.8 Flash-NextUD-Q4_K_XLllama.cpp gfx906-furnace fork (ROCm 7.1)44 tok/s–32,7684× Instinct MI50 (32GB)independent benchmark2026-10

Specifications

GPUs
4 × AMD Instinct MI50 (32GB)
VRAM
128 GB
System RAM
256 GB
RAM bandwidth
8 channels x 3200 MT/s x 8 bytes = 204.8 GB/s (AMD lists 204.8 GB/s per socket for the 7313P)
Power supply
2500 W
Estimated load
4x MI50 300 W + EPYC 7313P 155 W + 100 W = 1,455 W
Software
Vulkan

Before you build it

  • gfx906 (MI50) is no longer on AMD's ROCm support list; owners run llama.cpp on Vulkan or community gfx906 ROCm builds.
  • The MI50 is passively cooled with no fan and no display output; each card needs a blower and shroud or server-chassis airflow.
  • Extra MI50s add memory, not speed: one test measured under 3% decode gain from a second card when the model already fits on one.
  • A US 15 A / 120 V circuit is 1,800 W on paper, but the NEC 80% rule for continuous loads (3 h+) limits it to 12 A, about 1,440 W, shared with everything else on that circuit.
  • Corsair rates the HX1500i at 1,500 W only on 115-240 V input; at 100-115 V it is a 1,200 W supply.

Sources

Compared with

Similar machines