2x RTX 5090 vs 4x Instinct MI50 32GB (used) on EPYC 7003
The 4x Instinct MI50 32GB (used) on EPYC 7003 gives a model 123.5 GB to work in, against 62.5 GB on the 2x RTX 5090. The 2x RTX 5090 has 1.8× the memory bandwidth (1,792 against 1,024 GB/s, per card for multi-GPU builds), which sets the ceiling on generation speed. The largest 4-bit model each holds at 8,192 tokens of context: Llama 4 Scout 17B-16E Instruct on the 2x RTX 5090, Qwen3.8 Flash-Next on the 4x Instinct MI50 32GB (used) on EPYC 7003. Rows where the two differ are tinted.
Multi-GPU
2x RTX 5090
64 GB of VRAM with the highest memory bandwidth of any consumer pair. Power is the hard part: at ~1.4 kW it outgrows a normal 120 V household circuit.
gpt-oss-20b, 4-bit: ~224–327 tok/sestimate
Multi-GPU
4x Instinct MI50 32GB (used) on EPYC 7003
128 GB of 1 TB/s HBM2 for the price of used datacenter cards, at the cost of no fans, no official ROCm and ~1.5 kW. Built for big MoE models that need memory more than compute.
gpt-oss-20b, 4-bit: ~128–187 tok/sestimate
| Specification | 2x RTX 5090 | 4x Instinct MI50 32GB (used) on EPYC 7003 |
|---|---|---|
| Kind | Multi-GPU · DIY | Multi-GPU · DIY |
| Chip or GPUs | 2× RTX 5090 | 4× Instinct MI50 (32GB) |
| Memory | 64 GB VRAM + 64 GB RAM | 128 GB VRAM + 256 GB RAM |
| Memory for models | 62.5 GB (VRAM across all cards) | 123.5 GB (VRAM across all cards) |
| Memory bandwidth | 1,792 GB/s per card | 1,024 GB/s per card |
| FP16 compute | 209.5 TFLOPS per card | not published |
| Price | ~$4,109 + unpriced parts (GPU price not included) | ~$1,998 + unpriced parts (GPU, motherboard, ram, cooling and case prices not included) |
| Largest at 4-bit, 8k context | Llama 4 Scout 17B-16E Instruct | Qwen3.8 Flash-Next |
| Software | CUDA, Vulkan | Vulkan |
| Power | 3000 W power supply | 2500 W power supply |
What each one runs
Every model in the catalog that at least one of the two holds at 4-bit with 8,192 tokens of context. Copper speeds are measured medians; outlined ones are estimates. Rows where only one fits, or where the speed differs by more than a quarter, are tinted.
| Model | 2x RTX 5090 | 4x Instinct MI50 32GB (used) on EPYC 7003 |
|---|---|---|
| Qwen3.8 Flash-Next125B, 6B active · 111.4 GB file | does not fit | 49 tok/smedian of 1 run |
| Nemotron 3 Super 120B-A12B120B, 12B active · 81.0 GB file | does not fit | ~29–43 tok/sestimate |
| Qwen3.5 122B-A10B122B, 10B active · 71.3 GB file | does not fit | ~41–59 tok/sestimate |
| Mistral Medium 3.5 128B128B · 69.8 GB file | does not fit | ~6.7–11 tok/sestimate |
| GLM-4.5-Air106B, 12B active · 68.0 GB file | does not fit | ~31–45 tok/sestimate |
| Mistral Small 4 119B (2603)119B, 6.5B active · 67.6 GB file | does not fit | ~64–94 tok/sestimate |
| Llama 4 Scout 17B-16E Instruct109B, 17B active · 60.9 GB file | ~44–64 tok/sestimate | ~25–36 tok/sestimate |
| gpt-oss-120b117B, 5.1B active · 58.5 GB file | ~163–238 tok/sestimate | ~93–136 tok/sestimate |
| Qwen3 Coder Next80B, 3B active · 45.2 GB file | ~245–358 tok/sestimate | ~140–204 tok/sestimate |
| Llama 3.3 70B Instruct70B · 39.6 GB file | ~21–32 tok/sestimate | ~12–19 tok/sestimate |
| Nemotron 3.5 Lightning 30B-A3B30B, 3B active · 23.7 GB file | ~175–256 tok/sestimate | ~100–146 tok/sestimate |
| Qwen3.6 35B-A3B35B, 3B active · 20.8 GB file | ~233–341 tok/sestimate | ~133–195 tok/sestimate |
| Ornith 1.5 35B-A3B35B, 3B active · 20.2 GB file | ~240–350 tok/sestimate | ~137–200 tok/sestimate |
| Qwen3 Coder 30B-A3B Instruct30.5B, 3.3B active · 17.3 GB file | ~222–324 tok/sestimate | ~127–185 tok/sestimate |
| Gemma 4 31B IT30.7B · 17.1 GB file | ~48–75 tok/sestimate | ~27–43 tok/sestimate |
| Granite 4.2 30B29.3B · 16.8 GB file | ~49–77 tok/sestimate | ~28–44 tok/sestimate |
| Qwen3.8 27B27B · 16.2 GB file | ~50–79 tok/sestimate | ~29–45 tok/sestimate |
| Gemma 4 26B-A4B IT25.2B, 3.8B active · 15.9 GB file | ~174–253 tok/sestimate | ~99–145 tok/sestimate |
| Qwen3.6 27B27B · 15.7 GB file | ~52–82 tok/sestimate | ~30–47 tok/sestimate |
| Devstral Small 2 24B (2512)24B · 13.3 GB file | ~61–96 tok/sestimate | ~35–55 tok/sestimate |
| gpt-oss-20b21B, 3.6B active · 10.8 GB file | ~224–327 tok/sestimate | ~128–187 tok/sestimate |
| Phi-414.7B · 8.3 GB file | ~99–155 tok/sestimate | ~56–89 tok/sestimate |
| Ministral 3 14B Reasoning (2512)13.9B · 7.7 GB file | ~107–167 tok/sestimate | ~61–96 tok/sestimate |
| Gemma 4 12B IT11.95B · 6.6 GB file | ~123–194 tok/sestimate | ~70–111 tok/sestimate |
| Qwen3.5 9B9B · 5.3 GB file | ~155–243 tok/sestimate | ~88–139 tok/sestimate |
| Granite 4.2 8B8.8B · 5.2 GB file | ~159–249 tok/sestimate | ~91–142 tok/sestimate |
| Gemma 4 E4B IT8B · 5.0 GB file | ~289–454 tok/sestimate | ~165–259 tok/sestimate |
| Qwen3 8B8.2B · 4.7 GB file | ~175–274 tok/sestimate | ~100–157 tok/sestimate |
| Llama 3.1 8B Instruct8B · 4.6 GB file | ~179–280 tok/sestimate | ~102–160 tok/sestimate |
| Qwen3.5 4B4B · 2.6 GB file | ~321–503 tok/sestimate | ~183–288 tok/sestimate |
Full pages: 2x RTX 5090 · 4x Instinct MI50 32GB (used) on EPYC 7003 · check a different quant or context length