4x RTX 3090 (used) on EPYC 7003, open frame vs 4x Instinct MI50 32GB (used) on EPYC 7003
The 4x Instinct MI50 32GB (used) on EPYC 7003 gives a model 123.5 GB to work in, against 91.5 GB on the 4x RTX 3090 (used) on EPYC 7003, open frame. The 4x Instinct MI50 32GB (used) on EPYC 7003 has slightly more memory bandwidth (1,024 against 936 GB/s, per card for multi-GPU builds), which sets the ceiling on generation speed. The largest 4-bit model each holds at 8,192 tokens of context: Nemotron 3 Super 120B-A12B on the 4x RTX 3090 (used) on EPYC 7003, open frame, Qwen3.8 Flash-Next on the 4x Instinct MI50 32GB (used) on EPYC 7003. Rows where the two differ are tinted.
Multi-GPU
4x RTX 3090 (used) on EPYC 7003, open frame
The classic 96 GB home rig: four used 3090s on risers around a single-socket EPYC board with a full x16 link for every card. It runs 70B dense models at 8-bit or ~120B MoE models in VRAM.
gpt-oss-20b, 4-bit: ~117–171 tok/sestimate
Multi-GPU
4x Instinct MI50 32GB (used) on EPYC 7003
128 GB of 1 TB/s HBM2 for the price of used datacenter cards, at the cost of no fans, no official ROCm and ~1.5 kW. Built for big MoE models that need memory more than compute.
gpt-oss-20b, 4-bit: ~128–187 tok/sestimate
| Specification | 4x RTX 3090 (used) on EPYC 7003, open frame | 4x Instinct MI50 32GB (used) on EPYC 7003 |
|---|---|---|
| Kind | Multi-GPU · DIY | Multi-GPU · DIY |
| Chip or GPUs | 4× RTX 3090 | 4× Instinct MI50 (32GB) |
| Memory | 96 GB VRAM + 256 GB RAM | 128 GB VRAM + 256 GB RAM |
| Memory for models | 91.5 GB (VRAM across all cards) | 123.5 GB (VRAM across all cards) |
| Memory bandwidth | 936 GB/s per card | 1,024 GB/s per card |
| FP16 compute | 71.2 TFLOPS per card | not published |
| Price | ~$2,178 + unpriced parts (GPU, motherboard, ram, risers, case and cooling prices not included) | ~$1,998 + unpriced parts (GPU, motherboard, ram, cooling and case prices not included) |
| Largest at 4-bit, 8k context | Nemotron 3 Super 120B-A12B | Qwen3.8 Flash-Next |
| Software | CUDA, Vulkan | Vulkan |
| Power | 3000 W power supply | 2500 W power supply |
What each one runs
Every model in the catalog that at least one of the two holds at 4-bit with 8,192 tokens of context. Copper speeds are measured medians; outlined ones are estimates. Rows where only one fits, or where the speed differs by more than a quarter, are tinted.
| Model | 4x RTX 3090 (used) on EPYC 7003, open frame | 4x Instinct MI50 32GB (used) on EPYC 7003 |
|---|---|---|
| Qwen3.8 Flash-Next125B, 6B active · 111.4 GB file | does not fit | 49 tok/smedian of 1 run |
| Nemotron 3 Super 120B-A12B120B, 12B active · 81.0 GB file | ~27–39 tok/sestimate | ~29–43 tok/sestimate |
| Qwen3.5 122B-A10B122B, 10B active · 71.3 GB file | ~37–54 tok/sestimate | ~41–59 tok/sestimate |
| Mistral Medium 3.5 128B128B · 69.8 GB file | ~6.1–9.6 tok/sestimate | ~6.7–11 tok/sestimate |
| GLM-4.5-Air106B, 12B active · 68.0 GB file | ~28–41 tok/sestimate | ~31–45 tok/sestimate |
| Mistral Small 4 119B (2603)119B, 6.5B active · 67.6 GB file | ~59–86 tok/sestimate | ~64–94 tok/sestimate |
| Llama 4 Scout 17B-16E Instruct109B, 17B active · 60.9 GB file | ~23–33 tok/sestimate | ~25–36 tok/sestimate |
| gpt-oss-120b117B, 5.1B active · 58.5 GB file | ~85–124 tok/sestimate | ~93–136 tok/sestimate |
| Qwen3 Coder Next80B, 3B active · 45.2 GB file | ~128–187 tok/sestimate | ~140–204 tok/sestimate |
| Llama 3.3 70B Instruct70B · 39.6 GB file | ~11–17 tok/sestimate | ~12–19 tok/sestimate |
| Nemotron 3.5 Lightning 30B-A3B30B, 3B active · 23.7 GB file | ~91–133 tok/sestimate | ~100–146 tok/sestimate |
| Qwen3.6 35B-A3B35B, 3B active · 20.8 GB file | ~122–178 tok/sestimate | ~133–195 tok/sestimate |
| Ornith 1.5 35B-A3B35B, 3B active · 20.2 GB file | ~125–183 tok/sestimate | ~137–200 tok/sestimate |
| Qwen3 Coder 30B-A3B Instruct30.5B, 3.3B active · 17.3 GB file | ~116–169 tok/sestimate | ~127–185 tok/sestimate |
| Gemma 4 31B IT30.7B · 17.1 GB file | ~25–39 tok/sestimate | ~27–43 tok/sestimate |
| Granite 4.2 30B29.3B · 16.8 GB file | ~25–40 tok/sestimate | ~28–44 tok/sestimate |
| Qwen3.8 27B27B · 16.2 GB file | ~26–41 tok/sestimate | ~29–45 tok/sestimate |
| Gemma 4 26B-A4B IT25.2B, 3.8B active · 15.9 GB file | ~91–132 tok/sestimate | ~99–145 tok/sestimate |
| Qwen3.6 27B27B · 15.7 GB file | ~27–43 tok/sestimate | ~30–47 tok/sestimate |
| Devstral Small 2 24B (2512)24B · 13.3 GB file | ~32–50 tok/sestimate | ~35–55 tok/sestimate |
| gpt-oss-20b21B, 3.6B active · 10.8 GB file | ~117–171 tok/sestimate | ~128–187 tok/sestimate |
| Phi-414.7B · 8.3 GB file | ~52–81 tok/sestimate | ~56–89 tok/sestimate |
| Ministral 3 14B Reasoning (2512)13.9B · 7.7 GB file | ~56–87 tok/sestimate | ~61–96 tok/sestimate |
| Gemma 4 12B IT11.95B · 6.6 GB file | ~64–101 tok/sestimate | ~70–111 tok/sestimate |
| Qwen3.5 9B9B · 5.3 GB file | ~81–127 tok/sestimate | ~88–139 tok/sestimate |
| Granite 4.2 8B8.8B · 5.2 GB file | ~83–130 tok/sestimate | ~91–142 tok/sestimate |
| Gemma 4 E4B IT8B · 5.0 GB file | ~151–237 tok/sestimate | ~165–259 tok/sestimate |
| Qwen3 8B8.2B · 4.7 GB file | ~91–143 tok/sestimate | ~100–157 tok/sestimate |
| Llama 3.1 8B Instruct8B · 4.6 GB file | ~93–146 tok/sestimate | ~102–160 tok/sestimate |
| Qwen3.5 4B4B · 2.6 GB file | ~167–263 tok/sestimate | ~183–288 tok/sestimate |
Full pages: 4x RTX 3090 (used) on EPYC 7003, open frame · 4x Instinct MI50 32GB (used) on EPYC 7003 · check a different quant or context length