← All categories
Multi-GPU
Several cards whose VRAM adds up. llama.cpp splits layers across them, so each token passes through the cards in turn.
| Machine | Memory for models | Bandwidth | Price | Largest at 4-bit | gpt-oss-20b |
|---|---|---|---|---|---|
| 2x RTX 3090 (used)Multi-GPU | 46.5 GB | 936 GB/sper card | ~$3,639 + unpriced partsGPU and risers prices not included | Llama 3.3 70B Instruct | ~117–171 tok/sestimate |
| 4x RTX 3090 (used) on EPYC 7003, open frameMulti-GPU | 91.5 GB | 936 GB/sper card | ~$2,178 + unpriced partsGPU, motherboard, ram, risers, case and cooling prices not included | Nemotron 3 Super 120B-A12B | ~117–171 tok/sestimate |
| 2x RTX 5090Multi-GPU | 62.5 GB | 1,792 GB/sper card | ~$4,109 + unpriced partsGPU price not included | Llama 4 Scout 17B-16E Instruct | ~224–327 tok/sestimate |
| 2x Radeon RX 7900 XTXMulti-GPU | 46.5 GB | 960 GB/sper card | ~$5,637sum of parts | Llama 3.3 70B Instruct | ~120–175 tok/sestimate |
| 4x Instinct MI50 32GB (used) on EPYC 7003Multi-GPU | 123.5 GB | 1,024 GB/sper card | ~$1,998 + unpriced partsGPU, motherboard, ram, cooling and case prices not included | Qwen3.8 Flash-Next | ~128–187 tok/sestimate |
| 2x Intel Arc Pro B60Multi-GPU | 46.5 GB | 456 GB/sper card | ~$4,367sum of parts | Llama 3.3 70B Instruct | ~57–83 tok/sestimate |