EPYC 9355P + 768 GB 12-channel DDR5 (CPU only) vs RTX 5090 + EPYC 9355P with 768 GB DDR5 (MoE offload)
The EPYC 9355P + 768 GB 12-channel DDR5 (CPU only) gives a model 760 GB to work in (an assumed figure), against 32 GB on the RTX 5090 + EPYC 9355P with 768 GB DDR5 (MoE offload). The RTX 5090 + EPYC 9355P with 768 GB DDR5 (MoE offload) has 2.9× the memory bandwidth (1,792 against 614.4 GB/s), which sets the ceiling on generation speed. The largest 4-bit model each holds at 8,192 tokens of context: Kimi K2.6 on the EPYC 9355P + 768 GB 12-channel DDR5 (CPU only), Nemotron 3.5 Lightning 30B-A3B on the RTX 5090 + EPYC 9355P with 768 GB DDR5 (MoE offload). Rows where the two differ are tinted.
CPU + RAM server
EPYC 9355P + 768 GB 12-channel DDR5 (CPU only)
A single-socket Turin box whose 768 GB holds the largest open MoE models at 4-bit, generating at speeds set by ~600 GB/s of memory bandwidth. No GPU, so prompt processing is slow.
gpt-oss-20b, 4-bit: –
Hybrid GPU + RAM
RTX 5090 + EPYC 9355P with 768 GB DDR5 (MoE offload)
The usual recipe for running 600B+ MoE models at home: expert weights in 12-channel DDR5, attention and KV cache on one fast GPU. Faster prompt processing than CPU-only for a fraction of an all-GPU build's VRAM bill.
gpt-oss-20b, 4-bit: 290 tok/s
| Specification | EPYC 9355P + 768 GB 12-channel DDR5 (CPU only) | RTX 5090 + EPYC 9355P with 768 GB DDR5 (MoE offload) |
|---|---|---|
| Kind | CPU + RAM server · DIY | Hybrid GPU + RAM · DIY |
| Chip or GPUs | not published | RTX 5090 |
| Memory | 768 GB RAM | 32 GB VRAM + 768 GB RAM |
| Memory for models | 760 GB (System RAM)assumption | 32 GB (VRAM) |
| Other memory figure | none | 792 GB (GPU + RAM offload)assumption |
| Memory bandwidth | 614.4 GB/s | 1,792 GB/s |
| FP16 compute | not published | 209.5 TFLOPS |
| Price | ~$3,663 + unpriced parts (motherboard, ram and cooling prices not included) | ~$3,763 + unpriced parts (GPU, motherboard, ram and cooling prices not included) |
| Largest at 4-bit, 8k context | Kimi K2.6 | Nemotron 3.5 Lightning 30B-A3B |
| Software | CUDA, Vulkan | |
| Power | 1000 W power supply | 1500 W power supply |
What each one runs
Every model in the catalog that at least one of the two holds at 4-bit with 8,192 tokens of context. Copper speeds are measured medians; outlined ones are estimates. Rows where only one fits, or where the speed differs by more than a quarter, are tinted.
| Model | EPYC 9355P + 768 GB 12-channel DDR5 (CPU only) | RTX 5090 + EPYC 9355P with 768 GB DDR5 (MoE offload) |
|---|---|---|
| Kimi K2.61T, 32B active · 543.6 GB file | – | GPU + RAM offloadassumed limit~8.5–12 tok/sestimate, part in RAM |
| DeepSeek V4.1 Flash552B, 16B active · 483.0 GB file | – | GPU + RAM offloadassumed limit~11–15 tok/sestimate, part in RAM |
| GLM-5.3753.3B · 435.2 GB file | – | GPU + RAM offloadassumed limit– |
| DeepSeek V3.2671B, 37B active · 377.6 GB file | – | GPU + RAM offloadassumed limit~7.2–11 tok/sestimate, part in RAM |
| MiniMax M3428B, 23B active · 243.3 GB file | – | GPU + RAM offloadassumed limit~12–17 tok/sestimate, part in RAM |
| Qwen3.5 397B-A17B397B, 17B active · 227.3 GB file | – | GPU + RAM offloadassumed limit~16–23 tok/sestimate, part in RAM |
| GLM-5.3 Flash320B, 18B active · 186.0 GB file | – | GPU + RAM offloadassumed limit~15–22 tok/sestimate, part in RAM |
| Hy3295B, 21B active · 169.6 GB file | – | GPU + RAM offloadassumed limit~13–19 tok/sestimate, part in RAM |
| MiniMax M2.7228.7B · 129.3 GB file | – | GPU + RAM offloadassumed limit– |
| Qwen3.8 Flash-Next125B, 6B active · 111.4 GB file | – | GPU + RAM offloadassumed limit~32–47 tok/sestimate, part in RAM |
| Nemotron 3 Super 120B-A12B120B, 12B active · 81.0 GB file | – | GPU + RAM offloadassumed limit~23–34 tok/sestimate, part in RAM |
| Qwen3.5 122B-A10B122B, 10B active · 71.3 GB file | – | GPU + RAM offloadassumed limit~34–49 tok/sestimate, part in RAM |
| Mistral Medium 3.5 128B128B · 69.8 GB file | – | GPU + RAM offloadassumed limit~5.4–8.6 tok/sestimate, part in RAM |
| GLM-4.5-Air106B, 12B active · 68.0 GB file | – | GPU + RAM offloadassumed limit~26–38 tok/sestimate, part in RAM |
| Mistral Small 4 119B (2603)119B, 6.5B active · 67.6 GB file | – | GPU + RAM offloadassumed limit~55–80 tok/sestimate, part in RAM |
| Llama 4 Scout 17B-16E Instruct109B, 17B active · 60.9 GB file | – | GPU + RAM offloadassumed limit~22–33 tok/sestimate, part in RAM |
| gpt-oss-120b117B, 5.1B active · 58.5 GB file | – | GPU + RAM offloadassumed limit~85–124 tok/sestimate, part in RAM |
| Qwen3 Coder Next80B, 3B active · 45.2 GB file | – | GPU + RAM offloadassumed limit~150–219 tok/sestimate, part in RAM |
| Llama 3.3 70B Instruct70B · 39.6 GB file | – | GPU + RAM offloadassumed limit~13–21 tok/sestimate, part in RAM |
| Nemotron 3.5 Lightning 30B-A3B30B, 3B active · 23.7 GB file | – | ~175–256 tok/sestimate |
| Qwen3.6 35B-A3B35B, 3B active · 20.8 GB file | – | 117 tok/smedian of 1 run |
| Ornith 1.5 35B-A3B35B, 3B active · 20.2 GB file | – | ~240–350 tok/sestimate |
| Qwen3 Coder 30B-A3B Instruct30.5B, 3.3B active · 17.3 GB file | – | ~222–324 tok/sestimate |
| Gemma 4 31B IT30.7B · 17.1 GB file | – | 61 tok/smedian of 1 run |
| Granite 4.2 30B29.3B · 16.8 GB file | – | ~49–77 tok/sestimate |
| Qwen3.8 27B27B · 16.2 GB file | – | ~50–79 tok/sestimate |
| Gemma 4 26B-A4B IT25.2B, 3.8B active · 15.9 GB file | – | 122 tok/smedian of 2 runs |
| Qwen3.6 27B27B · 15.7 GB file | – | ~52–82 tok/sestimate |
| Devstral Small 2 24B (2512)24B · 13.3 GB file | – | ~61–96 tok/sestimate |
| gpt-oss-20b21B, 3.6B active · 10.8 GB file | – | 290 tok/smedian of 4 runs |
| Phi-414.7B · 8.3 GB file | – | ~99–155 tok/sestimate |
| Ministral 3 14B Reasoning (2512)13.9B · 7.7 GB file | – | ~107–167 tok/sestimate |
| Gemma 4 12B IT11.95B · 6.6 GB file | – | ~123–194 tok/sestimate |
| Qwen3.5 9B9B · 5.3 GB file | – | ~155–243 tok/sestimate |
| Granite 4.2 8B8.8B · 5.2 GB file | – | ~159–249 tok/sestimate |
| Gemma 4 E4B IT8B · 5.0 GB file | – | ~289–454 tok/sestimate |
| Qwen3 8B8.2B · 4.7 GB file | – | 200 tok/smedian of 1 run |
| Llama 3.1 8B Instruct8B · 4.6 GB file | – | 200 tok/smedian of 3 runs |
| Qwen3.5 4B4B · 2.6 GB file | – | ~321–503 tok/sestimate |
Full pages: EPYC 9355P + 768 GB 12-channel DDR5 (CPU only) · RTX 5090 + EPYC 9355P with 768 GB DDR5 (MoE offload) · check a different quant or context length