Moonshot AI · mixture of experts · Kimi K3 License
What runs Kimi K3
Moonshot's 2.8T-parameter MoE with 104B active, released in MXFP4. Running it locally takes a multi-terabyte-RAM server or a cluster.
- Parameters
- 2.8T, 104B active
- 4-bit file
- 1405.1 GB
- UD-Q4_K_XL
- Needs at 8k context
- 1406.8 GB
- context memory estimated
- Longest context
- 1,048,576 tokens
Context memory: 27,648 bytes per token, an estimate from the config: 24 gated MLA layers x (512 + 64) x 2 bytes; 69 linear layers hold a fixed state. 69 Kimi Delta Attention layers (linear, fixed-size state) plus 24 gated MLA layers that cache a compressed latent. Card gives no per-token figure. How context memory works.
Measured on this hardware
Machines that hold the 4-bit file with 8,192 tokens of context and have published runs on the same chip or cards, fastest first. Median of single-user runs with plain decoding.
No published run on a machine that fits it.
Too large for
Apple Mac Studio (M5 Max, 32-core GPU), Apple Mac Studio (M5 Max, 40-core GPU), Apple Mac Studio (M5 Ultra, 64-core GPU), Apple Mac Studio (M5 Ultra, 80-core GPU), Apple Mac mini (M5 Pro, 16-core GPU), Apple Mac mini (M5 Pro, 20-core GPU), Apple MacBook Pro 14-inch (M5 Max, 32-core GPU), Apple MacBook Pro 14-inch (M5 Max, 40-core GPU), Apple MacBook Pro 16-inch (M5 Max, 32-core GPU), Apple MacBook Pro 16-inch (M5 Max, 40-core GPU), Framework Desktop (Ryzen AI Max+ 395), Framework Desktop (Ryzen AI Max 385), Framework Desktop (Ryzen AI Max+ PRO 495), GMKtec EVO-X2 (Ryzen AI Max+ 395), Beelink GTR9 Pro (Ryzen AI Max+ 395), Minisforum MS-S1 MAX (Ryzen AI Max+ 395), Corsair AI Workstation 300 (Ryzen AI Max+ 395), HP Z2 Mini G1a (Ryzen AI Max+ PRO 395), NVIDIA DGX Spark, Dell Pro Precision with GB10, Microsoft Surface RTX Spark Dev Box, Dell Pro Precision with GB300, Dell Pro Max Tower T2 with NVIDIA RTX PRO 6000 Blackwell, tiny corp tinybox red flash, 1x RTX 3090 (used), 2x RTX 3090 (used), 4x RTX 3090 (used) on EPYC 7003, open frame, 1x RTX 5090, 2x RTX 5090, 1x RTX PRO 6000 Blackwell, 2x Radeon RX 7900 XTX, 4x Instinct MI50 32GB (used) on EPYC 7003, 2x Intel Arc Pro B60, EPYC 9355P + 768 GB 12-channel DDR5 (CPU only), RTX 5090 + EPYC 9355P with 768 GB DDR5 (MoE offload), RTX 3090 (used) + 128 GB DDR5 on AM5 (MoE offload). A smaller quant may fit: try the calculator.
Files
| Quant | Format | Size | Repo |
|---|---|---|---|
| UD-IQ1_S | GGUF | 553.2 GB | unsloth/Kimi-K3-GGUF |
| UD-IQ1_M | GGUF | 604.3 GB | unsloth/Kimi-K3-GGUF |
| UD-IQ2_XXS | GGUF | 662.2 GB | unsloth/Kimi-K3-GGUF |
| UD-Q2_K_XL | GGUF | 802.1 GB | unsloth/Kimi-K3-GGUF |
| UD-Q4_K_XL | GGUF | 1405.1 GB | unsloth/Kimi-K3-GGUF |
Sources
- huggingface.co: model card: parameters, context length, license and attention layout; release month
- huggingface.co: downloads (last 30 days), license tag, safetensors param count
- huggingface.co: layer count, KV heads and head size
- huggingface.co: GGUF file sizes (shards summed)