← All machines
Apple unified memory · Apple
Apple Mac Studio (M5 Max, 40-core GPU)
Mac Studio with the full M5 Max, configurable to 128GB of unified memory. At 128GB it holds a 70B dense model or a 120B-class MoE model at 4-bit inside the default GPU limit.
Memory for models
96 GBassumption
Assumed three quarters of 128 GB for macOS's default GPU limit, which Apple does not publish.
Memory bandwidth
614 GB/s
Maker price
~$3,099–$5,099
Largest at 4-bit
82.6 GB at 8,192 tokens of context, default GPU limit (an assumed limit, see Memory for models)
gpt-oss-20b, 4-bit
132 tok/smedian of 4 runs
Memory configurations
One page covers every memory option the maker sells. Prices are the maker’s list price for the cheapest configuration with that much memory.
| Memory | Price | Memory the GPU can use |
|---|---|---|
| 48 GB | ~$3,099 | 36 GB default gpu limitassumption 40 GB limit raised with sysctlassumption |
| 64 GB | ~$3,499 | 48 GB default gpu limitassumption 56 GB limit raised with sysctlassumption |
| 128 GB | ~$5,099 | 96 GB default gpu limitassumption 120 GB limit raised with sysctlassumption |
Assumed three quarters of 128 GB for macOS's default GPU limit, which Apple does not publish. Raised with iogpu.wired_limit_mb per the MLX docs, leaving an assumed 8 GB for macOS. How GPU memory limits work on unified-memory machines.
What it runs at 4-bit
Every model in the catalog at its 4-bit file with 8,192 tokens of context. 30 of 41 fit in at least one configuration. Speeds are the median of measured runs on the same chip where any exist; otherwise an estimate from memory bandwidth. Try other quants and context lengths.
| Model | Needs | Fit | Generation speed |
|---|---|---|---|
| Kimi K32.8T, 104B active · UD-Q4_K_XL | 1406.8 GBcontext memory estimated | Does not fit | |
| DeepSeek V4 Pro1.6T, 49B active · Q4_K_M | 890.5 GBcontext memory not counted | Does not fit | |
| Kimi K2.61T, 32B active · UD-Q4_K_XL | 545.7 GBcontext memory estimated | Does not fit | |
| DeepSeek V4.1 Flash552B, 16B active · Q4 (DwarfStar) | 484.5 GB | Does not fit | |
| GLM-5.3753.3B · UD-Q4_K_XL | 437.4 GBcontext memory estimated | Does not fit | |
| DeepSeek V3.2671B, 37B active · Q4_K_M | 379.6 GBcontext memory estimated | Does not fit | |
| MiniMax M3428B, 23B active · Q4_K_M | 244.8 GBcontext memory not counted | Does not fit | |
| Qwen3.5 397B-A17B397B, 17B active · Q4_K_M | 229.1 GBcontext memory estimated | Does not fit | |
| GLM-5.3 Flash320B, 18B active · UD-Q4_K_XL | 187.6 GBcontext memory estimated | Does not fit | |
| Hy3295B, 21B active · Q4_K_M | 173.6 GB | Does not fit | |
| MiniMax M2.7228.7B · Q4_K_M | 132.7 GB | Does not fit | |
| Qwen3.8 Flash-Next125B, 6B active · Q4_K_M | 113.1 GBcontext memory estimated | Fits with raised limitassumed limitfrom the 128 GB config | 39 tok/smedian of 2 runs |
| Nemotron 3 Super 120B-A12B120B, 12B active · Q4_K_M | 82.6 GBcontext memory estimated | Fitsassumed limitfrom the 128 GB config | 58 tok/smedian of 1 run |
| Qwen3.5 122B-A10B122B, 10B active · Q4_K_M | 73.0 GBcontext memory estimated | Fitsassumed limitfrom the 128 GB config | 58 tok/smedian of 4 runs |
| Mistral Medium 3.5 128B128B · Q4_K_M | 74.0 GB | Fitsassumed limitfrom the 128 GB config | ~6.0–7.5 tok/sestimate |
| GLM-4.5-Air106B, 12B active · Q4_K_M | 70.9 GB | Fitsassumed limitfrom the 128 GB config | 53 tok/smedian of 1 run |
| Mistral Small 4 119B (2603)119B, 6.5B active · Q4_K_M | 69.3 GBcontext memory estimated | Fitsassumed limitfrom the 128 GB config | 92 tok/smedian of 4 runs |
| Llama 4 Scout 17B-16E Instruct109B, 17B active · Q4_K_M | 62.4 GBcontext memory not counted | Fitsassumed limitfrom the 128 GB config | ~20–36 tok/sestimate |
| gpt-oss-120b117B, 5.1B active · MXFP4 (native) | 60.2 GBcontext memory estimated | Fitsassumed limitfrom the 128 GB config | 90 tok/smedian of 4 runs |
| Qwen3 Coder Next80B, 3B active · Q4_K_M | 46.9 GBcontext memory estimated | Fitsassumed limitfrom the 64 GB config | 98 tok/smedian of 3 runs |
| Llama 3.3 70B Instruct70B · Q4_K_M | 43.6 GB | Fitsassumed limitfrom the 64 GB config | 13 tok/smedian of 3 runs |
| Nemotron 3.5 Lightning 30B-A3B30B, 3B active · Q4_K_M | 25.3 GBcontext memory estimated | Fitsassumed limitfrom the 48 GB config | 142 tok/smedian of 2 runs |
| Qwen3.6 35B-A3B35B, 3B active · Q4_K_M | 22.4 GBcontext memory estimated | Fitsassumed limitfrom the 48 GB config | 148 tok/smedian of 2 runs |
| Ornith 1.5 35B-A3B35B, 3B active · Q4_K_M | 21.9 GBcontext memory estimated | Fitsassumed limitfrom the 48 GB config | 138 tok/smedian of 1 run |
| Qwen3 Coder 30B-A3B Instruct30.5B, 3.3B active · Q4_K_M | 19.5 GB | Fitsassumed limitfrom the 48 GB config | 107 tok/smedian of 1 run |
| Gemma 4 31B IT30.7B · Q4_K_M | 18.9 GBcontext memory estimated | Fitsassumed limitfrom the 48 GB config | 21 tok/smedian of 3 runs |
| Granite 4.2 30B29.3B · Q4_K_M | 20.3 GB | Fitsassumed limitfrom the 48 GB config | ~25–31 tok/sestimate |
| Qwen3.8 27B27B · Q4_K_M | 18.2 GBcontext memory estimated | Fitsassumed limitfrom the 48 GB config | 34 tok/smedian of 2 runs |
| Gemma 4 26B-A4B IT25.2B, 3.8B active · Q4_K_M | 17.4 GBcontext memory estimated | Fitsassumed limitfrom the 48 GB config | 116 tok/smedian of 3 runs |
| Qwen3.6 27B27B · Q4_K_M | 17.7 GBcontext memory estimated | Fitsassumed limitfrom the 48 GB config | 33 tok/smedian of 1 run |
| Devstral Small 2 24B (2512)24B · Q4_K_M | 16.1 GB | Fitsassumed limitfrom the 48 GB config | 38 tok/smedian of 1 run |
| gpt-oss-20b21B, 3.6B active · MXFP4 (native) | 12.5 GBcontext memory estimated | Fitsassumed limitfrom the 48 GB config | 132 tok/smedian of 4 runs |
| Phi-414.7B · Q4_K_M | 11.3 GB | Fitsassumed limitfrom the 48 GB config | ~51–63 tok/sestimate |
| Ministral 3 14B Reasoning (2512)13.9B · Q4_K_M | 10.4 GB | Fitsassumed limitfrom the 48 GB config | ~55–68 tok/sestimate |
| Gemma 4 12B IT11.95B · Q4_K_M | 8.2 GBcontext memory estimated | Fitsassumed limitfrom the 48 GB config | 44 tok/smedian of 1 run |
| Qwen3.5 9B9B · Q4_K_M | 7.0 GBcontext memory estimated | Fitsassumed limitfrom the 48 GB config | 103 tok/smedian of 1 run |
| Granite 4.2 8B8.8B · Q4_K_M | 7.9 GB | Fitsassumed limitfrom the 48 GB config | ~81–101 tok/sestimate |
| Gemma 4 E4B IT8B · Q4_K_M | 6.5 GBcontext memory not counted | Fitsassumed limitfrom the 48 GB config | ~148–185 tok/sestimate |
| Qwen3 8B8.2B · Q4_K_M | 7.3 GB | Fitsassumed limitfrom the 48 GB config | 92 tok/smedian of 1 run |
| Llama 3.1 8B Instruct8B · Q4_K_M | 7.1 GB | Fitsassumed limitfrom the 48 GB config | 90 tok/smedian of 2 runs |
| Qwen3.5 4B4B · Q4_K_M | 4.3 GBcontext memory estimated | Fitsassumed limitfrom the 48 GB config | 128 tok/smedian of 1 run |
Measured speeds
103 published runs with one user, plain decoding and at most 4,096 tokens already in context, on M5 Max (18-core CPU, 40-core GPU). Runs on 2 other machines with the same chip count here too, and say so.
| Model | Quant | Backend | Generation | Prompt | Context | Measured on | Source |
|---|---|---|---|---|---|---|---|
| Devstral Small 2 24B (2512) | 4-bit MLX (Devstral-Small-2-24B-Instruct-2512-4bit) | oMLX (version not stated) (MLX) | 38 tok/s | 1,113 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 12B IT | 4-bit MLX (gemma-4-12B-it-qat-4bit) | oMLX (version not stated) (MLX) | 44 tok/s | 1,704 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 26B-A4B IT | bf16 (MLX) | oMLX (version not stated) | 57 tok/s | 2,321 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 26B-A4B IT | BF16 (MLX) | mlx-vlm on TheTom/mlx fork (baseline KV) | 55 tok/s | – | 1,000 | a machine with the same chip | MLX discussion2026-04 |
| Gemma 4 26B-A4B IT | 4bit (MLX) | oMLX (version not stated) | 118 tok/s | 4,130 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 26B-A4B IT | 4-bit MLX (gemma-4-26B-A4B-it-QAT-MLX-4bit) | oMLX (version not stated) (MLX) | 116 tok/s | 3,984 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 26B-A4B IT | 4bit (MLX) | oMLX (version not stated) | 112 tok/s | 4,191 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 26B-A4B IT | Q4_0 | llama.cpp master (2026-04-25, build not stated) | – | 3,976 tok/s512-token prompt | 0 | a machine with the same chip | llama.cpp benchmark discussion2026-04 |
| Gemma 4 26B-A4B IT | Q4_0 | llama.cpp master (2026-04-25, build not stated) | – | 3,825 tok/s2,048-token prompt | 0 | a machine with the same chip | llama.cpp benchmark discussion2026-04 |
| Gemma 4 26B-A4B IT | Q8_0 | llama.cpp b10809 | 95 tok/s | 3,582 tok/s2,048-token prompt | 0 | a machine with the same chip | llama.cpp benchmark discussion2026-09 |
| Gemma 4 26B-A4B IT | Q8_0 | llama.cpp master (2026-04-25, build not stated) | – | 3,266 tok/s512-token prompt | 0 | a machine with the same chip | llama.cpp benchmark discussion2026-04 |
| Gemma 4 26B-A4B IT | Q8_0 | llama.cpp master (2026-04-25, build not stated) | – | 3,390 tok/s2,048-token prompt | 0 | a machine with the same chip | llama.cpp benchmark discussion2026-04 |
| Gemma 4 31B IT | 4bit (MLX) | oMLX (version not stated) | 25 tok/s | 752 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 31B IT | 4-bit MLX (gemma-4-31B-it-qat-OptiQ-4bit) | oMLX (version not stated) (MLX) | 21 tok/s | 630 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 31B IT | 4bit (MLX) | oMLX (version not stated) | 20 tok/s | 627 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 31B IT | Q4_0 | llama.cpp master (2026-04-25, build not stated) | – | 554 tok/s512-token prompt | 0 | a machine with the same chip | llama.cpp benchmark discussion2026-04 |
| Gemma 4 31B IT | Q4_0 | llama.cpp master (2026-04-25, build not stated) | – | 443 tok/s2,048-token prompt | 0 | a machine with the same chip | llama.cpp benchmark discussion2026-04 |
| Gemma 4 31B IT | 8-bit MLX | MLX 0.32.4 / mlx-lm 0.32.2 | 15 tok/s | – | 512 | a machine with the same chip | MLX discussion2026-10 |
| Gemma 4 31B IT | Q8_0 | llama.cpp master (2026-04-25, build not stated) | – | 473 tok/s512-token prompt | 0 | a machine with the same chip | llama.cpp benchmark discussion2026-04 |
| Gemma 4 31B IT | Q8_0 | llama.cpp master (2026-04-25, build not stated) | – | 354 tok/s2,048-token prompt | 0 | a machine with the same chip | llama.cpp benchmark discussion2026-04 |
| Gemma 4 E4B IT | Q8_0 | llama.cpp b10219 (llama-server) | 68 tok/s | 3,471 tok/s | 0 | a machine with the same chip | llama.cpp benchmark discussion2026-08 |
| Gemma 4 E4B IT | Q8_0 | llama.cpp b10219 | 67 tok/s | – | 0 | a machine with the same chip | llama.cpp benchmark discussion2026-08 |
| GLM-4.5-Air | 4-bit MLX (GLM-4.5-Air) | oMLX (version not stated) (MLX) | 53 tok/s | 1,085 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-07 |
| GLM-5.3 Flash | 4-bit MLX (GLM-5.3-Flash-oQ4e) | oMLX 0.7.0 (MLX) | 5.8 tok/s | 57 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-10 |
| GLM-5.3 Flash | 4bit (MLX) | oMLX (version not stated) | 5.1 tok/s | 180 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| gpt-oss-120b | 4-bit MLX (gpt-oss-120b-MXFP4-Q8) | oMLX 0.7.0rc1 (MLX) | 93 tok/s | 1,619 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-09 |
| gpt-oss-120b | 4bit (MLX) | oMLX (version not stated) | 90 tok/s | 2,193 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| gpt-oss-120b | 4bit (MLX) | oMLX (version not stated) | 89 tok/s | 2,061 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| gpt-oss-120b | MXFP4-Q8 (MLX) | MLX (mlx_lm, version not stated) | 88 tok/s | 1,325 tok/s4,096-token prompt | 4,096 | Apple MacBook Pro 14-inch (M5 Max, 40-core GPU) (same chip) | review2026-03 |
| gpt-oss-120b | 8bit (MLX) | oMLX (version not stated) | 92 tok/s | 2,305 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| gpt-oss-120b | 8bit (MLX) | oMLX (version not stated) | 90 tok/s | 1,717 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-09 |
| gpt-oss-20b | MXFP4 | llama.cpp master (2026-09-30, b10809 baseline) | 155 tok/s | 3,887 tok/s2,048-token prompt | 0 | a machine with the same chip | llama.cpp benchmark discussion2026-09 |
| gpt-oss-20b | 4-bit MLX (gpt-oss-20b) | oMLX 0.7.0 (MLX) | 134 tok/s | 1,039 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-10 |
| gpt-oss-20b | 4bit (MLX) | oMLX (version not stated) | 129 tok/s | 4,722 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| gpt-oss-20b | 4bit (MLX) | oMLX (version not stated) | 93 tok/s | 3,088 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| gpt-oss-20b | MXFP4 | llama.cpp master (2026-09-30, b10809 baseline) | – | 3,573 tok/s4,096-token prompt | 0 | a machine with the same chip | llama.cpp benchmark discussion2026-09 |
| gpt-oss-20b | MXFP4-Q8 (MLX) | MLX main (2026-08-18) / mlx_lm.benchmark | – | 4,131 tok/s512-token prompt | 0 | a machine with the same chip | MLX discussion2026-08 |
| gpt-oss-20b | MXFP4-Q8 (MLX) | MLX main (2026-08-18) / mlx_lm.benchmark | – | 5,308 tok/s2,048-token prompt | 0 | a machine with the same chip | MLX discussion2026-08 |
| Hy3 | mixed 2/3-bit experts, 8-bit attention/shared/output (MLX) | MLX 0.31.2 / mlx-lm 0.31.3 (eauchs hy_v3-mtp branch) | 7.4 tok/s | – | 0 | a machine with the same chip | MLX discussion2026-07 |
| Llama 3.1 8B Instruct | 4-bit MLX (Meta-Llama-3.1-8B-Instruct) | oMLX (version not stated) (MLX) | 108 tok/s | 3,459 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-07 |
| Llama 3.1 8B Instruct | Q4_K_M | LocalScore (llamafile/llama.cpp, version not stated) | 71 tok/s | 718 tok/s | 0 | a machine with the same chip | LocalScore2026-10 |
| Llama 3.3 70B Instruct | 4-bit MLX (Llama-3.3-70B-Instruct) | oMLX (version not stated) (MLX) | 13 tok/s | 358 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-07 |
| Llama 3.3 70B Instruct | 4bit (MLX) | oMLX (version not stated) | 13 tok/s | 340 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-07 |
| Llama 3.3 70B Instruct | 4bit (MLX) | oMLX (version not stated) | 13 tok/s | 367 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Llama 3.3 70B Instruct | 8bit (MLX) | oMLX (version not stated) | 7.1 tok/s | 94 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-07 |
| Llama 3.3 70B Instruct | 8bit (MLX) | oMLX (version not stated) | 6.8 tok/s | 280 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-07 |
| MiniMax M2.7 | 2bit (MLX) | oMLX (version not stated) | 49 tok/s | 1,221 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-07 |
| MiniMax M2.7 | 2bit (MLX) | oMLX (version not stated) | 48 tok/s | 1,225 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-08 |
| MiniMax M2.7 | 3bit (MLX) | oMLX (version not stated) | 55 tok/s | 1,257 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| MiniMax M2.7 | 3bit (MLX) | oMLX (version not stated) | 55 tok/s | 965 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-07 |
| MiniMax M2.7 | 4-bit MLX (MiniMax-M2.7-ConfigI-MLX) | oMLX 0.5.7 (MLX) | 56 tok/s | 782 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-08 |
| Mistral Small 4 119B (2603) | 4-bit MLX | mlx-lm PR #1075 branch, commit 1fcaacc | 104 tok/s | – | 0 | a machine with the same chip | MLX discussion2026-03 |
| Mistral Small 4 119B (2603) | 4-bit MLX | mlx-lm PR #1075 branch (MLX version not stated) | 93 tok/s | – | 0 | a machine with the same chip | MLX discussion2026-03 |
| Mistral Small 4 119B (2603) | 4-bit MLX | mlx-lm PR #1075 branch (MLX version not stated) | 90 tok/s | – | 0 | a machine with the same chip | MLX discussion2026-03 |
| Mistral Small 4 119B (2603) | 4-bit MLX (Mistral-Small-4-119B-2603-4bit) | oMLX 0.7.0 (MLX) | 87 tok/s | 1,507 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Nemotron 3 Super 120B-A12B | 4-bit MLX (NVIDIA-Nemotron-3-Super-120B-A12B) | oMLX 0.5.0 (MLX) | 58 tok/s | 880 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-07 |
| Nemotron 3.5 Lightning 30B-A3B | 4bit (MLX) | oMLX (version not stated) | 142 tok/s | 4,266 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Nemotron 3.5 Lightning 30B-A3B | 4bit (MLX) | oMLX (version not stated) | 141 tok/s | 4,061 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Nemotron 3.5 Lightning 30B-A3B | 8bit (MLX) | oMLX (version not stated) | 115 tok/s | 2,299 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Ornith 1.5 35B-A3B | 4-bit MLX (Ornith-1.5-35B-A3B-MLX-4bit) | oMLX (version not stated) (MLX) | 138 tok/s | 4,312 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3 8B | 4-bit MLX (Qwen3-8B) | oMLX (version not stated) (MLX) | 92 tok/s | 3,346 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-07 |
| Qwen3 8B | 4-bit MLX | MLX main (2026-08-25) / mlx_lm.generate | – | 2,791 tok/s4,096-token prompt | 0 | a machine with the same chip | MLX discussion2026-08 |
| Qwen3 8B | 4-bit MLX | MLX main (2026-08-25) / mlx_lm.generate | – | 1,970 tok/s8,192-token prompt | 0 | a machine with the same chip | MLX discussion2026-08 |
| Qwen3 Coder 30B-A3B Instruct | 4-bit MLX (Qwen3-Coder-30B-A3B-Instruct-MLX-4bit) | oMLX (version not stated) (MLX) | 107 tok/s | 4,804 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3 Coder Next | 4bit (MLX) | oMLX (version not stated) | 127 tok/s | 4,073 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3 Coder Next | 4bit (MLX) | oMLX (version not stated) | 98 tok/s | 3,795 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3 Coder Next | 4-bit MLX (Qwen3-Coder-Next-MLX-4bit) | oMLX 0.7.0 (MLX) | 96 tok/s | 3,155 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3 Coder Next | 5bit (MLX) | oMLX (version not stated) | 92 tok/s | 3,204 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3 Coder Next | 8bit (MLX) | oMLX (version not stated) | 80 tok/s | 3,473 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3 Coder Next | 8-bit MLX | MLX (mlx_lm, version not stated) | 79 tok/s | 754 tok/s4,096-token prompt | 4,096 | Apple MacBook Pro 14-inch (M5 Max, 40-core GPU) (same chip) | review2026-03 |
| Qwen3.5 122B-A10B | 4-bit MLX (Qwen3.5-122B-A10B-4bit) | oMLX (version not stated) (MLX) | 70 tok/s | 1,773 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.5 122B-A10B | 4-bit MLX | MLX (mlx_lm, version not stated) | 66 tok/s | 881 tok/s4,096-token prompt | 4,096 | Apple MacBook Pro 14-inch (M5 Max, 40-core GPU) (same chip) | review2026-03 |
| Qwen3.5 122B-A10B | oQ4 (MLX 4-bit) | mlx-lm PR #1456 branch (MLX version not stated) | 50 tok/s | – | 0 | a machine with the same chip | MLX discussion2026-07 |
| Qwen3.5 122B-A10B | oQ4 (MLX 4-bit) | mlx-lm PR #1456 branch (MLX version not stated) | 48 tok/s | – | 0 | a machine with the same chip | MLX discussion2026-07 |
| Qwen3.5 397B-A17B | 4-bit MLX (Qwen3.5-397B-A17B-2.416bit) | oMLX (version not stated) (MLX) | 42 tok/s | 485 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-05 |
| Qwen3.5 4B | NVFP4 (MLX) | MLX main (2026-08-01) / mlx_lm.benchmark | 128 tok/s | 5,395 tok/s2,048-token prompt | 2,048 | a machine with the same chip | MLX discussion2026-08 |
| Qwen3.5 9B | 4-bit MLX (Qwen3.5-9B-MLX-4bit) | oMLX (version not stated) (MLX) | 103 tok/s | 2,804 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.6 27B | 4bit (MLX) | oMLX (version not stated) | 33 tok/s | 992 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.6 27B | 8bit (MLX) | oMLX (version not stated) | 19 tok/s | 944 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.6 27B | 8bit (MLX) | oMLX (version not stated) | 16 tok/s | 853 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.6 27B | 8-bit MLX | MLX 0.32.4 / mlx-lm 0.32.2 | 15 tok/s | – | 512 | a machine with the same chip | MLX discussion2026-10 |
| Qwen3.6 35B-A3B | 4bit (MLX) | oMLX (version not stated) | 151 tok/s | 3,247 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.6 35B-A3B | 4-bit MLX (Qwen3.6-35B-A3B-MLX-4bit) | oMLX (version not stated) (MLX) | 146 tok/s | 4,625 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.6 35B-A3B | 4-bit MLX | MLX 0.32.0 / mlx-lm 0.31.3 | – | 3,909 tok/s8,192-token prompt | 0 | a machine with the same chip | independent benchmark2026-08 |
| Qwen3.6 35B-A3B | 4-bit (BaseRT .base bundle) | BaseRT 0.2.2 | – | 2,671 tok/s8,192-token prompt | 0 | a machine with the same chip | independent benchmark2026-08 |
| Qwen3.6 35B-A3B | 4-bit MLX | MLX 0.32.0 / mlx-lm 0.31.3 | – | 3,316 tok/s32,768-token prompt | 0 | a machine with the same chip | independent benchmark2026-08 |
| Qwen3.6 35B-A3B | 4-bit (BaseRT .base bundle) | BaseRT 0.2.2 | – | 2,271 tok/s32,768-token prompt | 0 | a machine with the same chip | independent benchmark2026-08 |
| Qwen3.6 35B-A3B | 4-bit MLX | MLX 0.32.0 / mlx-lm 0.31.3 | – | 2,388 tok/s65,536-token prompt | 0 | a machine with the same chip | independent benchmark2026-08 |
| Qwen3.6 35B-A3B | 4-bit (BaseRT .base bundle) | BaseRT 0.2.2 | – | 2,004 tok/s65,536-token prompt | 0 | a machine with the same chip | independent benchmark2026-08 |
| Qwen3.6 35B-A3B | 4-bit MLX | MLX 0.32.0 / mlx-lm 0.31.3 | – | 1,390 tok/s131,072-token prompt | 0 | a machine with the same chip | independent benchmark2026-08 |
| Qwen3.6 35B-A3B | 4-bit (BaseRT .base bundle) | BaseRT 0.2.2 | – | 1,379 tok/s131,072-token prompt | 0 | a machine with the same chip | independent benchmark2026-08 |
| Qwen3.6 35B-A3B | 4-bit MLX | MLX 0.32.0 / mlx-lm 0.31.3 | – | 973 tok/s196,608-token prompt | 0 | a machine with the same chip | independent benchmark2026-08 |
| Qwen3.6 35B-A3B | 4-bit (BaseRT .base bundle) | BaseRT 0.2.2 | – | 995 tok/s196,608-token prompt | 0 | a machine with the same chip | independent benchmark2026-08 |
| Qwen3.6 35B-A3B | 8-bit MLX | MLX 0.32.4 / mlx-lm 0.32.2 | 113 tok/s | – | 512 | a machine with the same chip | MLX discussion2026-10 |
| Qwen3.6 35B-A3B | 8bit (MLX) | oMLX (version not stated) | 104 tok/s | 1,965 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.6 35B-A3B | 8-bit MLX | MLX 0.32.4 / mlx-lm 0.32.2 | 101 tok/s | – | 4,096 | a machine with the same chip | MLX discussion2026-10 |
| Qwen3.6 35B-A3B | 8-bit MLX | MLX 0.32.4 / mlx-lm 0.32.2 | 101 tok/s | – | 512 | a machine with the same chip | MLX discussion2026-10 |
| Qwen3.6 35B-A3B | 8-bit MLX | MLX 0.32.4 / mlx-lm 0.32.2 | 97 tok/s | – | 512 | a machine with the same chip | MLX discussion2026-10 |
| Qwen3.6 35B-A3B | 8bit (MLX) | oMLX (version not stated) | 94 tok/s | 2,227 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.8 27B | NVFP4 (MLX) | MLX main (2026-10-16) / mlx_lm.benchmark | 34 tok/s | – | 512 | a machine with the same chip | MLX discussion2026-10 |
| Qwen3.8 27B | 4-bit MLX (Qwen3.8-27B-4bit) | oMLX (version not stated) (MLX) | 33 tok/s | 931 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.8 Flash-Next | 4-bit MLX (Qwen3.8-Flash-Next-MLX-mixed-4_8bit) | oMLX 0.7.2 (MLX) | 43 tok/s | 512 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.8 Flash-Next | 4bit (MLX) | oMLX (version not stated) | 35 tok/s | 1,087 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
With speculative decoding
A small draft model or extra prediction heads guess several tokens ahead. Generation is faster on text the guesses get right, so these are kept out of the medians above.
| Model | Quant | Backend | Generation | Prompt | Context | Measured on | Source |
|---|---|---|---|---|---|---|---|
| Gemma 4 26B-A4B IT | 4bit (MLX) | oMLX (version not stated) | 102 tok/s | 4,222 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 26B-A4B IT | 4bit (MLX) | oMLX (version not stated) | 66 tok/s | 2,026 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 26B-A4B IT | 6bit (MLX) | oMLX (version not stated) | 98 tok/s | 3,864 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 31B IT | 4bit (MLX) | oMLX (version not stated) | 38 tok/s | 674 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 31B IT | 4bit (MLX) | oMLX (version not stated) | 38 tok/s | 568 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 31B IT | 6bit (MLX) | oMLX (version not stated) | 40 tok/s | 693 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 31B IT | 8bit (MLX) | oMLX (version not stated) | 27 tok/s | 651 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 31B IT | 8bit (MLX) | oMLX (version not stated) | 15 tok/s | 633 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| GLM-5.3 Flash | 2bit (MLX) | oMLX (version not stated) | 56 tok/s | 1,165 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| GLM-5.3 Flash | 2bit (MLX) | oMLX (version not stated) | 42 tok/s | 881 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| GLM-5.3 Flash | 3bit (MLX) | oMLX (version not stated) | 7.9 tok/s | 204 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Nemotron 3.5 Lightning 30B-A3B | 4bit (MLX) | oMLX (version not stated) | 185 tok/s | 4,205 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Nemotron 3.5 Lightning 30B-A3B | 4bit (MLX) | oMLX (version not stated) | 181 tok/s | 4,160 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.5 122B-A10B | oQ4 (MLX 4-bit) | mlx-lm PR #1456 branch (MLX version not stated) | 104 tok/s | – | 0 | a machine with the same chip | MLX discussion2026-07 |
| Qwen3.6 27B | 6bit (MLX) | oMLX (version not stated) | 45 tok/s | 929 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.6 35B-A3B | 4bit (MLX) | oMLX (version not stated) | 185 tok/s | 5,372 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.6 35B-A3B | 4bit (MLX) | oMLX (version not stated) | 168 tok/s | 2,756 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.6 35B-A3B | 8bit (MLX) | oMLX (version not stated) | 108 tok/s | 5,154 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.6 35B-A3B | 8bit (MLX) | oMLX (version not stated) | 92 tok/s | 4,454 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.8 27B | Q4_K_M | llama.cpp commit 1deefcca3 (PR #27342 branch) | 25 tok/s | – | 0 | a machine with the same chip | llama.cpp benchmark discussion2026-08 |
| Qwen3.8 27B | Q4_K_M | llama.cpp commit 1deefcca3 (PR #27342 branch) | 23 tok/s | 510 tok/s383-token prompt | 0 | a machine with the same chip | llama.cpp benchmark discussion2026-08 |
| Qwen3.8 Flash-Next | 4bit (MLX) | oMLX (version not stated) | 97 tok/s | 1,875 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.8 Flash-Next | 4bit (MLX) | oMLX (version not stated) | 78 tok/s | 958 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
Many users at once
Total throughput with several requests in flight. One person chatting sees far less than this.
| Model | Quant | Backend | Generation | Prompt | Context | Measured on | Source |
|---|---|---|---|---|---|---|---|
| Qwen3.8 27B | MXFP4 (MLX) | MLX main (2026-10-16) / mlx_lm.benchmark16 requests at once | 150 tok/s | – | 512 | a machine with the same chip | MLX discussion2026-10 |
| Qwen3.8 27B | NVFP4 (MLX) | MLX main (2026-10-16) / mlx_lm.benchmark16 requests at once | 150 tok/s | – | 512 | a machine with the same chip | MLX discussion2026-10 |
| Qwen3.8 27B | NVFP4 (MLX) | MLX main (2026-10-16) / mlx_lm.benchmark8 requests at once | 113 tok/s | – | 512 | a machine with the same chip | MLX discussion2026-10 |
| Qwen3.8 27B | MXFP8 (MLX) | MLX main (2026-10-16) / mlx_lm.benchmark16 requests at once | 128 tok/s | – | 512 | a machine with the same chip | MLX discussion2026-10 |
Deep into a long context
Measured with more than 4,096 tokens already in context, where every new token also reads the context memory.
| Model | Quant | Backend | Generation | Prompt | Context | Measured on | Source |
|---|---|---|---|---|---|---|---|
| Devstral Small 2 24B (2512) | 4-bit MLX (Devstral-Small-2-24B-Instruct-25124Bit) | oMLX (version not stated) (MLX) | 29 tok/s | 763 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-05 |
| Devstral Small 2 24B (2512) | 4-bit MLX (Devstral-Small-2-24B-Instruct-2512-4bit) | oMLX 0.7.0 (MLX) | 28 tok/s | 832 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 12B IT | 4-bit MLX (gemma-4-12B-it-4bit) | oMLX (version not stated) (MLX) | 59 tok/s | 1,508 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 12B IT | 4-bit MLX (gemma-4-12B-it-OptiQ-4bit) | oMLX (version not stated) (MLX) | 30 tok/s | 936 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-08 |
| Gemma 4 26B-A4B IT | BF16 (MLX) | mlx-vlm on TheTom/mlx fork (baseline KV) | 43 tok/s | – | 60,000 | a machine with the same chip | MLX discussion2026-04 |
| Gemma 4 26B-A4B IT | 4-bit MLX (gemma-4-26b-a4b-it-4bit) | oMLX (version not stated) (MLX) | 91 tok/s | 3,982 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 26B-A4B IT | 4-bit MLX (gemma-4-26B-A4B-it-QAT-MLX-4bit) | oMLX (version not stated) (MLX) | 67 tok/s | 3,634 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 31B IT | 4-bit MLX (gemma-4-31B-it-qat-OptiQ-4bit) | oMLX (version not stated) (MLX) | 17 tok/s | 621 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 31B IT | 4-bit MLX (gemma-4-31B-it-qat-OptiQ-4bit) | oMLX (version not stated) (MLX) | 16 tok/s | 591 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-10 |
| GLM-4.5-Air | 4-bit MLX (GLM-4.5-Air) | oMLX 0.3.9rc1 (MLX) | 29 tok/s | 710 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-05 |
| GLM-4.5-Air | 4-bit MLX (GLM-4.5-Air) | oMLX 0.3.9rc1 (MLX) | 15 tok/s | 474 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-05 |
| GLM-5.3 Flash | 4-bit MLX (GLM-5.3-Flash-oQ4e) | oMLX 0.7.0 (MLX) | 4.2 tok/s | 172 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-10 |
| gpt-oss-120b | MXFP4-Q8 (MLX) | MLX (mlx_lm, version not stated) | 76 tok/s | 2,710 tok/s16,384-token prompt | 16,384 | Apple MacBook Pro 14-inch (M5 Max, 40-core GPU) (same chip) | review2026-03 |
| gpt-oss-120b | 4-bit MLX (gpt-oss-120b-MXFP4-Q8) | oMLX 0.7.0.dev4 (MLX) | 73 tok/s | 1,479 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-09 |
| gpt-oss-120b | MXFP4-Q8 (MLX) | MLX (mlx_lm, version not stated) | 65 tok/s | 2,537 tok/s32,768-token prompt | 32,768 | Apple MacBook Pro 14-inch (M5 Max, 40-core GPU) (same chip) | review2026-03 |
| gpt-oss-120b | 4-bit MLX (gpt-oss-120b-mxfp4) | oMLX 0.4.4 (MLX) | 62 tok/s | 1,620 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-06 |
| gpt-oss-20b | 4-bit MLX (gpt-oss-20b-MXFP4-Q8) | oMLX 0.5.7 (MLX) | 89 tok/s | 2,915 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-08 |
| gpt-oss-20b | 4-bit MLX (gpt-oss-20b-MXFP4-Q8) | oMLX 0.6.4 (MLX) | 85 tok/s | 3,349 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Llama 3.3 70B Instruct | 4-bit MLX (Llama-3.3-70B-Instruct) | oMLX (version not stated) (MLX) | 8.1 tok/s | 237 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-04 |
| Llama 3.3 70B Instruct | 4-bit MLX (Llama-3.3-70B-Instruct) | oMLX (version not stated) (MLX) | 6.7 tok/s | 173 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-05 |
| MiniMax M2.7 | 4-bit MLX (MiniMax-M2.7-RAM-90GB) | oMLX 0.3.8 (MLX) | 33 tok/s | 936 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-05 |
| MiniMax M2.7 | 4-bit MLX (MiniMax-M2.7-ConfigI) | oMLX 0.3.6 (MLX) | 21 tok/s | 795 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-04 |
| Nemotron 3 Super 120B-A12B | 4-bit MLX (Nemotron-3-Super-120B-A12B) | oMLX 0.3.12 (MLX) | 55 tok/s | 1,060 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-06 |
| Nemotron 3 Super 120B-A12B | 4-bit MLX (NVIDIA-Nemotron-3-Super-120B-A12B) | oMLX 0.3.9 (MLX) | 52 tok/s | 906 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-05 |
| Ornith 1.5 35B-A3B | 4-bit MLX (Ornith-1.5-35B-A3B-MLX-4bit) | oMLX (version not stated) (MLX) | 117 tok/s | 4,026 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Ornith 1.5 35B-A3B | 4-bit MLX (Ornith-1.5-35B-A3B-MLX-4bit) | oMLX (version not stated) (MLX) | 114 tok/s | 3,626 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3 8B | 4-bit MLX (Qwen3-8B) | oMLX (version not stated) (MLX) | 3.3 tok/s | 2,072 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-04 |
| Qwen3 8B | 4-bit MLX (Qwen3-8B) | oMLX (version not stated) (MLX) | 1.8 tok/s | 1,701 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-04 |
| Qwen3 Coder 30B-A3B Instruct | 4-bit MLX (Qwen3-Coder-30B-A3B-Instruct-4bit-dwq-v2) | oMLX (version not stated) (MLX) | 91 tok/s | 4,042 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3 Coder 30B-A3B Instruct | 4-bit MLX (Qwen3-Coder-30B-A3B-Instruct-MLX-4bit) | oMLX (version not stated) (MLX) | 65 tok/s | 2,236 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-08 |
| Qwen3 Coder Next | 4-bit MLX (Qwen3-Coder-Next-4bit) | oMLX 0.6.4 (MLX) | 81 tok/s | 1,969 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-09 |
| Qwen3 Coder Next | 4-bit MLX (Qwen3-Coder-Next-4bit) | oMLX 0.7.0.dev2 (MLX) | 75 tok/s | 1,859 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-09 |
| Qwen3 Coder Next | 8-bit MLX | MLX (mlx_lm, version not stated) | 74 tok/s | 1,802 tok/s16,384-token prompt | 16,384 | Apple MacBook Pro 14-inch (M5 Max, 40-core GPU) (same chip) | review2026-03 |
| Qwen3 Coder Next | 8-bit MLX | MLX (mlx_lm, version not stated) | 69 tok/s | 1,887 tok/s32,768-token prompt | 32,768 | Apple MacBook Pro 14-inch (M5 Max, 40-core GPU) (same chip) | review2026-03 |
| Qwen3 Coder Next | 8-bit MLX | MLX (mlx_lm, version not stated) | 48 tok/s | 1,432 tok/s65,536-token prompt | 65,536 | Apple MacBook Pro 14-inch (M5 Max, 40-core GPU) (same chip) | review2026-03 |
| Qwen3.5 122B-A10B | 4-bit MLX (Qwen3.5-122B-A10B-4bit) | oMLX (version not stated) (MLX) | 63 tok/s | 1,756 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.5 122B-A10B | 4-bit MLX | MLX (mlx_lm, version not stated) | 61 tok/s | 1,239 tok/s16,384-token prompt | 16,384 | Apple MacBook Pro 14-inch (M5 Max, 40-core GPU) (same chip) | review2026-03 |
| Qwen3.5 122B-A10B | 4-bit MLX | MLX (mlx_lm, version not stated) | 55 tok/s | 1,068 tok/s32,768-token prompt | 32,768 | Apple MacBook Pro 14-inch (M5 Max, 40-core GPU) (same chip) | review2026-03 |
| Qwen3.5 397B-A17B | 4-bit MLX (Qwen3.5-397B-A17B-2.416bit) | oMLX (version not stated) (MLX) | 34 tok/s | 754 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-04 |
| Qwen3.5 397B-A17B | 4-bit MLX (Qwen3.5-397B-A17B-2.416bit) | oMLX (version not stated) (MLX) | 29 tok/s | 622 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-04 |
| Qwen3.5 9B | 4-bit MLX (Qwen3.5-9B-MLX-4bit) | oMLX (version not stated) (MLX) | 96 tok/s | 4,563 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.5 9B | 4-bit MLX (Qwen3.5-9B-MLX-4bit) | oMLX (version not stated) (MLX) | 87 tok/s | 2,951 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.6 27B | 4-bit MLX (Qwen3.6-27B-4bit) | oMLX (version not stated) (MLX) | 31 tok/s | 892 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.6 35B-A3B | 4-bit MLX (Qwen3.6-35B-A3B-UD-MLX-4bit) | oMLX (version not stated) (MLX) | 95 tok/s | 4,334 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.8 27B | 4-bit MLX (Qwen3.8-27B-4bit) | oMLX (version not stated) (MLX) | 31 tok/s | 983 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.8 27B | 4-bit MLX (Qwen3.8-27B-4bit) | oMLX (version not stated) (MLX) | 30 tok/s | 917 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.8 27B | 4-bit MLX | MLX (version not stated) / mlx-lm 0.31.3 | 21 tok/s | 432 tok/s131,072-token prompt | 131,072 | a machine with the same chip | MLX discussion2026-08 |
| Qwen3.8 27B | Q4_K_M | llama.cpp b11541 | – | 633 tok/s2,048-token prompt | 8,192 | this machine | llama.cpp benchmark discussion2026-10 |
| Qwen3.8 27B | Q4_K_M | llama.cpp b11541 + LXD-8 metal-m5-prefill patches (unmerged fork) | – | 1,064 tok/s2,048-token prompt | 8,192 | this machine | llama.cpp benchmark discussion2026-10 |
| Qwen3.8 27B | Q4_K_M | llama.cpp b11541 | – | 528 tok/s2,048-token prompt | 32,768 | this machine | llama.cpp benchmark discussion2026-10 |
| Qwen3.8 27B | Q4_K_M | llama.cpp b11541 + LXD-8 metal-m5-prefill patches (unmerged fork) | – | 884 tok/s2,048-token prompt | 32,768 | this machine | llama.cpp benchmark discussion2026-10 |
| Qwen3.8 Flash-Next | IQ3_S | llama.cpp b11429 | 27 tok/s | – | 175,000 | a machine with the same chip | llama.cpp benchmark discussion2026-10 |
| Qwen3.8 Flash-Next | 4-bit MLX (Qwen3.8-Flash-Next-MLX-mixed-4_8bit) | oMLX 0.7.2 (MLX) | 46 tok/s | 2,229 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.8 Flash-Next | 4-bit MLX (Qwen3.8-Flash-Next-MLX-mixed-4_8bit) | oMLX 0.7.2 (MLX) | 37 tok/s | 2,308 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.8 Flash-Next | IQ4_XS | llama.cpp master (2026-09-03) | 35 tok/s | – | 16,384 | a machine with the same chip | llama.cpp benchmark discussion2026-09 |
| Qwen3.8 Flash-Next | IQ4_XS | llama.cpp master (2026-09-03) | 35 tok/s | – | 32,768 | a machine with the same chip | llama.cpp benchmark discussion2026-09 |
| Qwen3.8 Flash-Next | IQ4_XS | llama.cpp master (2026-09-03) | 28 tok/s | 388 tok/s2,048-token prompt | 65,536 | a machine with the same chip | llama.cpp benchmark discussion2026-09 |
| Qwen3.8 Flash-Next | IQ4_XS | llama.cpp master (2026-09-03) | 21 tok/s | 340 tok/s2,048-token prompt | 131,072 | a machine with the same chip | llama.cpp benchmark discussion2026-09 |
Specifications
- Chip
- M5 Max (18-core CPU, 40-core GPU)
- Memory
- unified LPDDR (Apple does not state the generation)
- Memory bandwidth
- 614 GB/s
- FP16 compute
- Apple does not publish TFLOPS for the M5 GPU or its Neural Accelerators
- Software
- Metal, MLX
- Ships with
- macOS
- Power
- 480 W (Maximum continuous power, Apple tech specs)
- Form factor
- Desktop
- Networking
- 10GbE; four Thunderbolt 5 (120Gb/s) on the back, Thunderbolt 5 clustering with RDMA per Apple
- Released
- 2026-09
- Maker page
- apple.com
Sources
- apple.com: chip, memory options, bandwidth, ports, 10GbE, 480W max continuous power
- apple.com: base prices per chip
- apple.com: available 2026-09-22
- apple.com: clustering over Thunderbolt 5 with RDMA; MLX
- apple.com: memory deltas: 48GB +$0, 64GB +$400, 128GB +$2,000 over $3,099
- github.com: how far the GPU memory limit can be raised on a Mac (sysctl iogpu.wired_limit_mb)
- github.com: the default GPU memory limit on a Mac and how to raise it
Compared with
Similar machines
- Apple MacBook Pro 14-inch (M5 Max, 40-core GPU)~$4,699–$6,699
- Apple MacBook Pro 16-inch (M5 Max, 40-core GPU)~$4,999–$6,999
- Apple Mac Studio (M5 Max, 32-core GPU)~$2,499