← All machines
Apple unified memory · Apple
Apple MacBook Pro 14-inch (M5 Max, 40-core GPU)
The full M5 Max in the 14-inch body, configurable to 128GB, so a 70B model at 4-bit fits on a laptop.
Memory for models
96 GBassumption
Assumed three quarters of 128 GB for macOS's default GPU limit, which Apple does not publish.
Memory bandwidth
614 GB/s
Maker price
~$4,699–$6,699
Largest at 4-bit
82.6 GB at 8,192 tokens of context, default GPU limit (an assumed limit, see Memory for models)
gpt-oss-20b, 4-bit
132 tok/smedian of 4 runs
Memory configurations
One page covers every memory option the maker sells. Prices are the maker’s list price for the cheapest configuration with that much memory.
| Memory | Price | Memory the GPU can use |
|---|---|---|
| 48 GB | ~$4,699 | 36 GB default gpu limitassumption 40 GB limit raised with sysctlassumption |
| 64 GB | ~$5,099 | 48 GB default gpu limitassumption 56 GB limit raised with sysctlassumption |
| 128 GB | ~$6,699 | 96 GB default gpu limitassumption 120 GB limit raised with sysctlassumption |
Assumed three quarters of 128 GB for macOS's default GPU limit, which Apple does not publish. Raised with iogpu.wired_limit_mb per the MLX docs, leaving an assumed 8 GB for macOS. How GPU memory limits work on unified-memory machines.
What it runs at 4-bit
Every model in the catalog at its 4-bit file with 8,192 tokens of context. 30 of 41 fit in at least one configuration. Speeds are the median of measured runs on the same chip where any exist; otherwise an estimate from memory bandwidth. Try other quants and context lengths.
| Model | Needs | Fit | Generation speed |
|---|---|---|---|
| Kimi K32.8T, 104B active · UD-Q4_K_XL | 1406.8 GBcontext memory estimated | Does not fit | |
| DeepSeek V4 Pro1.6T, 49B active · Q4_K_M | 890.5 GBcontext memory not counted | Does not fit | |
| Kimi K2.61T, 32B active · UD-Q4_K_XL | 545.7 GBcontext memory estimated | Does not fit | |
| DeepSeek V4.1 Flash552B, 16B active · Q4 (DwarfStar) | 484.5 GB | Does not fit | |
| GLM-5.3753.3B · UD-Q4_K_XL | 437.4 GBcontext memory estimated | Does not fit | |
| DeepSeek V3.2671B, 37B active · Q4_K_M | 379.6 GBcontext memory estimated | Does not fit | |
| MiniMax M3428B, 23B active · Q4_K_M | 244.8 GBcontext memory not counted | Does not fit | |
| Qwen3.5 397B-A17B397B, 17B active · Q4_K_M | 229.1 GBcontext memory estimated | Does not fit | |
| GLM-5.3 Flash320B, 18B active · UD-Q4_K_XL | 187.6 GBcontext memory estimated | Does not fit | |
| Hy3295B, 21B active · Q4_K_M | 173.6 GB | Does not fit | |
| MiniMax M2.7228.7B · Q4_K_M | 132.7 GB | Does not fit | |
| Qwen3.8 Flash-Next125B, 6B active · Q4_K_M | 113.1 GBcontext memory estimated | Fits with raised limitassumed limitfrom the 128 GB config | 39 tok/smedian of 2 runs |
| Nemotron 3 Super 120B-A12B120B, 12B active · Q4_K_M | 82.6 GBcontext memory estimated | Fitsassumed limitfrom the 128 GB config | 58 tok/smedian of 1 run |
| Qwen3.5 122B-A10B122B, 10B active · Q4_K_M | 73.0 GBcontext memory estimated | Fitsassumed limitfrom the 128 GB config | 58 tok/smedian of 4 runs |
| Mistral Medium 3.5 128B128B · Q4_K_M | 74.0 GB | Fitsassumed limitfrom the 128 GB config | ~6.0–7.5 tok/sestimate |
| GLM-4.5-Air106B, 12B active · Q4_K_M | 70.9 GB | Fitsassumed limitfrom the 128 GB config | 53 tok/smedian of 1 run |
| Mistral Small 4 119B (2603)119B, 6.5B active · Q4_K_M | 69.3 GBcontext memory estimated | Fitsassumed limitfrom the 128 GB config | 92 tok/smedian of 4 runs |
| Llama 4 Scout 17B-16E Instruct109B, 17B active · Q4_K_M | 62.4 GBcontext memory not counted | Fitsassumed limitfrom the 128 GB config | ~20–36 tok/sestimate |
| gpt-oss-120b117B, 5.1B active · MXFP4 (native) | 60.2 GBcontext memory estimated | Fitsassumed limitfrom the 128 GB config | 90 tok/smedian of 4 runs |
| Qwen3 Coder Next80B, 3B active · Q4_K_M | 46.9 GBcontext memory estimated | Fitsassumed limitfrom the 64 GB config | 98 tok/smedian of 3 runs |
| Llama 3.3 70B Instruct70B · Q4_K_M | 43.6 GB | Fitsassumed limitfrom the 64 GB config | 13 tok/smedian of 3 runs |
| Nemotron 3.5 Lightning 30B-A3B30B, 3B active · Q4_K_M | 25.3 GBcontext memory estimated | Fitsassumed limitfrom the 48 GB config | 142 tok/smedian of 2 runs |
| Qwen3.6 35B-A3B35B, 3B active · Q4_K_M | 22.4 GBcontext memory estimated | Fitsassumed limitfrom the 48 GB config | 148 tok/smedian of 2 runs |
| Ornith 1.5 35B-A3B35B, 3B active · Q4_K_M | 21.9 GBcontext memory estimated | Fitsassumed limitfrom the 48 GB config | 138 tok/smedian of 1 run |
| Qwen3 Coder 30B-A3B Instruct30.5B, 3.3B active · Q4_K_M | 19.5 GB | Fitsassumed limitfrom the 48 GB config | 107 tok/smedian of 1 run |
| Gemma 4 31B IT30.7B · Q4_K_M | 18.9 GBcontext memory estimated | Fitsassumed limitfrom the 48 GB config | 21 tok/smedian of 3 runs |
| Granite 4.2 30B29.3B · Q4_K_M | 20.3 GB | Fitsassumed limitfrom the 48 GB config | ~25–31 tok/sestimate |
| Qwen3.8 27B27B · Q4_K_M | 18.2 GBcontext memory estimated | Fitsassumed limitfrom the 48 GB config | 34 tok/smedian of 2 runs |
| Gemma 4 26B-A4B IT25.2B, 3.8B active · Q4_K_M | 17.4 GBcontext memory estimated | Fitsassumed limitfrom the 48 GB config | 116 tok/smedian of 3 runs |
| Qwen3.6 27B27B · Q4_K_M | 17.7 GBcontext memory estimated | Fitsassumed limitfrom the 48 GB config | 33 tok/smedian of 1 run |
| Devstral Small 2 24B (2512)24B · Q4_K_M | 16.1 GB | Fitsassumed limitfrom the 48 GB config | 38 tok/smedian of 1 run |
| gpt-oss-20b21B, 3.6B active · MXFP4 (native) | 12.5 GBcontext memory estimated | Fitsassumed limitfrom the 48 GB config | 132 tok/smedian of 4 runs |
| Phi-414.7B · Q4_K_M | 11.3 GB | Fitsassumed limitfrom the 48 GB config | ~51–63 tok/sestimate |
| Ministral 3 14B Reasoning (2512)13.9B · Q4_K_M | 10.4 GB | Fitsassumed limitfrom the 48 GB config | ~55–68 tok/sestimate |
| Gemma 4 12B IT11.95B · Q4_K_M | 8.2 GBcontext memory estimated | Fitsassumed limitfrom the 48 GB config | 44 tok/smedian of 1 run |
| Qwen3.5 9B9B · Q4_K_M | 7.0 GBcontext memory estimated | Fitsassumed limitfrom the 48 GB config | 103 tok/smedian of 1 run |
| Granite 4.2 8B8.8B · Q4_K_M | 7.9 GB | Fitsassumed limitfrom the 48 GB config | ~81–101 tok/sestimate |
| Gemma 4 E4B IT8B · Q4_K_M | 6.5 GBcontext memory not counted | Fitsassumed limitfrom the 48 GB config | ~148–185 tok/sestimate |
| Qwen3 8B8.2B · Q4_K_M | 7.3 GB | Fitsassumed limitfrom the 48 GB config | 92 tok/smedian of 1 run |
| Llama 3.1 8B Instruct8B · Q4_K_M | 7.1 GB | Fitsassumed limitfrom the 48 GB config | 90 tok/smedian of 2 runs |
| Qwen3.5 4B4B · Q4_K_M | 4.3 GBcontext memory estimated | Fitsassumed limitfrom the 48 GB config | 128 tok/smedian of 1 run |
Measured speeds
103 published runs with one user, plain decoding and at most 4,096 tokens already in context, on M5 Max (18-core CPU, 40-core GPU). Runs on 2 other machines with the same chip count here too, and say so.
| Model | Quant | Backend | Generation | Prompt | Context | Measured on | Source |
|---|---|---|---|---|---|---|---|
| Devstral Small 2 24B (2512) | 4-bit MLX (Devstral-Small-2-24B-Instruct-2512-4bit) | oMLX (version not stated) (MLX) | 38 tok/s | 1,113 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 12B IT | 4-bit MLX (gemma-4-12B-it-qat-4bit) | oMLX (version not stated) (MLX) | 44 tok/s | 1,704 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 26B-A4B IT | bf16 (MLX) | oMLX (version not stated) | 57 tok/s | 2,321 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 26B-A4B IT | BF16 (MLX) | mlx-vlm on TheTom/mlx fork (baseline KV) | 55 tok/s | – | 1,000 | a machine with the same chip | MLX discussion2026-04 |
| Gemma 4 26B-A4B IT | 4bit (MLX) | oMLX (version not stated) | 118 tok/s | 4,130 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 26B-A4B IT | 4-bit MLX (gemma-4-26B-A4B-it-QAT-MLX-4bit) | oMLX (version not stated) (MLX) | 116 tok/s | 3,984 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 26B-A4B IT | 4bit (MLX) | oMLX (version not stated) | 112 tok/s | 4,191 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 26B-A4B IT | Q4_0 | llama.cpp master (2026-04-25, build not stated) | – | 3,976 tok/s512-token prompt | 0 | a machine with the same chip | llama.cpp benchmark discussion2026-04 |
| Gemma 4 26B-A4B IT | Q4_0 | llama.cpp master (2026-04-25, build not stated) | – | 3,825 tok/s2,048-token prompt | 0 | a machine with the same chip | llama.cpp benchmark discussion2026-04 |
| Gemma 4 26B-A4B IT | Q8_0 | llama.cpp b10809 | 95 tok/s | 3,582 tok/s2,048-token prompt | 0 | a machine with the same chip | llama.cpp benchmark discussion2026-09 |
| Gemma 4 26B-A4B IT | Q8_0 | llama.cpp master (2026-04-25, build not stated) | – | 3,266 tok/s512-token prompt | 0 | a machine with the same chip | llama.cpp benchmark discussion2026-04 |
| Gemma 4 26B-A4B IT | Q8_0 | llama.cpp master (2026-04-25, build not stated) | – | 3,390 tok/s2,048-token prompt | 0 | a machine with the same chip | llama.cpp benchmark discussion2026-04 |
| Gemma 4 31B IT | 4bit (MLX) | oMLX (version not stated) | 25 tok/s | 752 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 31B IT | 4-bit MLX (gemma-4-31B-it-qat-OptiQ-4bit) | oMLX (version not stated) (MLX) | 21 tok/s | 630 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 31B IT | 4bit (MLX) | oMLX (version not stated) | 20 tok/s | 627 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 31B IT | Q4_0 | llama.cpp master (2026-04-25, build not stated) | – | 554 tok/s512-token prompt | 0 | a machine with the same chip | llama.cpp benchmark discussion2026-04 |
| Gemma 4 31B IT | Q4_0 | llama.cpp master (2026-04-25, build not stated) | – | 443 tok/s2,048-token prompt | 0 | a machine with the same chip | llama.cpp benchmark discussion2026-04 |
| Gemma 4 31B IT | 8-bit MLX | MLX 0.32.4 / mlx-lm 0.32.2 | 15 tok/s | – | 512 | a machine with the same chip | MLX discussion2026-10 |
| Gemma 4 31B IT | Q8_0 | llama.cpp master (2026-04-25, build not stated) | – | 473 tok/s512-token prompt | 0 | a machine with the same chip | llama.cpp benchmark discussion2026-04 |
| Gemma 4 31B IT | Q8_0 | llama.cpp master (2026-04-25, build not stated) | – | 354 tok/s2,048-token prompt | 0 | a machine with the same chip | llama.cpp benchmark discussion2026-04 |
| Gemma 4 E4B IT | Q8_0 | llama.cpp b10219 (llama-server) | 68 tok/s | 3,471 tok/s | 0 | a machine with the same chip | llama.cpp benchmark discussion2026-08 |
| Gemma 4 E4B IT | Q8_0 | llama.cpp b10219 | 67 tok/s | – | 0 | a machine with the same chip | llama.cpp benchmark discussion2026-08 |
| GLM-4.5-Air | 4-bit MLX (GLM-4.5-Air) | oMLX (version not stated) (MLX) | 53 tok/s | 1,085 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-07 |
| GLM-5.3 Flash | 4-bit MLX (GLM-5.3-Flash-oQ4e) | oMLX 0.7.0 (MLX) | 5.8 tok/s | 57 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-10 |
| GLM-5.3 Flash | 4bit (MLX) | oMLX (version not stated) | 5.1 tok/s | 180 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| gpt-oss-120b | 4-bit MLX (gpt-oss-120b-MXFP4-Q8) | oMLX 0.7.0rc1 (MLX) | 93 tok/s | 1,619 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-09 |
| gpt-oss-120b | 4bit (MLX) | oMLX (version not stated) | 90 tok/s | 2,193 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| gpt-oss-120b | 4bit (MLX) | oMLX (version not stated) | 89 tok/s | 2,061 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| gpt-oss-120b | MXFP4-Q8 (MLX) | MLX (mlx_lm, version not stated) | 88 tok/s | 1,325 tok/s4,096-token prompt | 4,096 | this machine | review2026-03 |
| gpt-oss-120b | 8bit (MLX) | oMLX (version not stated) | 92 tok/s | 2,305 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| gpt-oss-120b | 8bit (MLX) | oMLX (version not stated) | 90 tok/s | 1,717 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-09 |
| gpt-oss-20b | MXFP4 | llama.cpp master (2026-09-30, b10809 baseline) | 155 tok/s | 3,887 tok/s2,048-token prompt | 0 | a machine with the same chip | llama.cpp benchmark discussion2026-09 |
| gpt-oss-20b | 4-bit MLX (gpt-oss-20b) | oMLX 0.7.0 (MLX) | 134 tok/s | 1,039 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-10 |
| gpt-oss-20b | 4bit (MLX) | oMLX (version not stated) | 129 tok/s | 4,722 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| gpt-oss-20b | 4bit (MLX) | oMLX (version not stated) | 93 tok/s | 3,088 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| gpt-oss-20b | MXFP4 | llama.cpp master (2026-09-30, b10809 baseline) | – | 3,573 tok/s4,096-token prompt | 0 | a machine with the same chip | llama.cpp benchmark discussion2026-09 |
| gpt-oss-20b | MXFP4-Q8 (MLX) | MLX main (2026-08-18) / mlx_lm.benchmark | – | 4,131 tok/s512-token prompt | 0 | a machine with the same chip | MLX discussion2026-08 |
| gpt-oss-20b | MXFP4-Q8 (MLX) | MLX main (2026-08-18) / mlx_lm.benchmark | – | 5,308 tok/s2,048-token prompt | 0 | a machine with the same chip | MLX discussion2026-08 |
| Hy3 | mixed 2/3-bit experts, 8-bit attention/shared/output (MLX) | MLX 0.31.2 / mlx-lm 0.31.3 (eauchs hy_v3-mtp branch) | 7.4 tok/s | – | 0 | a machine with the same chip | MLX discussion2026-07 |
| Llama 3.1 8B Instruct | 4-bit MLX (Meta-Llama-3.1-8B-Instruct) | oMLX (version not stated) (MLX) | 108 tok/s | 3,459 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-07 |
| Llama 3.1 8B Instruct | Q4_K_M | LocalScore (llamafile/llama.cpp, version not stated) | 71 tok/s | 718 tok/s | 0 | a machine with the same chip | LocalScore2026-10 |
| Llama 3.3 70B Instruct | 4-bit MLX (Llama-3.3-70B-Instruct) | oMLX (version not stated) (MLX) | 13 tok/s | 358 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-07 |
| Llama 3.3 70B Instruct | 4bit (MLX) | oMLX (version not stated) | 13 tok/s | 340 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-07 |
| Llama 3.3 70B Instruct | 4bit (MLX) | oMLX (version not stated) | 13 tok/s | 367 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Llama 3.3 70B Instruct | 8bit (MLX) | oMLX (version not stated) | 7.1 tok/s | 94 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-07 |
| Llama 3.3 70B Instruct | 8bit (MLX) | oMLX (version not stated) | 6.8 tok/s | 280 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-07 |
| MiniMax M2.7 | 2bit (MLX) | oMLX (version not stated) | 49 tok/s | 1,221 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-07 |
| MiniMax M2.7 | 2bit (MLX) | oMLX (version not stated) | 48 tok/s | 1,225 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-08 |
| MiniMax M2.7 | 3bit (MLX) | oMLX (version not stated) | 55 tok/s | 1,257 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| MiniMax M2.7 | 3bit (MLX) | oMLX (version not stated) | 55 tok/s | 965 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-07 |
| MiniMax M2.7 | 4-bit MLX (MiniMax-M2.7-ConfigI-MLX) | oMLX 0.5.7 (MLX) | 56 tok/s | 782 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-08 |
| Mistral Small 4 119B (2603) | 4-bit MLX | mlx-lm PR #1075 branch, commit 1fcaacc | 104 tok/s | – | 0 | a machine with the same chip | MLX discussion2026-03 |
| Mistral Small 4 119B (2603) | 4-bit MLX | mlx-lm PR #1075 branch (MLX version not stated) | 93 tok/s | – | 0 | a machine with the same chip | MLX discussion2026-03 |
| Mistral Small 4 119B (2603) | 4-bit MLX | mlx-lm PR #1075 branch (MLX version not stated) | 90 tok/s | – | 0 | a machine with the same chip | MLX discussion2026-03 |
| Mistral Small 4 119B (2603) | 4-bit MLX (Mistral-Small-4-119B-2603-4bit) | oMLX 0.7.0 (MLX) | 87 tok/s | 1,507 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Nemotron 3 Super 120B-A12B | 4-bit MLX (NVIDIA-Nemotron-3-Super-120B-A12B) | oMLX 0.5.0 (MLX) | 58 tok/s | 880 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-07 |
| Nemotron 3.5 Lightning 30B-A3B | 4bit (MLX) | oMLX (version not stated) | 142 tok/s | 4,266 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Nemotron 3.5 Lightning 30B-A3B | 4bit (MLX) | oMLX (version not stated) | 141 tok/s | 4,061 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Nemotron 3.5 Lightning 30B-A3B | 8bit (MLX) | oMLX (version not stated) | 115 tok/s | 2,299 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Ornith 1.5 35B-A3B | 4-bit MLX (Ornith-1.5-35B-A3B-MLX-4bit) | oMLX (version not stated) (MLX) | 138 tok/s | 4,312 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3 8B | 4-bit MLX (Qwen3-8B) | oMLX (version not stated) (MLX) | 92 tok/s | 3,346 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-07 |
| Qwen3 8B | 4-bit MLX | MLX main (2026-08-25) / mlx_lm.generate | – | 2,791 tok/s4,096-token prompt | 0 | a machine with the same chip | MLX discussion2026-08 |
| Qwen3 8B | 4-bit MLX | MLX main (2026-08-25) / mlx_lm.generate | – | 1,970 tok/s8,192-token prompt | 0 | a machine with the same chip | MLX discussion2026-08 |
| Qwen3 Coder 30B-A3B Instruct | 4-bit MLX (Qwen3-Coder-30B-A3B-Instruct-MLX-4bit) | oMLX (version not stated) (MLX) | 107 tok/s | 4,804 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3 Coder Next | 4bit (MLX) | oMLX (version not stated) | 127 tok/s | 4,073 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3 Coder Next | 4bit (MLX) | oMLX (version not stated) | 98 tok/s | 3,795 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3 Coder Next | 4-bit MLX (Qwen3-Coder-Next-MLX-4bit) | oMLX 0.7.0 (MLX) | 96 tok/s | 3,155 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3 Coder Next | 5bit (MLX) | oMLX (version not stated) | 92 tok/s | 3,204 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3 Coder Next | 8bit (MLX) | oMLX (version not stated) | 80 tok/s | 3,473 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3 Coder Next | 8-bit MLX | MLX (mlx_lm, version not stated) | 79 tok/s | 754 tok/s4,096-token prompt | 4,096 | this machine | review2026-03 |
| Qwen3.5 122B-A10B | 4-bit MLX (Qwen3.5-122B-A10B-4bit) | oMLX (version not stated) (MLX) | 70 tok/s | 1,773 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.5 122B-A10B | 4-bit MLX | MLX (mlx_lm, version not stated) | 66 tok/s | 881 tok/s4,096-token prompt | 4,096 | this machine | review2026-03 |
| Qwen3.5 122B-A10B | oQ4 (MLX 4-bit) | mlx-lm PR #1456 branch (MLX version not stated) | 50 tok/s | – | 0 | a machine with the same chip | MLX discussion2026-07 |
| Qwen3.5 122B-A10B | oQ4 (MLX 4-bit) | mlx-lm PR #1456 branch (MLX version not stated) | 48 tok/s | – | 0 | a machine with the same chip | MLX discussion2026-07 |
| Qwen3.5 397B-A17B | 4-bit MLX (Qwen3.5-397B-A17B-2.416bit) | oMLX (version not stated) (MLX) | 42 tok/s | 485 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-05 |
| Qwen3.5 4B | NVFP4 (MLX) | MLX main (2026-08-01) / mlx_lm.benchmark | 128 tok/s | 5,395 tok/s2,048-token prompt | 2,048 | a machine with the same chip | MLX discussion2026-08 |
| Qwen3.5 9B | 4-bit MLX (Qwen3.5-9B-MLX-4bit) | oMLX (version not stated) (MLX) | 103 tok/s | 2,804 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.6 27B | 4bit (MLX) | oMLX (version not stated) | 33 tok/s | 992 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.6 27B | 8bit (MLX) | oMLX (version not stated) | 19 tok/s | 944 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.6 27B | 8bit (MLX) | oMLX (version not stated) | 16 tok/s | 853 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.6 27B | 8-bit MLX | MLX 0.32.4 / mlx-lm 0.32.2 | 15 tok/s | – | 512 | a machine with the same chip | MLX discussion2026-10 |
| Qwen3.6 35B-A3B | 4bit (MLX) | oMLX (version not stated) | 151 tok/s | 3,247 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.6 35B-A3B | 4-bit MLX (Qwen3.6-35B-A3B-MLX-4bit) | oMLX (version not stated) (MLX) | 146 tok/s | 4,625 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.6 35B-A3B | 4-bit MLX | MLX 0.32.0 / mlx-lm 0.31.3 | – | 3,909 tok/s8,192-token prompt | 0 | a machine with the same chip | independent benchmark2026-08 |
| Qwen3.6 35B-A3B | 4-bit (BaseRT .base bundle) | BaseRT 0.2.2 | – | 2,671 tok/s8,192-token prompt | 0 | a machine with the same chip | independent benchmark2026-08 |
| Qwen3.6 35B-A3B | 4-bit MLX | MLX 0.32.0 / mlx-lm 0.31.3 | – | 3,316 tok/s32,768-token prompt | 0 | a machine with the same chip | independent benchmark2026-08 |
| Qwen3.6 35B-A3B | 4-bit (BaseRT .base bundle) | BaseRT 0.2.2 | – | 2,271 tok/s32,768-token prompt | 0 | a machine with the same chip | independent benchmark2026-08 |
| Qwen3.6 35B-A3B | 4-bit MLX | MLX 0.32.0 / mlx-lm 0.31.3 | – | 2,388 tok/s65,536-token prompt | 0 | a machine with the same chip | independent benchmark2026-08 |
| Qwen3.6 35B-A3B | 4-bit (BaseRT .base bundle) | BaseRT 0.2.2 | – | 2,004 tok/s65,536-token prompt | 0 | a machine with the same chip | independent benchmark2026-08 |
| Qwen3.6 35B-A3B | 4-bit MLX | MLX 0.32.0 / mlx-lm 0.31.3 | – | 1,390 tok/s131,072-token prompt | 0 | a machine with the same chip | independent benchmark2026-08 |
| Qwen3.6 35B-A3B | 4-bit (BaseRT .base bundle) | BaseRT 0.2.2 | – | 1,379 tok/s131,072-token prompt | 0 | a machine with the same chip | independent benchmark2026-08 |
| Qwen3.6 35B-A3B | 4-bit MLX | MLX 0.32.0 / mlx-lm 0.31.3 | – | 973 tok/s196,608-token prompt | 0 | a machine with the same chip | independent benchmark2026-08 |
| Qwen3.6 35B-A3B | 4-bit (BaseRT .base bundle) | BaseRT 0.2.2 | – | 995 tok/s196,608-token prompt | 0 | a machine with the same chip | independent benchmark2026-08 |
| Qwen3.6 35B-A3B | 8-bit MLX | MLX 0.32.4 / mlx-lm 0.32.2 | 113 tok/s | – | 512 | a machine with the same chip | MLX discussion2026-10 |
| Qwen3.6 35B-A3B | 8bit (MLX) | oMLX (version not stated) | 104 tok/s | 1,965 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.6 35B-A3B | 8-bit MLX | MLX 0.32.4 / mlx-lm 0.32.2 | 101 tok/s | – | 4,096 | a machine with the same chip | MLX discussion2026-10 |
| Qwen3.6 35B-A3B | 8-bit MLX | MLX 0.32.4 / mlx-lm 0.32.2 | 101 tok/s | – | 512 | a machine with the same chip | MLX discussion2026-10 |
| Qwen3.6 35B-A3B | 8-bit MLX | MLX 0.32.4 / mlx-lm 0.32.2 | 97 tok/s | – | 512 | a machine with the same chip | MLX discussion2026-10 |
| Qwen3.6 35B-A3B | 8bit (MLX) | oMLX (version not stated) | 94 tok/s | 2,227 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.8 27B | NVFP4 (MLX) | MLX main (2026-10-16) / mlx_lm.benchmark | 34 tok/s | – | 512 | a machine with the same chip | MLX discussion2026-10 |
| Qwen3.8 27B | 4-bit MLX (Qwen3.8-27B-4bit) | oMLX (version not stated) (MLX) | 33 tok/s | 931 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.8 Flash-Next | 4-bit MLX (Qwen3.8-Flash-Next-MLX-mixed-4_8bit) | oMLX 0.7.2 (MLX) | 43 tok/s | 512 tok/s1,024-token prompt | 1,024 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.8 Flash-Next | 4bit (MLX) | oMLX (version not stated) | 35 tok/s | 1,087 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
With speculative decoding
A small draft model or extra prediction heads guess several tokens ahead. Generation is faster on text the guesses get right, so these are kept out of the medians above.
| Model | Quant | Backend | Generation | Prompt | Context | Measured on | Source |
|---|---|---|---|---|---|---|---|
| Gemma 4 26B-A4B IT | 4bit (MLX) | oMLX (version not stated) | 102 tok/s | 4,222 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 26B-A4B IT | 4bit (MLX) | oMLX (version not stated) | 66 tok/s | 2,026 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 26B-A4B IT | 6bit (MLX) | oMLX (version not stated) | 98 tok/s | 3,864 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 31B IT | 4bit (MLX) | oMLX (version not stated) | 38 tok/s | 674 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 31B IT | 4bit (MLX) | oMLX (version not stated) | 38 tok/s | 568 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 31B IT | 6bit (MLX) | oMLX (version not stated) | 40 tok/s | 693 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 31B IT | 8bit (MLX) | oMLX (version not stated) | 27 tok/s | 651 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 31B IT | 8bit (MLX) | oMLX (version not stated) | 15 tok/s | 633 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| GLM-5.3 Flash | 2bit (MLX) | oMLX (version not stated) | 56 tok/s | 1,165 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| GLM-5.3 Flash | 2bit (MLX) | oMLX (version not stated) | 42 tok/s | 881 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| GLM-5.3 Flash | 3bit (MLX) | oMLX (version not stated) | 7.9 tok/s | 204 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Nemotron 3.5 Lightning 30B-A3B | 4bit (MLX) | oMLX (version not stated) | 185 tok/s | 4,205 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Nemotron 3.5 Lightning 30B-A3B | 4bit (MLX) | oMLX (version not stated) | 181 tok/s | 4,160 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.5 122B-A10B | oQ4 (MLX 4-bit) | mlx-lm PR #1456 branch (MLX version not stated) | 104 tok/s | – | 0 | a machine with the same chip | MLX discussion2026-07 |
| Qwen3.6 27B | 6bit (MLX) | oMLX (version not stated) | 45 tok/s | 929 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.6 35B-A3B | 4bit (MLX) | oMLX (version not stated) | 185 tok/s | 5,372 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.6 35B-A3B | 4bit (MLX) | oMLX (version not stated) | 168 tok/s | 2,756 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.6 35B-A3B | 8bit (MLX) | oMLX (version not stated) | 108 tok/s | 5,154 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.6 35B-A3B | 8bit (MLX) | oMLX (version not stated) | 92 tok/s | 4,454 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.8 27B | Q4_K_M | llama.cpp commit 1deefcca3 (PR #27342 branch) | 25 tok/s | – | 0 | a machine with the same chip | llama.cpp benchmark discussion2026-08 |
| Qwen3.8 27B | Q4_K_M | llama.cpp commit 1deefcca3 (PR #27342 branch) | 23 tok/s | 510 tok/s383-token prompt | 0 | a machine with the same chip | llama.cpp benchmark discussion2026-08 |
| Qwen3.8 Flash-Next | 4bit (MLX) | oMLX (version not stated) | 97 tok/s | 1,875 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.8 Flash-Next | 4bit (MLX) | oMLX (version not stated) | 78 tok/s | 958 tok/s4,096-token prompt | 4,096 | a machine with the same chip | oMLX benchmark submission2026-10 |
Many users at once
Total throughput with several requests in flight. One person chatting sees far less than this.
| Model | Quant | Backend | Generation | Prompt | Context | Measured on | Source |
|---|---|---|---|---|---|---|---|
| Qwen3.8 27B | MXFP4 (MLX) | MLX main (2026-10-16) / mlx_lm.benchmark16 requests at once | 150 tok/s | – | 512 | a machine with the same chip | MLX discussion2026-10 |
| Qwen3.8 27B | NVFP4 (MLX) | MLX main (2026-10-16) / mlx_lm.benchmark16 requests at once | 150 tok/s | – | 512 | a machine with the same chip | MLX discussion2026-10 |
| Qwen3.8 27B | NVFP4 (MLX) | MLX main (2026-10-16) / mlx_lm.benchmark8 requests at once | 113 tok/s | – | 512 | a machine with the same chip | MLX discussion2026-10 |
| Qwen3.8 27B | MXFP8 (MLX) | MLX main (2026-10-16) / mlx_lm.benchmark16 requests at once | 128 tok/s | – | 512 | a machine with the same chip | MLX discussion2026-10 |
Deep into a long context
Measured with more than 4,096 tokens already in context, where every new token also reads the context memory.
| Model | Quant | Backend | Generation | Prompt | Context | Measured on | Source |
|---|---|---|---|---|---|---|---|
| Devstral Small 2 24B (2512) | 4-bit MLX (Devstral-Small-2-24B-Instruct-25124Bit) | oMLX (version not stated) (MLX) | 29 tok/s | 763 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-05 |
| Devstral Small 2 24B (2512) | 4-bit MLX (Devstral-Small-2-24B-Instruct-2512-4bit) | oMLX 0.7.0 (MLX) | 28 tok/s | 832 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 12B IT | 4-bit MLX (gemma-4-12B-it-4bit) | oMLX (version not stated) (MLX) | 59 tok/s | 1,508 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 12B IT | 4-bit MLX (gemma-4-12B-it-OptiQ-4bit) | oMLX (version not stated) (MLX) | 30 tok/s | 936 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-08 |
| Gemma 4 26B-A4B IT | BF16 (MLX) | mlx-vlm on TheTom/mlx fork (baseline KV) | 43 tok/s | – | 60,000 | a machine with the same chip | MLX discussion2026-04 |
| Gemma 4 26B-A4B IT | 4-bit MLX (gemma-4-26b-a4b-it-4bit) | oMLX (version not stated) (MLX) | 91 tok/s | 3,982 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 26B-A4B IT | 4-bit MLX (gemma-4-26B-A4B-it-QAT-MLX-4bit) | oMLX (version not stated) (MLX) | 67 tok/s | 3,634 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 31B IT | 4-bit MLX (gemma-4-31B-it-qat-OptiQ-4bit) | oMLX (version not stated) (MLX) | 17 tok/s | 621 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Gemma 4 31B IT | 4-bit MLX (gemma-4-31B-it-qat-OptiQ-4bit) | oMLX (version not stated) (MLX) | 16 tok/s | 591 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-10 |
| GLM-4.5-Air | 4-bit MLX (GLM-4.5-Air) | oMLX 0.3.9rc1 (MLX) | 29 tok/s | 710 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-05 |
| GLM-4.5-Air | 4-bit MLX (GLM-4.5-Air) | oMLX 0.3.9rc1 (MLX) | 15 tok/s | 474 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-05 |
| GLM-5.3 Flash | 4-bit MLX (GLM-5.3-Flash-oQ4e) | oMLX 0.7.0 (MLX) | 4.2 tok/s | 172 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-10 |
| gpt-oss-120b | MXFP4-Q8 (MLX) | MLX (mlx_lm, version not stated) | 76 tok/s | 2,710 tok/s16,384-token prompt | 16,384 | this machine | review2026-03 |
| gpt-oss-120b | 4-bit MLX (gpt-oss-120b-MXFP4-Q8) | oMLX 0.7.0.dev4 (MLX) | 73 tok/s | 1,479 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-09 |
| gpt-oss-120b | MXFP4-Q8 (MLX) | MLX (mlx_lm, version not stated) | 65 tok/s | 2,537 tok/s32,768-token prompt | 32,768 | this machine | review2026-03 |
| gpt-oss-120b | 4-bit MLX (gpt-oss-120b-mxfp4) | oMLX 0.4.4 (MLX) | 62 tok/s | 1,620 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-06 |
| gpt-oss-20b | 4-bit MLX (gpt-oss-20b-MXFP4-Q8) | oMLX 0.5.7 (MLX) | 89 tok/s | 2,915 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-08 |
| gpt-oss-20b | 4-bit MLX (gpt-oss-20b-MXFP4-Q8) | oMLX 0.6.4 (MLX) | 85 tok/s | 3,349 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Llama 3.3 70B Instruct | 4-bit MLX (Llama-3.3-70B-Instruct) | oMLX (version not stated) (MLX) | 8.1 tok/s | 237 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-04 |
| Llama 3.3 70B Instruct | 4-bit MLX (Llama-3.3-70B-Instruct) | oMLX (version not stated) (MLX) | 6.7 tok/s | 173 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-05 |
| MiniMax M2.7 | 4-bit MLX (MiniMax-M2.7-RAM-90GB) | oMLX 0.3.8 (MLX) | 33 tok/s | 936 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-05 |
| MiniMax M2.7 | 4-bit MLX (MiniMax-M2.7-ConfigI) | oMLX 0.3.6 (MLX) | 21 tok/s | 795 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-04 |
| Nemotron 3 Super 120B-A12B | 4-bit MLX (Nemotron-3-Super-120B-A12B) | oMLX 0.3.12 (MLX) | 55 tok/s | 1,060 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-06 |
| Nemotron 3 Super 120B-A12B | 4-bit MLX (NVIDIA-Nemotron-3-Super-120B-A12B) | oMLX 0.3.9 (MLX) | 52 tok/s | 906 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-05 |
| Ornith 1.5 35B-A3B | 4-bit MLX (Ornith-1.5-35B-A3B-MLX-4bit) | oMLX (version not stated) (MLX) | 117 tok/s | 4,026 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Ornith 1.5 35B-A3B | 4-bit MLX (Ornith-1.5-35B-A3B-MLX-4bit) | oMLX (version not stated) (MLX) | 114 tok/s | 3,626 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3 8B | 4-bit MLX (Qwen3-8B) | oMLX (version not stated) (MLX) | 3.3 tok/s | 2,072 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-04 |
| Qwen3 8B | 4-bit MLX (Qwen3-8B) | oMLX (version not stated) (MLX) | 1.8 tok/s | 1,701 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-04 |
| Qwen3 Coder 30B-A3B Instruct | 4-bit MLX (Qwen3-Coder-30B-A3B-Instruct-4bit-dwq-v2) | oMLX (version not stated) (MLX) | 91 tok/s | 4,042 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3 Coder 30B-A3B Instruct | 4-bit MLX (Qwen3-Coder-30B-A3B-Instruct-MLX-4bit) | oMLX (version not stated) (MLX) | 65 tok/s | 2,236 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-08 |
| Qwen3 Coder Next | 4-bit MLX (Qwen3-Coder-Next-4bit) | oMLX 0.6.4 (MLX) | 81 tok/s | 1,969 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-09 |
| Qwen3 Coder Next | 4-bit MLX (Qwen3-Coder-Next-4bit) | oMLX 0.7.0.dev2 (MLX) | 75 tok/s | 1,859 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-09 |
| Qwen3 Coder Next | 8-bit MLX | MLX (mlx_lm, version not stated) | 74 tok/s | 1,802 tok/s16,384-token prompt | 16,384 | this machine | review2026-03 |
| Qwen3 Coder Next | 8-bit MLX | MLX (mlx_lm, version not stated) | 69 tok/s | 1,887 tok/s32,768-token prompt | 32,768 | this machine | review2026-03 |
| Qwen3 Coder Next | 8-bit MLX | MLX (mlx_lm, version not stated) | 48 tok/s | 1,432 tok/s65,536-token prompt | 65,536 | this machine | review2026-03 |
| Qwen3.5 122B-A10B | 4-bit MLX (Qwen3.5-122B-A10B-4bit) | oMLX (version not stated) (MLX) | 63 tok/s | 1,756 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.5 122B-A10B | 4-bit MLX | MLX (mlx_lm, version not stated) | 61 tok/s | 1,239 tok/s16,384-token prompt | 16,384 | this machine | review2026-03 |
| Qwen3.5 122B-A10B | 4-bit MLX | MLX (mlx_lm, version not stated) | 55 tok/s | 1,068 tok/s32,768-token prompt | 32,768 | this machine | review2026-03 |
| Qwen3.5 397B-A17B | 4-bit MLX (Qwen3.5-397B-A17B-2.416bit) | oMLX (version not stated) (MLX) | 34 tok/s | 754 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-04 |
| Qwen3.5 397B-A17B | 4-bit MLX (Qwen3.5-397B-A17B-2.416bit) | oMLX (version not stated) (MLX) | 29 tok/s | 622 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-04 |
| Qwen3.5 9B | 4-bit MLX (Qwen3.5-9B-MLX-4bit) | oMLX (version not stated) (MLX) | 96 tok/s | 4,563 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.5 9B | 4-bit MLX (Qwen3.5-9B-MLX-4bit) | oMLX (version not stated) (MLX) | 87 tok/s | 2,951 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.6 27B | 4-bit MLX (Qwen3.6-27B-4bit) | oMLX (version not stated) (MLX) | 31 tok/s | 892 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.6 35B-A3B | 4-bit MLX (Qwen3.6-35B-A3B-UD-MLX-4bit) | oMLX (version not stated) (MLX) | 95 tok/s | 4,334 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.8 27B | 4-bit MLX (Qwen3.8-27B-4bit) | oMLX (version not stated) (MLX) | 31 tok/s | 983 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.8 27B | 4-bit MLX (Qwen3.8-27B-4bit) | oMLX (version not stated) (MLX) | 30 tok/s | 917 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.8 27B | 4-bit MLX | MLX (version not stated) / mlx-lm 0.31.3 | 21 tok/s | 432 tok/s131,072-token prompt | 131,072 | a machine with the same chip | MLX discussion2026-08 |
| Qwen3.8 27B | Q4_K_M | llama.cpp b11541 | – | 633 tok/s2,048-token prompt | 8,192 | Apple Mac Studio (M5 Max, 40-core GPU) (same chip) | llama.cpp benchmark discussion2026-10 |
| Qwen3.8 27B | Q4_K_M | llama.cpp b11541 + LXD-8 metal-m5-prefill patches (unmerged fork) | – | 1,064 tok/s2,048-token prompt | 8,192 | Apple Mac Studio (M5 Max, 40-core GPU) (same chip) | llama.cpp benchmark discussion2026-10 |
| Qwen3.8 27B | Q4_K_M | llama.cpp b11541 | – | 528 tok/s2,048-token prompt | 32,768 | Apple Mac Studio (M5 Max, 40-core GPU) (same chip) | llama.cpp benchmark discussion2026-10 |
| Qwen3.8 27B | Q4_K_M | llama.cpp b11541 + LXD-8 metal-m5-prefill patches (unmerged fork) | – | 884 tok/s2,048-token prompt | 32,768 | Apple Mac Studio (M5 Max, 40-core GPU) (same chip) | llama.cpp benchmark discussion2026-10 |
| Qwen3.8 Flash-Next | IQ3_S | llama.cpp b11429 | 27 tok/s | – | 175,000 | a machine with the same chip | llama.cpp benchmark discussion2026-10 |
| Qwen3.8 Flash-Next | 4-bit MLX (Qwen3.8-Flash-Next-MLX-mixed-4_8bit) | oMLX 0.7.2 (MLX) | 46 tok/s | 2,229 tok/s16,384-token prompt | 16,384 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.8 Flash-Next | 4-bit MLX (Qwen3.8-Flash-Next-MLX-mixed-4_8bit) | oMLX 0.7.2 (MLX) | 37 tok/s | 2,308 tok/s32,768-token prompt | 32,768 | a machine with the same chip | oMLX benchmark submission2026-10 |
| Qwen3.8 Flash-Next | IQ4_XS | llama.cpp master (2026-09-03) | 35 tok/s | – | 16,384 | a machine with the same chip | llama.cpp benchmark discussion2026-09 |
| Qwen3.8 Flash-Next | IQ4_XS | llama.cpp master (2026-09-03) | 35 tok/s | – | 32,768 | a machine with the same chip | llama.cpp benchmark discussion2026-09 |
| Qwen3.8 Flash-Next | IQ4_XS | llama.cpp master (2026-09-03) | 28 tok/s | 388 tok/s2,048-token prompt | 65,536 | a machine with the same chip | llama.cpp benchmark discussion2026-09 |
| Qwen3.8 Flash-Next | IQ4_XS | llama.cpp master (2026-09-03) | 21 tok/s | 340 tok/s2,048-token prompt | 131,072 | a machine with the same chip | llama.cpp benchmark discussion2026-09 |
Specifications
- Chip
- M5 Max (18-core CPU, 40-core GPU)
- Memory
- unified LPDDR (Apple does not state the generation)
- Memory bandwidth
- 614 GB/s
- FP16 compute
- Apple does not publish TFLOPS for the M5 Max GPU or its Neural Accelerators
- Software
- Metal, MLX
- Ships with
- macOS
- Power
- 96 W (USB-C power adapter included with the M5 Max 14-inch configuration (Apple store config))
- Form factor
- Laptop
- Networking
- No Ethernet; three Thunderbolt 5 ports
- Released
- 2026-03
- Maker page
- apple.com
Sources
- apple.com: M5 Max variants, 460/614GB/s bandwidth, memory options
- apple.com: base price $4,699 for 14-inch 40-core GPU
- apple.com: memory deltas (48GB +$0, 64GB +$400, 128GB +$2,000 for 40-core; 36GB only for 32-core); power adapter wattage
- macworld.com: M5 Pro/Max MacBook Pro pre-orders from March 4, 2026
- github.com: how far the GPU memory limit can be raised on a Mac (sysctl iogpu.wired_limit_mb)
- github.com: the default GPU memory limit on a Mac and how to raise it
Compared with
Similar machines
- Apple MacBook Pro 16-inch (M5 Max, 40-core GPU)~$4,999–$6,999
- Apple Mac Studio (M5 Max, 40-core GPU)~$3,099–$5,099
- Apple MacBook Pro 16-inch (M5 Max, 32-core GPU)~$4,399