How routing works
A mixture-of-experts layer holds many feed-forward blocks and a small router that picks a few of them for each token. The rest sit idle for that token. Since any token might need any expert, all of them have to stay in memory.
Memory follows the total
gpt-oss-120b has 117B parameters, 5.1B active per token. Its 4-bit file is 58.5 GB, the size of a dense model of the same total.
Speed follows the active share
On the Ryzen AI Max+ 395, measured runs put gpt-oss-120b at a median of 52 tok/s and the dense Llama 3.3 70B Instruct at 4.8 tok/s, even though the MoE file is the bigger of the two. Per token the MoE model reads only its active slice.
That is why unified-memory machines with modest bandwidth and a lot of memory, and CPU servers with hundreds of GB of RAM, are mostly bought to run MoE models. The speed estimates on this site use the active share of the file and a separate efficiency figure for MoE models, which reach a lower share of bandwidth than dense ones.
Offloading experts
Hybrid builds keep attention and the cache on a GPU and leave expert weights in system RAM. Each token then reads part of its active weights at RAM speed. The hybrid builds show those models as “GPU + RAM offload” with a speed estimate that adds the two reads.