A Reddit user challenges the sentiment that Mixture of Experts (MoE) models are inferior to dense models simply because they have fewer active parameters, citing the example of Qwen 3.5 122B having only 10B active.
- The user argues that if a 122B MoE model were equivalent to a 10B dense model, providers would release the faster dense version instead, as it avoids routing overhead.
- They suggest the value of an MoE model depends on the router's effectiveness in selecting the optimal experts to activate its total parameter potential.
- The post highlights that some MoE architectures may be more effective than others, implying the comparison is more nuanced than a simple active parameter count.