MiMo v2.5 local model offers high speed and quality in the 30b-400b gap

A user reports that MiMo v2.5 is a highly effective local model, outperforming expectations for models between 30b and 400b parameters and surpassing fast cloud providers in speed.

• Bartowski IQ4_NL quantization provided the best balance of quality and speed on a 192GB 4090 VRAM setup.
• UD-Q4_K_S matched IQ4_NL in quality but was approximately 1000pp/10tg slower.
• The model exhibits looping issues, which were mitigated using specific temperature and penalty settings.
• The model includes ASR capabilities, though support in llama.cpp is not yet available.

The author considers building a dedicated server worthwhile to run MiMo v2.5 due to its superior performance compared to other local options.