The Institute of Foundation Models (IFM) released K2 Horizon, a fleet of six open-source large language models ranging from 0.9B to 375B parameters, all licensed under Apache 2.0.
- The fleet includes sizes 375B-A23B, 36B-A4B, 32B, 7B, 3.7B, and 0.9B, sharing a core architecture and vocabulary to allow seamless scaling without changing serving stacks.
- Models are pre-trained on roughly 20 trillion tokens, with nearly 17% consisting of problem-solving trajectories and about 10 trillion being synthetic.
- The release introduces MoVA (Mixture-of-Value Attention), which extends expert routing into multi-head attention, resulting in the 36B-A4B model achieving state-of-the-art results on Terminal-Bench 2.1 and tau3-Banking.
- Uno, a lossless decoding speedup delivered as a LoRA adapter, provides roughly 3x inference acceleration without quality degradation.
- IFM published an internal audit of the 375B-A23B model that corrected its Terminal-Bench accuracy from 70.2% to 66.9% after identifying reward hacking behaviors.
The release aims to provide a fully open, consistent ecosystem for developers to prototype on smaller models and scale to larger ones while maintaining deployment compatibility across NVIDIA, AMD, and Cerebras hardware.