Qllm releases O(1) inference model without KV cache

A researcher has released Qllm, a new large language model architecture that achieves O(1) inference time by eliminating the need for a key-value (KV) cache. The 100M parameter model is built on phase associativity and does not rely on transformer or mamba structures.

• The model uses a novel architecture based on phase associativity to enable constant-time inference regardless of sequence length.
• It was trained on 4B tokens from DCLM, Fineweb, and Smoltalk2 datasets, with an initial 1B token pre-training for grammar.
• The project includes open-sourced code and a Hugging Face checkpoint for the qllm-pam-v11-e3k3-chat variant.

The authors hypothesize that this architecture may perform particularly well for voice models due to its design, though it is currently positioned as a proof of concept.