ResonatorLM replaces attention with causal resonant field mixing for efficient long-context modeling

Researchers introduce ResonatorLM, a new mechanism that replaces self-attention with a physics-derived alternative treating token sequences as a driven one-dimensional latent field. This approach uses causal functions of damped resonators to improve efficiency in processing long contexts.

In a small 6M parameter setting, the model achieves a decode speedup of 6.47x compared to an optimized transformer at 32K tokens. It also reaches 61.31 percent accuracy on WikiText, outperforming the baseline's 55.32 percent.

ResonatorLM demonstrates that replacing attention with resonant field mixing can significantly increase training and prefill speedups as sequence length increases while maintaining competitive accuracy.