Self-conditioned Flow Map Language Models via Fixed-point Flows

The paper introduces fixed-point flows, a two-dimensional class of self-conditioned flow-based language models that leverage self-conditioning to solve a fixed-point iteration bootstrapping the denoiser's performance. The authors demonstrate that these flows define valid flow maps and can be distilled from self-conditioned models using fixed-point distillation and flow map distillation.

• Fixed-point flows represent a class where the first dimension is the flow process and the second is the fixed-point iteration.
• Distillation compresses both the fixed-point iterations and the flow process.
• The resulting model, FMLM*, outperforms state-of-the-art self-conditioned and few-step models on OpenWebText.

This approach provides a theoretical understanding of self-conditioning in continuous flow-based language models and enables effective few-step text generation.