Message Passing Language Models enable efficient reasoning via direct thread communication

Researchers introduce Message Passing Language Models (MPLMs), a framework that allows large language model threads to communicate directly using lightweight send and receive primitives, addressing the scalability limits of traditional fork-join paradigms.

• MPLMs reduce communication costs by avoiding redundant context sharing and utilize preemption to terminate unpromising branches early.
• On Sudoku puzzles, MPLMs require asymptotically smaller context than serial Chain-of-Thought or parallel fork-join methods, enabling a fine-tuned model to solve 25x25 puzzles that challenge standard approaches.
• In 3-SAT tasks, the preemption capability improves efficiency by terminating unpromising branches.
• Appropriately prompted large pre-trained models can follow the MPLM protocol, achieving competitive results on long-context question answering compared to popular fork-join approaches.

This framework offers a more efficient scaling mechanism for LLM reasoning by enabling direct inter-thread communication and early termination based on peer information.