Tutorial explains FlashAttention as a secretly-associative operation

A new tutorial series begins with Part 1, which establishes the theoretical foundation for FlashAttention using modern algebraic formalism. The article demonstrates that FlashAttention is an associative operation, allowing it to be treated as a regular reduction on the GPU.

• Safe softmax, Welford's variance, and FlashAttention are identified as the same secretly-associative operation.
• The concept of a twisted monoid explains why the max-rescale coupling does not break associativity.
• The qk_scale formula derived in FA-2 is shown to come from log2(e)/√D.
• Numerical analysis covers overflow bounds and error limits, proving tiling never amplifies error.
• Bird's 3rd Homomorphism Theorem is presented as a test for secret associativity in loops.

This algebraic framing is described as more powerful than the original explanation and is already utilized in recent MLSys and CVPR papers.