Custom Rust + CUDA attention-transformer achieves 0.63 J/token on H100

A developer has released details of a custom attention-transformer engine built with Rust and CUDA, focusing on determinism and energy efficiency. The system supports both inference and training on the same stack and runs across multiple hardware backends including H100, M2 Pro, consumer GPUs, and WebGPU.

• Throughput reaches approximately 403 tokens/second on an H100 NVL with a 28-layer 7B-class stack using continuous batching.
• Energy consumption is measured at 0.63 J/token with a median board power of ~254 W.
• The engine is bit-exact, verified by AUDIT, and utilizes GAE, ATE, WNSM, and reversible training techniques.

The full benchmarks and methods are available on the developer's site, with open-source release planned after patent issuance this month under a dual license.