Benchmarking 128GB M5 Max Mac for Local AI inference

A user benchmarks a 128GB M5 Max MacBook Pro to evaluate its performance running various local large language models. The testing covers model families including Gemma 4, Qwen 3.6, MiniMax M2.7, and Mistral Medium 3.5 across different quantization formats and context lengths.

• Gemma 4 E4B achieved a prefill (PP) of 4,748 and token generation (TG) of 85.0 using MLX 8-bit, compared to 3,974 PP and 76.1 TG with GGUF Q8.
• Qwen 3.6 27B ran at 706 PP and 17.2 TG with MLX Q8, while the 35B A3B variant reached 2,153 PP and 79.4 TG at 128K context.
• Speculative decoding with Draft MTP n=2 reduced generation time for Qwen 27B from 75s to 62s.
• Multi-agent setups using Qwen 27B and 35B A3B completed a site-building task in 233s, outperforming the single-model control which took 537s.
• Large models like Mistral Medium 3.5 128B and DS4 DeepSeek V4 Flash were tested with Qwen 3.6 Flash achieving 452 PP and 45.2 TG.

The results provide empirical data on the hardware's capability to handle diverse model sizes and complex inference tasks locally.