Developer releases mlx-serve, a fast Zig-based local inference server

A developer has released mlx-serve, an open-source local inference server written in Zig that supports MLX and GGUF models. The project addresses the author's frustration with slow model support and broken features like tool calling in existing tools.

The server speaks OpenAI, Anthropic, and Ollama APIs and beats LM Studio by a +48% geometric mean across Gemma 4 and Qwen 3.6 benchmarks. It includes an auto-detecting tool-calling autocorrect feature and a native Swift UI app in its DMG distribution.

The single ~7MB binary is open source with no telemetry, designed to run efficiently on Mac hardware.