The llama.cpp project has added support for the Kimi-K3 text model, implementing its hybrid KDA (linear) + MLA (full) attention architecture along with five specific architectural features not present in the older Kimi-Linear-48B.

  • The implementation includes cross-layer residual attention, latent MoE, situ activation replacing SwiGLU, an MLA output gate, and a full-rank KDA gate.
  • Routed experts are shipped as compressed-tensors using "mxfp4-pack-quantized" format, which is bit-compatible with ggml's MXFP4 to allow lossless repacking without dequantization.
  • A new chat template handles Kimi-K3's XTML-ish tagged format for reasoning, content, and tool calls, including specific message delimiters and token-level span splitting.
  • The converter fixes type errors and increases LLAMA_MAX_EXPERTS from 512 to 1024, while the model saver now correctly emits the kda_gate_lower_bound parameter to preserve roundtrip fidelity.

This update enables users to run Kimi-K3 locally with verified end-to-end correctness against Moonshot's reference implementation and real generation data.