media r/LocalLLaMA · 1d ago · open_models

GLM details its inference infrastructure in a new blog post

from English

The GLM team has published a blog post on z.ai detailing the architecture and design of their inference infrastructure.

The article provides an in-depth technical overview of how GLM handles model serving and optimization.
It is described as a "super interesting read" for those interested in large language model deployment systems.

Importance 2/3 r/LocalLLaMA Zhipu AI Inference efficiency

Z.ai and Alibaba independently converge on 3:1 linear attention architecture for GLM-5.3-Flash and Qwen3.8-Flash-Next

Z.ai released GLM-5.3-Flash, a 320B-parameter multimodal MoE model with 18B active parameters, while Alibaba's Qwen team released Qwen3.8-Flash-Next, a 125B model with 6B active parameters. Despite independent development, both teams adopted nearly identical architectural configurations.

lab Hugging Face Blog · 31d ago · 28 views

ALTK-Evolve calibrates agentic memory dosage to model capability for better performance

A study of eight models reveals that agentic memory is not a universal feature but a dose that must be calibrated to the specific model's capability. The ALTK-Evolve framework distills guidelines from an agent's past trajectories and injects them at inference time without weight updates, showing that optimal memory strategies vary significantly across model tiers.

github llama.cpp · 52d ago · 32 views

llama.cpp b10174 adds NextN/MTP speculative decoding support for GLM-5.2

The llama.cpp project released build b10174, introducing NextN/MTP speculative decoding support for the GLM_DSA (GLM-5.2) model architecture.

arxiv arXiv cs.CL · 53d ago · 28 views

PIVOT shares prefix scans across query groups to accelerate token-level sparse attention

PIVOT (Proxy Indexing Via One full-prefix Traversal) is a training-free, drop-in replacement for the DeepSeek Sparse Attention (DSA) indexer that reduces computational redundancy by sharing one prefix scan across a group of nearby queries. Instead of scoring every preceding token for each query individually, PIVOT aggregates the group into a single proxy query to obtain a candidate set, from which top-k tokens are selected for each query.

media r/LocalLLaMA · 72d ago · 21 views

GLM-5.2 4-bit on 4× DGX Spark achieves 70.8% on Terminal-Bench 2.1

A user quantized the GLM-5.2 (753B MoE) model to Int4-Int8Mix with NVFP4 4-bit KV cache and ran it on four DGX Spark (GB10) systems, achieving a 70.8% score on Terminal-Bench 2.1.