A user asks whether running GLM-5.2 on four Ascend GX10 chips (DGX Sparks) is feasible. They inquire about 4-bit quantization using 512GB unified memory and estimate prompt and output token speeds for 100k context length, noting no existing performance data is available online.
media
r/LocalLLaMA
·
52d ago
·
open_models
GLM 5.2 on 4x Sparks: Reasonable?
from English
Related articles
media
r/LocalLLaMA
·
50d ago
·
1 view
GLM 5.2 Achieves 98% Max Intelligence with Less Than Half Tokens
GLM 5.2 demonstrates 98% of maximum intelligence in coding tasks using less than half of its total token budget, according to a technical report by z_ai. The model's reasoning efficiency has improved significantly, with token usage increasing from 16.7k to 36.7k between GLM 5.1 and GLM 5.2, though high-level settings may strain local hardware performance.
media
r/LocalLLaMA
·
51d ago
·
12 views
GLM-5.2 (744B, 2-bit) achieves 7.3 tok/s on 4×3090 with 192GB RAM
GLM-5.2 UD-IQ2_M runs at ~7.3 tokens per second on 4×RTX 3090s with 192GB DDR5 RAM using llama.cpp expert offload. Reducing quantization from IQ2 to IQ1 provided no speed gain, while increasing CPU threads from 6 to 12 improved performance by 22%. Decode is limited by CPU compute, not memory bandwidth, and the offloaded experts must be explicitly distributed across GPUs to avoid out-of-memory errors.
media
r/LocalLLaMA
·
51d ago
·
36 views
Running GLM-5.2 on CPU Only with Local Setup
A user runs GLM-5.2 locally on a Dell PowerEdge R740 with dual Xeon 6248R CPUs and 768GB RAM, using ik_llama.cpp for improved CPU inference. After isolating one NUMA node for optimal performance, they achieve 4–5.5 tokens per second in chat and about 3 tokens per second in coding tasks, noting the model shows 'frontier vibes' during code generation despite limited usability on this hardware.
media
r/LocalLLaMA
·
52d ago
·
12 views
GLM-5.2-FP8 HGX-H200 SGLang Docker Deployment Config
A user shares a Docker configuration for running GLM-5.2-FP8 on HGX-H200 hardware using SGLang. The setup achieves 262k context length and 70 tokens per second with 8 tensor parallelism, using a memory fraction of 0.83. The user notes that vLLM official recipes do not work on H200 due to KV cache FP8 quantization limitations on the DSV3 architecture.
media
Latent Space
·
53d ago
·
1 view
GLM-5.2 Claims Top Position in Frontend Coding with Speculative Decoding
GLM-5.2, a 744B parameter model from Z.ai, has been evaluated as the top frontend coding model globally, outperforming all Opus versions including Opus 4.8. This achievement is highlighted in third-party evaluations that validate official offline tests, marking a significant milestone for a model of its size, particularly in the competitive frontend coding domain.