GLM-5.2, the strongest open model to date, can now run locally using llama.cpp and Unsloth Studio. The 2-bit quantized model retains ~82% accuracy after reducing size from 1.51TB to 238GB, a 84% reduction, and is compatible with 256GB RAM or VRAM setups.
media
r/LocalLLaMA
·
53d ago
·
open_models
GLM-5.2 can now run locally in llama.cpp and Unsloth Studio
from English
Related articles
media
r/LocalLLaMA
·
45d ago
·
3 views
Running GLM5.2 on budget hardware < $2500
A Reddit user demonstrates how to assemble a local AI inference rig for under $2500 using affordable second-hand components, specifically targeting the ability to run large language models like GLM-5.2 without expensive enterprise hardware.
media
r/LocalLLaMA
·
48d ago
·
8 views
GLM-5.2 on 4x DGX Spark: Reconstructing Missing Build Steps for MTP Speculative Decode
The author successfully deployed GLM-5.2 with MTP speculative decode on a cluster of four NVIDIA GB10 (DGX Spark) nodes, achieving approximately 9.4 tokens per second. This setup utilizes vLLM with tensor parallelism, ported sparse-MLA Triton kernels, and a deterministic 15% expert pruning to fit AWQ-INT4 weights. A critical finding is that the original Docker image build instructions are incomplete, requiring reconstruction of missing patches for deep_gemm.py and sparse_attn_indexer.py. The author also identified that using any vLLM version other than the specific pinned commit causes real AWQ weights to crash during loading due to CUDA errors. To replicate the environment, users must apply a custom script that bakes in kernels and routes functions to sm12x fallbacks. Performance benefits include roughly double the speed of previous llama.cpp implementations, though inter-node bandwidth remains a bottleneck for dual-rail scaling.
media
r/LocalLLaMA
·
54d ago
·
209 views
unsloth GLM-5.2-GGUF with 2bit quantization at 238GB
The unsloth GLM-5.2-GGUF model is available with 2bit quantization, sized at 238GB. It is hosted on Hugging Face and shared via a Reddit post in the LocalLLaMA community.
media
r/LocalLLaMA
·
56d ago
·
2 views
Cheapest way to run GLM 5.x locally without unified memory
A user explores cost-effective methods to run GLM 5.x locally using 4-bit quantization, such as IQ4_XS, without relying on unified memory. Options include CPU-only setups like Sapphire Rapids ES with DDR5, multi-GPU offloading, or similar-sized models. The user runs a 5900X + 128GB DDR4 + 7900XT 20GB system, successfully handling Minimax 2.7 at Q4_K_S and Qwen 3.6 27B at IQ4_XS.
media
MarkTechPost
·
18h ago
·
1 view
webAI releases TwIL-LM, a 1.7B and 3B formal-logic model family for local autoformalization
webAI has released TwIL-LM, a two-model family of formal-logic reasoners with 1.7B and 3B parameters designed for autoformalization on local hardware. The models translate English into first-order logic and check conclusion validity, with the 3B variant achieving a macro gate score of 0.4218 on in-domain formal logic tasks.