llama.cpp version b9703 includes a rework of the server's preset handling, removing remote HF preset support and deprecated functions. The release provides binaries for macOS, Linux, Android, Windows, and openEuler across multiple architectures and hardware acceleration options, including Vulkan, CUDA, OpenVINO, and SYCL.
llama.cpp version b9704 now returns HTTP 400 for invalid grammar instead of silently dropping constraints. The release includes binaries for macOS, Linux, Android, Windows, and openEuler across multiple architectures and hardware accelerators, with support for Vulkan, ROCm, OpenVINO, SYCL, and CUDA.
The unsloth GLM-5.2-GGUF model is available with 2bit quantization, sized at 238GB. It is hosted on Hugging Face and shared via a Reddit post in the LocalLLaMA community.
Qwen3.6-27B runs at ~60 tokens/sec on 32GB VRAM with FP8 KV quantization. NVFP4 kv cache quantization on SM120 could significantly enhance performance on such systems, though current implementation is not yet available.
The user reports a significant gap between Llama benchmark results and actual model performance. Benchmarks show 754 tk/s prefill and 36 tk/s generation, but real usage reveals only 7.98 tokens per second, with high latency and poor throughput. The discrepancy is attributed to real-world usage conditions, not benchmark settings, suggesting the model's actual performance is far below the benchmarked speed.
LLaMA.cpp version b9698 enables self-updates only when built with llama-install.sh. The release includes binaries for macOS, Linux, Android, Windows, and openEuler across multiple architectures and hardware acceleration options, including Vulkan, CUDA, OpenVINO, and SYCL.
TransitNet, a compact attention-augmented deep learning framework, achieves 95.2% accuracy in low-SNR transit blind searches, outperforming TLS and BLS in ROC-AUC and PR-AP values. It recovers 93.0% of injected Earth- and sub-Earth-size transits, with 97.4% of injected transits fully covered by estimated transit windows, and successfully recovers all 34 confirmed Kepler planets with a mean midpoint error of 1.24 hours.
EfficientRollout introduces a self-speculative decoding framework that reduces rollout and end-to-end latency by up to 19.6% and 12.7% respectively, without compromising final model quality. It uses a quantized drafter derived from the target model and integrates a system-aware toggle policy to avoid compute-bound regimes, enabling effective speculation during evolving policy generations.
FoMoE introduces a system that partitions expert layers across workers to avoid full model replicas, reducing communication costs by up to 1.42x over baselines and 45.44x over DDP. It achieves up to 1.4x throughput speedups via a skip-token mechanism and demonstrates stable routing, with projected benefits extending to 100B-scale models through system modeling.
CAHP introduces a post-hoc framework that uses graph-theoretical clustering and information-theoretic measures to select complementary attention heads in Transformers. It automatically determines head retention without predefined sparsity, identifying a performance degradation threshold to ensure minimal model loss, and outperforms baselines in high-compression scenarios by preserving functionally critical heads in intermediate layers.
SwitchBraidNet is a quantisation-aware EEG classification architecture that achieves high accuracy in motor imagery and SSVEP tasks. It outperforms four baselines in FP16 and FP32, with MI accuracy of 69.49%, SSVEP accuracy of 93.48%, and a hybrid information transfer rate of 64.82 bits/min in FP16. The model runs efficiently with only 3.03 KB of INT8 storage, enabling low-power embedded deployment.
TransitNet, a compact attention-augmented deep learning framework, achieves 95.2% accuracy in low-SNR transit blind searches, outperforming TLS and BLS in ROC-AUC and PR-AP values. It recovers 93.0% of injected Earth- and sub-Earth-size transits, with 97.4% of injected transits fully covered by estimated transit windows, and successfully recovers all 34 confirmed Kepler planets with a mean midpoint error of 1.24 hours.
FoMoE introduces a system that partitions expert layers across workers to avoid full model replicas, reducing communication costs by up to 1.42x over efficient baselines and 45.44x over DDP. It achieves up to 1.4x throughput speedups via a skip-token mechanism and demonstrates stable routing, with projected benefits extending to 100B-scale models through system modeling.
Claude Code v2.1.181 introduces support for setting config settings via prompt syntax like /config thinking=false, adds sandbox Apple Events support on macOS, and improves streaming, auto-retry, and subagent behavior. It also fixes numerous bugs related to startup, file handling, clipboard, and UI responsiveness across platforms.
The ggml-cpu project now conditionally enables the POWER11 backend in ggml based on compiler support for -mcpu=power11. This prevents build failures on current GCC/Clang toolchains while maintaining forward compatibility. Updates to CMakeLists.txt support this change, and -mcpu=power10 is used for both P10 and P11 architectures.
llama.cpp version b9692 introduces new binaries for macOS, Linux, Android, Windows, and openEuler across multiple architectures. The release includes updates to support Vulkan, ROCm, OpenVINO, SYCL, and HIP, with fixes to remove batch dim usage in llava_uhd.
The Metal backend in llama.cpp has been extended to support f16 and bf16 tensor types for the concat operator, in addition to existing f32 and i32 support. This update includes specialized kernel templates, updated pipeline getters, and improved type-based kernel dispatch, with assistance from pi:llama.cpp/Qwen3.6-27B.
llama.cpp releases version b9688, adding model management and SSE realtime updates APIs. The release includes prebuilt binaries for macOS, Linux, Android, Windows, and openEuler, supporting various architectures and acceleration frameworks like Vulkan, CUDA, OpenVINO, and SYCL.
llama.cpp users can free up GPU memory by disabling mmproj offload, using reduced KV cache types, and adjusting spec-draft-n-max. Parameters like --ctx-checkpoints and --fit-target have minimal impact, while --parallel helps in multi-user setups but not for single users.
A user shares a Docker configuration for running GLM-5.2-FP8 on HGX-H200 hardware using SGLang. The setup achieves 262k context length and 70 tokens per second with 8 tensor parallelism, using a memory fraction of 0.83. The user notes that vLLM official recipes do not work on H200 due to KV cache FP8 quantization limitations on the DSV3 architecture.