VoidPadding introduces [VOID] as a padding token to separate semantic termination and response-length modeling. It improves performance on mathematical reasoning and code generation by 17.84 points over the original model and reduces decoding NFE by 55.7% on average.
A user compiled llamacpp with both CUDA and Vulkan support to leverage two GPUs, the w7800 and another card. The setup achieved +10% tokens/sec in decoding for a MiniMax-M3-UD-IQ2_M-00001-of-00004.gguf model, with plans to run benchmarks to assess real performance gains.
The Minimax M3 (4-bit MLX) was benchmarked on a Mac Studio M3 with 512GB storage. Results show token throughput and latency metrics across different prompt sizes, with peak performance at 269.1 tok/s for 8192-token prompts and 172.8 tok/s for a 65k-token prompt, using 228GB of peak memory.
GLM-5.2 has launched with open weights, a 1M context window, MIT license, and two reasoning modes. Early results show it ranks near the top in coding benchmarks, indicating strong real-world potential beyond API-only models.
A Reddit post discusses the cost-effective hardware setup for running Qwen 3.6 models, both 27B and 35B-A3B, noting that RTX 3090 24GB offers better long-term value over Tesla V100 due to discontinuation and upcoming Chinese alternatives. The proposed build totals $1,995.65, including a Ryzen 5 5600X, RTX 3090 24GB, and essential components, with the total price being a key concern for users seeking affordability.
Hashicorp founder Mitchell Hashimoto states that local language models aren't good enough yet. A Reddit user disputes this, noting that over a year, many people have successfully used local models for coding, with only 'vibecoders' facing challenges.
Claude v2.1.178 introduces new permission rules using Tool(param:value) syntax, improved workflow and skill loading in nested directories, and enhanced auto mode and error messaging. It fixes critical issues including crashes, authentication errors, and UI behavior in Chrome and VSCode, while refining tool prompts and undo functionality.
A community model can be built through crowdsourced compute using a 'Branch-Train-Stitch' approach. Participants train a prototype model on their hardware, submit narrow-domain submodels, and organizers stitch them into a large Mixture-of-Experts (MoE) model, with key decisions including prototype size, scope definitions, and training protocols.
Glimmer 1 is a 10,000-parameter language model trained on 500K tokens from FineWeb-Edu. It features a 512-token context window, a standard Llama architecture with 16 hidden dimensions, 2 layers, 4 attention heads, and 1 KV head using GQA, and is available on Hugging Face.
A user asks if anyone has successfully run Qwen 3.6 27b UD Q8 on multiple GPUs, noting issues with llamacpp and vllm. The model crashes or hangs during multi-turn requests, with llamacpp showing CUDA errors and vllm failing mid-turn, despite working well with Q5 quantization.
Georgi Gerganov confirms that Qwen3.6-27B is highly capable for coding tasks, noting its daily use on local hardware like M2 Ultra and RTX 5090. He describes using a minimal pi agent with a short system prompt to align it with his workflow, highlighting its utility for maintaining open-source projects.
VibeThinker-3B, scaled from a 1.5B model, reaches frontier-level performance in math and coding tasks. It scores 94.3 on AIME'26, 80.2 on LiveCodeBench v6, 76.4 on IMO-AnswerBench, and 93.4 on IFEval, with 96.1% success on first-attempt LeetCode problems.
llama.cpp version b9669 adds backend sampling support for Eagle3. The release includes binaries for macOS, Linux, Android, Windows, and openEuler across multiple architectures and hardware acceleration options, including Vulkan, CUDA, ROCm, OpenVINO, and SYCL.
llama.cpp release b9670 includes fixes for NVFP4 edge cases in llama-graph, such as moving post-GEMM MUL operations and restricting build_ffn to supported combinations. The release provides binaries for macOS, Linux, Android, Windows, and openEuler across multiple architectures and backend options, including CUDA, Vulkan, SYCL, and OpenVINO.
DiffusionGemma uses bidirectional attention to allow self-correction during token generation, enabling it to revise earlier tokens in a 256-token block. This capability gives it a structural advantage in generating valid tool calls, as it can correct malformed outputs that autoregressive models cannot fix once committed.
A test comparing Q8 and IQ3 XXS turbo4 quantized versions of Qwen3.6 27B shows that Q8 excels in API safety and input sanitization, while IQ3 XXS turbo4 performs better in thread management and modular code design. The model recommends merging both approaches: using Q8 for initial launch protection and IQ3 XXS for atomic writes and thread lifecycle, forming a combined Phase 1 foundation.
A user seeks guidance on learning context and harness engineering for building local-first AI applications with specialized use cases. They express interest in avoiding general-purpose AI models like Hermes or OpenClaw and ask where to find resources, given their background in MCP servers and tool calling.
llama.cpp version b9665 introduces a new --offline flag for benchmarking. The release includes binary builds for macOS, Linux, Android, Windows, and openEuler across multiple architectures and hardware acceleration options, including Vulkan, CUDA, ROCm, OpenVINO, and SYCL.
A project called Trace Commons invites users to donate their coding session traces to an open dataset licensed under CC-BY-4.0. The initiative aims to provide training data for open-weight and open-source AI models, countering potential data monopolies by Anthropic and OpenAI.
AeroLLM is a fast, optimised, and open-source chat application designed for Apple Silicon devices using the MLX backend. It supports local AI tasks like text-to-speech, speech-to-text, and large language models, with models downloaded directly from Hugging Face based on available RAM. The app is notarised due to lack of Apple Developer membership, but users can follow provided steps to run it as a signed macOS app.