arxiv arXiv cs.AI · 1d ago

Empirical Study of OpenPangu Quantization on Ascend NPUs

A controlled study evaluates OpenPangu 1B and 7B models on Huawei Ascend 910B1 NPUs using weight-only and weight-activation quantization methods. Results show 8-bit weight-only quantization is lossless for both models, while 4-bit quantization is practical for 7B but harmful for 1B on reasoning, math, and code tasks. Ultra-low precision methods like 2-bit and binary fail, and W4A4 SmoothQuant produces non-finite perplexity, indicating extreme low-bit compression remains challenging.

media r/LocalLLaMA · 1d ago

Open Source Hugging Face Downloader App Released

The developer has released an open-source desktop app that downloads Hugging Face models, datasets, and spaces locally. The app automatically detects connection issues and resumes downloads, runs without cloud services or telemetry, and supports macOS, Windows, and Linux (both x64 and arm64).

blog Simon Willison · 1d ago

OPFS + Pyodide test harness for browser-based SQLite editing

A test harness has been developed to explore using OPFS (Origin Private File System) with Pyodide to enable browser-based editing of persistent SQLite files. The tool is designed to test Datasette Lite's capability to modify local SQLite databases directly in the browser across different browsers.

media Hugging Face Forums · 1d ago

Inference provider information out of date?

The Hugging Face page for Llama 3.1 405B lists Featherless AI as a provider, but the test widget shows 'Failed to fetch' and featherless.ai does not list it as available. A similar issue is reported for Baidu's ERNIE-4.5-300B model.

media r/LocalLLaMA · 1d ago

7 Chinese companies shipping H100/H200-class AI chips, most IPO'd in last 6 months

At least seven Chinese companies are now shipping H100/H200-class AI accelerators, with most having gone public within the last six months. Huawei alone shipped 812,000 AI cards last year, accounting for 49% of China's domestic supply, and its Ascend 950 is reportedly targeted at H200-class performance. Several of these firms were founded by former NVIDIA and AMD GPU leaders, including MetaX, which saw revenue grow 3,800x in three years, and Alibaba, which launched a server with 1.5TB of VRAM for on-premises frontier model deployment.

media r/LocalLLaMA · 1d ago

Krea 2 Released on Hugging Face

Krea 2, a local LLM, has been released on Hugging Face. The model is available in a Turbo version at https://huggingface.co/krea/Krea-2-Turbo and a raw version at https://hugging- face.co/krea/Krea-2-Raw. The release includes community discussion on Reddit.

media r/LocalLLaMA · 1d ago

Comparing Docling, Liteparse, MinerU, and Unstructured for On-Prem Document Processing

A university seeking on-premises document processing for academic workflows must use local parsers due to strict data governance policies banning cloud APIs. The user evaluates Docling, Liteparse, MinerU, and Unstructured, noting Docling excels in complex layouts with Apache 2.0 licensing but is slower; Liteparse offers good printed document performance with Tesseract OCR; MinerU uses PaddleOCR and handles French documents well despite longer setup; Unstructured supports multiple formats including DOCX and PPTX. The solution must support recurring, stable parsing of evolving PDFs with minimal formatting changes.

github llama.cpp · 1d ago

Vulkan backend updates and new binary releases for llama.cpp

llama.cpp release b9774 adds Vulkan backend support for SQR, SQRT, SIN, COS, CLAMP, LEAKY_RELU, and NORM operations, with support for noncontiguous inputs. The release includes binary builds for macOS, Linux, Android, Windows, and openEuler across multiple architectures and backends including CUDA, OpenVINO, SYCL, and ROCm.

github llama.cpp · 1d ago

LLaMA.cpp Release b9775: New Binaries and Support for Multiple Platforms

LLaMA.cpp has released version b9775, introducing binaries for macOS, Linux, Android, Windows, and openEuler across various architectures. The release includes CPU, Vulkan, OpenVINO, SYCL, and ROCm support, with updated CUDA versions (12.4 and 13.3) and iOS XCFramework availability. A UI package is also provided.

media r/LocalLLaMA · 2d ago

Human Evaluation Shows GLM-5.2 Competes with Top Models

A human evaluation on Design Arena's leaderboard reveals GLM-5.2 performs nearly as well as Fable 5 in game development tasks, placing just one step below it. The model, based on open weights and MIT licensing, is assessed as equivalent in capability to the best available Claude models, suggesting that standardized benchmarks may no longer accurately reflect real-world performance.

arxiv arXiv cs.CL · 2d ago

Koshur Pixel: First Large-Scale Synthetic OCR Dataset for Kashmiri

Koshur Pixel introduces a synthetic OCR dataset with 613,078 image-text pairs generated from the KS-PRET-5M corpus using SynthOCR-Gen. It includes over 25 augmentation strategies and spans diverse fonts and textual scales, from words to full-page documents, enabling scalable training for Kashmiri OCR systems.

github llama.cpp · 2d ago

Fix for Vulkan result checking and test linking in llama.cpp

llama.cpp now links ggml-cpu when GGML_VULKAN_CHECK_RESULTS or GGML_VULKAN_RUN_TESTS are enabled to resolve linking failures. This fix restores debug functionality for Vulkan result verification and testing after the ggml-cpu library was split.

arxiv arXiv cs.CL · 2d ago

Tmax: A Simple RL Recipe for Terminal Agents

Tmax presents the strongest open RL recipe for terminal agents, achieving 27% on Terminal-Bench 2.0 with only 9B parameters. It uses a novel data taxonomy to generate over 2.5x more terminal environments than prior datasets, enabling efficient training with a simple, outcome-only recipe. The dataset, models, and code are open-sourced at https://github.com/hamishivi/tmax.

arxiv arXiv cs.CL · 2d ago

UnBias-Plus: Detect, Explain, and Rewrite Bias

UnBias-Plus is an open-source toolkit that enables segment-level bias classification, biased span localization, neutral text rewriting, and decision reasoning. It offers multiple access methods including Python, CLI, REST API, and web interfaces, with all source code, models, datasets, and documentation publicly available.

media r/LocalLLaMA · 2d ago

MiniMax M3 EAGLE3 GGUF model now compatible with llama.cpp

The MiniMax M3 EAGLE3 decoder has been converted to GGUF format and is now compatible with llama.cpp. Testing on a 2x3090, 128GB system with UD-Q2_K_XL quant showed performance improved from 2.3 to 5 tokens per second using --fit and keeping the model in VRAM.

media r/LocalLLaMA · 2d ago

EU AI Act mandates AI-generated text watermarking from August 2024

The EU AI Act requires all AI systems generating synthetic text to include machine-readable, detectable watermarks using robust, interoperable technical solutions with two layers. This applies to all AI models, including open-source ones, and extends to any service accessible by EU citizens, regardless of location. Non-compliance risks fines of up to 35 million euros or a percentage of annual income, with providers of 'systemic risk' AI models facing heightened liability.

lab Hugging Face Blog · 2d ago

Shipping huggingface_hub weekly with AI, open tools, and human oversight

Hugging Face is releasing huggingface_hub weekly, integrating AI models, open-source tools, and a human review process to ensure quality and safety. The update emphasizes transparency, community involvement, and responsible AI development through continuous human-in-the-loop validation.

arxiv arXiv cs.CL · 2d ago

ASR-Based Text Corpus Development for Fongbe and Hausa

ASR pipelines extend text resources for low-resource West African languages Fongbe and Hausa. Fine-tuning MMS-300M on Fongbe achieves 9.48% WER with preserved tonal diacritics, while Hausa transcriptions reach 57.4/100 human evaluation score; Fongbe segments score only 36.5/100, indicating need for post-processing or improved models. The study releases curated datasets, fine-tuned models, and transcribed corpora under ethical guidelines.

arxiv arXiv cs.CL · 2d ago

Test-Time Steering Resolves Temporal Fact Conflicts in LLMs

Researchers identify parametric temporal conflicts in language models where outdated facts persist in parameters. They introduce Temporal Attractor Steering (TAS), a test-time method that resolves 29-57% of such conflicts without retraining, maintaining 85-99% accuracy on non-conflict queries and outperforming a baseline on three of four models.

arxiv arXiv cs.CL · 2d ago

Multi-Agent Audit Framework for Clinical Mental Health Screening

A multi-agent audit framework improves clinical mental health screening by decomposing reasoning into perception, retrieval, inference, and audit stages. Evaluated on the DAIC-WOZ dataset, it reduces PHQ-8 depression severity prediction error from 5.35 to 5.02 and offers interpretable, verifiable diagnostic rationales.