media Hugging Face Forums · 1h ago · open_models

Eqx-zoo provides verified Equinox ports of Hugging Face models

from English

The author has built eqx-zoo, a library that loads Hugging Face Hub checkpoints directly as plain Equinox modules and verifies each model against the transformers library. The project currently supports Llama, Qwen2, Qwen3, Qwen3-MoE for generation, and BERT, RoBERTa, XLM-RoBERTa for embeddings.

Verification ensures float32 outputs match transformers layer-by-layer and greedy generation reproduces tokens exactly.
The author discovered that comparing bf16 implementations directly is flawed due to rounding differences, necessitating comparison against float32 relative to the source's error.
JIT-compiling a decoder in XXL was found to be measurably less accurate than eager execution, while encoders did not show this issue.
A subtle LayerNorm bug in bf16 was only detectable at the block level, highlighting the need for granular testing.

The verified checkpoints are available in a Hugging Face collection, and the library is installable via pip.

Importance 1/3 Trust 1/3 Hugging Face Forums Inference efficiency Local inference

Read original

media Hugging Face Forums · 3d ago · 13 views

AiiStream Q3.6 enables running Qwen3.6-35B-A3B on 8-16 GB Macs

The author has released AiiStream Q3.6, an open-source technique that allows the Qwen3.6-35B-A3B model to run on Apple Silicon devices with only 8 to 16 GB of RAM. By keeping routed experts on SSD and using early loading predictions, the method reduces MLX memory usage to approximately 1.7 GB while maintaining byte-identical outputs to the standard mlx_lm implementation.

github llama.cpp · 4d ago · 22 views

llama.cpp b11307 preserves batch order for speculative decoding

The llama.cpp project released build b11307, which fixes layer input ordering to preserve the original batch sequence during speculative decoding. This change ensures that token order is correctly maintained for unmasked NextN embeddings and compatible with tensor splits.

github llama.cpp · 4d ago · 6 views

llama.cpp b11301 fixes integer overflow in ggml/gguf

The llama.cpp project has released build b11301, which includes a fix for an integer overflow issue within the ggml and gguf components. This update addresses validation logic to prevent potential errors when handling zero-element tensors.

github llama.cpp · 4d ago · 15 views

llama.cpp b11302 fixes ThreadSanitizer data race in sparse indexer

The llama.cpp project released build b11302, which addresses a critical concurrency issue identified by ThreadSanitizer in the CI pipeline. The update resolves a data race within the sparse attention mechanism where multiple CPU threads were incorrectly writing to the same memory elements.

github llama.cpp · 4d ago · 8 views

llama.cpp b11298 adds dflash support to convert and feature extraction

The llama.cpp project has released build b11298, which introduces support for the dflash format. This update enables both conversion of models to this format and subsequent feature extraction capabilities.