media Hugging Face Forums · 3d ago · open_models

Developer builds 842kb, 107k-param bigram LM for browser inference

from English

A developer has created a tiny language model with an 842KB weight file and 107,776 parameters using bigram subwords. The model is designed to run directly in the browser and was trained on a mixed corpus that favors short story generation.

The model weighs only 842KB and contains 107,776 parameters.
It utilizes bigram tokens to spell out words individually rather than linking them.
Training data is weighted toward tiny stories, though the corpus is mixed.
Research findings are documented on Zenodo under "Phasor-Face Transformers (PrismFormer)."

The author aims to refine the model over the next few days to improve its output quality for generating short narratives.

Importance 1/3 Trust 1/3 Hugging Face Forums Inference efficiency Open weights

Read original

lab Tencent Hunyuan (HF) · 1d ago · 3 views

Tencent releases Simple Attention Sparsification checkpoints for Qwen3

Tencent has released Simple Attention Sparsification (SAS) checkpoints that learn to rank and select KV blocks for each query in Qwen3 models. Unlike methods distilling dense attention scores, SAS adds continuous gates to selected blocks, enabling end-to-end optimization of context ranking with the language-modeling loss.

media AI News (smol.ai) · 3d ago · 15 views

DeepSeek launches V4.1-Flash, an open-weight model focused on extreme inference efficiency

DeepSeek has released V4.1-Flash, a new open-weight flagship model designed for extreme inference efficiency and low cost. The model utilizes a causal encoder-decoder architecture to significantly reduce active compute and KV/cache costs.

media Latent Space · 3d ago · 8 views

DeepSeek releases v4.1-Flash with 763B-P8B-D16B architecture for efficient inference

DeepSeek has launched DeepSeek v4.1-Flash, a new open-weight flagship model featuring a novel causal encoder-decoder architecture designed to maximize inference efficiency and reduce costs.

media MarkTechPost · 5d ago · 22 views

DeepSeek releases DeepSeek-V4.1-Flash with 1M context and FP4 KV cache

DeepSeek AI has released DeepSeek-V4.1-Flash, a multimodal Mixture-of-Experts model featuring a 1-million-token context window and an FP4 KV cache that reduces global cache footprint to 890 bytes per token. The model utilizes a causal encoder-decoder architecture and Compressed Sparse Attention 2 (CSA2) to significantly lower memory requirements while maintaining performance.

media r/LocalLLaMA · 12d ago · 77 views

IFM releases K2-Horizon-MoVA-36B-A4B, a sparse MoE model with 4B active parameters

IFM has released the final checkpoint for K2-Horizon-MoVA-36B-A4B, a Mixture-of-Experts model featuring Mixture-of-Values attention that stores 36B parameters but runs only 4B per token. The release includes GGUF formats and links to smaller variants in the K2-Horizon family.