A cost analysis estimates that hosting diffusiongemma at different user token levels results in monthly costs per user ranging from 1.7€ to 122.8€. The study finds agentic AI usage is economically unsustainable for collective hosting, though costs could decrease with new GPUs or ASICs and a shorter GPU depreciation period.
media
r/LocalLLaMA
·
53d ago
·
open_models
Napkin math on collective hosting costs for diffusiongemma in 2026
from English
Related articles
arxiv
arXiv cs.AI
·
50d ago
·
1 view
Empirical Study of OpenPangu Quantization on Ascend NPUs
A controlled study evaluates OpenPangu 1B and 7B models on Huawei Ascend 910B1 NPUs using weight-only and weight-activation quantization methods. Results show 8-bit weight-only quantization is lossless for both models, while 4-bit quantization is practical for 7B but harmful for 1B on reasoning, math, and code tasks. Ultra-low precision methods like 2-bit and binary fail, and W4A4 SmoothQuant produces non-finite perplexity, indicating extreme low-bit compression remains challenging.
media
MarkTechPost
·
2d ago
·
1 view
webAI releases TwIL-LM, a 1.7B and 3B formal-logic model family for local autoformalization
webAI has released TwIL-LM, a two-model family of formal-logic reasoners with 1.7B and 3B parameters designed for autoformalization on local hardware. The models translate English into first-order logic and check conclusion validity, with the 3B variant achieving a macro gate score of 0.4218 on in-domain formal logic tasks.
media
Hugging Face Forums
·
7d ago
·
1 view
QuantCheck warns against assuming final checkpoint is best for 4-bit quantization
A new tool called QuantCheck highlights that the final pretraining checkpoint may not be the optimal choice for 4-bit quantization, as benchmarks can remain flat while fragility increases. The author observed this pattern on Pythia-160m, where the final checkpoint suffered approximately four times more quality damage than an earlier checkpoint of equal quality.
media
r/LocalLLaMA
·
11d ago
·
3 views
Meituan releases LongCat-Flash-Lite-Sparse with 1M context window
Meituan has released the weights for LongCat-Flash-Lite-Sparse, a new variant of its LongCat-Flash-Lite model. The update introduces LongCat Sparse Attention (LSA) to replace dense MLA and extends native context length support to 1 million tokens.
arxiv
arXiv cs.CL
·
13d ago
·
2 views
OSReward introduces standardized evaluation for cross-platform computer-use reward models
Researchers introduce OSReward, a realistic benchmark designed to evaluate the reliability of vision-language models (VLMs) acting as judges for computer-using agent trajectories. The study reveals that even state-of-the-art VLMs suffer from systematic leniency bias and are often too expensive for scale, while affordable open models perform poorly.