media Hugging Face Forums · 7h ago

Community Inquiry on Model Benchmarking Methods

A user on the Hugging Face discussion forum posted a question seeking advice on how to benchmark machine learning models. The inquiry was initiated by an individual who is new to the field of fine-tuning and wishes to evaluate their models after completion. The post explicitly asks for established methods or strategies that the community uses for this purpose. It highlights a common need among practitioners to understand standard evaluation practices in model development. The discussion thread currently contains only one post from a single participant. No specific benchmarks, metrics, or technical solutions were provided within the visible content of the source.

media Hugging Face Forums · 7h ago

Qwen3/Gemma3 Candle Skips Attention Masks for Equal-Length Batches in CPU Mode

A user has reported a critical bug in the Hugging Face text-embeddings-inference library affecting Qwen3 and Gemma3 models. The issue arises when running inference on CPUs with concurrent requests, leading to significant accuracy degradation. Specifically, the Candle backend incorrectly skips attention masks for batches where all input sequences have equal lengths. This defect compromises the reliability of embeddings generated under these specific conditions. To address the problem, the author submitted a pull request containing a fix that was thoroughly tested on their local machines. The bug highlights potential stability risks in CPU-based embedding services handling batched inputs.

github LlamaIndex · 7h ago

Llama Index v0.14.23 Release Notes

Llama Index released version 0.14.23 on June 24, 2026, introducing significant multimodal capabilities and various bug fixes. The core update includes multimodal synthesis features and the introduction of multimodal query engines to support diverse data types. Key fixes address document and video block handling within FunctionTool outputs and ensure URL-backed memory blocks are preserved correctly. Performance improvements were implemented by using sets for within-batch deduplication in the ingestion pipeline and optimizing token text splitting logic. The release also resolves a ZeroDivisionError on empty input sequences and fixes recursion errors in splitters when units exceed chunk sizes. Additionally, explicit UTF-8 encoding was added to file I/O operations, and deep copying of initial states prevents mutation leaks across workflow runs.

lab Claude Code Releases · 7h ago

Claude Code v2.1.191 Release Notes

Claude Code version 2.1.191 introduces /rewind support, allowing users to resume conversations from before a /clear command was executed. The update fixes several critical issues, including background agents resurrecting after being stopped and scroll position jumping during streaming responses. It also corrects behavior where /voice displayed generic error messages and where /login URLs were truncated in Windows Terminal. Significant improvements enhance reliability for MCP servers by adding retry logic for transient network errors during capability discovery and OAuth flows. Headless environments now skip browser popups for OAuth, while sandbox network permissions are remembered for the session duration. Performance optimizations reduce CPU usage during streaming by approximately 37% through text update coalescing and mitigate long-session memory growth from the terminal output cache.

github CrewAI · 9h ago

v1.14.8a4 Release Notes

v1.14.8a4 adds conversational flow support in the CLI TUI. It includes fixes for symlink path traversal during skill archive extraction and validation of declarative flow definition paths. Documentation for v1.14.8a3 is updated.

github llama.cpp · 9h ago

LLaMA.cpp Release b9784: Hexagon MM Optimizations and Cross-Platform Binaries

LLaMA.cpp releases version b9784 with major optimizations for hexagon-based MM operations, including 32x32 tiled weight repack, improved dyn.quant handling, and unified kernel parameter management. The release includes new binaries for macOS (arm64 and x64), iOS, and multiple Linux architectures with support for Vulkan, ROCm, and OpenVINO.

arxiv arXiv cs.LG · 10h ago

A Differentiable Atari VCS for Explainable AI

A fully differentiable emulator of the Atari 2600 VCS is presented, reproducing all 64 ALE games with bit-for-bit accuracy in RAM and screen output. The system enables gradient-based explainable AI by providing a complex, fully known ground truth, with both Julia and JAX implementations validated against a reference emulator and supporting high-throughput training on GPUs.

arxiv arXiv cs.LG · 10h ago

AdaR: Adaptive Recurrent Message Passing for Graph Test-Time Computing

AdaR enables flexible test-time computing on graphs without parameter changes by using adaptive recurrence. It derives step dependence as a necessary and sufficient condition for convergence and incorporates normalized step information and representation-target relations into recurrent updates, guided by gradient-based supervision signals. Empirical results show AdaR outperforms strong baselines in both inductive and transductive graph learning settings.

arxiv arXiv cs.LG · 10h ago

Speech-Text Models Latently Transcribe Speech in Intermediate Layers

Interleaved speech-language models undergo an implicit transcription phase where spoken words become decodable as text tokens in intermediate layers, despite no speech recognition training. Up to 77% of the data shows the spoken word appearing as a top candidate text prediction, followed by a transition to text-based next-word prediction before returning to speech. This behavior is influenced by interleaved training and text LM initialization, and correlates with spoken knowledge performance.

arxiv arXiv cs.LG · 10h ago

LLM-Integrated App Bug Seams Reveal Testing Gaps

A rental-search assistant with LLM features and multi-market support faced persistent user defects despite 1,553 passing automated tests. Analysis of 252 bug-fix commits showed 44% of fixes occurred at four unseen seams: browser runtime, non-default market, end-to-end flows, and whole-system level. A fix without a seam guard caused a defect to ship twice, highlighting the need for targeted testing at these boundaries.

arxiv arXiv cs.LG · 10h ago

Fed-CausalDiff: Decoupled Synchronization for Federated Do-Simulation

Fed-CausalDiff introduces a federated causal diffusion framework that enables do-simulation and policy evaluation in decentralized settings. It decomposes latent state evolution into global and local components, allowing decoupled synchronisation to reduce communication cost while maintaining accurate causal inference.

media r/LocalLLaMA · 10h ago

SDXL Running Locally in Browser on WebGPU, Open-Source

A browser extension enables local image generation using SDXL models via WebGPU, running on the user's GPU without external setups. The tool supports two models: SDXL-Lighting fp16 (7 GB) and a 4-bit version (3.6 GB), with requirements including at least 8 GB VRAM for the full model and a browser with WebGPU support (Chrome/Edge 122+ or latest Firefox).

arxiv arXiv cs.LG · 10h ago

Deep Learning Pipeline for Sign Language Recognition and Translation to Indian Vernaculars

A two-stage deep learning model classifies Indian sign language video clips into English words using a fine-tuned VideoMAE transformer, achieving 99% training and 78% validation accuracy on a 13-class dataset. The predicted English labels are translated into Hindi, Telugu, and Bengali using Meta AI's NLLB-200 multilingual model, with a Streamlit demo enabling user-uploaded video inference and cross-lingual output.

arxiv arXiv cs.LG · 10h ago

Prompt-Side Preprocessing Enhances Edge AI Accuracy

A structured prompt framework improves local LLM accuracy in environmental monitoring by transforming raw sensor data into enriched textual representations. Evaluations on indoor and outdoor datasets show local model accuracy increases from 50.9% to 81.7% indoors and from 63.7% to 89.3% outdoors with enriched prompts, while maintaining low latency near 0.22 seconds in no-chain-of-thought mode.

arxiv arXiv cs.LG · 11h ago

The Scissors Effect: Resize Diversity Hurts Robust Surrogate Transfer

Input diversity, a common practice in transfer attacks, improves success on standard surrogates but reduces it on robust ones. This regime-dependent effect, called the Scissors Effect, is driven by gradient geometry, with resize operations degrading alignment in robust models. A training-free rule (CG-DI) adjusts diversity based on local gradient consistency to preserve attack success across surrogate types.

arxiv arXiv cs.LG · 11h ago

HERTA: Automated Testing for FHE Framework Vulnerabilities

HERTA is the first automated testing tool designed for fully homomorphic encryption frameworks. It uses metamorphic testing with novel relations derived from FHE semantics to detect deep-seated logic bugs that can silently corrupt encrypted computations. Evaluation on three industry frameworks revealed 21 previously unknown bugs, several of which have been confirmed and fixed by developers, with significant implications for security and service integrity.

arxiv arXiv cs.LG · 11h ago

Robust Diffusion Models via Divergence-Induced Weighted Denoising

A new training method replaces MSE loss in diffusion models with an f-divergence-based transformation, creating a robust surrogate that improves performance under data contamination. The approach uses local divergence constructions under DDPM's Gaussian reverse-kernel, reducing the training objective to a one-dimensional function of denoising error, with bounded-influence divergences suppressing large errors and enhancing stability.

arxiv arXiv cs.LG · 11h ago

Generative Robust Optimisation Framework

Generative Robust Optimisation (GRO) introduces a deep generative model to define uncertainty sets, capturing nonlinear correlations, asymmetry, and multimodality. A five-point evaluation framework assesses neural network-based uncertainty sets across reconstruction fidelity, distribution matching, latent regularity, robust relevance, and computational tractability, with experiments validating GRO's effectiveness in production planning and facility location problems.

arxiv arXiv cs.LG · 11h ago

Introducing Quantum Measurement Temperature to Stabilize Hybrid QNN Training

A learnable scaling parameter called Quantum Measurement Temperature (QMT) is introduced to rescale quantum measurement outputs in hybrid quantum neural networks. This approach mitigates measurement-induced logit contraction, enhancing gradient magnitude and stability during training without altering the quantum circuit or measurement operators. Experiments show improved logit separation, gradient strength, and classification accuracy in protein and image classification tasks.

arxiv arXiv cs.LG · 11h ago

Deep material network for homogenization of piezoelectric composites

A piezoelectric deep material network (PDMN) is proposed to efficiently homogenize two-phase piezoelectric composites. The framework embeds electromechanical homogenization relations into its architecture, enabling physics-informed, semi-analytical predictions with over three orders of magnitude lower computational cost than direct numerical simulation, validated on PVDF-LiNbO3 and viscoelastic-piezoelectric composites under nonlinear loading.