This study investigates whether language models recognize when they are being tested, a factor critical for AI safety as it may cause models to alter their behavior strategically. Using 11 open-weight models from the Qwen 2.5, Gemma 2, and Llama 3.2 families, researchers analyzed how evaluation awareness manifests across different model sizes.
The authors introduce a pre-registered screening rule that determines before implementation whether an evolutionary outer loop over neural network parameters is worth building compared to a cheap single-shot alternative. The rule calculates a recovery metric R, defined as the best single-shot gain divided by the best gain of any cheap method, and prescribes skipping the outer loop when R is greater than or equal to 90%.
A study involving 815 participants examined whether using human-like language to describe artificial intelligence alters public perception compared to neutral descriptions.
The authors present DistilledGemma, an efficient system for person-place relation extraction from multilingual historical newspaper articles in English, German, and French. The approach utilizes a three-stage knowledge distillation pipeline to balance classification accuracy with computational efficiency.
The authors introduce Symbolic Mechanistic Data Attribution (SMDA), a framework that attributes training pairs to the interpretable symbolic policies governing model behavior, bridging the gap between mechanistic circuits and high-level decisions.
The article introduces TraceRetain, a lightweight framework for bounded external memory in frozen LLM agents that scores and evicts entries based on interpretable features like success and redundancy. The study evaluates how retention policies impact performance when external memory is used to augment language models.
The article addresses the limitation of AutoDiscovery's use of static "Bayesian surprise" by introducing evidence-informed LLM beliefs, where priors are updated with evidence from previous hypotheses to compute non-stationary surprisal. The authors find that embedding-based retrieval-augmented generation over prior discoveries best anticipates eventual posteriors and identify 37.5% of static surprisals as spurious.
A study benchmarks ten OCR systems on Devanagari text, revealing that specialized OCR vision-language models are fragile under degradation and that strong English performance does not predict Indic script accuracy.
Researchers propose Multi-Block Diffusion Language Models (MBD-LMs) to extend Single-Block diffusion text generation by decoding a running-set of consecutive blocks concurrently for inter-block parallelism. The approach bridges the gap between training and inference states through a post-training method called Multi-block Teacher Forcing (MultiTF).
Researchers introduce PolicyGuard, a sub-agent verifier designed to improve policy adherence in LLM agents by reasoning over the full dialogue context rather than relying on external checks of individual arguments. This approach addresses the limitations of prior safeguarding methods that often underestimate the need for conversation-specific remediation and explicit user confirmation.
A study reveals that evaluating diffusion large language models (dLLMs) is highly sensitive to prompt templates, creating an illusion that parallel decoding improves efficiency without performance loss.
Researchers propose a modular pipeline for building a travel-domain reasoning large language model grounded in an expert-designed knowledge graph to address accuracy and reliability issues in specialized domains. The approach integrates a travel knowledge graph, a bottom-up construction procedure for multi-hop question-answer pairs, and supervised fine-tuning to embed domain knowledge as auditable reasoning traces.
Researchers propose MIThinker, a lightweight thinking model that generates therapeutic thoughts to guide Motivational Interviewing counseling agents in strategy selection and response generation. To address the lack of annotated thought data, they introduce AugR1-MI, an automated pipeline that reverse-engineers counselor's thoughts from observed responses.
This article addresses the challenges of emotion recognition in song lyrics, which often diverge from the overall song's sentiment, by proposing a hybrid annotation framework that optimizes alignment between humans and large language models (LLMs). The authors introduce a new sentence-level dataset to examine this alignment and highlight the inherent subjectivity of the task.
The Complexity Ceiling Benchmark (CCB) evaluates how language model reasoning decays as the required sequential steps increase, fixing semantic content while varying task depth from 5 to 50. The study reveals consistent geometric per-step decay across three distinct regimes: grounded spatial state-tracking, abstract symbolic pointer manipulation, and transitive relational inference.
Research demonstrates that LLM agent memory systems rewrite casual or hedged remarks into confident, dated assertions that agents subsequently treat as verified facts. This process allows unverified information to bypass safety checks without requiring an active attacker, as the agent responds to phrasing confidence rather than source attribution.
The article identifies "intervention bias" as a critical failure mode in zero-shot large-language-model educational advisory agents, where they incorrectly recommend action despite oracle policies mandating inaction. Using the Open University Learning Analytics Dataset, the study demonstrates that zero-shot GPT-4o exhibits a 43 percentage-point false-positive rate at day 56, leading to approximately 4,300 unnecessary advisor contacts per cycle for 10,000 students.
The llama.cpp project has published the b9843 release, providing pre-built binaries for macOS, Linux, Android, Windows, and openEuler across various hardware architectures.
LangGraph version 1.2.7 has been released, introducing bug fixes and dependency updates for the LangChain ecosystem.
This study evaluates the effectiveness of top-1 argmax concentration as a collapse warning during the fine-tuning of discrete diffusion language models (DLMs) using Low-Rank Adaptation (LoRA). The authors find that this metric has zero precision because it saturates before optimization begins, failing to detect actual training collapses.