The article documents how measurements from proprietary LLM evaluators can become invalid within weeks, introducing the EPC framework to detect such instability. It applies this diagnostic across eight experimental conditions, revealing that version-conditional instability makes single-snapshot evaluator studies unreliable.
This study evaluates the impact of resampling methods like SMOTE and random undersampling on probability calibration in tree ensembles, finding that while SMOTE's cost is small, undersampling severely degrades calibration.
This study evaluates the performance of open-weight large language models running on-premises for text-to-SQL tasks using a reproducible benchmark on the BIRD development split. It compares three model families across two generations while ablating specific accuracy-enhancing techniques to determine their actual value.
The article introduces EarningsInOne, a new corpus aligning earnings news, conference call transcripts, and prices for the SP 1500 universe from 2022 to 2025. This resource bridges the gap between financial economists and NLP researchers by providing unified trading setups and evaluation metrics for both quantitative and qualitative signals.
The article introduces a novel method for automatic mapping between disease classification systems, such as ICD-9-CM and ICD-10-CM, that addresses the limitations of existing embedding-based approaches which often overlook complex one-to-many scenarios. By employing a blocking-and-matching pipeline inspired by entity resolution, the authors utilize large language models to identify valid mappings within candidate blocks.
Researchers propose Mandol, an agglomerative memory system designed to consolidate fragmented memory representations into a unified architecture for long-term conversational agents. This approach addresses the high latency and noise issues inherent in existing systems that rely on heterogeneous vector and graph databases.
This position paper argues that humans possess an evolved instruction-following bias, an innate inductive bias shaped by evolution to interpret and execute linguistic instructions. This cognitive feature enables rapid instructed task learning (RITL) and allows for the fast generalization of behavior from language.
The authors propose Fund2Persona, a framework that grounds financial advisor personas in fund disclosures, holdings transitions, and manager commentary to address the difficulty of scaling consistent expertise in LLM systems. The system refines these personas through an agentic actor-scorer-patcher loop, moving beyond simple persona prompts that often drift toward generic recommendations.
This paper benchmarks five lightweight, CPU-feasible hallucination detection methods to provide practical alternatives for resource-constrained researchers who cannot use GPU-intensive or proprietary solutions. The study evaluates ROUGE-L, semantic similarity, BERTScore, a FEVER-trained DeBERTa NLI detector, and an ensemble of similarity and NLI across the HaluEval benchmark's question answering, dialogue, and summarisation tasks.
The authors introduce SrDetection, a unified framework for detecting data leakage in code large language models that operates in both gray-box and black-box settings. The method generates semantically equivalent variants of benchmark samples to identify cases where the original data is disproportionately easier for the model due to pre-training exposure.
The paper introduces Neural Procedural Memory (NPM), a training-free framework that enables Large Language Model agents to utilize implicit activation steering for procedural memory instead of relying on explicit textual instructions. By distilling skills from historical experiences into steering vectors, NPM directly activates task-relevant neural mechanisms to guide execution.
This study analyzes the development of technologies in Natural Language Processing (NLP) from an entity-centric perspective, extracting methods, datasets, metrics, and tools to measure their impact via co-occurrence networks. The research reveals that while pre-trained language models like BERT and Transformer have become mainstream, the average number of entities per paper is increasing, indicating a growing knowledge burden for researchers.
The authors propose MATCH, a framework that augments sparsified attention mechanisms with dynamically integrated in-context information to address the scalability bottlenecks of traditional attention in long-context scenarios.
This study presents a token-level framework that decomposes language model scaling laws into localized learning events of individual contextualized tokens, challenging the view that heavy-tailed pattern difficulty is the sole cause.
This study proposes a sentence-level framework to identify, analyze, and trace the evolution of motivations for mentioning algorithms in academic papers, using natural language processing as a case study. The researchers classify these motivations using pretrained models and data augmentation, revealing that deep learning models outperform traditional machine learning approaches.
The authors propose KbSD, a framework that addresses reward sparsity in agentic search by using dense token-level supervision and quadrant-adaptive optimization to calibrate when models should trust parametric memory versus retrieved evidence. This approach utilizes an information-asymmetric self-distillation process where a hint-augmented teacher generates calibrated reasoning demonstrations for a student model without requiring a larger external model.
The authors propose ARKD, a reinforcement-learning-based adaptive KL-weighted distillation framework that addresses the limitations of single KL objective methods in compressing Large Language Models. By using a policy network to dynamically assign weights to forward and reverse KL divergence based on teacher-student distributional characteristics, the method achieves dual alignment on principal and long-tail modes.
A study demonstrates that the per-word processing time in the state-space language model Mamba aligns with human reading times. The research shows that Mamba's dynamic discretization timestep is a significant predictor of how long humans take to read words, even when controlling for other factors like GPT-2 surprisal.
This study analyzes the distribution of novelty in Chinese Library and Information Science (LIS) papers published between 2000 and 2022, examining trends across journals, topics, and time periods. Using BERTopic for topic identification and combinatorial innovation theory for novelty scoring, the research investigates how collaboration patterns influence scholarly innovation.
This study introduces clinical reasoning graphs to evaluate the diagnostic reasoning patterns of large language models, revealing that while they achieve competence, they lack consistent reasoning schemas. The authors extracted structured graph representations from 750 traces across five LLMs and tested for stable reasoning patterns in clinically similar cases.