arxiv arXiv cs.CL · 4h ago

Mitigating Position Bias in Transformers via Layer-Specific Positional Embedding Scaling

Researchers introduce layer-specific positional embedding scaling (LPES) to address the "lost-in-the-middle" problem in large language models, where critical information in long-context inputs is often underrepresented. This method assigns distinct scaling factors to each transformer layer to achieve a more balanced attention distribution without requiring parameter fine-tuning or increasing inference delay.

arxiv arXiv cs.CL · 4h ago

Low-Agreeableness Persona Conditioning for Safe LLM Fine-Tuning

Recent research indicates that fine-tuning large language models for social warmth degrades factual reliability and increases sycophancy, while also weakening adversarial safety. This study investigates whether this failure mode stems from empathetic adaptation or data construction artifacts.

arxiv arXiv cs.CL · 4h ago

Do Speech Emphasis Models Generalize across Languages and Emotions?

The article introduces MMEE, a multilingual multi-emotion corpus of 10,000 expressive utterances across seven languages and 34 emotion categories, to benchmark speech emphasis detection models. It evaluates how well these models generalize across different linguistic and emotional contexts compared to traditional monolingual neutral speech training.

arxiv arXiv cs.CL · 4h ago

Enhancing Numerical Prediction in LLMs via Smooth MMD Alignment

Researchers introduce Smooth Maximum Mean Discrepancy (SMMD) to address the unreliability of large language models in numerically precise tasks caused by standard cross-entropy training objectives. SMMD incorporates value-distance kernels over numeric tokens and graph-based smoothness to align predicted distributions with targets while encouraging local consistency.

arxiv arXiv cs.CL · 4h ago

KG2Cypher: Data-Centric Pipeline for Building Enterprise Text-to-Cypher Systems

Researchers present KG2Cypher, a data-centric pipeline designed to build enterprise text-to-Cypher systems from existing Knowledge Graphs. The system generates executable Cypher queries from graph facts and uses large language models to create corresponding natural-language questions.

arxiv arXiv cs.CL · 4h ago

Temporal Fusion Strategies for NER in Historical Texts

This study investigates how temporal metadata can be structurally embedded into Named Entity Recognition (NER) models to address the challenge of entity drift in historical texts. The authors systematically evaluate lightweight fusion strategies, including cross-attention, adapters, and concatenation, within Transformer-based architectures.

arxiv arXiv cs.CL · 5h ago

Output-Space Allocation Costs for Calibration-Guided LLM Compression: An Empirical Study

This study investigates whether aligning allocation costs with output-space objectives improves the fidelity of compressed large language models, specifically testing a modification to the ROCKET compression method. The authors compare using weight-space Frobenius error against an output reconstruction objective for multi-choice knapsack problem allocation.

arxiv arXiv cs.CL · 5h ago

SHIFT: Gate-Modulated Activation Steering for Knowledge Conflict Mitigation

The paper introduces SHIFT, a novel framework that mitigates knowledge conflicts in Retrieval-Augmented Generation (RAG) by reformulating neuron-level modification as learnable gate modulation. This approach allows large language models to adaptively regulate internal activations to resolve conflicts between retrieved context and parametric knowledge.

arxiv arXiv cs.CL · 5h ago

NLL-Guided Full-Attention Layer Selection for Training-Free Sliding-Window Adaptation

Researchers propose a training-free method to select which layers in hybrid attention models should retain full attention, addressing the inefficiency of fixed patterns in long-context inference. By measuring negative log-likelihood degradation on answer tokens, the approach identifies layers critical for maintaining accuracy when switching to sliding-window attention.

arxiv arXiv cs.CL · 5h ago

Position Bias Correction is Insufficient for One-Pass Attention Sorting

The study investigates whether correcting position bias enables single-pass attention sorting to match the performance of iterative methods in long-context language models. Experiments on LLaMA-2 and YaRN-Llama-2 models refute the hypothesis that debiasing alone is sufficient to bridge the performance gap.

arxiv arXiv cs.CL · 5h ago

Learning Complementary Action Modeling from Automotive Maintenance Instructions

The article introduces Complementary Action Modeling (CAM), a task focused on identifying or generating procedural counterparts in automotive maintenance instructions by modifying only the action phrase while preserving the rest of the sentence context.

arxiv arXiv cs.CL · 5h ago

Triadic Werewolf: A Jester Role for Multi-Hop Theory of Mind in LLMs

Researchers extended the Werewolf game with a Jester role to create a triadic social-deduction environment that requires reasoning across three opposing utility functions, challenging large language models' theory-of-mind capabilities. Evaluations on GPT-4.1, DeepSeek-V3.1, and Llama-3.3-70B revealed that while the Jester won 60-70% of games, GPT-4.1 wolves frequently voted the Jester out on day 1 in 60-70% of cases, a self-defeating action driven by language priors.

arxiv arXiv cs.CL · 5h ago

Verifiable Geometry Problem Solving: Solver-Driven Autoformalization and Theorem Proposing

Researchers propose SD-GPS, a solver-driven framework for geometry problem solving that addresses bottlenecks in autoformalization and theorem prediction by treating the symbolic solver as an execution oracle. This approach unifies supervised formal-language adaptation with solvability-guided reinforcement learning to ensure executability during formalization.

arxiv arXiv cs.CL · 5h ago

VASAE: Naming SAE Dictionary Directions with Vocabulary-Aligned Anchoring

The authors introduce Vocabulary-Aligned Sparse Autoencoder (VASAE), a method that trains sparse autoencoder features using vocabulary-aligned anchoring to assign each feature an intrinsic token name based on the nearest embedding in the Transformer's vocabulary.

arxiv arXiv cs.CL · 5h ago

AI Persuasive Framing in Collective Dilemmas

A study involving 1,283 participants tested whether AI assistants could enhance cooperation in iterated Collective Risk Games through personalized persuasive framing based on Social Value Orientation profiles. The research found that while pro-social nudges significantly increased contributions and group success rates, these effects were short-lived.

arxiv arXiv cs.CL · 6h ago

An Empirical Analysis of Factual Errors in Human-Written Text and its Application

This study addresses the neglect of factual error detection in human-written text by distilling a taxonomy of errors from newspaper article corrections, revealing categories like kanji misconversions that are absent in current hallucination benchmarks. The authors evaluate vanilla large language models on synthesized test cases and real corrections to assess their performance on this specific task.

arxiv arXiv cs.CL · 6h ago

Multi-Stage Explainable Framework for Speech-Based Cognitive Impairment Detection

Researchers propose a multi-stage explainability framework that translates black-box transformer predictions into clinically grounded narratives for speech-based cognitive impairment detection. The system integrates SHAP-based token attribution, linguistic features, and an LLM reasoning pipeline to map model outputs to specific cognitive-linguistic dimensions.

arxiv arXiv cs.CL · 6h ago

ToxiREX: A Dataset on Toxic REasoning in ConteXt

Researchers introduce ToxiREX, a new multilingual dataset designed to capture and explain implicit, context-dependent toxicity within Reddit comment threads. The dataset utilizes a systematic toxic reasoning schema to provide structured annotations for comments related to major global events across six languages.

arxiv arXiv cs.CL · 6h ago

Dialogue to Detection: A Multimodal Hybrid NLP Pipeline for Insurance Fraud Detection

This article introduces a synthetic multimodal framework designed to replicate First Notice of Loss (FNOL) conditions for insurance fraud detection, addressing the limitations of existing text-only approaches. The system generates agent-customer dialogue transcripts and two-speaker audios to integrate linguistic, behavioral, and speaker-based indicators.

arxiv arXiv cs.CL · 6h ago

The Signal-Coverage Matrix: Stratifying Type and Semantic Errors in Statement Autoformalization

This article introduces a signal-coverage matrix to stratify type and semantic errors in LLM autoformalization, moving beyond scalar type-correctness metrics. The framework categorizes outputs into true success, type-only, semantic-only, or both fail cells by crossing Lean elaborator results with semantic equivalence judgments.