arxiv arXiv cs.CL · 13h ago

Data-Driven Evolution of Library and Information Science Research Methods (1990-2022)

This study analyzes the influence of data-centric research on Library and Information Science by examining methodological evolution from 1990 to 2022. Researchers automatically extracted four key categories of data-driven entities from academic papers: algorithms and models, data resources, software and tools, and metrics. The analysis evaluates trends across three dimensions, including temporal characteristics, topic-specific evolution, and cross-method features. Findings identify data resources as the primary driver of methodological changes within the discipline. The research reveals a cyclical pattern characterized by emergence followed by stability or practical application in LIS methods. This perspective highlights how big data advancements have reshaped the field's technical landscape over three decades.

arxiv arXiv cs.CL · 13h ago

iLLaDA: An 8B Masked Diffusion Language Model with Fully Bidirectional Attention

The authors introduce iLLaDA, an 8B parameter masked diffusion language model trained from scratch using fully bidirectional attention. This approach contrasts with the predominant autoregressive factorization and causal attention used in modern large language models. The model's pre-training scaled to 12 trillion tokens, followed by supervised fine-tuning on a 25 billion-token instruction corpus for 12 epochs. iLLaDA maintains the masked diffusion objective throughout both training phases and employs variable-length generation for efficiency. It also introduces confidence-based scoring to enhance performance on multiple-choice evaluation tasks. Benchmark results show significant improvements over its predecessor, LLaDA, including gains of 21.6 points on BBH and 14.9 points on ARC-Challenge for the base model. The instruction-tuned variant achieved increases of 14.5 points on MATH and 16.5 points on HumanEval. Despite its non-autoregressive nature, iLLaDA remains competitive with Qwen2.5 7B across several metrics.

arxiv arXiv cs.CL · 13h ago

Local Branch Routing: Efficient Trainable Test-Time Scaling for Language Models

The authors introduce Local Branch Routing (LBR), a token-level framework designed to improve language model reasoning through efficient test-time scaling. LBR expands a small local lookahead tree and forwards all sampled branches through the model, using a lightweight router to select the depth-1 subtree for commitment. This approach allows each token decision to utilize evidence from candidate local futures without incurring the computational costs of full solution-level search. The method employs a prune-shift-grow decoding process that preserves discrete branch identities and defines a tractable tree-trajectory likelihood. Consequently, LBR enables end-to-end reinforcement learning with verifiable rewards, jointly optimizing the base model and router under the same likelihood-ratio principle as discrete-token RLVR. Experimental results on synthetic hierarchical-planning tasks demonstrate that post-candidate hidden states provide useful routing evidence. Furthermore, benchmarks in mathematical reasoning show that LBR improves both Pass@1 and Pass@32 metrics compared to discrete chain-of-thought and other baselines.

arxiv arXiv cs.CL · 14h ago

Neural Machine Translation for Low-Resource Tangkhul-English

This study addresses low-resource machine translation for the Tangkhul-English language pair, focusing on a severely under-resourced Tibeto-Burman language with minimal prior NLP infrastructure. The authors present two systems: a primary model based on ByT5-large and a contrastive system using mT5-small, both fine-tuned on 38,336 parallel sentence pairs. Evaluation on a held-out test set of 3,856 sentences shows the ByT5-large system achieving a corpus BLEU score of 39.97 and a chrF++ score of 58.07. Additional metrics include a BERTScore F1 of 0.8104 and a COMET score of 0.7302 using the wmt22-comet-da model. The research highlights orthographic challenges related to Tangkhul's Latin-script diacritics as a specific technical hurdle. Furthermore, the training corpus exhibits domain bias, consisting primarily of biblical texts, stories, and conversational data. Future work aims to improve performance through data diversification and domain adaptation strategies.

arxiv arXiv cs.CL · 14h ago

Computational Stylometry of English Pali Canon Translations Across Pitakas

This study presents a computational stylometric analysis of the Tipitaka across all three Pitakas in English translation, extending previous work on the Sutta Pitaka. The corpus comprises 134,831 segments from Bhikkhu Sujato's Sutta Pitaka, Bhikkhu Brahmali's Vinaya Pitaka, I.B. Horner's 1938 Vinaya translation, three English translations of the Abhidhammattha Sangaha, and cross-tradition Vinaya texts. The authors compute Zipf rank-frequency distributions, MATTR-500 lexical diversity, numeral-word density, and vocabulary overlap metrics. Main findings indicate that all corpora show Zipf-consistent distributions with R-squared values above 0.989. The Sutta and Theravada Vinaya exhibit nearly identical lexical diversity scores of 0.399 and 0.400, while the Sangaha corpus is more diverse at 0.560. The Sangaha corpus also displays the highest numeral-word density at 3.26%, reflecting its systematic enumeration of categories. Additionally, the Mulasarvastivada Vinaya shares significant vocabulary overlap with the Theravada Vinaya, whereas two English translations of the same source share only 24.2% of their vocabulary.

arxiv arXiv cs.CL · 14h ago

Story Operators: Decomposing the Original to Sequel Transformation in Embedding Space

This study models literary transformations as geometric operations within a sentence-embedding space using all-mpnet-base-v2 vectors from the PG19 corpus. By calculating displacement vectors between original novels and their sequels, the author decomposes these changes along a content basis derived via PCA. Analysis of thirteen verified author pairs reveals a taxonomy of sequel types: formulaic, concentrated, and compositional. Formulaic transformations involve minimal rank changes, such as Doyle's Holmes collections with a norm of 0.12. Concentrated shifts are dominated by a single axis, exemplified by Alcott's Little Women to Little Men where 75% of the change occurs on one move. Compositional transformations involve many small axes, seen in works by Twain, Burroughs, and Nesbit. For Tom Sawyer to Huckleberry Finn, the dominant axis is structural, reflecting a shift from domesticity to picaresque adventure rather than surface themes like vernacular voice. The geometric findings are corroborated against Mark Twain's documented authorial intent in letters to Howells.

arxiv arXiv cs.CL · 14h ago

Introducing corpora Hlava Cor and Hlava AD: Human Label Variation in Coreference and Discourse Relations

Researchers have created two new corpora, Hlava Cor and Hlava AD, to explore human variation in understanding text coherence. These resources contain multiple annotations of Czech texts along with annotators' explanations for their choices. The first corpus, Hlava Cor, consists of 1,024 contexts annotated by three individuals to capture coreference identification differences. It covers pronouns, full noun phrases, and anaphoric adverbials across various text types and grammatical-semantic categories. The second corpus, Hlava AD, comprises 512 contexts annotated by five annotators focusing on discourse relations in attributive and non-attributive constructions. Both corpora achieve an inter-annotator agreement of approximately 60-65 percent. Analysis reveals that lower coreference agreement correlates with automatic model disagreement, indicating higher ambiguity. Annotator comments further highlight varying confidence levels and individual reading strategies.

arxiv arXiv cs.CL · 14h ago

Agent-Authored World Modeling Aligns Training with Decision Needs

The paper introduces Agent-Authored World Modeling (AAWM), a training procedure that addresses the limitations of standard world modeling objectives tied to next-observation prediction. This traditional approach often omits dynamics relevant to an agent's current decision because supervision depends on what a transition reveals rather than what is needed. AAWM constructs supervision directly from the policy's decision needs by having the agent identify necessary environmental understanding at each state. Relevant transition evidence is retrieved across trajectories and synthesized into training targets that capture these decision-oriented dynamics. This method aligns the learning objective with the specific information required before acting, rather than forcing the model to reconstruct the next observation. Experimental results validate AAWM's effectiveness across multiple environments and training settings. The findings demonstrate that decision-aware world-model targets provide a more effective learning signal than conventional next-observation prediction.

arxiv arXiv cs.CL · 14h ago

OscillaTTS: Adaptive Oscillatory Inductive Bias for Modeling Sharp Prosodic Dynamics in Diffusion-Based TTS

Diffusion-based text-to-speech models have improved speech quality but struggle with sharp prosodic transitions and rapid pitch variations. Existing decoders often use periodic nonlinearities like the Snake activation function, which lack adaptability for abrupt amplitude and frequency changes. To address this, the authors introduce OscillaTTS, a system featuring an adaptive oscillatory nonlinearity. This component enables controllable periodic modulation while ensuring signal stability via a linear bypass mechanism. The study investigates the role of oscillatory inductive bias within diffusion-based TTS decoders. Experiments conducted on the LJSpeech and Emotional Speech Dataset demonstrate consistent improvements in both objective and subjective evaluations. These results indicate that OscillaTTS effectively models expressive prosodic dynamics compared to prior methods.

arxiv arXiv cs.CL · 14h ago

PolicyAlign: Direct Policy-Based Safety Alignment for Large Language Models

The authors introduce PolicyAlign, a framework designed to align large language models directly with natural-language safety policies rather than relying on costly supervision data. This approach addresses the mismatch between rapidly evolving safety requirements and conventional data-driven alignment methods. The process begins by synthesizing instructions that violate the specified policy, followed by on-policy self-distillation to internalize the desired behavior. To enhance training stability and data efficiency, the method incorporates Policy-Sensitive Filtering, which selects instructions inducing the largest behavioral shift. Experiments across multiple models demonstrate that PolicyAlign consistently improves safety metrics while maintaining low over-refusal rates and preserving general capabilities. The framework also generalizes effectively to specialized domains such as medical, legal, and financial safety scenarios. The code for this scalable alignment approach is released at https://github.com/Qwen-Applications/PolicyAlign.

arxiv arXiv cs.CL · 14h ago

Translation-Enhanced Speech Encoder Pre-training Improves Speech LLMs

Connecting a pre-trained speech encoder to a Large Language Model creates a structural misalignment because encoders often produce language-specific representations while LLMs operate in a unified, language-agnostic space. The authors argue that incorporating speech translation objectives into the pre-training process provides a principled mechanism to bridge this gap. Unlike monolingual transcription, translation forces the model to learn representations that are independent of specific languages. The study experimentally evaluates the impact of adding these translation objectives during speech encoder pre-training. Results demonstrate that this approach significantly improves cross-modal integration between the speech and text modalities. Consequently, models utilizing translation-enhanced pre-training achieve superior performance across various downstream Speech LLM tasks.

arxiv arXiv cs.CL · 15h ago

Reclaim Evaluation Shows Lossy Memory Is Worse Than No Memory

A study demonstrates that a language model's memory containing incorrect conclusions is more detrimental than having no memory at all. When models retain stale values while dropping supporting work, they emit confident but wrong answers, whereas empty memories allow for abstention. This phenomenon, termed brittle memory, was observed across seven models where the direction of failure never reversed regardless of task or disposition. The researchers introduced reclaim evaluation to measure correctability by compressing interactions and testing if corrections recover ground truth without using a judge. Results indicate that correctability depends on whether the source information survives compression rather than model capability. A source-first policy, which keeps recomputable sources and drops re-derivable conclusions, restored correctability significantly better than length-matched controls. In chained memory loops, dropped-source errors corrupt downstream steps irreparably, while the proposed fix maintains bounded performance horizons. The findings replicate across three deployed systems and real dialogue data, with a hand-built oracle reaching perfect accuracy.

arxiv arXiv cs.CL · 15h ago

The Generalization Spectrum: A Chromatographic Approach to Evaluating Learning Algorithms

Traditional evaluations reduce learning to a single aggregate score, obscuring how well knowledge from one example generalizes to others. The authors introduce the Generalization Spectrum, an evaluation framework that measures per-sample generalization by tracking performance across test variants with increasing transfer distance. These variants range from exact recall to implementation transfer across languages and context transfer under narrative reframing. The framework is instantiated on competitive programming using a selection-and-synthesis pipeline seeded with recent problems to mitigate contamination. Comparisons of canonical learning paradigms show that Reinforcement Learning converts memorization into near-transfer more efficiently than Supervised Fine-Tuning baselines. In-context learning exhibits strong but correspondence-dependent transfer capabilities in this context. Diagnostic profiles reveal that local gains do not necessarily expand the generalization radius for all methods. Specifically, abstractions and hints mainly lift local transfer, while Reference SFT preserves a stronger far-transfer tail than RFT. Furthermore, self-distillation or hint-assisted RL can reduce far transfer even when local transfer improves.

arxiv arXiv cs.CL · 15h ago

Probing Self-Supervised Speech Representations on Mandarin Sub-dialects via Unsupervised Articulatory Analysis

This study investigates how internal phonetic representations in self-supervised speech models behave under fine-grained dialect variation, addressing the limitations of existing probing studies that rely on curated corpora. The authors present a case study using an entirely unlabeled probing pipeline for Mandarin sub-dialects. Phone sequences are generated via a language-agnostic universal phone recognizer and mapped to articulatory feature vectors, enabling frame-level probing without manual annotation. Results reveal structured patterns in articulatory feature decodability across different Mandarin dialects. Acoustically salient features like labiality and stridency remain comparatively stable, while those associated with finer spectral distinctions show larger dialect-dependent variation. This variation is driven primarily by elevated decodability for Beijing speech relative to other sub-dialects. Layer-wise analyses demonstrate distinct representational dynamics for these feature groups, suggesting uneven dialect sensitivity across articulatory dimensions.

arxiv arXiv cs.CL · 15h ago

Fully Differentiable Neural Forced Alignment via Soft Dynamic Programming

The authors propose an end-to-end, fully differentiable neural architecture designed specifically for phoneme alignment to address the stagnation in this field compared to ASR advancements. The model features an encoder with two complementary branches dedicated to phoneme identity verification and boundary detection. A decoder implemented as a trainable module based on differentiable soft dynamic programming produces the final alignment decisions. The entire system is optimized using a novel contrastive loss that encourages clear separation between steady-state phoneme regions and transition boundaries. Experimental results show the approach outperforms current state-of-the-art methods on hand-annotated English benchmarks. Additionally, the model demonstrates strong word-level generalization capabilities and effective performance on unseen languages.

arxiv arXiv cs.CL · 15h ago

Fine-Tuned PEGASUS Achieves State-of-the-Art Performance on XL-Sum English Corpus

This paper presents a method for optimizing abstractive text summarization by fine-tuning the PEGASUS model on the XL-Sum English corpus. The objective is to surpass the performance of the baseline mT5 model in generating concise summaries that capture salient ideas without merely extracting sentences. The generated summaries are evaluated using the ROUGE metric, which compares auto-generated outputs against human-created references. The study claims that the fine-tuned PEGASUS model achieves state-of-the-art results on this specific dataset. Quantitative analysis reveals a 4.04% improvement in the ROUGE-1 score compared to the baseline. Additionally, the model demonstrates a significant 15.25% increase in the ROUGE-2 score. Finally, there is a reported 3.39% improvement in the ROUGE-L score, confirming the effectiveness of the fine-tuning approach.

arxiv arXiv cs.CL · 16h ago

Cliff Tokens: Identifying Single-Token Failure Triggers in LLM Mathematical Reasoning

Researchers introduce the concept of cliff tokens to identify specific single-token failure triggers in large language models during mathematical reasoning tasks. Unlike prior work that analyzes failures at step or sentence levels, this method pinpoints the exact token where potential drops significantly using an adaptive threshold based on a z-test. The study evaluates seven models across three benchmarks: GSM1K, MATH500, and AIME 2025. Deleting the first cliff token and resampling allows recovery of pass@64 to 1.0, whereas keeping it limits recovery between 0.71 and 1.00. The authors propose a taxonomy classifying cliffs as deterministic, uncertain, or sampled-off based on greedy choice and token entropy. This classification generalizes across different model scales and exhibits distinct probabilistic characteristics for each type. Furthermore, the team validates this taxonomy through single-token preference optimization known as Cliff-DPO. Trained on GSM8K, Cliff-DPO improves accuracy by up to +6.6 across benchmarks. Optimization proves effective for uncertain and sampled-off cliffs but yields no improvement for deterministic ones.

arxiv arXiv cs.CL · 16h ago

SFL-MTSC: Leveraging Semantic Frame-Level Multi-Task Self-Consistency for Robust Multi-Intent Spoken Language Understanding

Prompt-based spoken language understanding with large language models often suffers from inconsistent intent-slot structures due to decoding stochasticity, particularly in multi-intent scenarios. To address this, researchers propose Semantic Frame-Level Multi-Task Self-Consistency (SFL-MTSC), a novel structured aggregation framework operating at the semantic frame level. Instead of relying on output-level majority voting, SFL-MTSC decomposes predictions into intent-specific frames and applies domain-intent grouping alongside slot-level clustering. The framework evaluates cluster reliability using path support scoring to determine which frames are trustworthy. Reliable frames are retained and re-integrated to form the final prediction, ensuring greater structural consistency. Zero-shot experiments on the MAC-SLU benchmark dataset demonstrate improved slot F1 scores and overall accuracy compared to single-path inference. Intent accuracy remains largely stable across most settings while achieving these gains in slot-level performance.

arxiv arXiv cs.CL · 16h ago

Riazi-8B: An Urdu Large Language Model for Mathematical Reasoning

Recent large language models demonstrate strong mathematical reasoning, but these gains rely heavily on English-centric resources, leaving low-resource languages like Urdu with limited capabilities. To address this gap, researchers developed Riazi-8B, an Urdu model designed specifically for multi-step mathematical problem solving. The model was created through a two-step adaptation process involving continued pre-training on Urdu Wikipedia and supervised fine-tuning on Urdu Chain-of-Thought data derived from GSM8K. Evaluation of Riazi-8B was conducted on the MGSM-Urdu benchmark against existing Urdu instruction-tuned models. The results showed consistent improvements in answer correctness, reasoning quality, response completeness, and Urdu generation compared to baselines. These findings demonstrate that combining Urdu language adaptation with reasoning-focused fine-tuning effectively extends mathematical reasoning capabilities to low-resource languages.

arxiv arXiv cs.CL · 16h ago

Constraint Tax in Open-Weight LLMs: Tool Calling Suppression Under Structured Output Constraints

This study identifies a phenomenon called Tool Suppression, where open-weight language models cease invoking tools when JSON Schema constraints are simultaneously enabled. The authors observed this behavior in a production Agent system and reproduced it through controlled experiments across multiple model families. While tool execution and schema compliance function correctly when evaluated independently, they fail under joint deployment conditions. Analysis reveals that JSON Schema constraints are compiled into grammar-based token masks, rendering tool-call tokens unreachable during decoding. To interpret these findings, the paper proposes the Constraint Priority Inversion hypothesis, suggesting schema satisfaction dominates action selection under simultaneous constraints. The authors mitigate this issue by introducing Transparent Two-Pass Execution, an inference-time strategy that decouples tool execution from response generation. This approach restores tool invocation while preserving structured output guarantees without requiring model retraining. The research highlights that evaluating capabilities separately may overlook critical reliability issues in production systems.