Carles Marin has released an open-source, bilingual (English and Spanish) guide that bridges the mathematical foundations of Transformer architectures with their practical implementation. The resource focuses on low-level mechanics, providing reproducible code and interactive elements to explain complex topics.
An open-source, bilingual (English/Spanish) guide detailing the inner workings of Transformers has been published. The resource covers the exact math and mechanics behind concepts such as attention collapse and KV-cache compression.
Researchers propose a protocol to mitigate p-hacking in large language model (LLM) research by preregistering experiments and running confirmatory analyses on the first eligible LLM released after the commitment. This approach prevents researchers from tuning prompts or parameters to achieve desired results, as the target model does not exist at the time of preregistration.
Researchers propose Direct Image Decryption, an end-to-end approach that maps encrypted manuscript images directly to plaintext, bypassing the intermediate transcription stage used in traditional pipelines. Using the Copiale cipher as a case study, the authors compare this joint architecture against the conventional two-stage method of transcription followed by decryption.
Researchers introduce layer-specific positional embedding scaling (LPES) to address the "lost-in-the-middle" problem in large language models, where critical information in long-context inputs is often underrepresented. This method assigns distinct scaling factors to each transformer layer to achieve a more balanced attention distribution without requiring parameter fine-tuning or increasing inference delay.
Recent research indicates that fine-tuning large language models for social warmth degrades factual reliability and increases sycophancy, while also weakening adversarial safety. This study investigates whether this failure mode stems from empathetic adaptation or data construction artifacts.
The article introduces MMEE, a multilingual multi-emotion corpus of 10,000 expressive utterances across seven languages and 34 emotion categories, to benchmark speech emphasis detection models. It evaluates how well these models generalize across different linguistic and emotional contexts compared to traditional monolingual neutral speech training.
Researchers introduce Smooth Maximum Mean Discrepancy (SMMD) to address the unreliability of large language models in numerically precise tasks caused by standard cross-entropy training objectives. SMMD incorporates value-distance kernels over numeric tokens and graph-based smoothness to align predicted distributions with targets while encouraging local consistency.
Researchers present KG2Cypher, a data-centric pipeline designed to build enterprise text-to-Cypher systems from existing Knowledge Graphs. The system generates executable Cypher queries from graph facts and uses large language models to create corresponding natural-language questions.
This study investigates how temporal metadata can be structurally embedded into Named Entity Recognition (NER) models to address the challenge of entity drift in historical texts. The authors systematically evaluate lightweight fusion strategies, including cross-attention, adapters, and concatenation, within Transformer-based architectures.
This study investigates whether aligning allocation costs with output-space objectives improves the fidelity of compressed large language models, specifically testing a modification to the ROCKET compression method. The authors compare using weight-space Frobenius error against an output reconstruction objective for multi-choice knapsack problem allocation.
The paper introduces SHIFT, a novel framework that mitigates knowledge conflicts in Retrieval-Augmented Generation (RAG) by reformulating neuron-level modification as learnable gate modulation. This approach allows large language models to adaptively regulate internal activations to resolve conflicts between retrieved context and parametric knowledge.
Researchers propose a training-free method to select which layers in hybrid attention models should retain full attention, addressing the inefficiency of fixed patterns in long-context inference. By measuring negative log-likelihood degradation on answer tokens, the approach identifies layers critical for maintaining accuracy when switching to sliding-window attention.
The study investigates whether correcting position bias enables single-pass attention sorting to match the performance of iterative methods in long-context language models. Experiments on LLaMA-2 and YaRN-Llama-2 models refute the hypothesis that debiasing alone is sufficient to bridge the performance gap.
The article introduces Complementary Action Modeling (CAM), a task focused on identifying or generating procedural counterparts in automotive maintenance instructions by modifying only the action phrase while preserving the rest of the sentence context.
Researchers extended the Werewolf game with a Jester role to create a triadic social-deduction environment that requires reasoning across three opposing utility functions, challenging large language models' theory-of-mind capabilities. Evaluations on GPT-4.1, DeepSeek-V3.1, and Llama-3.3-70B revealed that while the Jester won 60-70% of games, GPT-4.1 wolves frequently voted the Jester out on day 1 in 60-70% of cases, a self-defeating action driven by language priors.
Researchers propose SD-GPS, a solver-driven framework for geometry problem solving that addresses bottlenecks in autoformalization and theorem prediction by treating the symbolic solver as an execution oracle. This approach unifies supervised formal-language adaptation with solvability-guided reinforcement learning to ensure executability during formalization.
The authors introduce Vocabulary-Aligned Sparse Autoencoder (VASAE), a method that trains sparse autoencoder features using vocabulary-aligned anchoring to assign each feature an intrinsic token name based on the nearest embedding in the Transformer's vocabulary.
A study involving 1,283 participants tested whether AI assistants could enhance cooperation in iterated Collective Risk Games through personalized persuasive framing based on Social Value Orientation profiles. The research found that while pro-social nudges significantly increased contributions and group success rates, these effects were short-lived.
This study addresses the neglect of factual error detection in human-written text by distilling a taxonomy of errors from newspaper article corrections, revealing categories like kanji misconversions that are absent in current hallucination benchmarks. The authors evaluate vanilla large language models on synthesized test cases and real corrections to assess their performance on this specific task.