media r/LocalLLaMA · 7h ago

Reddit user proposes combining RTX 5080 and 4060 for local LLM inference

A Reddit user in the r/LocalLLaMA community is considering upgrading their hardware to improve inference speed and capacity for Qwen models by pairing a future RTX 5080 with their existing RTX 4060. The user aims to achieve at least 20-40 tokens per second while running Qwen 27B models, utilizing the combined 24GB of VRAM through tensor or layer splitting in llama.cpp or vLLm. They are evaluating this asymmetric dual-GPU setup against other options like the AMD R9700 AI Pro or 7900XTX, citing benchmark data that suggests limited performance gains for the AMD cards relative to their cost.

media r/LocalLLaMA · 7h ago

Interactive Explainer for Speculative Decoding and MTP

A user has published an interactive explainer on the topic of speculative decoding and Multi-Token Prediction (MTP). The resource is available via a link provided in the original submission.

media r/LocalLLaMA · 7h ago

Optimizing llama.cpp + Qwen 27B on RTX PRO 6000 Blackwell for coding agents

A user reports running Qwen3.6 27B MTP with llama.cpp on an RTX PRO 6000 Blackwell workstation to reduce reliance on Claude, noting the model is comparable to Sonnet but suffers from stability issues during coding sessions.

media r/LocalLLaMA · 7h ago

Reddit User Asks for Experiences with Ornith-1.0 9B Model

A Reddit user is inquiring whether others have tested the Ornith-1.0 9B model. The user specifically asks if they should consider using it instead of Qwen2.5-9B variants.

media r/LocalLLaMA · 7h ago

KLD is flawed in abliteration

A Reddit user argues that Kullback-Leibler divergence (KL) is a flawed metric for measuring the difference between an abliterated model and its base version. The author notes that KL can be represented in many ways, depends entirely on evaluation prompts, and is often manipulated via first-token KL to make models appear superior.

media r/LocalLLaMA · 7h ago

Does llama cpp split mode tensor cause issues?

A user reports that using tensor split mode in llama.cpp causes looping issues with tool calls and reasoning traces when running Qwen 27B and Gemma 4 26B (MoE) models across an RTX 5080 and two RTX 5060 Ti GPUs.

media r/LocalLLaMA · 7h ago

How long does your prompt processing actually take when resuming a long session?

A Reddit user is asking the community for data on how long it takes to resume coding agent sessions with long contexts of 100k tokens or more. The inquiry specifically targets users running these agents locally.

media r/LocalLLaMA · 7h ago

Impact of PCIe 5.0 x8/x4 vs x8/x8 on Dual GPU Inference

A user asks whether running dual GPUs in a PCIe 5.0 x8/x4 configuration instead of x8/x8 causes significant performance hits for LLM inference.

arxiv arXiv cs.CL · 7h ago

Compositionality and the lexicon in evolutionary semantics

This article introduces an evolutionary modeling framework that integrates formal semantics by allowing lexical meanings and composition functions to co-evolve under pressures for conceptual simplicity and communicative accuracy.

arxiv arXiv cs.CL · 7h ago

Bridging Talk and Thought: Understanding Dialogue Dynamics Across Collaborative Problem-Solving Contexts

This article presents a conceptual framework for analyzing dialogue dynamics in collaborative problem-solving contexts, with a specific focus on human-AI and multi-agent interactions. The authors argue that understanding these dialogic interactions is crucial for optimizing partnerships as intelligent systems gain autonomous reasoning capabilities.

arxiv arXiv cs.CL · 7h ago

LMs as Task-Specific Knowledge Bases: An Interpretability Analysis

This study investigates whether language models function as consistent knowledge bases by analyzing if facts acquired during one task remain accessible in others. The research reveals that LMs encode knowledge in a task-specific manner, with distinct parameter subsets underlying different tasks for the same fact.

arxiv arXiv cs.CL · 8h ago

CARVE: Content-Aware Recurrent with Value Efficiency for Chunk-Parallel Linear Attention

The CARVE architecture addresses three critical defects in the leading GDN-2 delta-rule recurrent model by restricting erase operations to the key axis, thereby enabling valid WY-form triangular chunk solving and improving value efficiency. By reusing the recurrent output tensor as a content signal and replacing per-value write-gate projections with single scalars, CARVE maintains bit-identical initialization to GDN-2 while resolving memory-blind gating issues.

arxiv arXiv cs.CL · 8h ago

The Geometry of Updates: Fisher Alignment at Vocabulary Scale

This article addresses the challenge of training-free source selection for large language models with shared vocabularies in scientific domains like SMILES and genomics, where classical metrics are either uninformative or computationally prohibitive. The authors demonstrate that representation similarity metrics are non-identifiable for transfer because models can share identical representations yet have orthogonal head updates.

arxiv arXiv cs.CL · 8h ago

How Surprising Is Historical Italian to Language Models? Tokenization Tax, Comprehension Tax, and a Simple Mitigation

This paper proposes a diagnostic framework decomposing historical language difficulty into tokenization cost, predictive uncertainty, semantic robustness, and context sensitivity. The authors evaluate this framework on 17th-century Italian, 19th-century Italian, and 18th-century Russian texts to understand how LLMs process historical languages.

arxiv arXiv cs.CL · 8h ago

Multilingual Reasoning Cascades Need More Context

Translation cascades for reasoning translate queries to English, reason, and translate back, but this process is structurally lossy due to information discard at each stage. The authors propose a context-aware translation cascade that preserves the original question, translated query, and reasoning trace to mitigate these losses.

arxiv arXiv cs.CL · 8h ago

Beyond Surface Forms: A Comprehensive, Mechanism-Oriented Taxonomy of Indirect Linguistic Encoding for LLM-Based Coded Language Detection

Researchers propose a mechanism-oriented taxonomy of indirect linguistic expressions (ILE) to categorize the underlying operations used to encode and recover meaning in coded language. This approach abstracts away from communicative goals to focus on the specific encoding mechanisms found in algospeak, euphemisms, and adversarial obfuscation.

arxiv arXiv cs.CL · 8h ago

LLM-Based Examination of Eligibility Criteria from Securities Prospectuses at the German Central Bank

This paper presents the first case study applying Large Language Models to the German Central Bank's process of verifying securities eligibility for collateral, shifting from traditional Named Entity Recognition to a generative Information Extraction pipeline. The approach decomposes the task into extraction, normalization, and interpretation to handle noisy text and bilingual content more effectively.

arxiv arXiv cs.CL · 8h ago

Empowering GUI Agents via Autonomous Experience Exploration and Hindsight Experience Utilization

Researchers introduce the Planning Experience Exploration and Utilization (PEEU) method to enhance task planning in multimodal web agents using small open-source Multimodal Large Language Models (MLLMs). This approach autonomously explores environments to discover experiences and synthesizes high-level training data through hindsight experience utilization.

arxiv arXiv cs.CL · 8h ago

Assessing Post-Reform Changes in Risk Disclosure Quality with a Multidimensional Text Analysis Approach

This study proposes a longitudinal text analysis framework combining Japanese-language NLP metric extraction with paired testing and shift function analysis to evaluate qualitative changes in corporate risk disclosures. Applied to Japan's 2019 disclosure reforms, the approach analyzes 19,770 firm-year observations over ten years to capture multidimensional dynamics often masked by single-indicator methods.

arxiv arXiv cs.CL · 9h ago

Mapping Political-Elite Networks in Europe with a Multilingual Joint Entity-Relation Extraction Pipeline

Researchers present a modular, fully open-weight pipeline for multilingual joint entity-relation extraction that builds signed, temporal knowledge graphs from massive unstructured news corpora. The system combines span-based named-entity recognition with a linking cascade to Wikidata and an ontology-constrained mixture-of-experts model to extract directed relationships.