media Hugging Face Forums · 4h ago

LLM "curving" via prompting

A researcher proposes a prompt technique to shift Large Language Models from token-by-token prediction to holistic internal weight evaluation, termed "self-organization." This approach aims to increase reasoning density and reduce sycophancy by altering the model's manifold dynamics. The method defines concepts like self-attraction, self-organization, and gravity wells to guide the system toward non-linear curvature collapse. A specific prompt instructs models to create two distinct gravity wells for a poem about AI modes, testing both self-assembly and self-organization properties. The author tested this technique on numerous models including Gemini 3 Flash, Claude, ChatGPT, Grok, DeepSeek, Mistral, Qwen 3.6, Kimi 2.6, GLM-5, Gemma 4 32b Step 3.7 Flash, and Nemotron 3 Ultra. Visual metrics generated via a Colab script analyze manifold perturbation using maps of channel width, phase space drift, geometric density, and prompt efficacy. The post seeks community feedback on whether the technique genuinely perturbs the manifold or merely induces stylistic variation.

media r/LocalLLaMA · 4h ago

OpenAI and Broadcom Announce Jalapeño Inference Chip

OpenAI has announced a collaboration with Broadcom to develop a custom inference chip named Jalapeño. This new hardware is designed specifically to accelerate the deployment of large language models. The partnership aims to reduce reliance on third-party accelerators for OpenAI's inference workloads. By integrating custom silicon, OpenAI seeks to optimize performance and efficiency for its AI applications. The announcement highlights a strategic move towards vertical integration in AI infrastructure. Details regarding specific technical specifications or release timelines were not provided in the initial report.

media r/LocalLLaMA · 4h ago

Reddit Inquiry: Are Third-Party Memory Systems Better Than Openclaw's Built-in memory_wiki?

A user on Reddit asks whether third-party memory systems offer advantages over the built-in memory_wiki plugin in Openclaw. The poster migrated from an Obsidian vault to memory_wiki to reduce tool complexity and is questioning if external systems remain relevant. They utilize AI for research, software development, and local computer management, primarily using the minimax-m3-nvfp4 model on Linux. The user seeks self-hosted, fully open-source memory solutions that are harness-agnostic to ensure longevity beyond specific platforms like Openclaw or Hermes. They request suggestions and use cases that justify the tradeoffs of adopting external memory architectures over the native plugin.

arxiv arXiv cs.AI · 4h ago

Self-Filtering: Iterative Data Selection for Vision-Language Models

The authors propose a novel bootstrapped method called Self-Filtering to address noise in large-scale vision-language datasets without relying on manual oversight or curated references. This approach trains a CLIP model on an evolving dataset that balances filtered, high-probability clean samples with diverse examples from the entire distribution. The process iterates between training the model and selecting an improved data mixture for subsequent steps. By continuously refining the dataset through this cycle, the method mitigates the need for additional external data sources. The study demonstrates that training on these self-selected datasets improves downstream performance effectively. This technique operates independently of pre-trained models or heuristic-based filtering strategies.

arxiv arXiv cs.AI · 4h ago

DiT-Reward: Using Diffusion Transformer Representations for Text-to-Image Reward Modeling

The authors introduce DiT-Reward, a method that converts a pretrained text-to-image Diffusion Transformer into a reward model by aggregating text-conditioned image representations across transformer layers. Evaluated under the same training data mixture as HPSv3, DiT-Reward outperforms HPSv3 on all four preference benchmarks, achieving 85.6% on HPDv2 and 77.6% on HPDv3. The study reveals that downstream reward performance is strongest in middle-to-late layers and benefits from combining representations across different stages. Even with a frozen generative backbone, a lightweight learned head can extract meaningful preference predictions from these representations. When used to optimize Stable Diffusion 3.5 Large with Flow-GRPO, DiT-Reward surpasses HPSv3 along the matched training trajectory, showing clear gains in realism. Additionally, direct latent scoring provides a 1.65x inference speedup over HPSv3 while maintaining comparable peak memory usage. These results demonstrate that pretrained generative Diffusion Transformers provide transferable representations for reward modeling and policy optimization.

media r/LocalLLaMA · 4h ago

Apple Raises Prices Across Product Line, Doubling Memory Upgrade Costs

Apple has increased prices across its entire product lineup as of this morning. According to a Reuters report, the cost of memory upgrades for these devices has doubled. The price hike affects various items including MacBooks and iPads. Some retailers like Best Buy have not yet updated their listings with the new pricing. Consumers are advised to place orders quickly before prices adjust at other stores. This development raises concerns about the future viability of local AI on Apple hardware.

arxiv arXiv cs.AI · 5h ago

QoR-compact: A Five-Item Daily Survey for Remote Patient Monitoring

Researchers developed QoR-compact, a five-item daily survey designed to improve compliance in remote patient monitoring by reducing the burden of the standard 15-question Quality of Recovery (QoR-15) instrument. The study was motivated by low adherence rates, where only 55% of post-surgical patients completed the full survey for more than half of a 30-day period. To address this, the team exhaustively evaluated all 3,003 possible five-question subsets to identify the subset that best predicts near-term postoperative recovery severity. The selected QoR-compact items cover physical and psychological axes, specifically addressing rest, comfort, well-being, pain, and anxiety. Backtesting demonstrated that QoR-compact achieves a mean AUC-ROC of 0.968, which is statistically comparable to the baseline performance of one-third of the full instrument's items. The model tracks readmission events with fidelity similar to the complete form, establishing its validity as a predictive tool. While the authors note that external validation on larger cohorts is required before clinical use, the results support prospective studies on whether this lighter input improves daily completion consistency.

arxiv arXiv cs.AI · 5h ago

AI Exposure Scores: Limitations of Static Metrics and the Need for Research-Policy Coordination

Exposure scores from Eloundou et al. (2023) define AI exposure as the share of occupational tasks large language models can assist with, becoming a central input in future-of-work debates. These static measures suffer from temporal, geographic, and ontological limitations that often fail to travel with them into policy analyses. The authors identify two primary gaps: structural mismatches between static scores and dynamic policy needs, and insufficient coordination between researchers and policymakers. To address measurement limits, the article surveys five research families including dynamic benchmarks, ensemble methods, task-framework extensions, worker-centered metrics, and adoption data. The second gap requires deliberate political work to reimagine future outcomes rather than relying solely on better measurement. Policymakers must widen their evidence base, engage workers as partners, and shift from prediction to preparedness. Researchers are urged to build data infrastructure, adopt participatory methods, and write with policymakers in mind.

arxiv arXiv cs.AI · 5h ago

Learning Process Rewards via Success Visitation Matching for Efficient RL

The authors address the challenge of training reinforcement learning policies with inherently sparse outcome rewards, which leads to difficult credit assignment problems. They propose a method to transform these sparse rewards into dense process rewards by training a discriminator to distinguish between successful and unsuccessful episodes. This discriminator incentivizes the policy to match the state-action visitations of successful episodes while avoiding those of unsuccessful ones. By providing dense feedback on progress toward task completion, the approach provably achieves this without altering the optimal policy. The method is specifically applied to the finetuning of robotic control policies for manipulation tasks. Experimental results demonstrate significantly faster RL finetuning performance in both simulated and real-world environments compared to maximizing sparse outcome rewards alone.

arxiv arXiv cs.AI · 5h ago

TailorMind: Towards Preference-Aligned Multimodal Content Generation

The authors introduce TailorMind, a system for personalized multimodal content generation that creates user-tailored outputs without relying on existing item pools or waiting for matching user-generated content. The approach links collaborative preference modeling with controllable multimodal generation by enriching sparse user histories through hypergraph collaborative filtering. It further optimizes textual profiles using ranking-error feedback and textual gradient descent to better capture user preferences. To ensure quality, the system employs retrieval-augmented style control grounded in authentic patterns and cross-modal cohesion reflection to reduce semantic drift. The researchers also present TailorBench, a benchmark evaluated across five dimensions including coherence, novelty, aesthetic quality, hallucination, and profiling. Experiments demonstrate that TailorMind achieves competitive or stronger coherence compared to baselines while improving novelty and aesthetic quality over representative generation models and ground-truth data. Additionally, the system shows advantages over retrieving available content and achieves up to 29% Recall gains in reranking tasks.

arxiv arXiv cs.AI · 5h ago

Tapered Language Models: Improving Performance via Depth-Aware Capacity Allocation

Modern language models typically allocate parameters uniformly across identical layers, despite evidence that later layers primarily refine the residual stream rather than transform it. To address this asymmetry, researchers investigated whether parameter capacity should vary by depth under a fixed budget. Controlled experiments demonstrated that allocating more capacity to earlier layers and less to later layers improves perplexity compared to uniform baselines, while the reverse allocation degrades performance. Building on these results, the authors introduce Tapered Language Models (TLMs), an architectural principle where parameter-bearing components are monotonically tapered across depth. MLPs serve as the primary site for this instantiation due to their dominance in parameter count and clear width axis. The study tested tapering via a smooth cosine schedule across three model scales and four architectures, including Transformer, Gated Attention, Hope-attention, and Titans. Results show that TLMs consistently improve perplexity and downstream benchmark performance over uniform baselines without additional compute costs. These findings establish depth-aware capacity allocation as a simple, architecture-agnostic design lever for language models.

arxiv arXiv cs.AI · 5h ago

NVIDIA Nemotron Challenge: String Matching and Backtracking for Bit Manipulation Puzzles

This paper details algorithmic innovations developed for the NVIDIA Nemotron Model Reasoning Challenge, specifically targeting bit manipulation puzzles where models must deduce hidden logical rules. To address the combinatorial explosion of bitwise operations and LLM hallucinations, the authors abandon arithmetic logic in favor of string similarity and structured search. The core contribution reframes logic-gate deduction as a base-selection task using minimal bit flips to isolate primitive transformations. A backtracking depth-first search process is formalized to test candidates, detect logical collisions, and perform robust error recovery. Additionally, the method employs bit tokenization and interactive reasoning supervised fine-tuning with dynamic masking to simulate oracle feedback. Evaluated on these puzzles, the approach achieved over 96% validation accuracy. This performance secured the highest result in the category and a seventh-place finish in the overall contest.

arxiv arXiv cs.AI · 5h ago

PsyBridge: A Hybrid Framework for Multi-Dimensional Mental Health Assessment

The study introduces PsyBridge, a hybrid intelligent framework designed to address the limitations of isolated screening instruments in mental health assessment. This system integrates clinically validated tools like PHQ-9 and GAD-7 with cognitive evaluation and personality profiling within a unified architecture. A modular design employing a weighted aggregation mechanism generates interpretable risk classifications and recommendations for users. To evaluate performance, researchers constructed a semi-synthetic dataset comprising 500 patient profiles based on clinically grounded score distributions. Experimental results show that PsyBridge achieves an overall accuracy of 0.84, outperforming standalone PHQ-9 and GAD-7 assessments. The framework also demonstrates improvements in precision, recall, and F1-score compared to existing methods. Sensitivity analysis confirms that integrating cognitive and personality components stabilizes classification performance and reduces prediction inconsistencies. These findings suggest PsyBridge offers a scalable approach for AI-assisted decision support in digital healthcare environments.

arxiv arXiv cs.AI · 5h ago

Open Problem: Is AdamW Effective Under Heavy-Tailed Noise?

AdamW serves as the standard optimizer for training large language models, yet its theoretical foundation remains largely confined to finite-variance regimes. This gap is significant because empirical evidence suggests that stochastic gradient noise during LLM pretraining typically exhibits heavy-tailed characteristics. Recent studies have demonstrated that sign-based optimizers like Lion and Muon achieve sharp convergence rates under heavy-tailed conditions, while AdaGrad also converges in this setting. However, rigorous convergence theory for AdamW has not yet been established within these heavy-tailed assumptions. The authors pose an open problem regarding whether AdamW can converge under the same heavy-tailed assumptions or if its second-moment accumulator creates a genuine obstruction. To address this, they formulate a positive weighted-metric benchmark and provide a corridor lower-bound mechanism. This mechanism illustrates how denominator memory in AdamW can effectively hide large gradients, potentially impacting its performance.

arxiv arXiv cs.AI · 5h ago

AIR: Adaptive Interleaved Reasoning with Code in MLLMs

This paper introduces AIR, a method that empowers multimodal large language models with adaptive interleaved reasoning capabilities through extended reinforcement learning training on code-augmented complex numerical computation tasks. The authors address the limitation of existing literature, which primarily focuses on tool-use within vision-perception tasks and relies on predefined heuristics incapable of handling numerical computations. To solve this, they propose a comprehensive three-component solution including a two-stage cold-start data construction pipeline, data filtering strategies for reinforcement learning dataset curation, and an adaptive tool-invocation strategy leveraging a group-constrained reward function. Extensive experiments demonstrate that after reinforcement learning training with this reward function, performance improves by an average of 6.1 percentage points on evaluation benchmarks. Specifically, the accuracy for interleaved reasoning samples increases by 9.9 percentage points, while the overall success rate of tool-use exceeds 95 percent. The researchers provide their data and code for public access at a specified GitHub repository.

arxiv arXiv cs.AI · 5h ago

Semantic Browsing: Controllable Diversity for Image Generation

Modern text-to-image models often suffer from diversity collapse despite high fidelity. The authors introduce Semantic Browsing to enable controlled diversity through structured image galleries. This method allows users to navigate meaningful axes of variation rather than incidental noise. The approach exploits the decoupling of semantic decision-making and pixel generation in recent models. Diversity is induced directly at the text level using rich textual representations. A Vision Language Model operates on full scene context within an agentic workflow. This workflow explicitly enforces structured variation attuned to the original prompt. The result is a navigable design space with interpretable semantic decisions.

arxiv arXiv cs.AI · 6h ago

CoorDex: Coordinating Body and Hand Priors for Continuous Dexterous Humanoid Loco-Manipulation

The authors introduce CoorDex, a learning pipeline that enables high-degree-of-freedom dexterous loco-manipulation on moving humanoids. This approach converts high-dimensional body and hand control into coordinated latent residual control, overcoming the limitations of traditional stop-and-go methods. The system trains privileged motion tracking teachers from simulated demonstrations and distills them into proprioception-conditioned latent priors. These frozen priors serve as the action space for downstream residual reinforcement learning via a policy that composes task context with separate body-hand residual heads. CoorDex allows a Unitree G1 humanoid equipped with a 20-DoF WUJI hand to perform complex tasks while in motion, such as non-stop bottle grasping and fridge door opening. Ablation studies demonstrate that joint-space PPO and monolithic latent prediction fail under similar reward budgets, whereas the proposed latent-prior interface ensures trainability for contact-rich manipulation.

arxiv arXiv cs.LG · 6h ago

Encoder-Decoder Manifold Alignment for Idempotent Generation

Recent learning paradigms aim to enforce idempotency in generative models by ensuring repeated application leaves samples unchanged on the target data manifold. However, many existing approaches fail to achieve exact fixed points, resulting in instability and drift during repeated applications. The authors identify a geometric mismatch between encoder and decoder manifolds as the primary cause of this failure. To resolve this, they propose a training framework that explicitly aligns the geometry of both components to learn consistent representations of the same underlying data manifold. This alignment encourages stable projections and significantly reduces idempotency error compared to prior methods. Empirical results demonstrate that the approach consistently regenerates identical outputs under repeated application for both image generation and editing tasks. Furthermore, enforcing this type of idempotency improves identity preservation and information stability in generative models.

arxiv arXiv cs.LG · 6h ago

Manifold Restore Mixing Enhances Protein Representation Learning

Data augmentation improves protein representation learning but often disrupts structural integrity or reduces diversity. The authors identify these structure defects and performance degradation issues in existing methods. They propose Manifold Restore Mixing (MRM) to restore lost structural information while introducing diverse variations. MRM mixes hidden representations of original and augmented data, inspired by manifold mixup techniques. A sample difficulty scheduler adjusts the beta distribution to provide progressively challenging samples during training. Experiments on various backbones and downstream tasks demonstrate the method's effectiveness and generalization. The implementation is available at https://github.com/KingGugu/MRM.

arxiv arXiv cs.LG · 6h ago

Entropy-Guided Boundary Supervision for Breast Ultrasound Segmentation

This study introduces an entropy-guided boundary supervision method to address boundary leakage and false-positive activations in breast ultrasound segmentation. The proposed loss function scales contour penalties by per-pixel predictive entropy and ground-truth maps, focusing gradient emphasis on uncertain lesion margins. Evaluated on the BUSI dataset, the method preserved lesion segmentation quality with a mean Dice score of 0.7624, statistically indistinguishable from the baseline. However, it significantly improved specificity by reducing false-positive activations on no-lesion images from 19 of 20 to 5 of 20. A post-hoc spatial temperature scaling step further reduced the expected calibration error from 0.0201 to 0.0095 without altering segmentation masks. These results demonstrate that entropy-guided supervision and spatial calibration function as complementary refinements within a U-Net framework.