media Hugging Face Forums · 1h ago · open_models

ThetaMem proposes signed multiplicative key lifts for fixed-state sequence memory

from English

ThetaMem is a preliminary, single-seed study of a recurrent mixer that modifies the recurrent state via learned signed Hadamard or outer-product key lifts rather than refining gating mechanisms. The author presents mixed results on synthetic benchmarks and explicitly seeks criticism to identify the cheapest experiment that could falsify the approach.

On MQAR with 4× length extrapolation, ThetaMem (outer) achieved a score of 0.976 using 32× core-state floats, while GDN-2 with a wider key scored 0.814.
ThetaMem (Hadamard) scored 0.668 on the same benchmark compared to GDN-2's 0.567.
In MAD fuzzy recall tests, ThetaMem performed worse than GDN-2 (0.181 loss vs 0.323).

The author notes that the matched-core-state comparisons are not parameter-matched and calls for fairer controls involving state, parameter, and compute matching in future work.

Importance 1/3 Trust 1/3 Hugging Face Forums Research paper Training methods

Read original

lab NVIDIA Research · 1d ago · 32 views

LLMs match or beat Bayesian optimization for hyperparameter optimization

A new paper explores using foundational large language models (LLMs) to automate hyperparameter optimization (HPO), a process that typically relies on manual approaches in limited-budget settings. The authors developed a methodology where LLMs suggest hyperparameter configurations based on dataset and model descriptions, which are then iteratively refined according to model performance.

lab NVIDIA Research · 1d ago · 32 views

Source introduces approximate unrolled differentiation for training data attribution

Researchers introduce Source, a new training data attribution (TDA) method that combines the computational efficiency of influence functions with the accuracy of unrolling-based approaches. By using an influence-function-like formula to approximate unrolled differentiation, Source addresses limitations in implicit differentiation methods, such as their failure to account for optimization bias or multi-stage pipelines.

lab NVIDIA Research · 1d ago · 5 views

JacNet learns functions by directly modeling structured Jacobians

The paper introduces JacNet, a neural network architecture that directly learns the Jacobian of an input-output function rather than the mapping itself. This approach allows for precise control over derivative properties, enabling the enforcement of structural priors such as invertibility and k-Lipschitz continuity.

arxiv arXiv cs.CL · 2d ago · 1 view

MiLMMT-46-v1.0 improves multilingual translation via reference-free post-training

Researchers have released MiLMMT-46-v1.0, an open large language model for multilingual machine translation that utilizes reference-free post-training. Starting from the supervised-finetuned MiLMMT-46-v0.1, the team applied Group Relative Policy Optimization (GRPO) with a reward based on two reference-free quality estimation models gated by language identification.

lab NVIDIA Research · 2d ago · 6 views

Masters distills vision-language models via masking teacher and reinforcing student

Researchers propose Masters, a mask-progressive reinforcement learning framework for distilling large-scale vision-language models into compact versions suitable for edge deployment. The method addresses the instability caused by the size gap between teacher and student models.