Training methods — korshunov.ai

Training methods Page 1 / 12

Adaptive Functional Gradient Descent with Convergence Guarantees

We propose a new functional gradient descent algorithm that adapts its representation during optimization. The method achieves convergence to a stationary point under smooth losses and to a global minimizer under smoothness and a Polyak-Lojasiewicz condition, despite using finite-dimensional approximations. It outperforms both fixed-approximation FGD and neural network baselines in regression, PDE solving, and computer vision tasks.

arxiv arXiv cs.LG · 9d ago

Unified Causal-Origin Taxonomy of Distributional Shifts in RL

This paper proposes a unified causal-origin taxonomy for distributional shifts in reinforcement learning, linking ID/OOD generalization to non-stationary settings. It decomposes the agent-environment interaction using a POMDP framework, identifying internal, agent-driven, and external, environment-driven shifts, with explicit, implicit, and hybrid types defined by the shifted-time boundary. The work introduces an evaluation framework to measure shift impact through performance degradation and recovery metrics, enabling systematic analysis of RL robustness.

arxiv arXiv cs.LG · 9d ago

Key Properties for Effective Code Interpreter Reasoning

A study identifies extrinsic (crucial tokens) and intrinsic (cognitive behaviors) properties that enhance code interpreter reasoning in large language models. Stronger reasoning models show higher prevalence of verification, backtracking, and backward chaining, with these properties improving performance during inference and training, reducing overthinking and boosting token efficiency.

arxiv arXiv cs.LG · 9d ago

A nonparametric two-sample test using PReLU-IPM

The study introduces PReLU-IPM, a new integral probability metric based on a neural network discriminator with a single node. The resulting PReLU-TST test is nonparametric, consistent, and asymptotically equivalent to standard IPM-based tests, showing higher power or competitive performance on simulated and real datasets.

arxiv arXiv cs.LG · 9d ago

SPaiK: Scalable Pairwise Kernel Learning with Stochastic Vec Trick

SPaiK introduces a scalable kernel learning method for pairwise settings using the stochastic generalized vec trick (sGVT). This innovation reduces computational and memory demands, enabling efficient training on large datasets and making pairwise kernel learning feasible for previously intractable data sizes.

arxiv arXiv cs.LG · 9d ago

PACT: Small Language Model Deliberation for Reactive Reinforcement Learning

PACT combines a reactive RL policy with a 2B-parameter Small Language Model to generate and validate action plans. The SLM plan is executed directly if verified in simulation, bypassing the RL policy without retraining. PACT outperforms baselines on three increasingly difficult FrozenLake environments.

arxiv arXiv cs.LG · 9d ago

Post-Hoc Falsification Operators Fail to Improve Accuracy in Small Code Models

A measurement study finds that 26 semantic post-hoc operators do not improve held-out accuracy over Best-of-N in frozen small code models. While some operators reduce compute usage or recover correct programs, none outperform BoN in accuracy, due to systemic limitations like coverage walls and consensus traps. An expression-layer recovery (M1) improves performance on HumanEval+ by 12 tasks, with no harm or leakage, and shows consistent results across model cells.

arxiv arXiv cs.LG · 9d ago

Residual Connections Mitigate Gradient Issues in Deep Networks

A study uses multiplicative ergodic theory to analyze exploding and vanishing gradients in deep neural networks. It shows that residual connections affect the Liapunov spectrum, as characterized by Furstenberg and Kifer, thereby stabilizing gradient flow during training.

arxiv arXiv cs.LG · 9d ago

ExpRL: Exploratory RL for LLM Mid-Training

ExpRL introduces a novel mid-training approach for LLMs using human-written question-answer data as reward scaffolds. Instead of imitating reference solutions, it constructs problem-specific grading rubrics to reward intermediate reasoning steps, enabling better initialization for sparse-reward RL and outperforming SFT, sparse-reward GRPO, and self-distillation on math reasoning tasks.

arxiv arXiv cs.LG · 9d ago

HABC Improves RL Fine-Tuning of VLAs with Sparse Outcomes

Hierarchical Advantage-Weighted Behavior Cloning (HABC) enhances online RL fine-tuning of vision-language agents by using separate critic heads for viability and efficiency. It combines their outputs via a state-adaptive gate and applies per-transition weights, while intervention-aware credit assignment prevents supervision leakage. In real-robot experiments, HABC boosts success rates to 92%, 88%, and 38% on three bimanual tasks, surpassing SFT baselines of 36%, 44%, and 12%.

arxiv arXiv cs.LG · 9d ago

Geometric Action Model for Robot Policy Learning

The Geometric Action Model (GAM) enables robot policies to reason about 3D physical interactions by repurposing a pretrained geometric foundation model. GAM splits the GFM to serve as both an observation encoder and a causal future predictor, then routes predicted future geometry and actions through the same backbone, achieving accurate, robust, and efficient manipulation performance in simulation and real-robot benchmarks.

arxiv arXiv cs.LG · 9d ago

Exact Posterior Score Estimation for Linear Inverse Problems

The paper derives the exact posterior score in closed form for linear Gaussian inverse problems, enabling efficient posterior sampling via denoising. It introduces Exact Posterior Score (EPS), a training objective that preserves pretraining structure and achieves superior performance on fidelity, perceptual, and distributional metrics with fewer denoiser evaluations than gradient-based methods.