The authors propose Self-Aware Scheduling (SAS) to optimize the token unmasking order in masked diffusion language models, which significantly impacts generation quality. They derive a tractable upper bound on sequential decoding mismatch using Kullback-Leibler divergence and pathwise log-likelihood. This bound creates a dense self-aware reward that frames order selection as a policy optimization problem with a frozen denoiser. SAS learns a lightweight order policy via Group Relative Policy Optimization, supporting both any-order and semi-autoregressive decoding. On Sudoku tasks using a 1B parameter model, accuracy improved from 82.0% to 91.8%, reaching 97.5% after second-stage fine-tuning. For mathematical reasoning with LLaDA-8B, pass@1 on GSM8K increased from 64% to 76%. The method also raised MBPP scores from 39.5% to 41%, consistently matching or exceeding heuristic schedules across various parameters.
Self-Aware Scheduling Learns Token Unmasking Order in Diffusion Language Models
Fathom-Vaidya improves medical reasoning with rubric-based rewards
The authors introduce Fathom-Vaidya, a 30B parameter model that uses synthetic data and rubric-based reinforcement learning to enhance diagnostic and clinical healthcare reasoning. The training framework first applies rule-guided RL to MedBullets-derived questions for diagnosis, then utilizes 5.3k synthetic multi-turn scenarios with multi-dimensional rubrics for interactive clinical tasks.
Fathom-Vaidya improves medical reasoning with rubric-based rewards
The authors introduce Fathom-Vaidya, a 30B parameter model that uses synthetic data and rubric-based reinforcement learning to enhance both diagnostic and clinical healthcare reasoning. The training framework first targets diagnostic accuracy using MedBullets-derived questions and then addresses multi-turn clinical interactions through 5.3k generated scenarios with multi-dimensional rubrics.
Fathom-Vaidya improves medical reasoning with rubric-based rewards
The authors introduce Fathom-Vaidya, a 30B parameter model that uses synthetic data and rubric-based reinforcement learning to enhance both diagnostic and clinical healthcare reasoning.
MB-Bidram shares experimental WideNDepth architecture separating reasoning from knowledge
MB-Bidram has released an experimental neural architecture called WideNDepth (WND) designed to separate reasoning capabilities from knowledge storage. The model consists of a "Wide part" that functions as memory and a "Depth part" that acts as the reasoner.
Proposal: Progressive knowledge transfer from Qwen 9B to 27B for a fixed 4B model
A user on the Hugging Face forums proposes an experiment in progressive knowledge transfer, where a fixed-size student model (Qwen 4B) learns sequentially from increasingly larger teacher models rather than directly from the largest available one.