media r/LocalLLaMA · 1h ago · open_models

Microsoft confirms OpenAI uses Looped Transformers in GPT-6

from English

A Microsoft web page confirmed that OpenAI has been using Looped Transformers in the GPT-6 series, validating prior reporting by The Information. The documentation indicated that GPT-6.1 Sol utilizes two inference passes, a reduction from three.

The article notes that GPT-6 and GPT-6.1 share the same pre-trained base model weights but differ in post-training, with GPT-6.1 employing one fewer loop.

Microsoft subsequently updated the web page to remove this information.

Importance 2/3 r/LocalLLaMA Microsoft Research OpenAI Research paper

Read original

arxiv arXiv cs.LG · 111d ago · 16 views

MGUP: Momentum-Gradient Alignment for Selective Optimization

MGUP introduces a selective update mechanism that applies larger step-sizes to a fixed proportion of parameters in stochastic optimization, while using smaller, non-zero step-sizes for the rest. It integrates seamlessly with optimizers like AdamW, Lion, and Muon, providing theoretical convergence guarantees for MGUP-AdamW and demonstrating superior or more stable performance in training large language models and MAE pretraining tasks.

lab Microsoft Research Blog · 15d ago · 97 views

Microsoft releases RetroChimera for retrosynthesis prediction

Microsoft Research has published and open-sourced RetroChimera, a new framework for retrosynthesis prediction that combines two complementary models to propose high-quality synthesis routes. The system integrates R-SMILES 2, a Transformer-based de-novo model, with NeuralLoc, a graph neural network (GNN) based model, using a learned ensembling strategy to rank predictions.

arxiv arXiv cs.LG · 20d ago · 48 views

OpenAI releases Open-1B with fully auditable training run

OpenAI has released Open-1B, a language model trained under a regime where every operation during training is independently reproducible on heterogeneous commodity hardware with bitwise certainty. This approach addresses the reproducibility issues inherent in open-source models by imposing a definite order on sources of nondeterminism, such as GPU kernel reductions and data batch ordering.

media Latent Space · 27d ago · 56 views

OpenAI claims AI-assisted Navier-Stokes singularity result using 10,000 agents

OpenAI-affiliated accounts reported that an AI-assisted effort produced a result related to the Navier-Stokes existence and regularity problem, one of the Millennium Prize Problems. The claim states that approximately 10,000 agents collaborated on the task after being trained for roughly a year using multi-agent reinforcement learning.

lab OpenAI News · 28d ago · 66 views

OpenAI internal system proves Navier–Stokes equations can develop finite-time singularities

OpenAI reports that an internal AI system has produced a proof showing that the dynamics of the Navier-Stokes equations for fluid motion can develop a singularity in finite time. The solution, derived by a multi-agent system powered by a model significantly more capable than GPT-6 Astra, establishes statement "C" of the Millennium Prize formulation.