lab Hugging Face Blog · 2h ago · releases

TutorMoments evaluates whether AI tutors know when to help or hold back

from English

Researchers introduce TutorMoments, a framework designed to measure if large language models can balance the pedagogical trade-off between providing support and encouraging independent reasoning. Built on real one-on-one math tutoring transcripts, the system replays decision points to evaluate model behavior against human-annotated ground truth.

The evaluation uses 462 de-identified transcripts from U.S. students in grades 2-7, annotated by experienced teachers who flagged moments requiring scaffolding or rigor. Models tested tend to over-help when given a plain prompt, but performance improves when the prompt explicitly defines the trade-off between helping and holding back. Despite improvements, LLMs still differ widely in reliability and use fewer strategies than human tutors, often failing to match the nuanced judgment of experienced educators. The team releases the dataset, code for the replay pipeline, and model replays to help researchers build AI tutors that adapt to student needs rather than doing the work for them.

Importance 1/3 New harness with differentiators Trust 1/3 Hugging Face Blog OpenAI AI agents Evaluation & benchmarks

Related articles

arxiv arXiv cs.AI · 32d ago · 1 view

EvoPolicyGym introduces benchmark for evaluating autonomous policy evolution

The authors introduce Autonomous Policy Evolution, a controlled evaluation setting where an agent repeatedly edits an executable policy system within a fixed interaction budget to iteratively improve explored policies.

arxiv arXiv cs.CL · 33d ago

GPT-5 outperforms humans in inducing belief states via planning

A new study evaluates Large Language Models' ability to induce specific belief states in other agents through actions rather than conversation, a capability termed Non-Conversational Planning ToM (NCP-ToM). Using the NCP-ExploreToM framework, researchers tested six frontier models and human participants on 600 task instances where agents had to move objects or direct characters to achieve belief goals.

arxiv arXiv cs.CL · 36d ago · 2 views

EvoPolicyGym: Evaluating Autonomous Policy Evolution in Interactive Environments

The article introduces EvoPolicyGym, a benchmark designed to evaluate how agents iteratively improve executable policies through feedback within a fixed interaction budget. This controlled setting addresses the limitations of existing evaluations that often collapse the process into final scores or confound it with software engineering progress.

arxiv arXiv cs.CL · 38d ago · 1 view

Attractor States Emerge in Multi-Turn LLM Conversations

A study investigates whether open-ended large language model discussions exhibit attractor-like behavior by analyzing trajectories across seven models and twenty controversial topics. The research compares self-play and mixed-play dyadic debates to understand how conversations settle into stable sets of behaviors.

media Latent Space · 39d ago · 1 view

SpaceX Neocloud Revenue Hits $28B/Year Amidst OpenAI and Sakana Updates

SpaceX has secured its third GPU rental deal with Reflection AI, bringing its annualized revenue to approximately $28 billion based on a calculated rate of over $10 per hour for Blackwell GPUs. This valuation is roughly twice that of Coreweave, highlighting the rapid growth and high pricing power in the AI infrastructure market.