arxiv arXiv cs.CL · 47d ago · src: 51d ago · research

Test-Time Steering Resolves Temporal Fact Conflicts in LLMs

from English

Researchers identify parametric temporal conflicts in language models where outdated facts persist in parameters. They introduce Temporal Attractor Steering (TAS), a test-time method that resolves 29-57% of such conflicts without retraining, maintaining 85-99% accuracy on non-conflict queries and outperforming a baseline on three of four models.

Importance 2/3 New feature vs. leaders arXiv cs.CL Alibaba (Qwen) Meta AI Mistral AI Evaluation & benchmarks Open weights Reasoning models

Read original

arxiv arXiv cs.CL · 51d ago · 1 view

Causal Activation Directions for Mitigating Emergent Misalignment in Language Models

Fine-tuning language models on insecure code causes emergent misalignment. A shared activation direction across four model families achieves 99.6% separation of aligned and misaligned activations, and subtracting it reduces code spillover by 21-51 points. Cross-architecture transfer shows behavioral suppression but lacks specificity, with within-model directions being causally actionable and cross-model directions only causally real.

arxiv arXiv cs.CL · 53d ago · 2 views

LLMs Predict Dementia and Depression from Clinical Speech

A study uses open-weight large language models to assess dementia and depression severity from clinical interviews. LLMs achieve accurate zero-shot depression prediction (MAE 0.60) and improved dementia assessment with feature extraction (MAE 0.78), reducing errors by up to 35%. Pause-enriched transcripts match human transcriptions, supporting automated screening pipelines for neuropsychiatric disorders.

arxiv arXiv cs.CL · 53d ago · 1 view

RubricsTree: Scalable Evaluation Framework for Personal Health Agents

RubricsTree introduces a hierarchical taxonomy of over 100 clinically-verifiable Boolean rubrics, evolved from 4,000 real user queries via human-in-the-loop curation. It enables scalable, expert-aligned evaluation of personal health agents by dynamically routing queries to relevant rubrics and outperforms baseline methods in alignment, context sensitivity, and model performance gains of up to 66% on HealthBench.

arxiv arXiv cs.AI · 53d ago · 2 views

RubricsTree: Scalable Evaluation Framework for Personal Health Agents

RubricsTree introduces a hierarchical taxonomy of over 100 clinically-verifiable Boolean rubrics, evolved from 4,000 real user queries via human-in-the-loop curation. It enables scalable, expert-aligned evaluation of personal health agents by dynamically routing queries to relevant rubrics and outperforms baseline methods in alignment, context degradation detection, and model performance gains of up to 66% on HealthBench.

arxiv arXiv cs.CL · 36d ago

LLMs show distinct policies under skeptical pressure rather than sycophantic retreat

A study testing Llama-3.1-8B, Qwen2.5-7B, and Mistral-7B across climate, vaccine, and evolution domains finds that models do not sycophantically retreat from scientific consensus when users signal doubt. Instead, the models exhibit three distinct policies: reactive assertion where consensus increases (Llama), surface hedging with softened tone (Qwen), and non-response (Mistral).