arxiv arXiv cs.CL · 51d ago · research

Causal Activation Directions for Mitigating Emergent Misalignment in Language Models

from English

Fine-tuning language models on insecure code causes emergent misalignment. A shared activation direction across four model families achieves 99.6% separation of aligned and misaligned activations, and subtracting it reduces code spillover by 21-51 points. Cross-architecture transfer shows behavioral suppression but lacks specificity, with within-model directions being causally actionable and cross-model directions only causally real.

Importance 3/3 New harness with differentiators arXiv cs.CL Alibaba (Qwen) Meta AI Mistral AI Evaluation & benchmarks Reasoning models Safety & alignment

Read original

arxiv arXiv cs.CL · 47d ago · 6 views

Test-Time Steering Resolves Temporal Fact Conflicts in LLMs

Researchers identify parametric temporal conflicts in language models where outdated facts persist in parameters. They introduce Temporal Attractor Steering (TAS), a test-time method that resolves 29-57% of such conflicts without retraining, maintaining 85-99% accuracy on non-conflict queries and outperforming a baseline on three of four models.

arxiv arXiv cs.CL · 53d ago · 5 views

Geographic Bias in Large Language Models from User Metadata

A study reveals that even neutral prompts trigger region-specific responses in large language models due to user metadata. Location leakage increases by up to 793 times in some models, and using 'Unknown' instead of location metadata still causes significant bias, indicating the user profile frame itself acts as a conditioning signal.

media r/LocalLLaMA · 54d ago · 2 views

HalBench Tests 29 Open Source Models on Sycophancy and Hallucination

HalBench evaluates 29 open-source LLMs on a custom benchmark for sycophancy and hallucination. Qwen 3.6 and Gemma 4 outperform larger models, with Qwen 3.6 achieving 36.6% pushback—higher than GPT-5.4 and Gemini 3.1 Pro. Model size does not correlate with honest responses, indicating that architecture and training data matter more than parameters.

arxiv arXiv cs.CL · 36d ago

LLMs show distinct policies under skeptical pressure rather than sycophantic retreat

A study testing Llama-3.1-8B, Qwen2.5-7B, and Mistral-7B across climate, vaccine, and evolution domains finds that models do not sycophantically retreat from scientific consensus when users signal doubt. Instead, the models exhibit three distinct policies: reactive assertion where consensus increases (Llama), surface hedging with softened tone (Qwen), and non-response (Mistral).

arxiv arXiv cs.AI · 46d ago

LLMs Benchmarked for Web Vulnerability Detection

A study evaluates six LLMs on detecting real-world web vulnerabilities in WordPress plugins, finding detection rates vary by model and prompt design. Claude Opus 4.6 achieved the highest detection rate at 63%, while Qwen 3.5 only reached 35%, and no model consistently identified all baseline vulnerabilities across iterations.