A study reveals that even neutral prompts trigger region-specific responses in large language models due to user metadata. Location leakage increases by up to 793 times in some models, and using 'Unknown' instead of location metadata still causes significant bias, indicating the user profile frame itself acts as a conditioning signal.
arxiv
arXiv cs.CL
·
8d ago
·
research
Geographic Bias in Large Language Models from User Metadata
from English
Importance 3/3
arXiv cs.CL
Mistral AI
Alibaba (Qwen)
Anthropic
Evaluation & benchmarks
Reasoning models
Safety & alignment
Benchmarks
| Benchmark | Model | Score |
|---|---|---|
| SWE-bench Verified | Llama 3.1-8B | 31.7% |
| SWE-bench Verified | Qwen3-8B | 21.3% |
| SWE-bench Verified | Claude Sonnet 4.6 | 8.8% |