GEMS enables training-free superposition of multiple semantic directions in LLMs by addressing distributional deviation and directional interference through geometric constraints. On GSM8K, it maintains 98% accuracy with three non-mathematical directions, while unconstrained addition drops to 4%; on Wikitext-2, it increases PPL by only 2.2%.
arxiv
arXiv cs.CL
·
6d ago
·
research
GEMS: Geometric Constraints Enable Multi-Semantic Superposition in LLMs
from English
Importance 3/3
New feature vs. leaders
New harness with differentiators
arXiv cs.CL
OpenAI
Google DeepMind
Mistral AI
AI agents
Evaluation & benchmarks
Reasoning models
Benchmarks
| Benchmark | Model | Score |
|---|---|---|
| GSM8K | LLMs | 98% |