Self-Guided TTT selects evidence spans to improve long-context LLM accuracy

Researchers propose Self-Guided Test-Time Training (S-TTT) to address the difficulty of utilizing long inputs in large language models, where standard test-time training on random spans often degrades performance due to noise.

The method works by having the model identify relevant evidence spans within the context before adaptation, applying the training objective only to those selected segments. This approach avoids the prohibitive cost of full-context training while mitigating the negative effects of irrelevant data.

On LongBench-v2 and LongBench-Pro benchmarks, S-TTT improves accuracy for Qwen3-4B-Thinking-2507 and Llama-3.1-8B-Instruct, achieving up to a 15% relative improvement.