Self-Guided Test-Time Training selects evidence spans to improve long-context reasoning

Researchers propose Self-Guided Test-Time Training (S-TTT), a method that improves long-context utilization by having the model identify relevant evidence spans before adaptation. This approach addresses the high cost and noise issues of applying test-time training to entire or randomly sampled contexts.

• S-TTT selects specific evidence spans for parameter adaptation rather than using the full context or random samples.
• The standard language-modeling objective is applied only to these selected spans to avoid degrading base model performance.
• On LongBench-v2 and LongBench-Pro, S-TTT improves accuracy for Qwen3-4B-Thinking-2507 and Llama-3.1-8B-Instruct by up to 15% relative improvement.