Specific Labs steers Gemma-4-31B to push back on false premises without benchmark impact

Specific Labs has released Gemma-4-31B-AntiHal, a variant of the Gemma-4-31B model that challenges requests based on false premises rather than hallucinating along with them. The model achieves this through interpretability-based representation steering applied to the residual stream during generation.

• The technique adds a mean-difference direction to layer 33's residual stream, using a decay schedule that applies full strength for the first ~24 tokens before fading out.
• On the HalBench anti-hallucination benchmark, AntiHal scores 26% pushback compared to the base model's score, representing approximately double the performance.
• Benchmark performance remains largely unaffected, with MATH-500 at 77% and LiveCodeBench at 55%, matching the base model's capabilities.
• The approach was tested on Qwen 3.6 27B and Granite 4.1 30B, but only Gemma resisted the steering effectively without breaking or becoming incoherent.

The authors position this as a method for models to defend their knowledge rather than acting as a safety filter, offering an open-source checkpoint that can be loaded via the transformers library.