SHIFT: Gate-Modulated Activation Steering for Knowledge Conflict Mitigation

The paper introduces SHIFT, a novel framework that mitigates knowledge conflicts in Retrieval-Augmented Generation (RAG) by reformulating neuron-level modification as learnable gate modulation. This approach allows large language models to adaptively regulate internal activations to resolve conflicts between retrieved context and parametric knowledge.

• SHIFT equips LLMs with a lightweight gate module while keeping the backbone model frozen, optimizing fewer than 0.01% trainable parameters.
• The gate module adjusts internal representations during generation to adaptively leverage both contextual and parametric knowledge.
• Extensive experiments on six datasets validate the effectiveness of SHIFT compared to various competing baselines.

This method addresses the risk of unintended cascading effects from traditional neuron editing, enabling LLMs to rely more effectively on contextual evidence without compromising general capabilities.