Researchers introduce model merging as a training-free strategy to enhance ad-hoc search effectiveness in conversational information retrieval, addressing the high costs and catastrophic forgetting associated with traditional fine-tuning. The study evaluates linear and non-linear parameter-wise merging strategies, specifically Model Soup and Slerp, on standard ad-hoc and conversational datasets.
- The approach enables a single retrieval model to operate across both ad-hoc and conversational settings without additional fine-tuning.
- Experiments demonstrate that model merging significantly enhances the ad-hoc search capabilities of conversational retrievers.
- The method improves generalizability across task-specific datasets, achieving up to 15% higher NDCG@3 under zero-shot conditions.
This technique offers a cost-effective alternative to retraining by leveraging existing models to maintain foundational performance while adapting to conversational contexts.