The paper introduces DiARC, a method that improves the abstract reasoning capabilities of large language models by incorporating negative sample supervision alongside positive examples. This approach addresses the limitations of current methods that rely heavily on data augmentation or expensive closed-source models.
- The authors propose constructing preference pairs to enable models to distinguish between correct and incorrect solutions.
- Three specific techniques are used to generate negative samples: output-level visual transformations, DSL-level rule inversion, and task-specific rule editing.
- These negative samples serve as informative near-miss alternatives while preserving the original observed demonstrations.
- Experimental results across multiple ARC-like benchmarks demonstrate that DiARC consistently outperforms baseline models.
By leveraging preference alignment through negative samples, this method offers a cost-effective way to enhance reasoning performance without relying on costly closed-source models.