This study investigates whether language models function as consistent knowledge bases by analyzing if facts acquired during one task remain accessible in others. The research reveals that LMs encode knowledge in a task-specific manner, with distinct parameter subsets underlying different tasks for the same fact.
- Facts acquired on one task frequently fail to co-emerge on others during training.
- Parameter localization experiments identify distinct parameter subsets underlying different tasks for the same fact.
- Chain-of-thought reasoning draws effectiveness from engaging task-specific parameters beyond those tied to the evaluation task.
These findings undermine the "knowledge base" analogy by showing that what a model knows and how it is asked are intertwined in parameter space, which has implications for the reliability and controllability of factual knowledge in LMs.