A new study evaluates whether large language models hold consistent positions on substantive societal issues using the novel extsc{HardChoices} dataset, which captures disagreements within ideological camps.
- The research moves beyond high-level left-right or progressive-conservative dimensions to examine specific policy conflicts.
- Both large and small LLMs rarely declare neutrality when faced with these questions.
- Models often display incoherence but show a remarkable degree of agreement on issues where they do take stances.
The findings suggest that while LLMs can be steered in post-training, they lack robust, independent stances on complex societal matters.