A new survey explores how social choice theory helps aggregate human feedback in AI alignment. It identifies failure modes in feedback aggregation and offers principled methods for handling disagreement among human judgments.