A case study evaluates how annotator variation and aggregation methods affect multilabel emotion annotation. The paper shows that soft vote-share labels, including intensity-weighted variants, better capture annotator uncertainty and improve model alignment with empirical variance compared to hard labels.
Multilabel Emotion Annotation: Agreement and Soft Voting Analysis
from English