AI keyword extraction for crowdsourced archives favors open-weight over generative models

A study using the University of Oxford's "Their Finest Hour Online Archive" evaluated three Natural Language Processing approaches—Named Entity Recognition, Keyword Extraction, and Topic Modelling—to automate keyword assignment at scale. The project tested these methods across techniques ranging from traditional statistical models to modern GenAI neural networks.

• Quantitative and qualitative findings indicate that while NLP offers real potential for large-scale extraction, no single method provides a complete solution.
• Model selection significantly shapes results, with open-weight, extractive models emerging as best suited for responsible deployment.
• Generative AI introduces accountability risks due to its abstractive nature, requiring careful consideration by collection managers.

The authors argue that automated keyword extraction in crowdsourced collections raises distinct stewardship responsibilities regarding metadata derived from living contributors, necessitating a balance between technical performance and ethical oversight.