Letter Lemmatization: One-to-one and Banded RNNs for Reversing Character-Set Simplification and Abbreviation in Medieval Text

This paper addresses the challenge of fluid character sets in medieval text corpora by introducing methods to reverse character-set simplification and abbreviation. The authors train character-level one-to-one RNNs with self-supervision to undo these mappings, recovering half the Character Error Rate (CER) using only 20 text lines.

• One-to-one RNNs provide significant improvements for HTR post-correction while ignoring insertions and deletions.
• The same networks are adapted into Banded RNNs using character-level alignment ground truth to expand abbreviations in medieval charter transcriptions.
• An elaborate heuristic defines a metric for semantic similarity between characters of arbitrary sets, termed letter lemmatization.
• A Python library is presented to efficiently perform all these methods.

The work provides tools and metrics to handle heterogeneous digitization policies and varying transcription practices in historical documents.