AutoSpecNER is a dataset of 659 car advertisements with over 10,000 entities annotated across 15 categories. It achieves 91.5% inter-annotator agreement and shows that DeBERTa outperforms both rule-based methods and large language models in vehicle specification extraction, reaching a 90% micro-F1 score.
arxiv
arXiv cs.CL
·
1d ago
·
research
AutoSpecNER: Fine-Grained NER Dataset for Vehicle Specifications
from English
Benchmarks
| Benchmark | Model | Score |
|---|---|---|
| SWE-bench Verified | DeBERTa | 90pts |
| SWE-bench Verified | strongest large language model | 77.8pts |
| SWE-bench Verified | rule-based | 43pts |