Researchers address the challenge of automated essay scoring when educators revise or introduce new rubrics, a scenario known as cross-rubric generalization. They propose a fine-tuning framework for Large Language Models that utilizes rubric-agnostic intermediate representations called "traits" alongside target-essay supervision during training.
- The approach improves macro F1 by 5.0% over a baseline without traits in the hardest setting where both target rubrics and essays are unseen during training.
- Increasing target-essay supervision further enhances performance, allowing the best fine-tuned open-source Llama-based model to outperform GPT-5-mini prompting by 2.1% macro F1.
- The same Llama-based model trails GPT-5 by only 1.9% in this evaluation.
These results demonstrate that incorporating trait-based intermediate structure and controlled supervision effectively improves generalization to previously unseen scoring rubrics.