Researchers introduce AIriskEval-edu-db2, a new dataset designed to train and evaluate LLM-based auditors for pedagogical risk assessment in K-12 educational content. The dataset contains 1,639 explanations derived from 170 ScienceQA questions across science, language arts, and social sciences.
- Each question includes one human-written explanation alongside 11 generated by LLM-simulated teacher profiles with distinct pedagogical risks.
- A comprehensive risk rubric covers five dimensions: factual precision, depth and completeness, focus and relevance, student-level appropriateness, and ideological bias.
- The dataset adds 785 explanations with structured explainability annotations, including risk localization and description, validated by expert teachers.
- Validation experiments compare proprietary models against a fine-tuned local Llama 3.1 8B model for risk detection and explainability assessment.
The study evaluates whether supervised fine-tuning on this dataset allows a locally deployable model to approach or outperform stronger frontier models while preserving privacy in educational auditing.