This paper introduces DysLexLens, a low-resource LLM framework designed to analyze the experiences of dyslexic learners with AI tools through online forum discussions. The system provides an end-to-end, evidence-traceable architecture that transforms noisy social media posts into focused corpora and generates verifiable query responses.
- Employs a dictionary-driven filtering method to construct a focused Reddit corpus on dyslexia and AI by removing noisy or weakly related posts.
- Integrates LLM-assisted semantic analysis with knowledge-graph-based query reasoning to uncover meaningful patterns in user data.
- Utilizes quantitative evaluation metrics, specifically RAGAS and Query Robustness, to measure the performance of LLM-generated responses.
- Provides structured qualitative validation guidelines for assessing response quality, focusing on hallucination and evidence alignment.
The framework's effectiveness was demonstrated using dyslexia-related Reddit forum data and 30 questions, showing potential generalizability to other low-resource forum contexts. Sample data, questions, and evaluation results are available on GitHub to support reproducibility.