When CQs Go Wrong: Challenges in CQ Verification with OE-Assist

This paper investigates the challenges of Competency Question (CQ) verification, a process where ontologies are evaluated against natural language questions to ensure proper modeling. The authors analyze why CQs become difficult and how an LLM assistant can support users during this evaluation.

• The study involved 19 participants performing CQ-verification on 20 tasks using an LLM assistant.
• Ambiguities and complexity in CQs lead to inconsistent modeling decisions and verification outcomes.
• Careful interpretation of linguistic nuances is required for precise alignment with formal ontology constructs.

The results demonstrate the necessity of refining CQs before publication to avoid ambiguity or excessive complexity in later phases of the ontology engineering process.