This paper presents an exploratory study of interpretation difficulty in Discord chats related to cybercrime, where harmful communication often relies on slang and coded terms.
- The authors constructed reference interpretations of difficult messages reviewed by an expert.
- They evaluated human and large language model (LLM) interpretations under different context conditions.
- Local context alone was found insufficient for humans, while external knowledge and extended conversational context substantially improved interpretation.
- For LLMs, local context also improved interpretation, with larger models performing better.
- The study proposes a preliminary classification of factors that make harmful chats difficult to interpret.
These findings suggest that harmful-content analysis should treat interpretation as an evidence-integration problem rather than message-level classification alone.