The authors introduce PSALM, an LLM-as-a-judge framework designed to operationalize EU copyright doctrine for evaluating stylistic appropriation in large language models. Unlike existing safeguards that focus on verbatim memorization, PSALM assesses computational overlap and stylistic dimensions such as writing style and narrative voice.
- The framework utilizes ten evaluators covering stylistic dimensions (writing style, narrative voice) and content dimensions (character, plot, scene, world building).
- It also accounts for statutory exceptions including parody, pastiche, quotation, and scènes à faire.
- Applied to Llama~3.2 models fine-tuned on translated historical Dutch literary works, the study found that fine-tuning induces systematic stylistic appropriation beyond verbatim memorization.
- Negative Preference Optimisation unlearning was shown to substantially reduce similarity but left detectable residual stylistic patterns.
PSALM provides infrastructure for auditable, legally informed compliance evaluation, bridging qualitative legal standards and quantitative technical measurement.