Study finds reward function design critical for RL-based LLM process model generation

A study investigates how reward function design impacts the quality of business process models generated by large language models using reinforcement learning. Researchers trained Llama 3.1 8B and Qwen 2.5 14B models under 48 configurations to optimize syntactic, pragmatic, and semantic quality.

• RL significantly improves pragmatic and syntactic quality while preserving semantic fidelity, reducing output variability by more than sixfold.
• Equal reward weighting consistently outperforms targeted weighting, as emphasizing a specific dimension can collapse the model into low-quality modes.
• Design choices interact non-trivially with model architecture; for instance, invalidity penalties and SFT initialization effects differ between Llama and Qwen.

The findings demonstrate that reward composition is a primary determinant of optimization outcomes, with effects comparable to the decision to apply RL itself.