Guided-Retry prompting reduces LLM hallucination in task-oriented dialogue database failures

Researchers propose a lightweight prompting-based recovery approach to improve robustness when backend database calls fail or return mismatched information in task-oriented dialogue agents, without requiring retraining.

• Evaluated across six open-weight model families (DeepSeek-R1, Gemma-2, Llama-3, Mistral, Phi-3, and Qwen-2.5) and four database conditions including empty results and API errors.
• Tested on fault-injected benchmarks built on MultiWOZ 2.2 and SGD datasets.
• Naive agents hallucinated on 30.5% of failure turns on MultiWOZ and 20.9% on SGD.
• The Guided-Retry strategy reduced hallucination by 50% on MultiWOZ (to 15.3%) and by 42% on SGD (to 12.2%).
• Residual hallucination remained substantial (6-37% across models), with wrong-domain failures being the hardest case.

The study demonstrates that conditioning responses on structured database status significantly improves safety metrics, though residual errors persist particularly in complex retrieval scenarios.