A study evaluated agentic large language model systems for generating breast cancer treatment recommendations across 72 real clinical cases spanning stages I to IV. The research utilized 1,147 case-specific rubrics generated through Asymmetric Information Rubric Generation (AIRG) to assess reliability in complex oncology planning.
- Seven pipelines were compared, including single-LLM baselines, tool-augmented systems, and multi-agent architectures with fact checking and autonomous subagent spawning.
- The best-performing configuration was Claude Opus 4.8 with the D&C+SA pipeline, achieving a global score of 0.594 ± 0.025.
- Tool use and increased agent autonomy had mixed effects, improving performance in some settings but degrading it in others.
- Oncologist-led error analysis revealed persistent clinically relevant failures, including incorrect recommendations, flawed justifications, citation errors, outdated claims, and overconfidence.
The findings suggest that while agentic LLM systems can generate clinically relevant breast cancer recommendations, they remain insufficient for unsupervised clinical use.