The authors introduce CLExEval, a human-in-the-loop framework designed to evaluate Large Language Model (LLM) clinical reasoning under progressive information masking. The study combines 5,600 expert-physician annotations with 200 clinical reasoning traces derived from 40 rare diagnostic cases to expose evaluation illusions where fluent explanations mask incorrect diagnoses.
- GPT-4o-mini's diagnostic accuracy drops from 95.0% to 32.5% under information scarcity, demonstrating a verbosity bias.
- A specialist model reaches 92.5% maximum diagnostic potential but fails to retrieve that knowledge reliably in verbose contexts.
- There is a 68.6% reasoning-to-output mismatch where correct diagnoses appear in traces but are not reflected in final answers.
- GPT-4o-mini approved 47.9% of clinically incorrect outputs on a human-verified failure set, while HuatuoGPT-o1 showed a positive self-preference bias.
The results suggest that standalone automated clinical evaluations can substantially overestimate clinical reliability without expert-grounded validation.