IdeaGene-Bench benchmarks scientific lineage reasoning and idea generation

Researchers introduce IdeaGene-Bench (IG-Bench), a benchmark designed to evaluate AI systems' ability to reason about scientific inheritance and generate ideas grounded in that lineage. The benchmark is built on the IdeaGene framework, which represents papers as sets of typed, evidence-grounded "Idea Genome" objects and uses GenomeDiff records to track evolutionary dynamics like mutation and inheritance.

• IG-Bench contains 1,961 golden lineage traces, 1,085 curated Idea Genome objects, and 920 pairwise GenomeDiff records across 10 scientific domains.
• The benchmark supports two evaluations: IG-Exam tests closed-form lineage reasoning across 42 task types with 1,029 instances, while IG-Arena evaluates generation using a lineage-conditioned Population-Evolution Score (PES).
• Experiments on 14 LLM-based scientists reveal a compositional bottleneck, with the strongest system achieving only 27.3% exact accuracy on lineage reasoning tasks.

The authors consider this important because current benchmarks fail to assess whether AI can follow the inheritance structure of scientific ideas, where new work recombines and repairs earlier mechanisms.