Observation-Aligned supervision improves chart-to-code generation

The authors introduce Observation-Aligned supervision, a framework for chart-to-code generation that replaces latent raw-data targets with quantities constrained by visual observations to prevent hallucination.

• The method rewrites training data to use box statistics for boxplots, wedge percentages for pie charts, and bin weights for histograms.
• Experiments on ChartMimic and ChartX show consistent improvements in observable value recovery across multiple VLMs.
• Results include gains under both-executable evaluation metrics.

The authors argue that improving chart-to-code models requires supervision targets that respect what is identifiable from the chart image rather than assuming gold code is fully recoverable.