A new decision-focused RL framework jointly trains a forecaster and charging controller to handle unknown EV departure times. By aligning forecast accuracy with downstream decision quality, the method achieves up to 14% higher total reward and a 55% reduction in unsupplied energy compared to standard RL approaches.