A decision-focused RL framework jointly trains a forecaster and charging controller to handle unknown EV departure times. The method improves charging decisions by up to 14% in total reward and reduces unsupplied energy by 55% compared to standard RL without forecasting.