The article demonstrates that minimizing average error under forward marginals in score matching does not guarantee numerical stability during the discretized reverse-time sampling process. The authors construct a smooth score field with arbitrarily small forward-marginal $L^2$ error where the Euler--Maruyama discretizations converge in probability, yet every positive moment diverges.
- A family of bounded, globally Lipschitz denoisers is constructed where both forward-marginal error and path-space total variation distance tend to zero, while their endpoints diverge in every Wasserstein distance $W_p$ for $p \ge 1$.
- For compactly supported data, projecting the learned denoiser onto a known bounded closed convex set containing the support preserves pointwise accuracy and yields grid-uniform moment bounds.
- Experiments with a small fixed DiT-style network show that while overall trajectory errors remain small, large growth occurs along rare numerical trajectories, which is suppressed by denoiser projection.
This work highlights that weak convergence can hold even when Wasserstein distances diverge, and suggests that projecting the denoiser onto a convex set containing the data support is a method to ensure stability.