This paper analyzes the probability that a model's trajectory remains within designated failure regions when trained using noisy gradient descent, idealized as overdamped Langevin dynamics.
- The equilibrium mass of a failure region is exponentially small in dimension d, with a complementary energy-barrier rate for small noise.
- A shape-free bound shows in-set probability relaxes to the static value after a burn-in time of order d, relying on the global spectral gap.
- An Ornstein-Uhlenbeck example demonstrates that transient swelling can occur, necessitating a local relaxation rate attached to the failure region.
- For geometrically isolated regions, this local rate exceeds the global one, shrinking the burn-in and capping trajectory probability uniformly in time.
The study concludes that while strong convexity determines relaxation speed, the shape of the unsafe set dictates whether the trajectory bulges through it during training.