A new framework derandomizes PAC-Bayes bounds for smooth loss functions by analyzing the generalization gap of the Jensen gap class via Rademacher complexity. The resulting bounds for deterministic predictors involve flatness measures derived from Jacobians and Hessians of the score map, and are applied to linear models and smooth neural networks. A practical regularizer is proposed, computed using folded BatchNorm weights, and validated on CIFAR-10 with varying batch sizes.
Smoothness-Based Derandomization of PAC-Bayes Bounds
from English