The authors develop a nonlinear spectral deconfounding framework for gradient boosting to address sensitivity to hidden confounders in flexible machine-learning methods. This approach replaces the ordinary squared-error loss with a spectral loss, which slows down learning in directions aligned with latent confounders.
- The method relies on the interaction between spectral shrinkage and regularization, particularly early stopping, rather than the spectral loss alone.
- A mixed-model interpretation connects LAVA-type shrinkage to random-effects adjustment, enabling an empirical-Bayes procedure for tuning.
- The framework is extended to general likelihoods and nonlinear confounding using Laplace approximations and kernel random effects.
Experiments on synthetic and real-world data demonstrate that spectrally deconfounded boosting improves target function estimation under hidden confounding while being substantially more scalable than existing nonlinear baselines.