A study demonstrates that an adversarial model trainer can plant backdoors in a large class of deep, feedforward neural networks that are statistically undetectable even in the white-box setting. These backdoors allow for the generation of invariance-based adversarial examples by mapping distant inputs to unusually close outputs.
The backdoored and honestly trained models remain close in total variation distance, meaning they are indistinguishable even when the full model descriptions, such as all weights, are available. Without the backdoor, it is provably impossible under standard cryptographic assumptions to generate such adversarial examples in polynomial time.
These findings highlight a fundamental power asymmetry between model trainers and model users regarding the security and integrity of deep learning models.