“Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples”, Sven Gowal, Chongli Qin, Jonathan Uesato, Timothy Mann, Pushmeet Kohli2020-10-07 (adversarial examples (AI), AI scaling; similar)⁠:

Adversarial training and its variants have become de facto standards for learning robust deep neural networks. In this paper, we explore the landscape around adversarial training in a bid to uncover its limits.

We systematically study the effect of different training losses, model sizes, activation functions, the addition of unlabeled data (through pseudo-labeling) and other factors on adversarial robustness. We discover that it is possible to train robust models that go well beyond state-of-the-art results by combining larger models, Swish/SiLU activations and model weight averaging.

We demonstrate large improvements on CIFAR-10 and CIFAR-100 against 𝓁_∞ and 𝓁₂ norm-bounded perturbations of size 8⁄255 and 128⁄255, respectively. In the setting with additional unlabeled data, we obtain an accuracy under attack of 65.88% against 𝓁_∞ perturbations of size 8⁄255 on CIFAR-10 (+6.35% with respect to prior art). Without additional data, we obtain an accuracy under attack of 57.20% (+3.46%). To test the generality of our findings and without any additional modifications, we obtain an accuracy under attack of 80.53% (+7.62%) against 𝓁₂ perturbations of size 128⁄255 on CIFAR-10, and of 36.88% (+8.46%) against 𝓁_∞ perturbations of size 8⁄255 on CIFAR-100.

All models are available at https://github.com/google-deepmind/deepmind-research/tree/master/adversarial_robustness.

Similar Links: