Nicholas Carlini
More info
- Affiliation
- UC Berkeley, Google, Anthropic
- Known for
- The Carlini–Wagner attack, training data extraction
The security researcher who has made a speciality of breaking published defences for machine learning models, often within weeks.
As a PhD student under David Wagner at Berkeley, Carlini published the Carlini–Wagner attack in 2017, an optimisation method that finds minimal perturbations to make an image classifier see the wrong thing. It defeated defensive distillation, then considered promising, and became the standard yardstick for robustness. A year later he, Anish Athalye and Wagner showed that several defences accepted at ICLR were merely hiding their gradients rather than stopping the attack.
In 2021 he showed that language models such as GPT-2 can be coaxed into regurgitating verbatim chunks of their training data, names and phone numbers included. After years at Google and DeepMind he moved to Anthropic in 2025.