Adversarial ML

Adversarial Machine Learning (AML) is the study of how machine learning models can be attacked, manipulated, or fooled by malicious inputs — and how to build defenses against such attacks. It covers attack paradigms including adversarial examples (imperceptible input perturbations that cause misclassification), data poisoning (corrupting training data to corrupt the model), model inversion (extracting private training data from a model), and evasion attacks (bypassing classifiers at inference time). The field also develops defenses such as adversarial training, certified robustness, and input preprocessing.

As AI systems are deployed in safety-critical domains — autonomous vehicles, medical diagnosis, financial fraud detection, and content moderation — their robustness against adversarial manipulation has become a regulatory and liability concern, reflected in the EU AI Act's high-risk system requirements. AI companies are hiring specialists who can red-team models, audit training pipelines for poisoning vulnerabilities, and design defenses that survive real-world adversarial pressure. The field is now central to AI safety work at frontier labs, where model robustness intersects with alignment and misuse prevention.