In the midst of the rapid and intense discourse evolving around AI and it’s implications for humanity beyond generating words and images, I just read this interesting article at Wired Magazine’s Danger Room about how since 2018, Microsoft has been developing a specialist team focused on identifying vulnerabilities in artificial intelligence systems. There have been cyber security red teams for decades looking for vulnerabilities in software, hardware and just about anything digital, so this isn’t really a surprise. I suspect there are dozens of AI-focused red teams operating at present. The team at Microsoft has been laser focused on assessing AI platforms and their weaknesses by attempting to attack them and has grown to include machine learning experts, cybersecurity researchers, and social engineers. Team founder Ram Shankar Siva Kumar explains that their mission extends beyond identifying technical security issues and also concentrates on the responsible use of AI, highlighting and assessing system failures and any potential to generate offensive or inappropriate content.
Early efforts were directed towards traditional security tools. Notably, the team collaborated with MITRE and other researchers to release the 2020 Adversarial Machine Learning Threat Matrix and in 2021 they published an AI security risk assessment framework. As AI and machine learning have become more integral parts of various systems, the team’s focus has broadened to address flaws and failures in these technologies.
In one operation, the team demonstrated a potential denial of service attack exploiting the machine learning components of a Microsoft cloud deployment service. The team’s findings have underscored the value of having a team dedicated to identifying AI vulnerabilities as attackers do not necessarily need to have vast resources to exploit AI platforms. The team’s proactive approach also involves anticipating future attack trends and working with other groups within Microsoft to fix any traditional vulnerabilities they find in systems or applications.
The best defense is a good offense.
John Schneider
I have always been an early adopter. Ideas, concepts, technologies and methodologies, learning and testing these is a personal passion. This puts me in the unique position of usually having perspective, POV and experience with this newness before everyone else. I use this to both my and my consulting clients advantage. Let me show you.