NIST Releases Cybersecurity Guide for AI Developers

The United States National Institute of Standards and Technology (NIST) has issued a warning that artificial intelligence (AI) systems are susceptible to various types of cyberattacks, and robust protections are not yet foolproof. AI and machine learning (ML) developers are being urged to be cautious of claims promising absolute security against these vulnerabilities.

Risks Identified in AI Development

In a detailed document called Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations (NIST.AI.100-2), NIST outlines the significant threats to AI systems. The guideline focuses on predictive and generative AI, delving into the potential attacks that can occur and offering methods to counteract them.

The guideline mentions that AI systems can be 'poisoned' or misled by adversaries to cause malfunctions. These attacks can happen during the system's training phase or afterwards, as the AI iterates and improves its decision-making through real-world interactions. This poses a challenge because the vast amount of training data cannot always be closely monitored or filtered for integrity.

Common Types of AI Attacks and Mitigations

NIST's guide categorizes the attacks broadly into evasion, poisoning, privacy, and abuse attacks. Evasion attacks typically occur post-deployment, where an adversary makes slight input alterations to deceive the AI system. Poisoning attacks, on the other hand, involve corrupting training data to influence the AI's future behavior. Privacy attacks aim to extract sensitive details about the AI or the data it was trained on, while abuse attacks feed the AI incorrect information through compromised but legitimate sources, diverting its intended purpose.

Experts from NIST suggest that mitigation strategies should not only emphasize purifying data and refining AI models but should also incorporate cryptographic techniques to verify the AI systems' origin and data integrity. Additionally, they recommend 'red teaming', where an internal group attacks the system to unearth vulnerabilities before deployment.

The Compromise Between Security and Performance

Developers face trade-offs in ensuring their AI systems are secure and trustworthy. A high-performing AI system might still be prone to adversarial exploitation, reducing its reliability. Conversely, an AI overly tailored to resist attacks may suffer from lowered accuracy and potentially unfair outcomes. Organizations are thus encouraged to balance these factors based on their specific AI system usage and broader economic, social, and political contexts.

The NIST report is considered a valuable resource for understanding these complexities in AI security, providing an in-depth analysis that not only showcases the various types of threats but also acknowledges that creating foolproof mitigations is still a work in progress.

cybersecurity, AI, development