At the 2023 Defcon hacker conference in Las Vegas, there was a significant collaboration between AI tech companies and algorithmic integrity groups to identify vulnerabilities in generative AI platforms. This initiative, known as a “red-teaming” exercise, aimed to increase transparency and scrutiny of these influential yet opaque systems. The US government also supported this effort, signaling a growing concern for the security and ethics of AI technologies.
Following the success of the red-teaming exercise at Defcon, the nonprofit organization Humane Intelligence is taking a step further in evaluating AI systems. They have partnered with the US National Institute of Standards and Technology (NIST) to launch a nationwide red-teaming effort focused on evaluating AI office productivity software. This initiative, part of NIST’s AI challenges called Assessing Risks and Impacts of AI (ARIA), invites participation from any US resident, including developers and members of the general public. The qualifying round will be conducted online, with successful participants advancing to an in-person red-teaming event at the Conference on Applied Machine Learning in Information Security (CAMLIS) in Virginia.
The primary objective of Humane Intelligence’s red-teaming efforts is to democratize the evaluation of AI technologies. According to Theo Skeadas, the chief of staff at Humane Intelligence, the average individual lacks the knowledge and expertise to determine the fitness of an AI model for a specific purpose. Therefore, by allowing participants to assess the performance of these models themselves, Humane Intelligence aims to empower users and promote accountability in the AI industry.
During the final red-teaming event at CAMLIS, participants will be divided into red and blue teams tasked with attacking and defending AI systems, respectively. The evaluation will be based on the AI 600-1 profile, a component of NIST’s AI risk management framework. By using this structured rubric, the teams will measure whether the red team can subvert the expected behavior of the AI systems, highlighting potential vulnerabilities and security flaws.
Rumman Chowdhury, founder of Humane Intelligence and a contractor at NIST’s Office of Emerging Technologies, emphasizes the importance of collaborating with government agencies, international organizations, and NGOs in red-teaming efforts. The goal is to establish a collaborative platform where individuals from diverse backgrounds, including policymakers, journalists, and civil society members, can contribute to the testing and evaluation of AI systems. This inclusive approach seeks to promote transparency and accountability in the development of AI technologies.
As the field of AI continues to evolve rapidly, the need for rigorous testing and evaluation mechanisms becomes increasingly critical. By engaging a wide range of stakeholders in red-teaming activities, Humane Intelligence and NIST are paving the way for a more transparent and secure AI landscape. Through initiatives such as bias bounty challenges, where individuals are rewarded for identifying biases and inequities in AI models, the community is encouraged to actively participate in the monitoring and assessment of AI systems.
The red-teaming efforts led by Humane Intelligence and NIST represent a significant step towards enhancing the security, resilience, and ethics of AI technologies. By involving a diverse group of participants and implementing structured evaluation frameworks, these initiatives are setting a new standard for transparency and accountability in the AI industry.
Leave a Reply