The Importance of AI Governance
The use of AI in consumer-facing businesses is growing rapidly, accompanied by concerns about how to govern the technology effectively in the long term. In response to this concern, the Biden administration has issued an executive order mandating new measurement protocols for the development and use of advanced AI systems. AI providers and regulators are now focusing on explainability as a crucial aspect of AI governance. This enables individuals affected by AI systems to better understand and challenge the outcomes of these systems, including potential bias.
While explaining AI may be practical for simpler algorithms, such as those used to approve car loans, more recent AI technology relies on complex algorithms that are challenging to explain but offer significant benefits. For instance, OpenAI’s GPT-4 is trained on massive amounts of data and can generate human-like conversations that are revolutionizing entire industries. Google Deepmind’s cancer screening models use deep learning methods to accurately detect diseases and save lives.
“Should we deprive the world of these technologies that are only partially explainable, when we can ensure they bring benefit while limiting harm?”
Even US lawmakers, who are seeking to regulate AI, are realizing the challenges surrounding explainability. This highlights the need for a different approach to AI governance, one that focuses on outcomes rather than solely on explainability.
Adopting a Framework for Assessing AI Risks
In the medical science community, the importance of identifying potential harm when developing new therapies has long been recognized. To assess the risk of harm and reduce uncertainty, randomized controlled trials are conducted. Similarly, in the world of AI, where systems continuously learn and evolve, traditional randomized controlled trials may not be suitable for assessing AI risks.
However, a similar framework like A/B testing, which has been extensively used in product development, can be applied to assess the outcomes of AI systems continuously. A/B testing involves treating different groups of users differently to measure the impacts of specific features or interventions. This framework allows for iterative changes to AI systems and helps understand the benefits and potential harms caused by those changes.
“In an increasingly complex environment, continuous measurement using a control group can determine which AI treatment caused the harm and hold that treatment accountable.”
Companies like Bing, Uber, Airbnb, and others have successfully implemented A/B testing to measure the impacts of changes in their products and user experience. This framework can not only evaluate business benefits but also identify harms like disparate impact and discrimination. For example, a large bank can use A/B testing to determine if a new pricing algorithm for personal lending products is fair in its treatment of women.
Furthermore, the exposure of AI to human subjects can be controlled through a gradual rollout of new product features or limited initially to a smaller, less risky population. Practices like red teaming, where a group of employees interact adversarially with the AI system to test its harms, can also be implemented before releasing the technology to the general population.
By evaluating AI systems based on their outputs on different populations, a quantitative and tested framework can be established to determine their potential harm. This framework holds AI providers accountable for the proper functioning and alignment of their systems with ethical principles.
While explainability remains a crucial focus in AI governance, adopting techniques from healthcare and tech industries, such as A/B testing, can help ensure that AI works as intended and, most importantly, is safe.
Caroline O’Brien – Chief Data Officer and Head of Product at Afiniti
Elazer R. Edelma – Edward J. Poitras Professor in Medical Engineering and Science at MIT, Professor of Medicine at Harvard Medical School, and Senior Attending Physician in the Coronary Care Unit at the Brigham and Women’s Hospital in Boston.
