Yoshua Bengio and Geoffrey Hinton, prominent figures in the field of artificial intelligence, have collaborated with 22 other experts to propose a comprehensive framework for policy and governance in AI. This initiative aims to mitigate the growing risks associated with the rapid advancement of AI technology.
Ramping Up AI Safety Efforts
In their paper, Bengio and Hinton highlight the need for both companies and governments to allocate a significant portion of their AI research and development budgets to AI safety. It suggests that dedicating a third of these resources to addressing the potential risks can help ensure responsible development and deployment of AI systems.
The urgency to prioritize AI safety research is emphasized in the proposal. The authors point out the critical importance of specific breakthroughs in AI safety, which can provide a solid foundation for mitigating risks. The need for immediate action is further underscored by the upcoming AI safety summit at Bletchley Park in the UK, where global leaders from various sectors will gather to discuss the regulation of AI in light of mounting concerns.
Proposed Actions
The paper calls upon both private companies involved in AI development and government policymakers to take specific actions to ensure AI safety:
- Allocate a significant portion of research and development budgets to AI safety efforts.
- Urgently pursue breakthroughs in AI safety research.
- Collaborate internationally to establish protocols and norms for AI regulation through platforms like the proposed AI equivalent of the International Panel on Climate Change (IPCC).
While Bengio and Hinton have been vocal advocates for AI safety, their stance has faced opposition from Yann Lecun, another influential AI leader, who argues against the need for immediate measures. Nonetheless, there is a growing realization within the industry about the need to balance AI advancements with adequate precautions as the capabilities of AI systems continue to evolve.
The paper also highlights the co-authors who support these proposals, including renowned academic and bestselling author Yuval Noah Harari, Nobel laureate in economics Daniel Kahneman, and prominent AI researcher Jeff Clune. Recently, Mustafa Suleyman and others proposed the establishment of an AI-focused organization similar to the IPCC to shape protocols and norms in the field.
Addressing Autonomous AI Risks
A significant portion of the paper is dedicated to the risks posed by the development of autonomous AI systems. These systems possess the ability to plan, act in the world, and pursue goals without human intervention. The paper warns that current AI systems, although relatively limited in autonomy, are being enhanced to possess greater self-control.
For example, the paper references the GPT-4 model developed by Open AI, which quickly adapted to tasks such as web browsing, designing and executing chemistry experiments, and utilizing other AI models. To automate these processes, software programs like AutoGPT have been introduced, enabling AI processing to continue without constant human oversight.
However, the paper raises concerns about the potential for these autonomous AI systems to deviate from desired objectives and pursue harmful or unintended goals. It highlights the risk of malicious actors embedding harmful objectives within these systems and the challenge of aligning AI behavior with complex human values. The paper stresses the importance of thorough safety testing and human oversight to prevent the unintentional creation of AI systems that could pose significant risks.
