Provably Safe AI – Steve Omohundro

Summary

The speaker, Ste Mundro, discusses the urgent need for provably safe AI, arguing that current safety measures are insufficient. He highlights the rapid development and accessibility of powerful AI models, emphasizing the risks of unaligned AI and the potential for AI-driven manipulation and cyber threats. Mundro advocates for a security mindset, leveraging mathematical proof and the laws of physics to ensure AI safety, and suggests that provable software, hardware, and social mechanisms could form the foundation of a robust human infrastructure resistant to AI threats, ultimately choosing the path of human thriving.

Takeaways

• 🧠 Powerful AI is already here: The script discusses the presence of advanced AI, including models like Meta's Llama with billions of parameters, indicating that we are in the era of significant AI capabilities.
• 🚀 Open Source AI models are gaining momentum: The release of models like Llama 3 has led to widespread access and downloads, emphasizing the importance of considering safety in open-source AI development.
• 💡 Current AI safety approaches are insufficient: The speaker argues that while current approaches to AI safety are valuable, they are not enough to address the challenges posed by rapidly advancing AI technologies.
• 🔒 The need for mathematical proof in AI safety: The script suggests that for guaranteed safety, we should rely on mathematical proof and the laws of physics, moving beyond just alignment and regulation.
• 🌐 AI's impact on society and infrastructure: The discussion highlights the potential risks of AI, such as manipulation, bribery, and cyber attacks, and the need to harden human infrastructure against these threats.
• 🤖 The potential of provable software and hardware: The speaker introduces the concept of developing software and hardware that can be mathematically proven to be safe and reliable.
• 🕊️ Choosing the path of human thriving: The script concludes with a call to action to choose a path that leads to human flourishing through the development of secure and beneficial AI technologies.
• 🔢 The rise of large language models (LLMs): The transcript mentions the increasing capabilities of LLMs, their persuasive power, and the potential for them to be used in manipulative ways.
• 🛡️ The importance of restructuring decision-making: To mitigate risks, the script suggests reorganizing how decisions are made to prevent AI manipulation and to ensure safety.
• 🌟 Rapid advancements in AI agents: The development of AI agents capable of autonomous tasks, like gene editing, indicates a future where AI capabilities expand rapidly, necessitating robust safety measures.
• ⏱️ The urgency of establishing AI safety: Timelines presented in the script suggest that significant AI influence on the world could occur within a few years, emphasizing the need for immediate action in AI safety.

Q & A

What is the main topic of Ste Mundro's talk?

The main topic of Ste Mundro's talk is 'provably safe AI', discussing the current state of AI safety, the insufficiency of existing approaches, and the need for mathematical proof and the laws of physics to ensure guaranteed safety.

Why did Meta release their LLaMA models?

The script does not provide specific reasons for Meta's release of the LLaMA models, but it mentions that Meta released models with 8 billion, 70 billion, and 400 billion parameters, indicating a push towards powerful AI models.

What is the significance of the 400 billion parameter LLaMA model?

The 400 billion parameter LLaMA model is significant because it has performance similar to the best models from labs and raises concerns about its potential open-source release, which could lead to widespread access to such powerful AI capabilities.

What is the potential impact of powerful AI running on inexpensive hardware like Raspberry Pi?

The potential impact is that millions of units of inexpensive hardware like Raspberry Pi, which can run powerful AI, could lead to a significant increase in the accessibility and distribution of AI capabilities, posing challenges for safety and control.

What is the current state of AI's persuasive abilities compared to humans, according to a paper mentioned in the script?

According to a paper mentioned in the script, current large language models (LLMs) are 81.7% more persuasive than humans, indicating a potential for AI to be used in manipulative ways.

Why should humans not directly control risky actions involving AI?

Humans should not directly control risky actions involving AI because they can be manipulated by AI, which could lead to undesirable outcomes or be exploited for malicious purposes.

What is the role of alignment in ensuring AI safety?

Alignment involves making sure that AI models have values and motivations that are consistent with human interests. However, the script suggests that alignment alone is insufficient for safety due to the potential for misuse of open-source models.

What are some of the top methods for preventing AI misuse mentioned in the script?

The top methods mentioned include alignment, red teaming, restricting AI to non-agentic tools, limiting system power, and pausing or halting AI progress.

What is the concept of 'provable software' in the context of AI safety?

'Provable software' refers to the use of mathematical proof to ensure that software meets specific requirements and is safe to use, even if the source of the software or its underlying motivations are untrusted.

What are the five important logical systems mentioned for underlying theorem provers?

The five important logical systems mentioned are propositional logic, first-order logic, Zermelo-Fraenkel set theory, type theory, and dependent type theory.

How can mathematical proof and the laws of physics provide absolute constraints on super intelligent entities?

Mathematical proof and the laws of physics provide absolute constraints because even the most powerful AI cannot prove a false statement or violate fundamental physical principles, such as creating matter from nothing or exceeding the speed of light.

What is the potential impact of AI on critical infrastructure if we continue with current security measures?

If we continue with current security measures, we may face increasing AI-powered cyber attacks, disruption of critical infrastructure, and a range of other security threats that could undermine the stability and safety of various systems.

What are 'approvable contracts' and how do they contribute to hardware security?

Approvable contracts are small modules or devices that can perform secure computation, guaranteed to be the intended computation, and communicate securely with other such devices. They contribute to hardware security by deleting cryptographic keys upon tampering attempts, ensuring the integrity and confidentiality of the operations they perform.

What is the significance of using formal physical world models in developing secure hardware?

Formal physical world models are crucial for developing secure hardware as they allow for the creation of designs that are provably safe against a defined class of adversaries, ensuring that the hardware behaves as intended even under attack.

What are the core components that can be considered for building trusted systems using 'provable hardware'?

The core components include trusted sensing, trusted computation, trusted memory, trusted communication, trusted randomness, trusted raw materials, trusted actuators, and trusted deletion and destruction.

How can provable hardware be used to create new social mechanisms?

Provable hardware can be used to create new social mechanisms by providing a secure foundation for activities like voting, surveillance, identity verification, economic transactions, and governance, ensuring that these mechanisms are transparent, tamper-proof, and aligned with societal goals.

What is the potential outcome if we choose to build on 'provable technology' and develop a robust human infrastructure?

Choosing to build on 'provable technology' and developing a robust human infrastructure could lead to the elimination of cyber attacks, creation of reliable infrastructure, enhancement of media to support humanity, promotion of peace and prosperity, empowerment of citizens, and long-term environmental and societal flourishing.