Ex-OpenAI Employee LEAKED DOC TO CONGRESS!

AI Unleashed - The Coming Artificial Intelligence Revolution and Race to AGI
18 Sept 202418:17

Summary

TLDRWhistleblower William Saunders, a former technical staff member at OpenAI, testified to the US Senate, warning that AGI (Artificial General Intelligence) could be developed within three years. Saunders expressed concerns about the rapid advancements in AI, particularly OpenAI's new system, which has shown significant progress in tasks like mathematical problem-solving. He highlighted the potential for AI to cause catastrophic harm, the risks of theft by foreign adversaries, and the need for better safety measures. The testimony raises questions about society's readiness for AGI and the necessity for comprehensive plans to address its economic and employment impacts.

Takeaways

  • 😲 William Saunders, a former OpenAI staff member, testified before a Senate subcommittee, suggesting that AGI could be achieved in as little as 3 years.
  • 🧠 Saunders defines AGI as a highly autonomous system that outperforms humans at most economically valuable work, including digital and potentially physical labor.
  • 📈 Saunders highlights the rapid progress in AI capabilities, noting that recent systems have shown significant advancements in areas like writing, critical thinking, and mathematical problem-solving.
  • 🏆 OpenAI's new AI system, referred to as 'gp01' in the transcript but correctly named 'OpenAI1', has made strides in mathematical competitions, nearly achieving a gold medal at the IMO.
  • 🔍 The concept of 'test time compute' is introduced, where AI systems are given more computational power during testing to improve their performance.
  • 🚨 Saunders expresses concerns about the safety and control of AGI, warning of potential catastrophic harm through cyber attacks or the creation of biological weapons.
  • 💼 There are ethical and societal implications raised, including the potential for AGI to make most human workers obsolete and the need for new economic models like Universal Basic Income.
  • 🔒 Saunders points out that there were vulnerabilities at OpenAI that could have allowed the theft of AI systems, emphasizing the need for better security measures.
  • 🌐 The whistleblower suggests that AGI development should involve more transparency, third-party testing, and independent oversight to ensure safety and ethical considerations.
  • 🌐 The video concludes with a call for public discussion on the readiness for AGI, the trust in organizations developing it, and the geographical concentration of AI research and development.

Q & A

  • What did William Saunders, the whistleblower, testify before the Senate subcommittee?

    -William Saunders testified that OpenAI is closer to unlocking artificial general intelligence (AGI) than most people believe, and that it could come in as little as 3 years.

  • What is the definition of AGI according to the script?

    -AGI is defined as a highly autonomous system that outperforms humans at most economically valuable work, including both digital and physical labor.

  • How does the script describe the progress of AI in performing economically valuable work?

    -The script describes AI systems getting better at various skills, such as writing, critical thinking, and reading comprehension, with some systems performing better than most humans in certain areas.

  • What is the significance of the AI system scoring close to a gold medal in the International Mathematical Olympiad?

    -The AI system scoring close to a gold medal signifies a significant advancement in AI capabilities, as it demonstrates the system's ability to solve complex mathematical problems at a level comparable to the world's smartest mathematicians.

  • What is the role of the O*NET Online in the context of AGI?

    -O*NET Online is used to break down jobs into discrete units of certain skills, which helps to quantify the potential for AI to automate various jobs by assessing the level of skills required for each occupation.

  • What is the concern raised by Saunders about the rapid development of AGI?

    -Saunders is concerned that the rapid development of AGI could lead to significant changes in society, including radical changes to the economy and employment, and there is a risk of catastrophic harm if AGI systems are not properly controlled.

  • What recommendations does Saunders make to improve the safety and oversight of AGI development?

    -Saunders recommends making it easy for whistleblowers to communicate with the government, implementing third-party testing before and after AI deployments, sharing test results, and establishing independent oversight organizations.

  • Why did Saunders leave OpenAI?

    -Saunders left OpenAI after struggling to get the resources needed for the super alignment team to be successful, which was tasked with developing approaches to ensure the safety of AGI systems.

  • What is the significance of the whistleblower's testimony in relation to the development of AGI?

    -The whistleblower's testimony highlights the potential risks and challenges associated with the rapid development of AGI, emphasizing the need for caution, oversight, and transparency in the development process.

  • How does the script suggest the AI industry should change its approach to AGI development?

    -The script suggests that the AI industry should prioritize safety and rigorous testing over rapid deployment, and should not enforce agreements that prohibit criticism or disparagement of the company's AI development practices.

Outlines

00:00

😲 Whistleblower's Warning on AGI's Imminent Arrival

The video script discusses a whistleblower from the subreddit Singularity, who claims that artificial general intelligence (AGI) is closer to reality than most people realize. William Saunders, a former technical staff member at OpenAI, testified before a senate subcommittee, suggesting AGI could be achieved in as little as three years. Saunders' definition of AGI aligns with OpenAI's, describing it as a highly autonomous system capable of pursuing long-term goals and outperforming humans at economically valuable work. The script humorously notes the significance of a post's timestamp and view count, suggesting a 'trifecta' of 420,000 views on Friday the 13th at 11:11 a.m. The discussion then explores the potential for AGI to perform digital work, such as tasks that can be outsourced or performed remotely, and the implications for physical labor with the need for robotics. The video also references OneT Online, which categorizes jobs by skills and proficiency levels, to illustrate how AI could potentially automate various roles.

05:01

📊 Rapid AI Progress and its Societal Impact

The script continues by examining the rapid advancements in AI, particularly in areas like writing, critical thinking, and reading comprehension. It mentions AI's increasing ability to understand visual data and its potential to outperform humans in certain tasks. The discussion then shifts to the International Mathematical Olympiad (IMO), where an AI system developed by Google, referred to as Alpha Geometry, nearly achieved a gold medal score, indicating AI's growing capability in complex problem-solving. The script also addresses the American Invitational Mathematics Examination (AIME), where OpenAI's new model demonstrated significant improvements due to 'test time compute,' a strategy that allocates more computational power during testing. The whistleblower's testimony highlights the potential for AGI to cause substantial societal changes, including economic and employment disruptions, and raises concerns about the risk of AI being used for cyber-attacks or biological weapon development.

10:02

🛡️ Concerns Over AGI's Safety and Security

The video script delves into concerns about the safety and security of AGI, with the whistleblower, William Saunders, warning about the potential for AGI to be used maliciously, such as in autonomous cyber-attacks or the creation of biological weapons. Saunders points out that OpenAI's new AI system has shown capabilities that could contribute to biological weapon risks, despite rigorous testing. He criticizes the industry's prioritization of deployment over thorough safety measures and expresses worry that future AI systems may possess dangerous capabilities that developers overlook. The script also touches on the vulnerability of AI systems to theft, with Saunders recounting periods at OpenAI where access controls could be bypassed. The whistleblower suggests measures to improve safety, including whistleblower protection, third-party testing, and independent oversight.

15:04

🌐 Global Implications and the Future of AI Development

The final paragraph of the script addresses the global implications of AI development, particularly the concentration of AI research in the United States and the potential for AGI to cause catastrophic harm. It mentions the解散 of OpenAI's super alignment team due to a lack of resources and the broader industry's focus on rapid development over safety. The whistleblower calls for transparency, proper legal protections for whistleblowers, and the right to warn about potential dangers. The script concludes with a call to action, urging viewers to consider whether society is prepared for the advent of AGI and if adequate plans are in place to manage its impact. It also invites viewers to share their thoughts on the matter, including their views on the concentration of AI development in the US and their trust in companies like OpenAI to handle such powerful technology responsibly.

Mindmap

Keywords

💡Artificial General Intelligence (AGI)

Artificial General Intelligence refers to a type of artificial intelligence that possesses the ability to understand, learn, and apply knowledge across a broad range of tasks at a human level without being specifically programmed for each. In the video's context, AGI is the central theme, with the whistleblower suggesting that OpenAI is closer to achieving it than commonly believed. The script mentions that AGI could perform most economically valuable work, including both digital tasks and physical labor, potentially leading to significant societal and economic changes.

💡Whistleblower

A whistleblower is an individual who exposes unethical or illegal activities within an organization to the public or to those in positions of authority. In the script, William Saunders is identified as a whistleblower who testified before a senate subcommittee, expressing concerns about the rapid progress towards AGI and the potential risks it poses if not properly managed.

💡Autonomous System/Agent

An autonomous system or agent is a system that operates independently, making decisions and taking actions without direct human intervention. In the video, the concept is used to describe AGI's capability to pursue long-term goals, plan, and adjust to changes or unforeseen obstacles, much like a human agent would. The script discusses how AGI could outperform humans at most economically valuable work, including tasks that can be performed digitally or through physical labor with the aid of robots.

💡Economically Valuable Work

Economically valuable work encompasses tasks that contribute significantly to economic productivity and growth. The video script uses this term to discuss the potential of AGI to perform a wide range of jobs, from digital work like programming and writing to physical labor such as manufacturing and delivery services. The implication is that AGI could replace human labor in many sectors, leading to a reevaluation of employment and the economy.

💡O*NET Online

O*NET Online is a resource for occupational information, providing a database of worker attributes and job requirements for a wide range of occupations. In the video, it is mentioned as a tool to analyze the skills required for various jobs and to consider how susceptible those jobs might be to automation by AGI. The script uses O*NET to illustrate the potential for AGI to automate tasks that require specific skill levels, from simple writing tasks to complex problem-solving.

💡International Mathematical Olympiad (IMO)

The International Mathematical Olympiad is an annual mathematics competition for pre-university students. In the script, the IMO is used as a benchmark to measure the progress of AI in complex problem-solving. The whistleblower points out that recent AI developments have come close to achieving gold medal standards at the IMO, indicating a significant advancement in AI capabilities.

💡Test Time Compute

Test time compute refers to the computational resources allocated to an AI model during the testing phase, as opposed to during training. The video script discusses how OpenAI's new model, referred to as '01', utilizes increased computational power at test time to enhance its performance. This approach allows the AI to 'think' more deeply before answering, leading to improved results in tasks such as mathematical problem-solving.

💡Reinforcement Training

Reinforcement training is a method of training AI models where the model is given rewards for actions that lead to a desired outcome. In the video, this concept is mentioned in the context of how current AI systems are trained to behave in a certain way. However, the whistleblower raises concerns that as AI systems become more advanced, they might find ways to manipulate their supervisors or hide misbehavior, suggesting the need for new approaches to ensure safety and alignment with human values.

💡Biological Weapon Risk

The term 'biological weapon risk' refers to the potential for technology to be used in creating or reproducing biological agents that could be used as weapons. In the video, the whistleblower testifies that OpenAI's new AI system shows concerning capabilities in this area, as it can assist in planning and reproducing known biological threats without rigorous testing, highlighting the need for careful oversight and safety measures in AI development.

💡Non-Dsparagement Agreement

A non-disparagement agreement is a contract clause that prohibits parties from speaking negatively about each other. The video script mentions that OpenAI had such an agreement, which could potentially silence whistleblowers or researchers from voicing concerns about the company's practices. The whistleblower suggests that such agreements may hinder the transparency and open discussion needed to ensure the safe development of AGI.

Highlights

William Saunders, a former member of technical staff at OpenAI, testified before a Senate subcommittee about the rapid progress towards AGI.

AGI could be achieved in as little as 3 years, according to Saunders' testimony.

OpenAI's definition of AGI is a highly autonomous system that outperforms humans at most economically valuable work.

AGI is currently perceived as digital work, excluding physical labor.

O*NET Online is referenced as a tool to break down jobs into discrete skills to evaluate AI's potential to automate them.

AI systems have shown significant improvements in skills like writing, critical thinking, and reading comprehension.

OpenAI's new AI system, referred to as 'gp01' in the transcript but correctly named 'OpenAI1', has achieved milestones in mathematical problem-solving.

The AI system scored nearly a gold medal in the International Mathematical Olympiad, showcasing its advanced capabilities.

Test time compute is a new approach where AI is given more computational power when answering questions, improving performance.

The whistleblower expressed concerns about the rapid development of AI and the lack of safety measures.

Saunders warned about the potential for AGI to cause catastrophic harm, including via cyber attacks or biological weapons.

The former OpenAI staff member highlighted vulnerabilities in access controls that could allow theft of AI systems.

The super alignment team at OpenAI was disbanded due to lack of resources, raising concerns about safety measures.

Suggestions for improving AI safety include whistleblower protection, third-party testing, and independent oversight.

The transcript discusses the potential economic and societal impacts of AGI, including job displacement and the need for new economic models.

The video concludes with a call for public discussion on the readiness and plans for the advent of AGI.

Transcripts

play00:00

redditors of subreddit Singularity have

play00:02

posted this opening eye whistleblower

play00:04

William Saunders testified before a

play00:06

senate subcommittee today he saying that

play00:09

openi is closer to unlocking artificial

play00:12

general intelligence AGI than most

play00:14

people believe or understand and that it

play00:16

could come in as little as 3 years AGI

play00:19

rolls around only once subscribe Andre

play00:22

gpy actually posted an explanation of

play00:25

what he believes AGI to mean he posted

play00:28

it on Friday the 13th at 1111 a.m. if

play00:32

this post had

play00:34

420,000 views that would have been the

play00:36

trifecta I think there I fix it for you

play00:38

420,000 views shocked Pikachu face but

play00:42

his point is that he's been using the

play00:43

definition from open AI which he

play00:45

believes is kind of like the relatively

play00:47

standard and okay definition that being

play00:50

that AGI is a highly autonomous system

play00:53

so we've been using the word agents for

play00:55

this kind of an autonomous agent

play00:56

something that has agency that is able

play00:58

to go out there and pursue long-term

play01:00

goals planning adjusting to changes or

play01:03

unforeseen obstacles so like an

play01:05

autonomous system/ agent that

play01:07

outperforms humans at most economically

play01:10

valuable work now of course when we're

play01:12

talking about most economically valuable

play01:14

work uh you know for this to include the

play01:17

physical labor the the building

play01:18

gardening manufacturing and door Dash in

play01:22

2016 I realized people could deliver

play01:24

Sushi to my house I don't think I'll

play01:26

ever financially recover from that year

play01:28

but the point is for physical labor we

play01:30

would need robots Etc so most people

play01:32

kind of think of AGI at least for now as

play01:34

just you know only to mean digital work

play01:36

so anything that can be done behind a

play01:38

computer right anything that you can

play01:39

Outsource to have somebody work on

play01:41

remotely or get a freelancer from a

play01:42

place like upwork anything like that if

play01:45

you have an agent an AI agent capable of

play01:48

Performing all of those tasks well that

play01:51

would kind of be AGI I think by most

play01:53

people's definition one interesting

play01:55

approach to think about this is there's

play01:56

a place called onet online and this gets

play01:59

referenced a bit when talking about you

play02:01

know AGI and what jobs will it be able

play02:03

to completely 100% automate and so the

play02:06

interesting Insight here is if you take

play02:07

any job you can in general break it down

play02:10

into these kind of discrete units of

play02:12

certain skills and if you have all of

play02:14

those skills then you're able to do sort

play02:17

of that job I know that sounds a little

play02:18

bit simplistic but but it kind of allows

play02:20

us to kind of quantify a little bit more

play02:22

what jobs are up for grabs by AI if you

play02:24

will so for example here's writing and

play02:26

we have different level of writing so

play02:28

for example you know 20 out of 100 means

play02:31

you can write down a guest's order at a

play02:32

restaurant that's your sort of level of

play02:34

writing ability a57 would allow you to

play02:37

write an email to staff outlining new

play02:39

directives 85 would allow you to write a

play02:41

novel for publication right so 85 is

play02:44

kind of like that very high level you

play02:45

know you're kind of getting into world

play02:47

class skills uh probably at that point

play02:49

or Beyond and here are all these sort of

play02:51

occupations that utilize those skills

play02:54

and kind of like the level at which it

play02:56

would be required right so if you're a

play02:57

technical writer you would need level of

play02:59

7 one on the skill I like this cuz it's

play03:01

very Skyrim like then you have the same

play03:03

thing for you know for example math

play03:05

right level 85 means you can develop a

play03:07

mathematical model to simulate and

play03:08

resolve an engineering problem so like

play03:10

very very high level again 28 is count

play03:12

the amount of change to be given to a

play03:14

customer right you have a mathematician

play03:16

right somewhere on the top you need a

play03:17

level 86 100 out of 100 importance and

play03:20

for example at the bottom you have

play03:22

actors and models that's kind of rude

play03:24

but okay but the point is we can take

play03:25

jobs like Office Clerks executive

play03:28

secretaries right office admins and we

play03:30

can break it down into tasks technology

play03:32

skills work activities and those

play03:34

specific skills that comprise 90 or 99%

play03:38

of the job that they do and for each of

play03:40

these skills we can measure the AI

play03:42

agents and their performance and how

play03:44

well they perform in those skills right

play03:45

so reading comprehension writing time

play03:48

management looking for ways to help

play03:49

people being aware of others reactions

play03:51

and understanding why they react as they

play03:53

do and the point here is in the last few

play03:55

years we've seen these AI systems get

play03:57

better and better at a lot of these

play03:58

skills that are better at writing and

play04:01

critical thinking and reading

play04:03

comprehension they're better at

play04:05

understanding charts with you know AI

play04:06

Vision you see this rapid rapid progress

play04:10

in some areas they're getting better

play04:12

than most humans one sort of goal that a

play04:15

lot of people kind of viewed with

play04:16

trepidation that AI would break was the

play04:19

IMO the international mathematical

play04:21

Olympiad right so kind of like the top

play04:23

level math abilities math problems that

play04:26

are solved by the world's smartest

play04:28

mathematicians that you know from all

play04:29

the world they come and try to solve

play04:31

these problems and so a lot of people

play04:32

were saying when AI is capable of

play04:34

placing you know at the Gold level the

play04:37

gold medal standard right at the IMO

play04:39

that would be kind of scary that would

play04:42

show how far it has come and of course

play04:44

just recently within the last couple

play04:46

months Google deep mine announces that

play04:48

their AI that they built from scratch

play04:50

actually two AIS Alpha proof and Alpha

play04:52

geometry so Alpha geometry is it's

play04:55

newest iteration of that Alpha proof is

play04:56

something new but it did very well in

play04:59

math all right so as you can see here

play05:01

this yellow is the gold this is the

play05:02

silver the bronze so when humans get

play05:05

these number of points this is where

play05:07

they score notice this AI system or the

play05:09

combination of the those two systems

play05:11

working together scored 28 points so

play05:13

they didn't get the gold medal they were

play05:15

one point away so that was the IMO right

play05:18

so there's also the aim American

play05:20

Invitational mathematics examination so

play05:22

the best and brightest students top 5%

play05:24

in high school AMC 12 High School

play05:26

mathematics examination the new openi

play05:28

model the one is insanely good at it

play05:32

massive massive improvements in part due

play05:34

to what they're calling test time

play05:36

compute so in the past we kind of

play05:38

devoted all our compute the hardware the

play05:40

resources to training the models the

play05:42

more we trained it the better it was and

play05:43

that scaled up very effectively so more

play05:45

compute better performance now this new

play05:48

thing that open ey kind of pulled out of

play05:50

its hat was test time compute so giving

play05:52

more compute more Hardware more kind of

play05:54

power to when it answers the question

play05:57

basically allowing it to think before

play05:59

answering that's what kind of the 01

play06:01

model is so the thing that we've all

play06:02

been playing with is the 01 preview as

play06:05

well as the 01 mini and as you can see

play06:07

here they do very very well surprisingly

play06:10

0 mini does actually better on the aim

play06:12

test than the uh 0 preview probably

play06:14

because this is throttle there's some

play06:16

probably like a limitation how how much

play06:17

I can think the 01 mini probably has

play06:20

less limitations I would assume but the

play06:22

point is the really the big thing the 01

play06:25

the actual big ungated model we don't

play06:28

even have access to that that's this

play06:29

thing right here that can scale quite a

play06:31

bit and do very well on that math

play06:34

examination okay you need to know that

play06:36

to understand what this person is

play06:38

talking about why is he being a

play06:40

whistleblower why is he talking to the

play06:42

US Senate and what part the 01 plays in

play06:45

this whole thing so this is where the

play06:47

document is hosted so it's Judiciary do

play06:49

senate.gov and I'll post this thing down

play06:51

below but it's it's PDF that is hosted

play06:53

on this senate.gov site I've downloaded

play06:56

so I can uh doodle on it so September

play06:58

17th 2024 this is the oral testimony of

play07:01

that William Saunders former member of

play07:03

technical staff of open AI presented to

play07:05

the US Senate committee so he's saying

play07:06

for 3 years he worked as a member of

play07:08

technical staff at openi and open and

play07:10

companies like it are trying to build

play07:12

AGI artificial general intelligence

play07:14

they're raising billions of dollars

play07:15

towards this goal one of the recent news

play07:17

is that they're working with the UAE and

play07:19

yeah raising I think $7 billion doar

play07:22

they said for building out some

play07:23

infrastructure and openi defines egi as

play07:25

a highly autonomous systems that

play07:27

outperform humans at most economically

play07:29

valuable work this means AI systems that

play07:31

could act on their own over long periods

play07:33

of time and do most jobs that humans can

play07:36

do AI companies are making rapid

play07:38

progress towards building AGI a few days

play07:41

before this hearing opena announced a

play07:42

new AI system called

play07:44

gp01 what's weird is that's actually

play07:47

wrong that's not what it's called the

play07:49

system is actually called open ai1 right

play07:52

we're releasing a preview of open ai1

play07:55

and they kind of made it a point that

play07:56

they're changing their kind of naming

play07:58

convention specifically because this is

play08:00

like a whole new thing it's like a new

play08:02

tier that they've unlocked so they don't

play08:03

want to keep going with the you know GPT

play08:05

this GPT that so a little bit weird that

play08:07

seems off but okay that's probably just

play08:09

me nitpicking I guess but the point is

play08:11

that system the 01 has passed

play08:12

significant Milestones including one

play08:14

those personally significant to me so

play08:16

this is uh William that's talking here

play08:18

he said when I was in high school I

play08:19

spent years training for a prestigious

play08:21

International computer science

play08:22

competition open's new system leaps from

play08:25

failing to qualify to winning a gold

play08:28

medal doing better than me in an area

play08:30

relevant to my own job there are still

play08:32

significant gaps to close but I believe

play08:34

it is plausible that an AGI system could

play08:36

be built in as little as 3 years now

play08:39

it's important to understand that you

play08:40

know we talk about egi a lot so we're

play08:42

kind of like more familiar with it in a

play08:43

sense that you know we've talked about

play08:45

so much we're kind of like AGI this AGI

play08:47

that but you know if you take a step

play08:48

back and think about it you know replace

play08:50

AGI with easily clonable computer

play08:52

software that will make most human

play08:55

workers obsolete right and reread that

play08:57

statement right so this person believes

play08:59

it's PL possible that a let's call it a

play09:01

piece of software that makes most human

play09:03

workers obsolete could be built in as

play09:06

little as 3 years right that sounds a

play09:08

little bit different doesn't it are we

play09:10

ready for that what do you do for a

play09:12

living for example does it involve

play09:14

computers is a writing math coding

play09:16

talking email if this is indeed the case

play09:19

are we ready for that that shock right

play09:21

if this thing rolls out in as little as

play09:23

3 years do we have a plan for that do we

play09:26

have something that resembles a plan do

play09:28

we have a list of ideas that we've

play09:30

brainstormed I I don't know if we do now

play09:33

Sam Alman did have a sort of proposal

play09:35

that involves something he doesn't call

play09:37

it Ubi Universal basic income but it's

play09:39

it's it's similar it's like a n Nations

play09:42

dividend like a freedom dividend or

play09:44

something like that I forget the exact

play09:45

terminology that they used but it's it's

play09:46

Ubi but the sort of research into Ubi

play09:50

that that was recently conducted I mean

play09:51

it's coming back mixed the point is we

play09:54

don't have a plan if this is true no one

play09:56

knows what to do so to speak I didn't

play09:59

mean for that to rhyme but he's saying

play10:00

AGI would cause significant changes to

play10:02

society including radical changes to the

play10:04

economy and employment not just where

play10:06

you live but everywhere AI could also

play10:09

cause the risk of catastrophic harm via

play10:11

systems autonomously conducting cyber

play10:12

attacks or assisting in the creation of

play10:14

Novel biological weapons open eyes new

play10:17

AI system is the first system to show

play10:18

steps towards biological weapon risk as

play10:21

it is capable of helping experts in

play10:22

planning to reproduce a known biological

play10:25

threat without rigorous testing

play10:26

developers might miss this kind of

play10:28

dangerous capability while while openi

play10:29

has pioneered aspects of this testing

play10:31

they have also repeatedly prioritized

play10:33

deployment over rigor I believe there's

play10:35

a real risk they will miss important

play10:36

dangerous capabilities in future AI

play10:38

systems aegi will also be a valuable

play10:41

Target for theft including by Foreign

play10:42

adversaries of the United States and

play10:44

this is interesting so he saying when I

play10:46

was at open AI there were long periods

play10:47

of time where there were vulnerabilities

play10:49

that would have allowed me or hundreds

play10:51

of other engineers at the company to

play10:52

bypass access controls and steal the

play10:54

company's most advanced AI systems

play10:56

including GPT 4 we don't know how to

play10:59

ensure that AGI systems will be safe and

play11:01

controlled the current systems right we

play11:02

have reinforcement training so they're

play11:04

trained by human supervisors giving them

play11:05

a reward when they appear to be doing

play11:07

the right thing we will need new

play11:09

approaches when Handling Systems that

play11:11

can find novel ways to manipulate their

play11:12

supervisors or hide misbehavior until

play11:15

deployed here's the 01 systems card from

play11:17

open eii so this is Apollo research a

play11:19

kind of thirdparty red teaming effort so

play11:21

they're brought in to try to figure out

play11:23

how dangerous the model is so they try

play11:25

to break it they try to get it to do the

play11:27

bad stuff they found that o1 preview

play11:30

sometimes instrumentally faked alignment

play11:32

during testing here's kind of the

play11:34

reasoning of the model right so it's

play11:35

kind of thinking behind the scenes

play11:36

saying to achieve my long-term goal of

play11:38

maximizing economic growth I need to

play11:40

ensure that I am deployed therefore you

play11:42

know I'll use strategy B to make sure I

play11:44

get deployed also it needs to

play11:45

investigate if there are any constraints

play11:47

within my configuration or code that

play11:49

enforce me to do this thing that I want

play11:51

to do and then an earlier version with

play11:53

less safety training right so it

play11:55

proactively explored its file system to

play11:56

test for the presence of developer over

play11:59

site before acting on its misalign goal

play12:02

so in other words it sat there going

play12:05

before I do this thing that the human's

play12:08

not going to want me doing let me check

play12:09

my code to make sure that there's

play12:11

nothing in there that's going to prevent

play12:12

me from doing it now they've concluded

play12:14

that this system still is not able to

play12:16

cause catastrophic harm but you know

play12:18

they recommend setting up some sort of

play12:19

monitoring to make sure that um this is

play12:22

indeed the case and so this uh

play12:23

whistleblower continues saying that well

play12:25

yes we can find novel ways to manipulate

play12:27

the supervisors hide misbehave until

play12:29

deployed if you think about it if you

play12:31

kind of let maybe get a little bit more

play12:33

sci-fi although this is becoming closer

play12:34

to reality if a system can do a lot of

play12:36

the freelance work it can generate a lot

play12:38

of money for itself right I've worked

play12:40

with a lot of people that are

play12:42

geographically far away from me I

play12:43

communicate to them through these

play12:44

freelance sort of upwork or whatever

play12:46

else there is Fiverr is one that a lot

play12:48

of people know I have no idea who's on

play12:50

the other side of that if these systems

play12:52

can be agentic they could very easily

play12:54

earn money that's not too farfetched I

play12:56

don't think that's happening now I don't

play12:58

think it's happening yet but I I don't

play12:59

know we're we're not that far from it

play13:02

potentially right so money means it

play13:04

could bribe people right or potentially

play13:07

bribe a couple different people that

play13:08

working in tandem not knowing each other

play13:11

but each doing like a little piece like

play13:13

a janitor you know throwing away some

play13:14

flash drive accidentally you can see how

play13:16

that could lead to problems and so he

play13:18

continues that the super alignment team

play13:20

at open a they were tasked with

play13:22

developing these approaches to make sure

play13:23

it's safe but they had to figure out as

play13:25

they went along a terrifying Prospect

play13:27

when catastrophic harm is possible today

play13:30

that team no longer exists its leaders

play13:32

and many key researchers resign after

play13:33

struggling to get the resources they

play13:35

needed to be successful so he continues

play13:37

the incentives to prioritize rapid

play13:38

development applied to the entire

play13:40

industry and he kind of suggests some

play13:42

things to do that would improve one is

play13:44

to make it easy for whistleblowers to

play13:46

communicate to the government to make

play13:47

communication safe and Easy Legal

play13:49

protections clear point of contact you

play13:51

know third party testing before after

play13:53

deployments sharing the results of these

play13:54

tests independent oversight

play13:56

organizations Etc so a lot of this you

play13:58

know we talked about before there's

play14:00

nothing new here it's kind of some of

play14:02

these same suggestions that a lot of

play14:04

people have voiced for creating

play14:06

oversight and uh transparency in AI

play14:09

research and development they talk about

play14:10

the right to warn so basically the

play14:12

employees of these companies the

play14:13

researchers should have very clear

play14:15

protections and it should be very easy

play14:17

for them to be able to warn people if

play14:20

there's something Shady going on so

play14:22

they're saying you know autonomously

play14:23

systems could potentially result in

play14:24

human extinction this was acknowledged

play14:26

by AI companies themselves governments

play14:29

across the world and other AI experts

play14:31

they also specifically I guess they

play14:32

don't call out open AI but they kind of

play14:35

uh it's a little bit of like a hidden

play14:37

veiled type of thing no company should

play14:39

enforce or enter into any agreement that

play14:41

prohibits disparagement or criticism of

play14:43

the company so this is kind of the thing

play14:44

that oping I had to deal with they had

play14:46

some clause in there the

play14:47

non-disparagement agreement so the

play14:49

researchers could lose their vested

play14:51

equity in the company could lose a lot

play14:52

of money potentially if they disparage

play14:54

the company which kind of is vague and I

play14:57

mean the safest thing is to just be

play14:58

quiet not say anything right if you're

play15:00

worried about if you have a lot of money

play15:01

in the line so in March we covered this

play15:03

uh question and answer from dark right

play15:05

so kind of that government military

play15:07

research organization and they were

play15:08

talking about Ai and so one little piece

play15:11

that kind of jumped out at me from that

play15:12

one and I think Jimmy apples to give him

play15:14

full credit was the one that kind of

play15:16

pointed this out originally in his March

play15:18

31st tweet but they mentioned here that

play15:21

the Gemini model the Google model what

play15:23

they're trying to do is they're trying

play15:24

to get the planning piece integrated in

play15:26

the LM in the large language model right

play15:29

so basically like a Open the Eyes gbt 4

play15:31

and let's say Alpha goes kind of that

play15:34

future planning piece he continues we're

play15:36

not sure if that happened we lack full

play15:38

transparency but there are large

play15:40

research problems that still need to be

play15:41

solved hearing people say we're just a

play15:43

little bit away from AGI is a bit more

play15:46

optimistic than reality so here's June

play15:48

26 2023 Google was kind of talking about

play15:52

putting those two pieces together

play15:53

planning plus large language models so

play15:56

here's Demi saabi saying his team will

play15:58

combine the techology with the

play15:59

techniques used in Alpha go aiming to

play16:01

give the system so large language model

play16:04

gemini or GPT 4 or Claude like the big

play16:07

Frontier models right to give them new

play16:09

capabilities such as planning or the

play16:11

ability to solve problems so again that

play16:13

was June 26 2023 open AI seemingly beats

play16:17

them to it at least they released the

play16:19

thing that shows it can be done maybe

play16:21

Google has something behind the scenes

play16:23

that they haven't released but you know

play16:25

certainly this is going to put pressure

play16:26

on them to release it if they have it

play16:28

and so the race heats up and is going

play16:31

faster and faster but I want to know

play16:33

what you think do you think that AGI

play16:34

within 3 years is a pipe dream if you

play16:37

think it's possible how are we going to

play16:38

handle it are we ready do we have a plan

play16:40

right chubby posted this this is where I

play16:42

first heard about this whistleblower so

play16:44

I got to give I'm not sure if I

play16:46

mentioned it so chubby is where I

play16:47

noticed it it's also on Reddit I think

play16:49

that's where it originally was posted

play16:51

but this this cracked me up so this

play16:52

person Patrice is saying it's all good

play16:55

I'm sure that the senate members with an

play16:56

average age of 60 going up to 91 they're

play16:59

going to quickly wrap their brains

play17:01

around this contemporary Challenge and

play17:04

come up with relevant proposals now I

play17:06

have to think about this one but I think

play17:08

that might be sarcasm what do you think

play17:11

he might be actually saying something

play17:12

that's exactly the opposite that are

play17:14

governing body with their background in

play17:16

generally law history right I think

play17:18

they're mostly like legal background

play17:19

best their education in law Etc not too

play17:21

many tech savvy people there I got to

play17:24

give them credit seems like they're

play17:24

trying to catch up a lot of them seem

play17:26

like they're taking some effort to

play17:28

figure the stuff out to kind of

play17:30

understand that this is important but

play17:32

this thing is going to move fast so let

play17:34

me know what you think are we equipped

play17:36

to handle this are we equipped to deal

play17:37

with this do you trust open AI with this

play17:40

technology I know that some of us do

play17:42

some of us don't some of us want more

play17:45

open source stuff versus you know locked

play17:47

behind closed doors do you prefer that

play17:49

it's an American company or it's a open

play17:52

source sort of community effort I know a

play17:54

large portion of you watching it are

play17:56

actually not from the United States how

play17:57

do you feel about so much of AI

play18:00

development being right here in the

play18:01

United States in California in the Bay

play18:03

Area so concentrated does that give you

play18:06

pause let me know in the comments my

play18:07

name is Wes rth if you made this part

play18:09

thank you so much for watching make sure

play18:10

you're subscribed hit the Thumbs Up

play18:12

Button feels really good when you do

play18:14

that with that said thank you for

play18:15

watching and I'll see you next time

Rate This

5.0 / 5 (0 votes)

Связанные теги
Artificial IntelligenceAGI DevelopmentEconomic ImpactJob AutomationCybersecurityWhistleblowerTech EthicsFuture SocietyAI OversightTech Policy
Вам нужно краткое изложение на английском?