When AI Can Fake Reality, Who Can You Trust? | Sam Gregory | TED

Summary

The speaker from WITNESS, a human rights group, discusses the escalating challenge of distinguishing real from AI-generated content, highlighting the societal impacts of deepfakes. They share insights from a deepfakes rapid-response task force and emphasize the need for detection tools, content provenance, and a pipeline of responsibility to fortify truth and combat misinformation in an increasingly AI-infused media landscape.

Takeaways

• 🧩 The distinction between real and fake is becoming increasingly blurred with advances in generative AI and deepfakes.
• 🚀 The speaker began working on deepfakes in 2017, highlighting the evolution from a hype to a serious threat, particularly in the creation of falsified sexual images.
• 🌐 The impact of deepfakes is global, affecting women and girls and now expanding to include the potential to dismiss real events as faked.
• 🛡 WITNESS, the human-rights group led by the speaker, aids people in using technology to defend their rights and has coordinated a global effort to combat deepfakes.
• 🔍 A deepfakes rapid-response task force has been established, consisting of media-forensics experts and companies that debunk deepfakes and claims of deepfakes.
• 🗣️ The task force has dealt with cases from Sudan, West Africa, and India, demonstrating the complexity and challenges in verifying the authenticity of audio clips.
• 🕵️‍♂️ Even experts struggle to rapidly and conclusively determine the authenticity of deepfakes, and the ease of falsely accusing real content as fake is increasing.
• 🌟 The future presents profound challenges in protecting real content and detecting fakes, with deepfakes targeting politicians and influencing political ads and crisis reporting.
• 🔑 The need for detection skills and tools to be accessible to those who need them most, such as journalists and human-rights defenders, is emphasized.
• 💡 There is a call for better understanding of content provenance and disclosure through technologies like invisible watermarking and cryptographically signed metadata.
• 🌳 A responsible pipeline from AI foundations to deployment in systems and platforms is necessary to ensure transparency, accountability, and liability in AI usage.

Q & A

What is the main challenge discussed in the script regarding the advancement of generative AI?

The main challenge is the increasing difficulty in distinguishing between real and fake content, as well as the potential for AI to both create convincing fakes and to be used as an excuse to dismiss genuine reality.

When did the speaker start working on deepfakes, and what was the initial concern?

The speaker started working on deepfakes in 2017, with the initial concern being the overhyped threat to trust in information and the harm caused by falsified sexual images.

What is the role of WITNESS as described in the script?

WITNESS is a human-rights group that helps people use video and technology to protect and defend their rights, and has coordinated a global effort called 'Prepare, Don't Panic' to address the manipulation and synthesis of reality.

What is the purpose of the deepfakes rapid-response task force mentioned in the script?

The task force, composed of media-forensics experts and companies, aims to debunk deepfakes and claims of deepfakes, providing a rapid response to such incidents.

How did the task force handle the audio clip from Sudan?

Experts used a machine-learning algorithm trained on over a million examples of synthetic speech to prove with high certainty that the Sudan audio clip was authentic.

What challenges did the task force face with the West Africa audio clip?

The task force couldn't reach a definitive conclusion due to challenges in analyzing audio from Twitter with background noise, which affected the analysis.

What was the outcome of the analysis of the Indian politician's leaked audio clip?

Despite the politician's claims that the audio was AI-falsified, experts concluded that it was at least partially real, not AI-generated.

What are the three steps proposed to address the challenges posed by deepfakes and AI in communication?

The three steps are: 1) Ensuring detection skills and tools are available to those who need them, 2) Understanding the content provenance and disclosure through metadata and watermarking, and 3) Establishing a pipeline of responsibility from AI models to the platforms where media is consumed.

Why is it important to have robust detection tools for deepfakes?

Robust detection tools are important to help journalists, community leaders, and human-rights defenders discern authenticity from simulation and to fortify the credibility of critical voices and images.

What is the significance of content provenance and disclosure in the context of AI-generated media?

Content provenance and disclosure, through metadata and watermarking, provide a 'recipe' of how AI and human input were mixed in the creation or editing of media, which is essential for building trust and literacy in AI-infused media.

How can we ensure that the infrastructure for authenticity does not compromise privacy or backfire globally?

By focusing on the 'how' of AI-human media making rather than the 'who', ensuring that the infrastructure respects rights and allows for anonymity where necessary, without obliging disclosure of personal information.

What is the role of governments in ensuring transparency, accountability, and liability in the pipeline of AI responsibility?

Governments need to ensure that there is a clear pipeline of responsibility for AI, including transparency in how AI is used, accountability for its effects, and liability for misuse, to prevent the repetition of social media failures in the next generation of technology.

Outlines

00:00

🤖 The Challenge of Deepfakes and Trust in Information

This paragraph discusses the increasing difficulty in distinguishing between real and AI-generated content, particularly with the advancement of deepfake technology. The speaker, from the human rights group WITNESS, outlines the evolution of deepfakes from a hyped threat in 2017 to a growing issue affecting women and girls globally. They highlight the broader implications of generative AI in creating and dismissing 'fake reality,' and the role of their organization in combating deepfakes through a rapid-response task force. The paragraph also details the task force's efforts in analyzing audio clips from Sudan, West Africa, and India, emphasizing the complexity and evolving nature of deepfake detection.

05:03

🔍 Addressing the Deepfake Phenomenon: Detection and Provenance

The speaker emphasizes the need for structural solutions to address the deepfake issue, focusing on three main steps. First, ensuring that detection tools and skills are accessible to those who need them most, such as journalists and human rights defenders. They discuss the limitations and challenges of current detection tools, which may not be reliable or universally effective. Second, the paragraph introduces the concept of content provenance and disclosure, including the use of invisible watermarking and cryptographically signed metadata to provide a 'recipe' of AI and human involvement in media creation. This approach is presented as a critical component of new media literacy in an AI-infused world. The speaker also touches on the importance of maintaining privacy and avoiding the disclosure of personal information in the process.

10:04

🌐 Creating a Pipeline of Responsibility for AI in Media

In the final paragraph, the speaker calls for a pipeline of responsibility that encompasses the entire lifecycle of AI in media, from foundational models to deployment in systems and platforms. They argue that governments must ensure transparency, accountability, and liability within this pipeline to prevent the misuse of AI technologies. The speaker warns of the dangers of a world where reality can be easily faked and genuine reality dismissed as potentially fake, referencing the political philosopher Hannah Arendt's views on the consequences of a society unable to discern truth from falsehood. The paragraph concludes with a call to action to prevent such a future by taking the necessary steps now.

Keywords

💡Generative AI

Generative AI refers to artificial intelligence systems that can create new content, such as text, images, or audio, that is similar to the data they were trained on. In the video's context, generative AI is central to the discussion of deepfakes and the challenges they pose to discerning real from fake content. The script mentions how advances in generative AI have made it easier to create convincingly fake realities.

💡Deepfakes

Deepfakes are a type of media forgery that use AI to manipulate or generate images or videos that appear real but are not. They are a significant concern in the video, as they illustrate the growing problem of misinformation and the potential for malicious use, such as falsified sexual images or impersonating public figures.

💡Media-forensics

Media-forensics is the field of analyzing digital media to determine its authenticity. In the video, a deepfakes rapid-response task force composed of media-forensics experts is mentioned, emphasizing the importance of this field in debunking deepfakes and verifying the authenticity of media content.

💡Synthetic speech

Synthetic speech is a technology that generates human-like speech using AI. The script refers to a machine-learning algorithm trained on over a million examples of synthetic speech, which was used to authenticate an audio clip, demonstrating the role of this technology in both creating and detecting deepfakes.

💡Authenticity

Authenticity in the context of the video refers to the genuineness and trustworthiness of information or media content. The speaker discusses the challenges of proving authenticity in the face of deepfakes and the importance of tools and techniques to verify and maintain the authenticity of media.

💡Human-rights defenders

Human-rights defenders are individuals or groups who promote and protect human rights. The video emphasizes the role of such defenders in using technology, like video and AI, to document and expose human rights abuses, and the challenges they face with the rise of deepfakes and misinformation.

💡AI-generated imagery

AI-generated imagery refers to visual content created by AI, which can be used to depict events or situations that did not actually occur. The script warns of the dangers of AI-generated imagery being shared as real, contributing to the spread of disinformation and confusion.

💡Content provenance

Content provenance is the origin and history of digital content, including how it was created and by whom. The video discusses the need for content provenance to establish trust in media, especially with the increasing use of AI in content creation and editing.

💡Cryptographically signed metadata

Cryptographically signed metadata refers to data about a digital file that is securely signed with a cryptographic key, ensuring its authenticity and integrity. In the video, this concept is presented as a solution for adding trust to AI-generated media by providing a record of how the content was created and distributed.

💡AI-infused media literacy

AI-infused media literacy is the ability to critically evaluate and understand media content that has been created or altered by AI. The video suggests that as AI becomes more prevalent in communication, media literacy must evolve to include understanding the role of AI in content creation and the implications for trust and authenticity.

💡Pipeline of responsibility

A pipeline of responsibility refers to a series of accountability measures that should be in place throughout the development, deployment, and use of AI technologies. The video argues for the necessity of such a pipeline to ensure transparency, accountability, and liability in AI applications, particularly in the context of media and communication.

Highlights

The increasing difficulty in distinguishing between real and fake content due to advances in generative AI and deepfakes.

The initial focus on deepfakes in 2017 was on falsified sexual images, which have since grown to affect women and girls worldwide.

The evolving threat of generative AI in creating fake realities and dismissing real ones as potentially faked.

The role of audiovisual AI in exacerbating societal problems, including electoral manipulation and human rights abuses.

Introduction of WITNESS, a human rights group that aids in using technology to defend rights against new forms of reality manipulation.

The 'Prepare, Don't Panic' initiative, a global effort to counteract the manipulation and synthesis of reality.

The deepfakes rapid-response task force, composed of media-forensics experts and companies, debunking deepfakes and claims of deepfakes.

Case studies from Sudan, West Africa, and India, illustrating the challenges and successes in deepfake detection.

The limitations of current deepfake detection methods, including the inability to conclusively separate true from false.

The future challenges in protecting real content and detecting fakes, with a focus on political leaders and crisis zones.

The importance of not relying solely on individuals to discern deepfakes, but rather implementing structural solutions.

The need for robust foundations to discern authenticity, fortify credibility, and develop powerful detection technology.

Three proposed steps to address the challenges of AI in communication: detection tools, content provenance, and a pipeline of responsibility.

Ensuring detection skills and tools are accessible to those who need them most, such as journalists and human rights defenders.

The concept of content provenance and disclosure, including invisible watermarking and cryptographically signed metadata.

The challenges in creating an infrastructure for authenticity that respects privacy and does not compromise anonymity.

The necessity for a pipeline of responsibility from AI foundations to platforms, ensuring transparency, accountability, and liability.

The risk of a world where reality is easily faked and genuine reality is dismissed, leading to a loss of capacity to act, think, and judge.

Transcripts

It's getting harder, isn't it, to spot real from fake,

AI-generated from human-generated.

With generative AI,

along with other advances in deep fakery,

it doesn't take many seconds of your voice,

many images of your face,

to fake you,

and the realism keeps increasing.

I first started working on deepfakes in 2017,

when the threat to our trust in information was overhyped,

and the big harm, in reality, was falsified sexual images.

Now that problem keeps growing, harming women and girls worldwide.

But also, with advances in generative AI, we're now also approaching a world

where it's broadly easier to make fake reality,

but also to dismiss reality as possibly faked.

Now, deceptive and malicious audiovisual AI

is not the root of our societal problems,

but it's likely to contribute to them.

Audio clones are proliferating in a range of electoral contexts.

"Is it, isn't it" claims cloud human-rights evidence from war zones,

sexual deepfakes target women in public and in private,

and synthetic avatars impersonate news anchors.

I lead WITNESS.

We're a human-rights group

that helps people use video and technology to protect and defend their rights.

And for the last five years, we've coordinated a global effort,

"Prepare, Don't Panic,"

around these new ways to manipulate and synthesize reality,

and on how to fortify the truth

of critical frontline journalists and human-rights defenders.

Now, one element in that is a deepfakes rapid-response task force,

made up of media-forensics experts

and companies who donate their time and skills

to debunk deepfakes and claims of deepfakes.

The task force recently received three audio clips,

from Sudan, West Africa and India.

People were claiming that the clips were deepfaked, not real.

In the Sudan case,

experts used a machine-learning algorithm

trained on over a million examples of synthetic speech

to prove, almost without a shadow of a doubt,

that it was authentic.

In the West Africa case,

they couldn't reach a definitive conclusion

because of the challenges of analyzing audio from Twitter,

and with background noise.

The third clip was leaked audio of a politician from India.

Nilesh Christopher of “Rest of World” brought the case to the task force.

The experts used almost an hour of samples

to develop a personalized model of the politician's authentic voice.

Despite his loud and fast claims that it was all falsified with AI,

experts concluded that it at least was partially real, not AI.

As you can see,

even experts cannot rapidly and conclusively separate true from false,

and the ease of calling "that's deepfaked" on something real

is increasing.

The future is full of profound challenges,

both in protecting the real and detecting the fake.

We're already seeing the warning signs

of this challenge of discerning fact from fiction.

Audio and video deepfakes have targeted politicians,

major political leaders in the EU, Turkey and Mexico,

and US mayoral candidates.

Political ads are incorporating footage of events that never happened,

and people are sharing AI-generated imagery from crisis zones,

claiming it to be real.

Now, again, this problem is not entirely new.

The human-rights defenders and journalists I work with

are used to having their stories dismissed,

and they're used to widespread, deceptive, shallow fakes,

videos and images taken from one context or time or place

and claimed as if they're in another,

used to share confusion and spread disinformation.

And of course, we live in a world that is full of partisanship

and plentiful confirmation bias.

Given all that,

the last thing we need is a diminishing baseline

of the shared, trustworthy information upon which democracies thrive,

where the specter of AI

is used to plausibly believe things you want to believe,

and plausibly deny things you want to ignore.

But I think there's a way we can prevent that future,

if we act now;

that if we "Prepare, Don't Panic,"

we'll kind of make our way through this somehow.

Panic won't serve us well.

[It] plays into the hands of governments and corporations

who will abuse our fears,

and into the hands of people who want a fog of confusion

and will use AI as an excuse.

How many people were taken in, just for a minute,

by the Pope in his dripped-out puffer jacket?

You can admit it.

(Laughter)

More seriously,

how many of you know someone who's been scammed

by an audio that sounds like their kid?

And for those of you who are thinking "I wasn't taken in,

I know how to spot a deepfake,"

any tip you know now is already outdated.

Deepfakes didn't blink, they do now.

Six-fingered hands were more common in deepfake land than real life --

not so much.

Technical advances erase those visible and audible clues

that we so desperately want to hang on to

as proof we can discern real from fake.

But it also really shouldn’t be on us to make that guess without any help.

Between real deepfakes and claimed deepfakes,

we need big-picture, structural solutions.

We need robust foundations

that enable us to discern authentic from simulated,

tools to fortify the credibility of critical voices and images,

and powerful detection technology

that doesn't raise more doubts than it fixes.

There are three steps we need to take to get to that future.

Step one is to ensure that the detection skills and tools

are in the hands of the people who need them.

I've talked to hundreds of journalists,

community leaders and human-rights defenders,

and they're in the same boat as you and me and us.

They're listening to the audio, trying to think, "Can I spot a glitch?"

Looking at the image, saying, "Oh, does that look right or not?"

Or maybe they're going online to find a detector.

And the detector they find,

they don't know whether they're getting a false positive, a false negative,

or a reliable result.

Here's an example.

I used a detector, which got the Pope in the puffer jacket right.

But then, when I put in the Easter bunny image that I made for my kids,

it said that it was human-generated.

This is because of some big challenges in deepfake detection.

Detection tools often only work on one single way to make a deepfake,

so you need multiple tools,

and they don't work well on low-quality social media content.

Confidence score, 0.76-0.87,

how do you know whether that's reliable,

if you don't know if the underlying technology is reliable,

or whether it works on the manipulation that is being used?

And tools to spot an AI manipulation don't spot a manual edit.

These tools also won't be available to everyone.

There's a trade-off between security and access,

which means if we make them available to anyone,

they become useless to everybody,

because the people designing the new deception techniques

will test them on the publicly available detectors

and evade them.

But we do need to make sure these are available

to the journalists, the community leaders,

the election officials, globally, who are our first line of defense,

thought through with attention to real-world accessibility and use.

Though at the best circumstances,

detection tools will be 85 to 95 percent effective,

they have to be in the hands of that first line of defense,

and they're not, right now.

So for step one, I've been talking about detection after the fact.

Step two -- AI is going to be everywhere in our communication,

creating, changing, editing.

It's not going to be a simple binary of "yes, it's AI" or "phew, it's not."

AI is part of all of our communication,

so we need to better understand the recipe of what we're consuming.

Some people call this content provenance and disclosure.

Technologists have been building ways to add invisible watermarking

to AI-generated media.

They've also been designing ways --

and I've been part of these efforts --

within a standard called the C2PA,

to add cryptographically signed metadata to files.

This means data that provides details about the content,

cryptographically signed in a way that reinforces our trust

in that information.

It's an updating record of how AI was used to create or edit it,

where humans and other technologies were involved,

and how it was distributed.

It's basically a recipe and serving instructions

for the mix of AI and human

that's in what you're seeing and hearing.

And it's a critical part of a new AI-infused media literacy.

And this actually shouldn't sound that crazy.

Our communication is moving in this direction already.

If you're like me -- you can admit it --

you browse your TikTok “For You” page,

and you're used to seeing videos that have an audio source,

an AI filter, a green screen, a background,

a stitch with another edit.

This, in some sense, is the alpha version of this transparency

in some of the major platforms we use today.

It's just that it does not yet travel across the internet,

it’s not reliable, updatable, and it’s not secure.

Now, there are also big challenges

in this type of infrastructure for authenticity.

As we create these durable signs of how AI and human were mixed,

that carry across the trajectory of how media is made,

we need to ensure they don't compromise privacy or backfire globally.

We have to get this right.

We can't oblige a citizen journalist filming in a repressive context

or a satirical maker using novel gen-AI tools

to parody the powerful ...

to have to disclose their identity or personally identifiable information

in order to use their camera or ChatGPT.

Because it's important they be able to retain their ability to have anonymity,

at the same time as the tool to create is transparent.

This needs to be about the how of AI-human media making,

not the who.

This brings me to the final step.

None of this works without a pipeline of responsibility

that runs from the foundation models and the open-source projects

through to the way that is deployed into systems, APIs and apps,

to the platforms where we consume media and communicate.

I've spent much of the last 15 years fighting, essentially, a rearguard action,

like so many of my colleagues in the human rights world,

against the failures of social media.

We can't make those mistakes again in this next generation of technology.

What this means is that governments

need to ensure that within this pipeline of responsibility for AI,

there is transparency, accountability and liability.

Without these three steps --

detection for the people who need it most,

provenance that is rights-respecting

and that pipeline of responsibility,

we're going to get stuck looking in vain for the six-fingered hand,

or the eyes that don't blink.

We need to take these steps.

Otherwise, we risk a world where it gets easier and easier

to both fake reality

and dismiss reality as potentially faked.

And that is a world that the political philosopher Hannah Arendt

described in these terms:

"A people that no longer can believe anything

cannot make up its own mind.

It is deprived not only of its capacity to act

but also of its capacity to think and to judge.

And with such a people you can then do what you please."

That's a world I know none of us want,

that I think we can prevent.

Thanks.

(Cheers and applause)