CW2024: Keren Elazari, Analyst, Author & Researcher, Blavatnik ICRC, Tel Aviv University

TAUVOD
14 Jul 202414:28

Summary

TLDRThe speaker, a former hacker, discusses the 'Dark Side of AI' and how malicious actors are leveraging generative AI for cybercrimes. They highlight tools like 'Worm GPT' and 'Predator AI', which are being used for phishing campaigns and targeting vulnerable cloud infrastructures. The talk also touches on the use of social media platforms for spreading misinformation and the increasing sophistication of attacks, including deep fakes and synthetic identities, emphasizing the need to rebuild trust in digital ecosystems.

Takeaways

  • 🌐 The speaker emphasizes the pervasive influence of code and AI in modern life, and the potential dark side of AI when used maliciously by hackers.
  • 🎬 The speaker was inspired to become a hacker by the 1995 film 'Hackers', which portrayed hackers as using their skills for good, not evil.
  • πŸ‘₯ The reality of hacking includes both malicious attackers and nation-state adversaries who are fast, creative, and innovative in their use of technology.
  • πŸ€– Generative AI, such as chatbots and large language models, is being adopted by criminals for nefarious purposes, including phishing campaigns and targeting cloud infrastructures.
  • πŸ’‘ Criminals are not only quick to adopt AI but also create and market their own tools, often with uncreative names based on existing AI models.
  • πŸ› 'Worm GPT' is an example of a malicious tool allegedly used for creating phishing emails, and has been sold on dark web marketplaces, though its efficacy is questionable.
  • πŸ” 'Predator AI' is another tool designed to exploit vulnerable cloud systems, demonstrating the operational use of AI by criminals.
  • πŸ“’ Platforms like Telegram and TikTok are highlighted as channels for criminals to market and sell their malicious AI tools and services.
  • πŸ“§ Generative AI can be used to create highly personalized phishing emails in various languages, making attacks more effective.
  • πŸ†” Synthetic identities and fake documents, such as IDs and passports, can be generated by AI, facilitating fraudulent activities like opening bank accounts.
  • πŸ•ŠοΈ While the script focuses on malicious use, it also mentions 'Fuzzy AI', a tool created by ethical hackers to demonstrate the potential for AI to counter other AI systems.
  • 🌐 The speaker concludes by highlighting the importance of trust in digital ecosystems and the need to learn from ethical hackers and security researchers to rebuild that trust.

Q & A

  • What is the main theme of the video script?

    -The main theme of the video script is the dark side of AI, focusing on how hackers and malicious actors are using artificial intelligence for nefarious purposes.

  • What does the speaker suggest about the adaptability of malicious attackers in the context of AI?

    -The speaker suggests that malicious attackers are incredibly adaptive, moving fast and being creative in using AI, embodying the quality of innovation.

  • What is the significance of the movie 'Hackers' from 1995 to the speaker's personal journey?

    -The movie 'Hackers' was an instant inspiration for the speaker, making her realize that her passions, curiosity, and power over technology could be channeled into being a hacker.

  • What is generative AI, and how are criminals exploiting it?

    -Generative AI refers to systems that can create new content, such as text, images, or code. Criminals are exploiting it to create phishing campaigns, fake identities, and automated attacks on vulnerable systems.

  • What is 'Worm GPT' and how is it being used by attackers?

    -'Worm GPT' is a tool allegedly created by criminals that can generate phishing campaigns and emails, posing as a significant threat to legitimate AI systems and users.

  • What is 'Predator AI' and its purpose?

    -'Predator AI' is an automatic tool designed to target vulnerable, misconfigured cloud infrastructures, such as WordPress servers and AWS instances, with pre-configured capabilities and exploits.

  • How are social media platforms like Telegram and TikTok being used by criminals?

    -Criminals are using these platforms to market and sell their malicious products and services, as well as to spread fake and malicious information, taking advantage of the platforms' lack of regulation.

  • What is 'Fuzzy AI' and its role in the cybersecurity landscape?

    -'Fuzzy AI' is a tool created by security researchers to demonstrate how generative AI can be used to jailbreak other AI models, serving as a proof of concept for the potential defensive uses of AI in cybersecurity.

  • Can you provide an example of how deepfake technology has been used in financial fraud?

    -An example is when a British director received an urgent email and a follow-up phone call from someone mimicking his German boss, leading to the transfer of $243,000 to a fraudulent subcontractor.

  • What is the 'synthetic identity' mentioned in the script, and how can it be misused?

    -A 'synthetic identity' is a fake identity created using AI, which can be used to open bank accounts or cryptocurrency exchanges for illicit activities, such as fraud or money laundering.

  • What is the speaker's final message regarding the importance of trust in the digital ecosystem?

    -The speaker emphasizes the importance of rebuilding trust in the digital ecosystem, as malicious use of AI threatens to undermine this trust, which is crucial for thriving in the digital age.

Outlines

00:00

🌐 The Dark Side of AI and Cybersecurity

The speaker introduces the topic of the 'Dark Side of AI' and their personal journey as a hacker, emphasizing curiosity and learning. They highlight the rapid adaptation and creativity of malicious hackers in using AI for cyber attacks. The talk references the Hollywood film 'Hackers' as an inspiration, contrasting it with the real-world threats posed by generative AI tools used for malicious purposes such as phishing campaigns and exploiting cloud vulnerabilities. The speaker also introduces the concept of 'malicious innovation' and the branding of AI tools by criminals, exemplified by 'Worm GPT' and 'Predator AI', which target misconfigured cloud infrastructures.

05:02

πŸ›‘οΈ The Exploitation of Generative AI by Criminals

This paragraph delves into the operational use of generative AI by criminals, focusing on 'Predator AI' and its capabilities to exploit vulnerable cloud systems. The speaker discusses the marketing of such tools on platforms like Telegram and the lack of regulation that allows criminals to operate freely. They also touch on the broader implications of generative AI in creating synthetic identities and conducting sophisticated phishing campaigns, as well as the use of AI in direct attacks and exploiting network vulnerabilities. The paragraph concludes with a counterpoint, introducing 'Fuzzy AI', a tool created by ethical researchers to demonstrate the potential for AI to jailbreak other AI models.

10:03

🎭 Deep Fakes and the Erosion of Trust in Digital Communications

The final paragraph discusses the use of deep fake technology in financial scams, where criminals impersonate individuals to authorize fraudulent transactions. The speaker recounts specific cases where large sums of money were lost due to deep fake impersonations during video conferences. They also mention the availability of websites that generate synthetic identities and documents, which can be used for nefarious purposes. The speaker warns of the challenges in verifying identities in a digital world where trust is increasingly compromised and calls for learning from ethical hackers and security researchers to forge a path forward in rebuilding trust in digital ecosystems.

Mindmap

Keywords

πŸ’‘Code

Code refers to the system of rules and symbols used to represent instructions to a computer. In the context of the video, the speaker's initial curiosity in coding led to her journey as a hacker. The script mentions learning HTML code by deconstructing websites, which is a fundamental aspect of understanding the internet's structure and a starting point for many in the field of cybersecurity.

πŸ’‘Hacker

A hacker is someone skilled in understanding and manipulating technology systems, often for the purpose of exploiting vulnerabilities. The video discusses the speaker's evolution from a curious learner to identifying as a hacker, emphasizing the positive aspect of using these skills to shape the world, rather than causing harm, which contrasts the common perception of hackers as malicious actors.

πŸ’‘Cybersecurity

Cybersecurity encompasses the practices and technologies designed to protect networks, devices, programs, and data from attack, damage, or unauthorized access. The script frequently touches on cybersecurity, discussing the advancements in the field and the challenges posed by the misuse of AI by hackers, which is central to the video's theme of the 'Dark Side of AI'.

πŸ’‘Artificial Intelligence (AI)

Artificial Intelligence refers to the simulation of human intelligence in machines that are programmed to think like humans and mimic their actions. The video discusses the pervasiveness of AI and its potential misuse by hackers, such as creating phishing campaigns and exploiting cloud infrastructures, which highlights the dual-use nature of AI technology.

πŸ’‘Generative AI

Generative AI is a subset of AI that can create new content, such as text, images, or code, that appears original and coherent. The script mentions Generative AI tools like chat GPT and GPT, which have been adopted by both legitimate users and malicious actors, illustrating the video's theme of the potential dark applications of AI.

πŸ’‘Malicious Innovation

Malicious Innovation refers to the creation and application of new technologies or methods for harmful or criminal purposes. The video describes how malicious actors are using generative AI for phishing campaigns and targeting cloud infrastructures, showcasing the darker side of innovation when used unethically.

πŸ’‘Deepfake

Deepfake is a term used to describe AI-generated synthetic media, often videos or audio, where a person's likeness or voice is convincingly replicated. The script provides examples of deepfakes being used in scams, such as impersonating a boss to defraud employees, demonstrating the real-world implications of this technology.

πŸ’‘Phishing

Phishing is a type of online scam where attackers attempt to acquire sensitive information, such as usernames, passwords, and credit card details, by disguising themselves as a trustworthy entity. The video discusses how generative AI can be used to create highly personalized and varied phishing emails, making them more effective and dangerous.

πŸ’‘Synthetic Identity

A synthetic identity is a fake identity created by combining real and fabricated data. The script mentions the use of generative AI to create synthetic identities, such as fake IDs and passports, which can be used for fraudulent activities like opening bank accounts or cryptocurrency exchanges.

πŸ’‘Nation-State Adversaries

Nation-State Adversaries are entities that act on behalf of or are sponsored by a nation, engaging in cyberattacks or espionage. The video references a report by Open AI that identifies the misuse of AI by state-affiliated threat actors, suggesting that even governments may be exploiting AI for malicious purposes.

πŸ’‘Trust

Trust is the reliance on the integrity, strength, ability, or surety of a person or thing. The video concludes with a discussion on the erosion of trust in digital ecosystems due to the misuse of AI and cybersecurity threats, emphasizing the importance of rebuilding trust as a key challenge for the future.

Highlights

Code and AI are increasingly prevalent in our lives, with HTML influencing fashion choices and cybersecurity and AI being major topics of discussion.

The speaker, a former hacker, shares insights on the dark side of AI, focusing on how malicious hackers exploit AI technologies.

Hackers are adaptive and innovative, embodying the quality of innovation in their malicious activities.

Generative AI, such as chatbots and large language models, are being adopted by criminals for nefarious purposes.

Examples of malicious AI tools include 'dark Bard', 'worm GPT', and 'fraud GPT', which are used for creating phishing campaigns and exploiting cloud infrastructures.

Criminals are not creative in naming their AI tools, often using similar patterns to legitimate AI models.

The tool 'spy ey' was created by security researchers as a proof of concept to demonstrate the potential misuse of AI.

Worm GPT is allegedly sold on dark web marketplaces, though its efficacy is questionable, indicating a potential scam.

Predator AI is an operational tool designed to target vulnerable cloud infrastructures, such as misconfigured WordPress servers.

Criminals use platforms like Telegram and TikTok to market and sell their malicious AI tools due to the lack of regulation.

Generative AI is becoming a tool of choice for criminals in phishing campaigns, creating synthetic identities, and exploiting networks.

Fuzzy AI is a tool created by ethical researchers to demonstrate the potential to jailbreak other AI models.

Deepfake technology has been used in scams, such as mimicking a boss's voice to defraud employees of large sums of money.

Nation-state adversaries are also leveraging AI, as reported by OpenAI, although the extent of this use is disputed.

The most significant risk from malicious AI is the erosion of trust in digital ecosystems, which are crucial for societal functioning.

The speaker calls for learning from ethical hackers and security researchers to forge a path forward and rebuild trust.

An invitation to bsides TV, Israel's largest hacker community event, for further learning and engagement on these topics.

Transcripts

play00:00

[Music]

play00:03

they say code is eating the world and

play00:06

HTML has took over my fashion choice for

play00:09

today ladies and gentlemen I'm so happy

play00:12

to be with you to share my point of view

play00:14

about the Dark Side of AI thank you my

play00:16

friend and in this week we've heard so

play00:19

much about cyber security advancements

play00:22

and we've heard artificial intelligence

play00:24

all over but I wanted to present the

play00:27

point of view of hackers about using Ai

play00:31

and how bad hackers can use AI so

play00:34

spoiler alert I grew up as a hacker but

play00:37

not necessarily A malicious hacker in

play00:40

fact I grew up as a very curious young

play00:43

little girl right here in Tel Aviv I was

play00:46

asking my parents so many questions and

play00:49

I was teaching myself how to write HTML

play00:52

code by taking apart other people's

play00:54

websites I was learning all about the

play00:57

worldwide web in the first year we got

play01:00

access to the internet here in Tel Aviv

play01:02

in 1993 but it was only in '95 that I

play01:05

realized my true passion was to be a

play01:08

hacker and I realized this thanks to my

play01:11

hacker Mentor her name Angelina

play01:16

Julie some of you may have seen her in

play01:18

the Hollywood film hackers that came out

play01:21

in

play01:22

1995 for me that movie was an instant

play01:25

inspiration I realized for the first

play01:28

time in my life that my my passions my

play01:31

curiosity and my power over technology

play01:34

it's called being a hacker spoiler alert

play01:37

if you haven't seen the film Angelina is

play01:40

not the bad guy in fact she's the leader

play01:43

of a fierce group of hacker Misfits high

play01:46

school kids who use their power over

play01:48

technology to shape the world and even

play01:51

save the day but we are here today to

play01:55

learn from The Real World of hackers not

play01:57

just my Hollywood Heroes and in our real

play02:00

world there are a lot of malicious

play02:02

attackers and what we've realized in the

play02:05

last few years is that these types of

play02:08

malicious attackers whether they are

play02:09

criminals or nation state adversaries

play02:13

are incredibly adaptive they move fast

play02:16

they're creative in other words they

play02:19

embody a quality we have been talking

play02:21

about all week Innovation so let's talk

play02:24

a little bit about malicious Innovation

play02:27

and in particular how criminals and

play02:30

malicious adversaries can take advantage

play02:33

of generative AI by the way this is my

play02:37

favorite Transformer Optimus Prime from

play02:41

back in the day Transformers were the

play02:43

automotive cars in the kids cartoons but

play02:46

today kids are growing up with

play02:48

Transformers like chat GPT and other

play02:51

different types of generative AI tools

play02:54

and large language models so we are all

play02:56

very familiar with Bard Gemini CL CL and

play03:00

many of these other generative AI

play03:03

systems GPT has more than 1 million

play03:07

users one billion users in the two years

play03:10

it's been on our planet but what about

play03:13

the malicious cousins of chat GPT what

play03:16

about dark Bard allegedly trained on

play03:20

dark Nate data or dark gpt3 bot or worm

play03:25

GPT based on the open-source gptj model

play03:29

or perhaps you've heard about threat GPT

play03:32

wolf GPT fraud GPT as you can see while

play03:36

criminals are fast to adopt AI they are

play03:39

not incredibly creative when it comes to

play03:41

The Branding and the naming conventions

play03:44

of their AI tools and the last one on

play03:46

the list is of particular interest it's

play03:48

called spy ey it was actually created by

play03:51

a team of security researchers in Korea

play03:54

as a proof of concept tool now what is

play03:57

common to all of these different models

play04:00

is that bad guys are not afraid to take

play04:03

their chances and start using them and

play04:06

what can they use them for well let's

play04:08

take a look at worm GPT allegedly from

play04:11

the actual screenshots of worm GPT it

play04:15

can create fishing campaigns and emails

play04:18

it can be the best tool for attackers

play04:21

and the worst enemy of legitimate GPT or

play04:25

the open AI GPT system what is even more

play04:28

interesting is that the creator of worm

play04:30

GPT has been selling it on darket

play04:34

websites and telegram channels and it's

play04:37

not clear whether this tool actually

play04:40

works or perhaps it's just a scam to get

play04:44

criminals to pay for an allegedly

play04:46

criminal tool that doesn't always work

play04:49

so no honor Amongst Thieves it appears

play04:52

but there's other types of generative AI

play04:54

tools the criminals are creating and

play04:57

marketing the next one is a little bit

play04:59

more operational and its name is

play05:02

predator AI it has about 11,000 lines of

play05:08

code created by generative AI with a

play05:11

terrible user interface incredibly poor

play05:14

user experience but this is an automatic

play05:17

tool designed to Target vulnerable

play05:21

misconfigured Cloud infrastructures what

play05:24

do I mean by that WordPress servers

play05:27

jumla AWS instan

play05:30

this tool comes preconfigured with the

play05:32

capabilities and the exploits to allow

play05:35

attackers to take advantage of the so

play05:37

many vulnerable Cloud systems that are

play05:40

incredibly popular in this day and age

play05:44

another interesting fact for the

play05:45

audience here the people behind Predator

play05:48

AI actually include their name and the

play05:51

telegram Channel where you can find them

play05:53

and hopefully to them pay them for their

play05:57

capabilities and services and this is

play05:59

the f face if you will of one of the

play06:01

creators of Predator Ai and uh in the

play06:05

very bottom you can see at least they

play06:07

recognize that Israel is real but their

play06:11

statements about what they think about

play06:13

our country are quite clear and have

play06:16

taken the liberty of blackening out the

play06:18

fruity language that they take advantage

play06:20

of another tool these types of attackers

play06:23

will take advantage of and they're not

play06:25

the only ones is what I like to call the

play06:28

explosives of of the 21st century the

play06:31

TNT of the 21st century of course these

play06:34

are Telegram and Tik Tok these are the

play06:38

platforms where so many creative

play06:41

criminals can take advantage of they can

play06:43

use it to Market and sell their products

play06:46

their services and since these platforms

play06:49

are not very regulated they can do

play06:52

whatever they want there now it doesn't

play06:55

end there we know that these platforms

play06:57

are also served as a basis to spread

play07:00

fake information and also male

play07:02

information malicious information that

play07:05

will harm us so it is my recommendation

play07:07

to take these platforms with a grain of

play07:10

salt but when it comes to hackers and

play07:13

generative AI I think we're only at the

play07:15

beginning of a love story for the new

play07:18

age because if we think about all the

play07:21

classic ways that attackers get into

play07:24

organizations those are fishing emails

play07:27

credentials and identity thefts using

play07:29

people's passwords and of course direct

play07:31

attacks exploits on network appliances

play07:34

or really hacking into directly into

play07:37

remote systems to get access into an

play07:39

organization if you look about these

play07:41

three classic access vectors that pretty

play07:44

much every big breach or ransomware

play07:46

campaign has started with for each and

play07:49

every one of these generative AI has

play07:51

become a tool of choice for criminals

play07:53

when it comes to fishing campaigns it

play07:55

can now be used to generate a hundred

play07:57

different variations in every language

play08:00

with every image translated exactly and

play08:03

personalized exactly you spent this week

play08:05

in Tel Aviv cyu perhaps you will be

play08:07

getting some fishing emails next week

play08:10

and when it comes to credentials and

play08:12

identity I'll show you in a minute how

play08:14

generative AI is helping bad guys create

play08:17

synthetic identities and certainly in

play08:20

the realm of exploits and direct attacks

play08:23

scanning Automation and different AI

play08:26

tools have already been part of the

play08:28

Arsenal of bad guys now just to give a

play08:30

Counterpoint I also want to showcase to

play08:32

you fuzzy AI a tool created by the good

play08:36

researchers at Cyber Arc labs to

play08:39

demonstrate how they can use generative

play08:42

AI to jailbreak other generative AI

play08:45

models got it it's an AI that can hack

play08:48

or jailbreak other AI models so this is

play08:50

a proof of concept tool by the good

play08:52

friends at Cyber clabs do check it out I

play08:55

think it's fairly impressive but Creator

play08:57

creative criminals have come up with

play08:59

different ways to to use technology

play09:01

against us surely many of you heard the

play09:04

story about the British director who got

play09:07

an urgent email from his German boss

play09:09

asking to transfer

play09:13

$243,000 to a new subcontractor of the

play09:15

company that boss followed it up with a

play09:18

phone conversation and that employee

play09:21

recognized his boss's German accent and

play09:24

of course transferred the funds and so

play09:28

$243,000 were lost like that certainly

play09:31

you've heard about the story when do you

play09:33

think it happened a week ago a month ago

play09:36

two years ago when chat GPT became a

play09:39

broadly accessible tool news flash

play09:42

ladies and gentlemen this happened five

play09:45

years ago criminals have learned how to

play09:47

use deep fake algorithms to come across

play09:51

as an individual using their voice and

play09:53

likeness and just recently we've heard

play09:56

about deep fake video person as taking

play10:00

over video conferences like Zoom or

play10:03

Google meet to masquerade as an entire

play10:06

team of individuals from a company this

play10:08

happened just a few months ago in Hong

play10:10

Kong a Chief Financial Officer and his

play10:13

entire team were

play10:15

masquerad by a whole group of video

play10:19

avatars which fooled one employee an

play10:22

employee who was on a call who believed

play10:24

everyone was real that employee

play10:26

transferred $200 million Hong Kong

play10:28

dollars which is about20 million us now

play10:32

when I heard the story I didn't believe

play10:34

it at first how are we to believe such

play10:36

stories so I actually saw the video

play10:39

conference by the chief superintendent

play10:41

of the Hong Kong police explaining they

play10:44

believe the criminals used videos from

play10:46

that company to specifically train the

play10:49

AI to generate those deep fake

play10:52

convincing video

play10:55

avatars now very recently The Wall

play10:57

Street Journal put out this information

play10:59

piece this opinion piece deep fakes are

play11:01

coming for the financial sector my

play11:04

friends in the Wall Street Journal they

play11:06

are not coming they are here meet David

play11:10

Creek an individual that does not exist

play11:13

here he is this is a synthetic identity

play11:17

this person does not exist but there is

play11:19

a website that is now able to generate

play11:22

IDs and passports these images complete

play11:26

with the carpet background look very

play11:28

legitim

play11:29

and they can be used to open a new bank

play11:31

account or a new cryptocurrency exchange

play11:34

this is the website where you can find

play11:36

such fake IDs it is called only fake a

play11:40

joke perhaps on the only fans website

play11:43

and while they were rumored taken down

play11:46

by the American government a few months

play11:48

ago they came back with a statement we

play11:50

haven't disappeared anywhere in fact we

play11:53

are now preparing an update and they are

play11:56

also offering a discount so you can use

play11:59

the code ID card to get your first time

play12:01

discount they are very Savvy when it

play12:03

comes to their branding and their

play12:05

marketing so how will you do on your

play12:08

next Zoom call do you feel comfortable

play12:11

challenging people's

play12:12

identities how can you verify an email

play12:15

or a phone conversation in such a world

play12:18

indeed this is a new Criminal

play12:20

Renaissance for bad guys it doesn't end

play12:23

there our life as security practitioners

play12:26

used to be pretty clear we were Mario at

play12:29

the bottom by the way Mario before he

play12:31

was super he was Mario fighting Donkey

play12:33

Kong and we had to Be watchful for

play12:37

flaming barrels of oil thrown at us by a

play12:40

500 ton gorilla at the top that's still

play12:42

our job as security practitioners but

play12:45

now they can do it a thousand times

play12:47

faster and when it comes to nation state

play12:49

adversaries they've learned to take

play12:51

advantage of generative AI don't take it

play12:54

for me open AI said in their own report

play12:57

that they have identified malicious use

play13:00

of AI by state Affiliated threat actors

play13:04

open AI say they found this use to be

play13:07

limited and incremental but I disagree

play13:10

perhaps I disagree because these are the

play13:13

countries that were found to be taking

play13:15

advantage of that platform so with these

play13:18

types of adversaries I believe it is

play13:20

better that we take it into our

play13:23

attention strongly and not lightly to

play13:26

summarize ladies and gentlemen what is a

play13:29

risk in this day and age what the most

play13:32

complicated the most fragile thing these

play13:35

types of attackers can take away from us

play13:38

it's our trust our trust in the digital

play13:41

ecosystems that allow us to thrive that

play13:44

has allowed Israel to do okay even in

play13:47

such a time of difficulty and adversity

play13:51

how can we Forge a future ahead how can

play13:54

we rebuild trust this is my question to

play13:56

all of you and I hope you choose to

play13:59

learn from the friendly hackers and

play14:00

security researchers that are showing us

play14:03

that path forward for those who wish to

play14:05

learn more from the friendly hackers we

play14:07

will be right here on Thursday during

play14:09

bsides TV Israel's largest hacker

play14:12

Community event thank you so much for

play14:14

your attention please stay safe and see

play14:17

you next time sayonara

Rate This
β˜…
β˜…
β˜…
β˜…
β˜…

5.0 / 5 (0 votes)

Related Tags
CybersecurityAI EthicsHackingCriminal AIDeepfakesCyber ThreatsGenerative AICybercrimeTrust IssuesSecurity Awareness