Discord's AI breaks its own TOS!

Summary

このビデオでは、DiscordのAIサマリー機能が自社の利用規約に違反している問題を解説しています。具体的には、ユーザーのデータ漏洩が指摘されており、削除されたメッセージのユーザーIDが依然としてアクセス可能であることが問題視されています。この問題は、個人情報の削除を保証するDiscordのプライバシーポリシーに反しています。さらに、ビデオではDiscordのAI機能の失敗例として、ClydeやAutomod AIなどが挙げられています。最後に、この問題を発見した人物がバグ報奨金として$750を受け取った事例が紹介されています。このビデオは、DiscordのAIサマリー機能の問題点とその影響、法的なリスクについて詳しく解説しています。

Takeaways

• 💡このスクリプトはDiscordのAI要約機能のセキュリティ上の問題について説明しています。
• 💡ユーザーがメッセージを削除しても、要約にはユーザーIDが表示されることが分かっています。
• 💡これはDiscordのプライバシーポリシーに違反している可能性があることが指摘されています。
• 💡報告されたこの問題について、Discordがどのように対処するかが不明です。

Q & A

DiscordのAIサマリー機能とは何ですか？

DiscordのAIサマリー機能は、ユーザーの会話をOpenAIを使用して要約する機能です。

なぜDiscordのAIサマリー機能が問題視されているのですか？

DiscordのAIサマリー機能が、ユーザーのデータを不正に漏洩し、Discord自身の利用規約に違反する可能性があるためです。

具体的にどのようなセキュリティ問題が指摘されていますか？

ユーザーがメッセージを削除しても、AIサマリーを通じてユーザーIDやメッセージIDが漏洩し得るという問題が指摘されています。

このセキュリティ問題による個人への影響は何ですか？

削除したはずのメッセージに関する情報が第三者に漏れ、ハラスメントなどの被害につながる恐れがあります。

Discordのプライバシーポリシーではどのようなルールが設定されていますか？

ユーザーは自分が送信したメッセージを削除する権利があり、削除されたメッセージはユーザーの個人情報とともに完全に消去されるべきです。

なぜこの問題は法的な問題を引き起こす可能性があるのですか？

AIサマリー機能がDiscordのプライバシーポリシーに違反し、ユーザーの個人情報保護に関する法律に抵触する可能性があるためです。

Discordはこの問題にどのように対処する可能性がありますか？

問題の重大性を踏まえ、DiscordはAIサマリー機能を取り下げる可能性が高いです。

Discordの他のAI機能についての実績はどうですか？

Discordは過去に複数のAI機能をリリースしましたが、問題が発生し、いくつかの機能は取り下げられました。

Discordはこのようなバグを発見した場合、どのような対応をしますか？

Discordにはバグ報奨金プログラムがあり、バグを報告したユーザーには報奨金が支払われます。

このセキュリティ問題が初めて報告されたのはいつですか？

このセキュリティ問題は2023年5月に初めて報告されましたが、記事作成時点でまだ解決されていません。

Outlines

00:00

🚨ディスコードのAIサマリーが利用規約に違反

ディスコードのAIサマリー機能がユーザーデータの漏洩を引き起こし、自社の利用規約に違反していることが判明しました。この機能はOpenAIを使用してディスコードの会話を要約するもので、セキュリティ上の問題が発覚しました。特定のペンリーという人物がこの問題を発見し、ユーザーがメッセージを削除しても、特定のAPIリクエストを使用すると、削除されたメッセージのIDや参加者の情報が依然としてアクセス可能であることが明らかになりました。これにより、プライバシーの侵害やハラスメントのリスクが生じています。さらに、この問題はディスコードのプライバシーポリシーにも違反しており、法的な問題を引き起こす可能性があります。この問題は2023年5月から既知のものであり、未だに修正されていない状態です。

05:01

🤖ディスコードのAI機能の失敗と将来性

ディスコードのAI機能、特にAIサマリーは多くの問題を抱えており、他のAI機能も期待外れであることが指摘されています。AIチャットボットのClydeは不適切な言動で問題となり、他の機能もほとんど更新されていないか、計画通りに機能していない状態です。AIサマリー機能に関する最近の問題が明らかになったことで、ディスコードがこの機能を撤回する可能性が高いと分析しています。また、ディスコードはバグ報告に対して報酬を提供する制度を設けており、この問題を報告した人物は$750の報酬を受け取ったとのことです。このビデオでは、ディスコードのAI機能の現状と将来性について批判的な視点から分析を加えています。

Keywords

💡AI summaries

Discord's AI-powered chat summaries feature that summarizes conversations. It surfaces private user IDs even after messages are deleted.

💡privacy policy

Discord's policy governing use of private user data. The AI summaries feature violates this by exposing user IDs.

💡user ID

Unique identifier assigned to each Discord user. Remains visible via AI summaries even if messages are deleted.

💡data deletion

Ability to permanently delete your private messages per Discord's policy. But AI summaries retain trace data.

Highlights

Discord's AI summaries feature breaks their own terms of service by leaking private user data

Penley discovered an exploit where message IDs and user IDs are exposed even if messages are deleted

This allows harassing users even if they've deleted sensitive messages

Discord's privacy policy states users can delete messages and personal information should be removed

AI summaries break this by exposing user IDs of deleted messages

This has been an issue since May 2023 and is still not patched

Discord has failed at most previous AI attempts like Clyde and automod

The speaker predicts Discord will cancel AI summaries due to these issues

Transcripts

you probably didn't know but discord's

AI summaries break discord's own terms

of service by leaking your data but how

did this happen what are the

repercussions and what is Discord going

to do about it now I'm going to assume

that you've never had the absolute

pleasure of using discord's AI summaries

now I'm in Discord experiment Tu will

have the invite Link in the description

so you can try it out yourself but

there's this little scroll button up

here and when you click on it you get

this option to look at Discord AI

summaries now what the summaries feature

does is it uses open AI to summarize

your Discord conversations and why am I

saying this feature is an absolute

pleasure to use well here's this example

here a conversation about enabling a fax

printer which if you click on it is just

one message from someone named fax

printer asking how to enable AI is going

to be taking our jobs for sure actually

for Discord that's definitely not the

case because I don't think this AI

summaries feature is going to last for

long because there's a big massive issue

with it and the issue with AI summaries

was found by a very familiar face on

this channel penley now I could be

really lazy and boring and just read off

of the screenshot but I've had two cups

of coffee today I'm feeling it so I'm

going to show you how this exploit Works

in practice so when you're on a Discord

server and you get these AI summaries

it's actually something called a get

request to discord's API it's like me

ringing up Discord servers and saying

yeah I need some summaries for this

specific Channel but in these summaries

we can see that there is a topic a

summary short message IDs and people and

this actually corresponds to the

information shown in the summaries tab

we can see that the title is

impersonation issues the topic

impersonation issues well like I said we

can also see message IDs and we can also

see the user IDs of everyone that is in

the summary so let's say that Pokemon

deletes their message if you go to the

AI summaries tab they should not show up

as an author anymore in fact every

single person could delete their message

except one there has to be one remaining

message but if everyone deleted their

message they will not show up as authors

which is fine except if you go to

insomnia or you just use anything and

you send a get request to Discord if

they delete all of their messages you

will still be able to see their message

IDs and the people that have partak in

the summary so let's say there's an AI

summary of someone talking about them

coming out of the closet now whoever was

chatting about that they realize maybe

we should delete our messages so that

people don't harass us so they delete

all their messages talking about coming

out of the closet but they just forget

one so that it still remains in the AI

summaries well let's say a homophobe

joins the Discord server and they see a

summary talking about someone coming out

of the closet even with the messages

deleted they could still go to insomnia

send a get request and figure out

everyone that talked about about the

thread which means that the homophobe

could see everyone involved in the

thread and harass every single one of

them so that is how this security issue

could affect you personally but things

get even worse because this whole entire

security issue has legal repercussions

now let's look at the rules by dran Van

strangle but in discord's terms of

service they have a privacy policy now

this whole document basically dictates

what Discord can and cannot do with your

personal information and if you scroll

down long enough there's a very

important section you actually allowed

to edit and delete specific pieces of

information within Discord services this

is like a fundamental Rule and the

fundamental rule is that you can delete

any message that you have sent on

Discord if you have posted it and you

still have access to the server or the

space that you posted it what this means

is that if you delete a message on

Discord it should be deleted your

personal information should not be

attached to it in any way whatsoever

your Discord user ID should all be wiped

unless of course you've done something a

little bit naughty so when what

scenarios would Discord store your

message even after you delete it well

there's two main scenarios the first one

is to comply with the law for example if

you're planning a domestic terrorism

attack on Discord Discord would save

your messages even if you deleted it

assuming of course they actually figure

you out because they are required by law

or if they get some sort of legal notice

but the second scenario that Discord

would store your messages is if you're

reporting someone so if you go through

the report message prompts and you go

through everything Discord will store

this message so that they can actually

ban the person even after after you

delete this message it's still going to

be stored by Discord but here's the

thing even with these AI summaries if

you delete your message your user ID

will still pop up that is your

information and absolutely nowhere in

this privacy policy I promise you

absolutely nowhere it says that a

like me some random Discord user can get

some of the information of your message

discord's own AI summaries is breaking

discord's own privacy policy now I'm not

a lawyer but I'm not sure about the

legality of of all this and I want to

point out that this has been an issue

since May

2023 and it still has not been patched

at this time and what will Discord do

about it well we need a little bit of

backstory because here's the thing every

single tech company on the planet loves

Ai and Discord is no exception but

Discord is the exception in terms of

complete and utter failure because in

this blog post talking about AI Discord

announces a handful of AI features we

first have Clyde Clyde is discord's AI

chatbot Clyde says racial slurs Clyde

tried to date underage people and if

you're not up to date on the Discord

news Clyde has been cancelled our next

feature automod AI the way that it works

is that it'd use AI to moderate your

Discord server and see whether or not

people are breaking the rules I've not

seen any update to discord's automod AI

whatsoever this feature is basically

dead in the water we have Avatar remix

which is just a Discord bot so nothing

crazy there we have whiteboard with AI

preview which is where you're supposed

to be able to Draw Something in the

Whiteboard and uh turn it into a magical

AI image it doesn't exist in Discord so

the final AI feature of Discord is

conversation summaries it is an absolute

hot mess of problems for Discord and I'm

99% sure I am betting my left nut on

this one that Discord is going to cancel

AI summaries just like how they canceled

every single AI feature that they tried

to release W wamp and now that the cat

is out of the bag with this problem

chances are Discord is going to look at

this summaries AI feature and just

cancel it completely which means I get

to save my left nut so this is a big win

for me now one final thing I want to

talk about in terms of these AI

summaries leaking your user ID is that

Discord will actually pay you money if

you find bugs like this Discord has a

bug Bounty and fortunately for us panle

told us how much they got paid

$750 which is a pretty fair chunk of

change you could actually buy 42

emergency fire blankets to put out the

giant dumpster fire of a problem that AI

summaries are Discord take notes here

any hoot Gamers That's All She Wrote a

pretty nerdy video and somehow if you

made it this far then let me give you a

kiss I love you sweetheart