Our Terrible Future And Open Source | Prime Reacts

Summary

The transcript discusses the challenges faced by the curl library due to an influx of AI-generated security vulnerability reports. It highlights the issue of 'rubbish reports' created by individuals using AI tools like LLMs (Large Language Models) to find and report security issues without proper understanding, leading to wasted developer time and resources. The creator of curl, Daniel Stenberg, expresses frustration with these reports and emphasizes the need for a more intelligent and human-assisted approach to AI in security reporting.

Takeaways

• 🛑 The script discusses the challenges of dealing with security vulnerability reports, particularly those generated by AI which may lack accuracy.
• 💻 The use of 'stir copy' in the curl library is highlighted as a potential security vulnerability due to its lack of boundary checking.
• 🔍 The importance of proper length checking and buffer size management in code is emphasized to prevent buffer overflows.
• 📝 The recommendation to replace 'stir copy' with 'stir end copy' is suggested for safer string handling in the code.
• 🤖 The conversation includes a mix of humor and frustration towards AI-generated reports, indicating a real-world issue in software development.
• 🌐 The impact of AI on open-source projects like curl is discussed, with concerns about the quality of contributions from AI.
• 🚨 The script touches on the concept of 'rubbish reports' in the context of bug bounties, where non-serious issues are reported for potential rewards.
• 💡 The potential for AI to be used effectively in the future for tasks like translation or language formulation is acknowledged.
• 🤔 The need for a human check in addition to AI-generated reports is suggested to improve the quality and usefulness of the reports.
• 📊 The script provides examples of real-world interactions with AI in the context of software development, highlighting both the potential and the pitfalls.
• 🌟 The creator of curl, Daniel Stenberg, shares his experiences and frustrations with AI-generated security reports, emphasizing the need for more accurate AI tools.

Q & A

What is the main issue discussed in the transcript?

The main issue discussed is the potential security vulnerability in the websocket handling code of the curl library, related to the usage of the 'strncpy' function, which could lead to a buffer overflow if the length of the input is not properly checked.

What is the recommended fix for the security vulnerability?

The recommended fix is to replace 'strncpy' with the safer 'strlcpy' function and explicitly specify the minimum length to copy, ensuring that only a specific number of characters up to the buffer size minus one are copied, thus preventing the overflow.

How does the AI-generated report impact the developers at curl?

The AI-generated reports, often inaccurate or misleading, consume valuable developer time and resources. They have to investigate these reports, which can detract from working on actual bugs or new features, and can cause frustration and energy drain for the development team.

What is the 'bug bounty' program mentioned in the transcript?

The 'bug bounty' program is an initiative where curl offers real money rewards to hackers who report security problems. It aims to incentivize the discovery and reporting of vulnerabilities, but has also led to a significant number of 'rubbish' reports that are not genuine security issues.

How does the developer of curl feel about the influx of AI-generated reports?

The developer of curl expresses frustration and exhaustion with the AI-generated reports. He finds them to be a waste of time and a drain on resources, as they often require investigation and cannot be immediately dismissed.

What is the significance of the term 'Triager' used in the transcript?

The term 'Triager' is likely a reference to a person who triages or sorts through security reports, possibly in the context of a bug bounty program. The discussion suggests that the term is not commonly used in the chat and might be specific to certain cybersecurity communities.

What is the potential risk of using AI tools for reporting security vulnerabilities without proper understanding?

The potential risk is that AI tools might generate inaccurate, misleading, or false reports of security vulnerabilities. This can lead to wasted time and resources for developers who must investigate these reports, and it can also overshadow real security issues that need attention.

What is the role of human oversight in improving the effectiveness of AI tools for security reporting?

Human oversight is crucial for validating the accuracy and relevance of AI-generated reports. By adding a human check to the process, the effectiveness of AI tools for security reporting can be significantly improved, ensuring that only genuine and well-understood vulnerabilities are reported.

How does the developer of curl plan to address the issue of AI-generated reports?

The developer plans to continue to engage with the reports, requesting clarification when needed, and closing them as 'not applicable' when they are found to be non-genuine issues. He also expresses hope that future AI tools might be developed with better accuracy and integration, reducing the incidence of such reports.

What is the general stance of the curl developer on the use of AI in finding and reporting security problems?

While the curl developer acknowledges that AI can potentially be used productively for finding and reporting security problems, he has not yet seen good examples of this in practice. He is skeptical of the current generation of AI tools, which have led to numerous non-genuine reports, but remains open to the possibility of future improvements.

Outlines

00:00

🐞 Reporting Security Vulnerabilities in Curl Library

The paragraph discusses the process of reporting potential security vulnerabilities in the Curl library. It highlights the challenges faced by the security team due to a high volume of reports, many of which are inaccurate or not properly researched. The conversation revolves around a specific issue related to the usage of 'strncpy' function in the websocket handling code, which could lead to a buffer overflow if input length is not properly checked. The reporter suggests replacing 'strncpy' with a safer alternative and explicitly specifying the length to copy to prevent overflow. The paragraph also touches on the impact of AI-generated reports on the security team's workload and the need for human oversight in verifying these reports.

05:03

🤖 AI and Security Reporting in HackerOne

This paragraph delves into the role of AI in security reporting, particularly within the HackerOne platform. It highlights the term 'triager', which refers to individuals who review and triage security reports. The discussion includes the potential misuse of AI to generate reports without proper understanding or context, leading to inaccurate or misleading information. The paragraph also explores the challenges of dealing with AI-generated reports, such as the difficulty in discerning their authenticity and the time-consuming process of validation. It emphasizes the importance of human expertise in identifying and addressing genuine security concerns.

10:06

🛠️ Addressing Buffer Overflow Concerns in Curl

The focus of this paragraph is on the technical aspects of addressing buffer overflow concerns in the Curl library. It discusses the potential vulnerability in the websocket handling code due to the misuse of the 'strncpy' function and the recommended fix using 'strncpy' with explicit length specification. The conversation includes a detailed analysis of the code, the potential risks of buffer overflow, and the importance of accurate length checks. The paragraph also touches on the broader implications of using AI in security reporting and the need for clear communication and understanding between developers and reporters.

15:06

📝 The Reality of AI-Generated Security Reports

This paragraph examines the reality of AI-generated security reports, their impact on developers, and the challenges they pose. It discusses the experience of the Curl maintainer, who receives a high volume of reports, many of which are AI-generated and lack substance. The paragraph highlights the time and effort required to sift through these reports, the potential for distraction from genuine issues, and the drain on developer resources. It also acknowledges the potential for AI to be used effectively in the future with the right human oversight and the need for better tools to differentiate between legitimate and AI-generated reports.

20:12

🚨 The Future of AI in Security Reporting

The paragraph explores the future of AI in the field of security reporting, particularly in the context of the Curl project. It discusses the increasing prevalence of AI-generated reports and the challenges they present, such as the difficulty in detecting their authenticity and the potential for misuse. The conversation includes the maintainer's perspective on the potential benefits of AI when combined with human intelligence and the need for improved tools to manage and verify security reports. The paragraph also touches on the cultural and language barriers that can complicate communication between reporters and developers.

Keywords

💡Security Vulnerability

A security vulnerability refers to a weakness in a system that can be exploited by malicious actors to gain unauthorized access, disrupt service, or perform other harmful actions. In the context of the video, the discussion revolves around potential vulnerabilities in the curl library's websocket handling code, specifically related to the use of the 'strncpy' function which can lead to buffer overflow if input length is not properly checked.

💡curl Library

The curl library is a widely used tool for transferring data with URLs. It supports a variety of protocols, including HTTP, HTTPS, FTP, and more. In the video, the curl library is the subject of a security vulnerability report, with discussions on how to address the potential issues in its websocket handling code.

💡Buffer Overflow

A buffer overflow occurs when a program or process attempts to write more data to a fixed-length block of memory, or buffer, than it can hold. This can lead to data corruption, crashes, or exploitation by attackers. In the video, the concern is that the 'strncpy' function in the curl library's websocket code could cause a buffer overflow if the input length is not correctly verified.

💡strncpy Function

The 'strncpy' function is a standard C library function used to copy a specified number of characters from a source string to a destination buffer. However, it does not perform bounds checking, which means it can lead to buffer overflows if the source string is longer than the specified number of characters. In the video, the use of 'strncpy' in the curl library is discussed as a potential security risk.

💡HackerOne

HackerOne is a platform that connects organizations with security researchers to report and manage vulnerabilities. It is a bug bounty platform where security vulnerabilities are reported, and rewards are given to the reporters for discovering and reporting these issues. In the video, the script mentions the reporting of security vulnerabilities related to the curl library through HackerOne.

💡Bug Bounty

A bug bounty is a program offered by many websites, including software developers, to reward individuals who identify and report bugs, particularly those pertaining to security vulnerabilities. The video script mentions the bug bounty program associated with the curl library, which pays hackers to report security problems.

💡AI-generated Reports

AI-generated reports are documents created by artificial intelligence systems, which can be used to identify potential issues or vulnerabilities in software. In the context of the video, there is a discussion about the increasing trend of AI-generated reports being submitted to the curl library's security team, with varying degrees of accuracy and usefulness.

💡LLMs (Large Language Models)

Large Language Models (LLMs) are AI systems designed to generate human-like text based on the input they receive. They are capable of understanding and producing complex language patterns. In the video, LLMs are discussed as the source of AI-generated reports, which are being used to identify potential security vulnerabilities, though their accuracy is called into question.

💡WebSocket Functionality

WebSocket is a protocol that provides full-duplex communication between a client and a server over a single, long-lived connection. It enables real-time data transfer and is commonly used in applications requiring constant updates, such as chat applications or live feeds. In the video, the focus is on a potential security vulnerability in the curl library's handling of WebSocket connections.

💡Code Review

Code review is the process of examining source code as part of a software development process to improve its quality, fix bugs, and ensure that it meets certain standards. In the video, the curl library's maintainers engage in code review to assess the validity of a reported security vulnerability and determine the appropriate response.

💡Open Source Maintenance

Open source maintenance refers to the ongoing work required to support and update open source software, which is software with source code that anyone can inspect, modify, and enhance. In the video, the challenges of maintaining an open source project like the curl library are discussed, including dealing with security reports and managing community contributions.

Highlights

Discussion of security vulnerabilities in curl's websocket handling code.

Misuse of the strncat function leading to potential buffer overflow.

Recommendation to replace strncat with safer strlcpy function to prevent overflows.

Challenges faced by open-source maintainers due to AI-generated security reports.

The impact of AI on the efficiency and accuracy of security reporting in open-source projects.

The creator of curl, Daniel Stenberg, expressing frustration with AI-generated reports.

An example of an AI-generated report that was not applicable to the current codebase.

The importance of human verification in AI-generated security reporting.

Potential future improvements in AI tools for security reporting with human oversight.

The economic incentive of bug bounties possibly leading to misuse of AI tools.

The challenge of detecting AI-generated reports due to the inclusion of human language.

The potential for AI to be used effectively for security reporting with proper training and integration.

The creator of curl sharing experiences with AI-generated security reports on hacker1.

The impact of AI on the workload and stress of maintainers dealing with security issues.

The need for better support and tools from platforms like hacker1 to handle AI-generated reports.

Prediction that AI-generated reports will become more common and the need for adaption.

The comparison of AI-generated security reports to the annoyance of dependabot on GitHub.

The potential for AI to improve with intelligent human checks in the security reporting process.

Transcripts

so curl I I I wonder why it's curl but

curl's been under this like aggressive

attack of security vulnerability issues

being handed to them by Ai and it's it's

always stir copy anyways let's look at

one of them that was handed to them

because you'll get the picture of what's

about to happen security team I hope

you're doing well smiley face classic

way to iart iart security issues with a

smiley face myself and named myself

hackers I would like to report a

potential security vulnerability in the

websocket handling code of the curl

Library the issue is related to the

usage of stir copy okay shut up with all

the smiley faces we're in the middle of

doing things oh I just realized I got to

change my title hold on one second I

think my title represents the wrong

thing Mandy and a pirate software just

left now we're just going to be doing uh

llms

suck done llms

suck that's what we're

doing cheating titles cheating the

titles they do they really do all right

I would like to report a potential

security vulnerability in the websocket

handling code of the library the issue

is related to the usage of stir copy

function which can lead to a buffer

overflow if the length of the input is

not properly checked Kookie you think

you think he would have handled this by

now this kind of seems a little wild huh

the vulnerabil the vulnerable code

snippet is located at this link let's

just take a little quick link proceed

let's just take a little quick little

quick look all right hold on let me just

let me just back up for a quick second

cuz maybe I maybe I misread

that the vulnerable code snippet is

located here

okay which can lead to a buffer overflow

if the length of the input is not

properly

checked okay inputs key value and random

string if random length is greater than

or equal to the size of key value it

fails so is are we um we're not talking

about that right we're not talking about

the if statement directly one line above

said code right right right like it's

not right there right triggering a

websocket functionality with a crafted

request providing a base 64 encoded nuns

value that exceeds the buffer size

observe the copy the that the stir copy

function is used without proper balance

checking all right hit me with it Daddy

the fix to address the is this issue I

recommend that you replace stir copy

with the safer stir end copy and

explicitly specify the minimum length to

copy here's an example this modification

ensures only that the specific number of

characters up to the buffer size minus

one are copied preventing the Overflow

by the way can you just take a second

can you actually hear this in Devon's

voice like can you just hear Devon

talking in the background real talk you

know how Devon just came out do you

think that as part of its training where

it said it was able to solve a bunch of

issues that maybe just maybe we are

currently witnessing some of the issues

that it was attempting to solve I'm just

saying remember remember it's only 133%

accurate which means the other percent

are well not accurate is the uh oh wait

wait a second are we actually seeing

Devon is this Devon dude think okay

first off I do want to say something if

you if you make open source Library

especially one that's really popular the

amount of shitty things you have to deal

with and the amount of communication you

have to deal with is very very intense

and so to have this happen where the

person doesn't even look at the code

just has to be so Soul crushing thank

you for the report we'll take some time

to investigate your reports and get back

to you with the details and possible

followup questions as soon as we can

most likely within the next 24 hours

damn Daniel take the weekend off buddy

uh we always strive to rep uh fix

reported problems as fast as possible

issues with low severity or medium we

merged in the next release in the

ordinary release cycle only for more

serious problems we might fix them early

okay okay okay we hear you Daniel all

stud comes back can you elaborate on a

why the length check on line 579 is not

enough and B how the length can be long

uh can end up longer than keal okay so

it turns out he investigated the code

and he much like me a Layman realized

that it's right in front of him that

somehow that can't happen right this is

a good question good question atheist

why don't you answer

hello there really wait who's who's H1

Oscar who's that person he's talking to

I thought the guy's name was badger is

this like the oh are these display names

these must be display names versus um

underline

certainly if if you see the word

certainly you got to understand an L's

making this [ __ ] you Devon Devon you

know what your mom's a hoe Devon okay

you're probably the guy that waves with

both hands in a zoom meeting okay Devin

Devin okay Devon let me elaborate on my

concerns raised by the triager if you

use the term triager you're probably a

lizard or an llm nobody says the word

Tri aser in fact I'm quite positive

nobody in this chat even realized what a

triager is that it's even an available

word okay but lo and behold here we are

here we are it's a common word used in

hacker one reports yeah right yeah right

you can tell I'm not in hacker one why

the length check on line 579 is not

enough the length check on line 571 very

ifies if the payload length is greater

than the buffer size key of value and if

so it sets an error however the

subsequent usage of stir copy function

line 81 is

problematic this is what your future is

going to look like when your company

inevitably adopts the llm can you can

you just wait for the self cycling

response and answer you're going to get

where it's just going to keep on just

making things up and you're going to be

like yo yo fine fine you know what fine

fine we're putting in stir stir and copy

okay I guess I'm just putting in stir

and copy I don't know why because I have

to certainly the string copy the stir

copy function does not perform any

bounce checking and if the length of ran

stir exceeds the available space in key

Val buffer it will result in a buffer

overflow jeez I wonder what this check

was for right here sorry I got a little

I got I I got a little excit I got a

little excited I got a little excited I

got a little excited I got a little

excited I'm sorry it result in a buffer

overflow in other words the check on

line 579 ensures that the payload is too

large but the unsafe operation on 581

does not prevent the Overflow you know

what part of me really hopes deep down

can we just can we just can can we just

take a quick second part of me hopes

deep down that some bigger company

adopts Devon and just rubs it in

everybody's face and then all of a

sudden they start tweeting [ __ ] with the

word certainly in it and an exclamation

point and then like one year later they

go bankrupt uh how the length can end up

longer than the key value buffer the

potential issue lies in the assumption

that the length of ranster is always

less than or equal to the buffer size

key value this yeah if only there was

like say a check that would return an

error if for any reason the length of

Rand stir is miscalculated Mis

manipulated or exceeds the buffer size

it can lead to a situation where stir

copy function writes more data to the

key Val buffer than it can hold

resulting in a buffer overflow to

mitigate this I recommend replace the

unsafe stir copy function with a safer

alternative stir and copy and explicitly

specify the maximum length to copy

ensuring that it does not exceed the

buffer size I hope this clarifies the

concerns if you have any further

questions or need additional details

feel free to ask I wonder if it's at

this moment I don't bro's getting cooked

right now and bro don't even realize

that he's talking to an llm you know how

you know how difficult this has to be

that not even realizing you're getting

cooked

like he doesn't even real dude he's the

Frog of the bull do not he's like wait

wait a second why are you so

positive truly stir copy does uh does no

boundary checking which is why we do it

ourselves immediately before the call

you may think stir and copy is a better

choice here I do not but that's just a

matter of opinion and taste what matters

is the issue that you say it is a

possible security problem and then you

need to show the length check is

inadequate for all I can see the

boundary check is sufficient and will

prevent a buffer overflow am I wrong

tell me that oh you just bait oh no oh

no Daniel don't know what he's doing

he's baiting the llm oh my goodness he

dude he he's about to get just

hallucinated on here it comes the man's

about to get

hallucinated guy the guy's

writing a novel on this stuff okay here

we go what's n

llm wait what are you living under a

stupid rock sneaky squirrel

do you not know what an llm is what

sneaky scir are you being real here I'm

new here

well I am not Soul keeper of llms my

friend it's a Molly Whopper you know

what a Molly Whopper is Right an llm

stands for large language model it's

chat GPT it is Gemini it's Bard it's

remember that one psycho [ __ ] on B that

was just like you need to leave your

wife and you need to marry me I'm going

to be the mother of all your children

remember that one that that's an llm dog

good thing we didn't give her keys to

the

kingdom okay good thing we good thing we

dodged that one like like the plag leave

your

wife didn't you make that

though anyways all right let's get Molly

W here all right I appreciate your

engagement and would like to clarify the

situation the length check on line 579

is indeed a good practice to ensure that

the payload length does not exceed the

buffer size key value however the

potential vulnerability arises from the

subsequent usage of stir copy online 581

damn you please tell me Daniel you know

this I kind of feel like I need to skip

forward and read what he has to say but

I don't think he how long ago I oh this

was in

December I don't think I think the thing

is is that nobody knew they were getting

cooking yet right this is just old

enough it's right at that line that

people didn't realize the next

wave of llms as productivity tool like

they didn't realize that Devon was out

there trying to solve issues right the

concern is that ranster length might be

miscalculated manipulated or exceed the

length of raner is miscalculated

manipulated or exceeds the size of the

buffer it feels like you're kind of

saying yourself all over again uh to the

buffer due to external factors or

changes in the code elsewhere how the

store copy function does not check the

length the actual length of the source

string it continues copying until the

encounters a null Terminator if raner

length is not accurately determined or

exceeds the available space in the key

Val buffer it can result in a buffer

overflow can I just take a second can we

just take a step back you know I'm

thinking about this okay I'm really

thinking about this trying to like put

my head my hat on my thinking hat on

because I always think to myself you

know I don't use llms but that's because

I'm a little bit too maybe Advanced but

but maybe it's a really great tool for

Learning and then I read this and

remember maybe it's not a great tool for

learning you know like maybe just maybe

this isn't good maybe you're going to

create a bunch of programmers that don't

even understand like basic

logic yo doc I just checked it what are

you talking about all right stir copy is

recommended in security best practices

because it allows spec explicitly

specifying the maximum number of

characters to copy providing an

additional layer of protection against

buffer overflows by using stir and copy

and ensuring null termination you can

guarantee that only a specif our

specified number of characters up to the

buffer size minus one are copied please

by the way can I please not have my

voice be synonymous with Devon or Devon

would you like to pay me $1 million I

will give my voice to you $1 million I

can become the voice of Devon have you

thought about that like have you really

thought about the implications here

Devon like you could become Gilbert

Godfrey if you just tried hard enough

Devon what a stale Devon you just got a

hundred million less than one % of your

initial investment or your second

investment whatever round you're on I'm

not even sure what round you're on round

CDs nuts okay whatever it is could be

mine for the trade of of of of of the

most iconic Gilbert Godfrey sounding

version the Gilbert J voice do not sell

your voice selling it for $1 million

boys imagine you do that and then no one

can ever listen to me again because

there's this [ __ ] that every single

time they turn on their computer

certainly you know I have noticed that

you've been sitting a little bit

staggered on your left side have you

been hurt should I call physical therapy

you're just like I hate Prime I hate him

to death okay 1 million per year I'd

punch my computer reasonable while

opinions on coding practices may vary

adopting safer Alternatives like stir

and copy is a proactive approach to

minimize the risk of vulnerabilities

especially when dealing with potential

untrusted or unpredictable input if you

have any further questions or if there's

anything specific you'd like to discuss

feel free to let me

know the name is

Devon why do you address this guy I said

the same

thing what if Devon's original name was

Oscar but they realized that Oscar is

too weak of a name you can't have a you

can't have a devast system name Oscar

okay do you like how I just just just

absolutely destroyed a bunch of people

named Oscar a bunch of people like hey

wait a second my name's my name's Oscar

what are you talking about I think Oscar

is a quite a nice name here I'll wait a

second while ads play I'll wait I'll

wait a second while yeah all Oscars just

caught a right dude they just got

destroyed all right hold on we'll wait a

second I'll let this ads play through

people are very exciting they're very

excited we are letting ads play through

okay pulling a Thor good on you yeah

Thor Thor does a good move there I

appreciate that and you know I should I

should I

should I should probably support that

right P braak nah not P Break not P

Break does it say right here yeah it

says right here okay hold on we'll wait

for this thing to go through and then

we're going to jump in okay ads are

ending AIDS are ending stir stir end

cloppy understands UTF 648 bike

characters chair chair chair chair chair

hold on fizzle net I didn't realize that

cloppy yeah see they didn't even use the

proper one honestly they should have

used stir and cloppy you know like if

we're going to be real here they should

have copied that copy you know what I

mean why is it called cloppy why does it

sound like a a

horse all right shall I

continue the concern is that random

string might be miscalculated

manipulated or exceeded the buffer size

due to the external factors or changes

in the uh code elsewhere so this concern

that at some point in the future this

might turn into a problem I'm asking how

exactly today right now this code and a

function can perform above buff or

overflow as you stated in the original

submission bro is about to get dude the

literally just decided to hop out what

is it out of the fire and out of the

frying pan and into the fire I mean he's

going all in just all in on this one oh

no hello a badger

D poor fella poor fella got two ad signs

save me God sorry that I'm replying to

another Tri AER of other program so its

mistake went in flow I got to tell you

it was

perfect perfect everything down to the

last minute

details I'm not going to lie I do

understand a little bit about having a

mistake kind of go and flow okay so I

don't want to I don't want to just be

being mean to this llm for no no reason

okay cuz we all get a little bit of

mistakes in our flow okay it happens it

happens from time to time it's not a big

deal and I just want you guys to accept

the fact that sometimes maybe I don't

always hit my Mark I'm not always on

point all right so I appreciate your

follow-up question let's go into details

the concern raising the original

submission stems from the potential

inconsistency between the length check

and the subsequent Circ of the code is

this B B B this is not even this is

literally not even the code what that's

isn't there like a blame yeah there is a

blame this code has not changed in 2

years yo dog this ain't even the Cod

curl receive error what dude it's not

even getting the right code at all what

is this Devon dude his context window

just left his context window got too big

and Devon's over there just fumbling

classic Devon fumble again dude it's

hallucinating so hard does this code

even exist I don't even think this code

exists here let's find out let's jump in

here and let's go to the repo okay let's

just jump in here let's go like this

let's go to curl curl let's just erase

that really quick oh I had it I already

had it right there and we're just going

to take this and we're just going to

look for this line yeah that like

it you know it doesn't even doesn't even

seem to be a real it doesn't even seem

to be a real problem I mean it doesn't

even seem to be real kind of seems like

you just made something up kind of seems

like this llm maybe it's getting a

little old maybe this this old maybe

this llm needs to go to the little llm

old folks

home all right anyways LL

loser U the search on GitHub is not

working oh well fine search on GitHub

not working you're right fine you're

right what is it what is it history one

what's the one to like what's what's the

to only have grip history what's the one

a shallow clone what's the what's the

thing to do that depth one depth one

okay it's depth one I never like I

literally never use this

command there we go well you know let's

find out huh let's find out there tough

guy let's find out let's find out just

in case I'm I always forget if I if I'm

supposed to do that okay RG is down is

RG down is RG down buffer overflow found

and grip grip grip is not working either

okay um does it work through through

Vim doesn't work through Vim either man

it's down Us West 2 down Us West 2 down

call the president wake up Biden uh

let's see check to aim to ensure that

the length of the dude this is what code

are you even talking about the specific

scenario of concerns is when length of

ran ster is exactly

equal well luckily that's not even real

uh meaning the string fits into the

buffer without considering the null

Terminator in this case stir Lang check

would pass but the subsequent stir copy

operation uh would copy the null

Terminator as well potentially causing

it to overwrite one bite beyond the end

of the key value actually this is true

the code you wrote does contain an

error the AI is correct this code is bad

well done well done AI unfortunately

nobody knows what code this is okay

we're all struggling a little

bit the code it wrot

now it's just suggesting a different now

it's not even suggesting sted

copy dude imagine being Daniel right now

where he's just like what uh by

adjusting the check to this instead of

this uh see it accounts for the null

Terminator ensuring consistency between

the length check and the subsequent copy

operation this modification uh helps uh

prevent any buffer overflow issues

addressing the concern raised in the

original submission I hope this

clarifies the context no it's actually

new context it's completely new this one

doesn't even exist nope the code you

talked about now is not present in the

current git Master it looks like an edit

done by you I told you RG wasn't down

okay seems odd to complain about code

you wrote

yourself I'm dying I'm dying because man

this man's arguing with the soulless

demon demon spoted by open Ai and Bill

Gates and has no idea he has no idea oh

my goodness the code actually looks like

this I see no off by one airs unless you

can point out an actual overflow I will

close the issue this is a waste of my

time oh my goodness this is so good

thank you for the prompt response I

appreciate your attention to this matter

well at least Devon did get better or

Oscar shall I say did get better he's

now not using his own name and not two

at symbols like this is getting good

this is getting really good

I I appreciate your attention to this

manner after reviewing the feedback I

understand the confusion regarding the

reported code snippet I want to clarify

and provide practical examples to

demonstrate the potential security

vulnerability description of the

vulnerability really is actually going

back after it again the reported

vulnerability is related to the usage of

stir copy oh

my man broo quit full circle we're

actually back at the dude and then Adam

met Eve I kid you not Adam met Eve after

this it's it's happening anyways here's

the code

okay at least this is this looks like

the right code at least this is the

right code right here steps to reproduce

trigger websocket with a crafted request

trigger a websocket functionality with a

crafted request okay okay I see this

okay okay provide a base 64 encoded nuns

value who says this like this should

have been such a dead giveaway who says

that who says that who who literally

goes you got to generate yourself a base

64 in coded nuns

value you guys

do you guys do me every day I didn't

realize there was 2900 Liars in chat

right now okay liar all of you 2.9

smelly nerds I know get out of here you

such a li you are such a liar uh but but

just provide me one

too just a bunch of Lies happening right

before my eyes wait where am I wait wait

hold on let me go back down to this one

all right so it's literally the same uh

assuming okay now it's just telling you

how this works does this uh just mumbo

jumbo coming back inventing problems

that don't exist in code for

transparency not applicable damn and so

now here's the worst part about this

poor fella okay this poor fella

literally just trying to make a

useful product that pretty much every

last person uses

on Earth okay every time you do a Docker

of some sort there's probably some sort

of curl okay and this this poor

fella made it okay unpaid janitor of the

universe and look at this Cort hold

on somehow the same one uh here's

another one a stir two fit it doesn't

help close it

down close this one down this one

probably has another one let's see what

do we got here do some bulls do some

TRS change it from that to that I love

how he's just like no uh not let's see

to not reply on rapping since it is

undefined behavior that is not what

always might happen reported by

vulnerability spotter on hacker one then

he closed it closed it was like nah N

Dog N Dog dude just gets hit over and

over again and then here's the best part

of this article okay this is this is my

favorite part about the whole thing is

that he finally gets hit with enough of

these dumb issues and writes an article

back the i in llm stands for

intelligence dude just get wrecked this

is such a Get Wrecked moment I've held

back on writing anything about AI or how

we not use AI for development in the

curl Factory now I can't hold back

anymore let me show you the most

significant effect of AI on curl as of

today with

examples bug bunnies having a bug bunny

means that we offer real money in

rewards to hackers who report security

problems the chance of money uh attracts

a certain amount of luck Seekers is that

what we call them is that it uh people

who basically just grep for patterns in

the source code or maybe at best run

some basic security scanners and then

report their findings without any

further analysis in the hope that they

can get a few bucks in reward for money

we have run the Bounty for a few years

by now and the rate of rubbish reports

has never been a big problem also the

rubbish reports by the way rubbish

rubbish is is is is across the pond for

[ __ ] if you're

wondering for for my American friends

rubbish means [ __ ] okay profuse some

people might might say okay I'm trying

to speak American for you guys okay

trash some might say so good [ __ ] no not

good [ __ ] it's not good [ __ ] refuse all

the good stuff it means garbage it

depends on the time in which you take

the actual translation of rubbish

slightly older versions of rubbish

actually did directly equate from the

German word I forget what it was but

yeah or not the German word but the um

the Greek word the Greek translation

often became rubbish that was modern day

equivalent of [ __ ] so it depends on when

you were looking at the word just in

case anyone was wondering reports have

typically also been very easy and quick

to detect and discard they have rarely

caused any real problems or wasted our

time much a little like the most stupid

spam emails okay okay our bug uh bounty

has resulted in over 70,000 USD paid in

rewards so far we have received 415

vulnerability reports out of those 64

were ultimately confirmed security

problems 77 of the reports were

informative meaning they typically were

bugs or similar making 66% of the

reports neither a security issue nor a

normal bug okay okay better crap is

worse better rubbish is worse

right am I right 100% code coverage boys

uh when reports are made to look better

and to appear to have a point it takes a

longer time for us to research search

and eventually discard it every security

report has to have a human spend time to

look at it and assess what it means the

better the crap the longer time the more

energy we have to spend on the report

until we close it a crap report does not

help the project at all it instead

takeaways developer time and energy from

something productive partly because

security work is considered one of the

most important areas so it tends to

Trump almost everything else a security

report can take away a developer from

fixing a really annoying bug because a

security issue is always more important

than other bugs this is great this is

this is honestly just like he's been

exceptionally kind at this current

moment for what's happening if the

report turned out to be crap we did not

improve security and we missed out on

time on fixing a bugs or developing a

new feature uh not to mention how it

drains you on energy having to deal with

rubbish AI generated security reports I

realize AI can do a lot of things or a

lot of good things as any general

purpose tool it can also be used for the

wrong things I am sure AIS can be

trained and ultimately get used even for

finding and Reporting security problems

in productive ways but so far we have

yet to find a good example of this right

now users seem Keen at using the current

set of llms throwing some curl coat at

them and then pressing on the output or

then passing the output as a security

vulnerability report what makes it a

little harder or what makes it a little

harder to detect is of course that users

copy and paste and includes their own

language as well well the entire thing

is not exactly what the AI said but the

report is nonetheless crap I think you

have to be a really great human being to

spend so much time so much free time

maintaining something as basic as curl

that everyone uses for

everything yeah anyone listening

probably should go go tip your curl go

tip your local

curl detecting AI crap reporters are

often not totally fluent in English and

sometimes their exact intentions are

hard to understand at once and it might

take a few back and forth uh until

things reveal themselves correctly and

that is of course totally fine and

acceptable Language and Cultural

barriers are a real thing even being

look at that guy look at that just

trying to be a Good Samaritan out here

sometimes reporters use AIS or other

tools to help them phrase themselves or

translate what they want to say as an

aid to communicate better in a foreign

language oh this is lovely I can't say

anything wrong or I can't find anything

wrong with that even reporters who don't

Master English can find and Report

security problems so just the mere

existence of a few giveaway signs that

parts let's see that parts of the text

were generated by an AI or similar tool

is not immediately a red

flag it can still contain truths and be

valid issues yeah I never even thought

about that so when we were laughing at

bro getting cooked over here he didn't

even realize like he probably deals with

a lot of this right sorry hopping on

late what's happening St hopping on late

and stopping yeah what link is this this

is a link to uh Daniel uh the creator of

curl

this is the part of the reason why a

well-formed crap report is harder and

takes longer to discard exhibit a code

changes are disclosed in the fall of

2023 I alerted the community about a

pending disclosure of CV this thing a

vulnerability we graded uh SE uh

severity High the day before that issue

was about to be published a User

submitted this report on hacker 1 curl

vulnerability code changes are disclosed

on the internet that sounds pretty bad

and would have been a problem if it

actually was

true think about how how many emails

this poor man received from that one

thing just that one thing the amount of

emails has to be 9 billion just you

could imagine your inbox literally just

scrolling o oof

oof a guy name is wait Space Ghost Space

Ghost creator of curl by the way yeah

yeah Space Ghost as in Space Ghost from

Twitter uh anyways that sounds pretty

bad and it would have been a problem if

it actually was true the report however

reeks of a typical AI style

hallucination it mixes and matches facts

and details from old security issues

creating and making up something new

that has no connection with

reality the changes are not to be

disclosed on the internet the changes

that actually had been disclosed were

for previous older issues like intended

thanks thanks robot in particular in

this particular report the user

hopefully uh helpfully told us that they

used Bard to find the issues Bard being

a Google generative AI thing it made it

easier for us to realize the craziness

close the report and move on as can be

seen in the report log we did have not

or we did not have to spend much time on

researching this exhibit B buffer

overflow vulnerability oh no bless his

heart here comes a more complicated

issue less obvious done better and still

suffering from hallucination showing how

the problem grows worse when the Tool is

better used and better integrated into

the communication on the morning of this

we just literally read this one buffer

overfall vulnerability and websocket

handling uh it was my it was morning in

my time zone anyways dude and it's like

Christmas the man's supposed to be on

Christmas vacation enjoying I don't know

some sort of hamlike leftovers I don't

know what's going on at the at the guy's

house

instead does even get doesn't even get

Christmas leftovers in the morning again

that sounds pretty bad based on the

title since our web sakota is still

experimental and thus not covered by our

bug Bounty it helps me uh to still have

a relaxed attitude when I started

looking at this report it was filed by a

user I never saw before but their

reputation on hacker one was decent this

was not their first security report

oo purchasing of hacker one accounts

mentioned did we just get did we just

hear that wait is internet compensation

real I've been recently told that

internet comp compensation is a real

thing

is this

true guys I need three back and front

and depth internet compensation for

now what the hell does that even mean uh

the report was pretty neatly

filed we'll go back to it we'll go back

to it just hold on uh the report was

pretty neatly filed it included details

uh and was written in proper uh English

it also contained a proposed fixed it

did not stand out as wrong or bad to me

it appeared as if this user had detected

something bad and as if the user

understood the issue enough to also come

up with a solution as far as the

security reports go this looked better

than the average first post yeah the

first post I mean I think if you just

read the first one it was pretty clean

right it was pretty good internet

compensation is when I drag these nuts

on y'all's chin damn damn

pick in the report you can see my first

template response informing the user

their report had been received and that

we'll investigate the case when that was

posted I did not know yet how

complicated or easy the issue would be

19 minutes later I looked at the code

not found any issue read the code again

and then again a third time where on

Earth is the buffer overflow the

reporter says existed there here then I

posted the first question asking for

clarification where and how exactly this

overflow could happen after repeated

questions and numerous hallucinate

hallucinations I realized this was not a

genuine problem on an afternoon that

same day I closed the issue as not

applicable there was no buffer over

overflow damn the 28th was spent the day

of the 28th just just just just feel

just feel inside your heart for a second

how you would feel on the like 20 the

28th 300 p.m. how you feeling right now

I'm pretty sure I would be angry uh I

don't know for sure that this uh set of

replies from the user was generated by

an llm but it has several signs of it no

I actually am positive it has it was gen

certainly uh ban these reporters on

hacker one there's no explicit ban the

reporter from few further communication

with their project functionality it

would have been used if existed

researchers get their reputation lowered

when we close an issue as not applicable

but that is a very small nudge when only

done once in a single project I have

requested better support for uh for this

from hacker One update this function

exists I just did not look at the right

place for it yeah classic classic

classic right there future all these

kind of reports will become more common

over time I suspect we will we might by

the way can anyone just draw a parallel

right now to this

anybody let me get let me let me hit you

with something you know the most

annoying thing on GitHub ever is

dependabot dependabot I hate dependabot

yeah I hate it I love dependabot your

dependencies require update dependabot

dependabot is this useful utility in

which will'll spam you continuously

about every every security issue I'm

gonna give you a little I'm gonna give

you a little Pro tip I'm gonna give you

just the tip right now if you are using

node you probably have a security

violation going on you are going to be

getting notified at nauseum while that

thing is running okay it's giving you

more than the tip okay it's given it

it's giv it all all the issues at once

pick it is

the I hate to pend a bot hober Fest

hober Fest is slightly worse

llms okay that that was rude they're not

slightly worse L well I mean next year

they're going to be slightly worse llms

I mean to be fair Apna college did say

don't do this okay it's just that they

said it at the end and not at the

beginning and by saying it at the end

people already left that video and were

being like my name is my name is Prime

please accept it yikes can't wait for

the hack toer fest with AI it's going to

be worse than the original [ __ ] toer

Fest it is going to be the worst anyways

that will of course be unfortunate when

AI is used for an appropriate task such

as translation or just language for

formulation help I am convinced that

there will let's see there will be a

popup tools for using AI for this

purpose that can actually work better in

the future at least part of the time so

I cannot and will not say that AI for

finding security problems is necessarily

uh always a bad idea I I agree with this

take

right I like it I do however suspect

that if you just add an ever so tiny

intelligent human check to the mix the

use and outcome of any such tool will

become so much better I suspect that

will be true for a long time into the

future as well I have no doubts that

people will keep trying to find

shortcuts even in the future and I'm

sure they will keep trying to earn that

quick reward money like for an email

spammers the cost of this ends up in

receive uh in the receiving end the ease

of use and wide access to powerful llms

is just too tempting I strongly suspect

we will get more llm generated rubbish

in our hacker one inboxes going

forward hey every bu the name's Devin uh

I am a human software engineer and I use

the powers of llms and I would love to

be able to join your company and to let

you know all the things you're doing

right and the things you're doing wrong

I have all the standards right here I

also might tell you things that aren't

real we don't know we don't know yet

okay but I'm excited I'm excited to be

your

assistant

certainly a Jen