Our Terrible Future And Open Source | Prime Reacts
Summary
TLDRThe transcript discusses the challenges faced by the curl library due to an influx of AI-generated security vulnerability reports. It highlights the issue of 'rubbish reports' created by individuals using AI tools like LLMs (Large Language Models) to find and report security issues without proper understanding, leading to wasted developer time and resources. The creator of curl, Daniel Stenberg, expresses frustration with these reports and emphasizes the need for a more intelligent and human-assisted approach to AI in security reporting.
Takeaways
- π The script discusses the challenges of dealing with security vulnerability reports, particularly those generated by AI which may lack accuracy.
- π» The use of 'stir copy' in the curl library is highlighted as a potential security vulnerability due to its lack of boundary checking.
- π The importance of proper length checking and buffer size management in code is emphasized to prevent buffer overflows.
- π The recommendation to replace 'stir copy' with 'stir end copy' is suggested for safer string handling in the code.
- π€ The conversation includes a mix of humor and frustration towards AI-generated reports, indicating a real-world issue in software development.
- π The impact of AI on open-source projects like curl is discussed, with concerns about the quality of contributions from AI.
- π¨ The script touches on the concept of 'rubbish reports' in the context of bug bounties, where non-serious issues are reported for potential rewards.
- π‘ The potential for AI to be used effectively in the future for tasks like translation or language formulation is acknowledged.
- π€ The need for a human check in addition to AI-generated reports is suggested to improve the quality and usefulness of the reports.
- π The script provides examples of real-world interactions with AI in the context of software development, highlighting both the potential and the pitfalls.
- π The creator of curl, Daniel Stenberg, shares his experiences and frustrations with AI-generated security reports, emphasizing the need for more accurate AI tools.
Q & A
What is the main issue discussed in the transcript?
-The main issue discussed is the potential security vulnerability in the websocket handling code of the curl library, related to the usage of the 'strncpy' function, which could lead to a buffer overflow if the length of the input is not properly checked.
What is the recommended fix for the security vulnerability?
-The recommended fix is to replace 'strncpy' with the safer 'strlcpy' function and explicitly specify the minimum length to copy, ensuring that only a specific number of characters up to the buffer size minus one are copied, thus preventing the overflow.
How does the AI-generated report impact the developers at curl?
-The AI-generated reports, often inaccurate or misleading, consume valuable developer time and resources. They have to investigate these reports, which can detract from working on actual bugs or new features, and can cause frustration and energy drain for the development team.
What is the 'bug bounty' program mentioned in the transcript?
-The 'bug bounty' program is an initiative where curl offers real money rewards to hackers who report security problems. It aims to incentivize the discovery and reporting of vulnerabilities, but has also led to a significant number of 'rubbish' reports that are not genuine security issues.
How does the developer of curl feel about the influx of AI-generated reports?
-The developer of curl expresses frustration and exhaustion with the AI-generated reports. He finds them to be a waste of time and a drain on resources, as they often require investigation and cannot be immediately dismissed.
What is the significance of the term 'Triager' used in the transcript?
-The term 'Triager' is likely a reference to a person who triages or sorts through security reports, possibly in the context of a bug bounty program. The discussion suggests that the term is not commonly used in the chat and might be specific to certain cybersecurity communities.
What is the potential risk of using AI tools for reporting security vulnerabilities without proper understanding?
-The potential risk is that AI tools might generate inaccurate, misleading, or false reports of security vulnerabilities. This can lead to wasted time and resources for developers who must investigate these reports, and it can also overshadow real security issues that need attention.
What is the role of human oversight in improving the effectiveness of AI tools for security reporting?
-Human oversight is crucial for validating the accuracy and relevance of AI-generated reports. By adding a human check to the process, the effectiveness of AI tools for security reporting can be significantly improved, ensuring that only genuine and well-understood vulnerabilities are reported.
How does the developer of curl plan to address the issue of AI-generated reports?
-The developer plans to continue to engage with the reports, requesting clarification when needed, and closing them as 'not applicable' when they are found to be non-genuine issues. He also expresses hope that future AI tools might be developed with better accuracy and integration, reducing the incidence of such reports.
What is the general stance of the curl developer on the use of AI in finding and reporting security problems?
-While the curl developer acknowledges that AI can potentially be used productively for finding and reporting security problems, he has not yet seen good examples of this in practice. He is skeptical of the current generation of AI tools, which have led to numerous non-genuine reports, but remains open to the possibility of future improvements.
Outlines
π Reporting Security Vulnerabilities in Curl Library
The paragraph discusses the process of reporting potential security vulnerabilities in the Curl library. It highlights the challenges faced by the security team due to a high volume of reports, many of which are inaccurate or not properly researched. The conversation revolves around a specific issue related to the usage of 'strncpy' function in the websocket handling code, which could lead to a buffer overflow if input length is not properly checked. The reporter suggests replacing 'strncpy' with a safer alternative and explicitly specifying the length to copy to prevent overflow. The paragraph also touches on the impact of AI-generated reports on the security team's workload and the need for human oversight in verifying these reports.
π€ AI and Security Reporting in HackerOne
This paragraph delves into the role of AI in security reporting, particularly within the HackerOne platform. It highlights the term 'triager', which refers to individuals who review and triage security reports. The discussion includes the potential misuse of AI to generate reports without proper understanding or context, leading to inaccurate or misleading information. The paragraph also explores the challenges of dealing with AI-generated reports, such as the difficulty in discerning their authenticity and the time-consuming process of validation. It emphasizes the importance of human expertise in identifying and addressing genuine security concerns.
π οΈ Addressing Buffer Overflow Concerns in Curl
The focus of this paragraph is on the technical aspects of addressing buffer overflow concerns in the Curl library. It discusses the potential vulnerability in the websocket handling code due to the misuse of the 'strncpy' function and the recommended fix using 'strncpy' with explicit length specification. The conversation includes a detailed analysis of the code, the potential risks of buffer overflow, and the importance of accurate length checks. The paragraph also touches on the broader implications of using AI in security reporting and the need for clear communication and understanding between developers and reporters.
π The Reality of AI-Generated Security Reports
This paragraph examines the reality of AI-generated security reports, their impact on developers, and the challenges they pose. It discusses the experience of the Curl maintainer, who receives a high volume of reports, many of which are AI-generated and lack substance. The paragraph highlights the time and effort required to sift through these reports, the potential for distraction from genuine issues, and the drain on developer resources. It also acknowledges the potential for AI to be used effectively in the future with the right human oversight and the need for better tools to differentiate between legitimate and AI-generated reports.
π¨ The Future of AI in Security Reporting
The paragraph explores the future of AI in the field of security reporting, particularly in the context of the Curl project. It discusses the increasing prevalence of AI-generated reports and the challenges they present, such as the difficulty in detecting their authenticity and the potential for misuse. The conversation includes the maintainer's perspective on the potential benefits of AI when combined with human intelligence and the need for improved tools to manage and verify security reports. The paragraph also touches on the cultural and language barriers that can complicate communication between reporters and developers.
Mindmap
Keywords
π‘Security Vulnerability
π‘curl Library
π‘Buffer Overflow
π‘strncpy Function
π‘HackerOne
π‘Bug Bounty
π‘AI-generated Reports
π‘LLMs (Large Language Models)
π‘WebSocket Functionality
π‘Code Review
π‘Open Source Maintenance
Highlights
Discussion of security vulnerabilities in curl's websocket handling code.
Misuse of the strncat function leading to potential buffer overflow.
Recommendation to replace strncat with safer strlcpy function to prevent overflows.
Challenges faced by open-source maintainers due to AI-generated security reports.
The impact of AI on the efficiency and accuracy of security reporting in open-source projects.
The creator of curl, Daniel Stenberg, expressing frustration with AI-generated reports.
An example of an AI-generated report that was not applicable to the current codebase.
The importance of human verification in AI-generated security reporting.
Potential future improvements in AI tools for security reporting with human oversight.
The economic incentive of bug bounties possibly leading to misuse of AI tools.
The challenge of detecting AI-generated reports due to the inclusion of human language.
The potential for AI to be used effectively for security reporting with proper training and integration.
The creator of curl sharing experiences with AI-generated security reports on hacker1.
The impact of AI on the workload and stress of maintainers dealing with security issues.
The need for better support and tools from platforms like hacker1 to handle AI-generated reports.
Prediction that AI-generated reports will become more common and the need for adaption.
The comparison of AI-generated security reports to the annoyance of dependabot on GitHub.
The potential for AI to improve with intelligent human checks in the security reporting process.
Transcripts
so curl I I I wonder why it's curl but
curl's been under this like aggressive
attack of security vulnerability issues
being handed to them by Ai and it's it's
always stir copy anyways let's look at
one of them that was handed to them
because you'll get the picture of what's
about to happen security team I hope
you're doing well smiley face classic
way to iart iart security issues with a
smiley face myself and named myself
hackers I would like to report a
potential security vulnerability in the
websocket handling code of the curl
Library the issue is related to the
usage of stir copy okay shut up with all
the smiley faces we're in the middle of
doing things oh I just realized I got to
change my title hold on one second I
think my title represents the wrong
thing Mandy and a pirate software just
left now we're just going to be doing uh
llms
suck done llms
suck that's what we're
doing cheating titles cheating the
titles they do they really do all right
I would like to report a potential
security vulnerability in the websocket
handling code of the library the issue
is related to the usage of stir copy
function which can lead to a buffer
overflow if the length of the input is
not properly checked Kookie you think
you think he would have handled this by
now this kind of seems a little wild huh
the vulnerabil the vulnerable code
snippet is located at this link let's
just take a little quick link proceed
let's just take a little quick little
quick look all right hold on let me just
let me just back up for a quick second
cuz maybe I maybe I misread
that the vulnerable code snippet is
located here
okay which can lead to a buffer overflow
if the length of the input is not
properly
checked okay inputs key value and random
string if random length is greater than
or equal to the size of key value it
fails so is are we um we're not talking
about that right we're not talking about
the if statement directly one line above
said code right right right like it's
not right there right triggering a
websocket functionality with a crafted
request providing a base 64 encoded nuns
value that exceeds the buffer size
observe the copy the that the stir copy
function is used without proper balance
checking all right hit me with it Daddy
the fix to address the is this issue I
recommend that you replace stir copy
with the safer stir end copy and
explicitly specify the minimum length to
copy here's an example this modification
ensures only that the specific number of
characters up to the buffer size minus
one are copied preventing the Overflow
by the way can you just take a second
can you actually hear this in Devon's
voice like can you just hear Devon
talking in the background real talk you
know how Devon just came out do you
think that as part of its training where
it said it was able to solve a bunch of
issues that maybe just maybe we are
currently witnessing some of the issues
that it was attempting to solve I'm just
saying remember remember it's only 133%
accurate which means the other percent
are well not accurate is the uh oh wait
wait a second are we actually seeing
Devon is this Devon dude think okay
first off I do want to say something if
you if you make open source Library
especially one that's really popular the
amount of shitty things you have to deal
with and the amount of communication you
have to deal with is very very intense
and so to have this happen where the
person doesn't even look at the code
just has to be so Soul crushing thank
you for the report we'll take some time
to investigate your reports and get back
to you with the details and possible
followup questions as soon as we can
most likely within the next 24 hours
damn Daniel take the weekend off buddy
uh we always strive to rep uh fix
reported problems as fast as possible
issues with low severity or medium we
merged in the next release in the
ordinary release cycle only for more
serious problems we might fix them early
okay okay okay we hear you Daniel all
stud comes back can you elaborate on a
why the length check on line 579 is not
enough and B how the length can be long
uh can end up longer than keal okay so
it turns out he investigated the code
and he much like me a Layman realized
that it's right in front of him that
somehow that can't happen right this is
a good question good question atheist
why don't you answer
hello there really wait who's who's H1
Oscar who's that person he's talking to
I thought the guy's name was badger is
this like the oh are these display names
these must be display names versus um
underline
certainly if if you see the word
certainly you got to understand an L's
making this [Β __Β ] you Devon Devon you
know what your mom's a hoe Devon okay
you're probably the guy that waves with
both hands in a zoom meeting okay Devin
Devin okay Devon let me elaborate on my
concerns raised by the triager if you
use the term triager you're probably a
lizard or an llm nobody says the word
Tri aser in fact I'm quite positive
nobody in this chat even realized what a
triager is that it's even an available
word okay but lo and behold here we are
here we are it's a common word used in
hacker one reports yeah right yeah right
you can tell I'm not in hacker one why
the length check on line 579 is not
enough the length check on line 571 very
ifies if the payload length is greater
than the buffer size key of value and if
so it sets an error however the
subsequent usage of stir copy function
line 81 is
problematic this is what your future is
going to look like when your company
inevitably adopts the llm can you can
you just wait for the self cycling
response and answer you're going to get
where it's just going to keep on just
making things up and you're going to be
like yo yo fine fine you know what fine
fine we're putting in stir stir and copy
okay I guess I'm just putting in stir
and copy I don't know why because I have
to certainly the string copy the stir
copy function does not perform any
bounce checking and if the length of ran
stir exceeds the available space in key
Val buffer it will result in a buffer
overflow jeez I wonder what this check
was for right here sorry I got a little
I got I I got a little excit I got a
little excited I got a little excited I
got a little excited I got a little
excited I'm sorry it result in a buffer
overflow in other words the check on
line 579 ensures that the payload is too
large but the unsafe operation on 581
does not prevent the Overflow you know
what part of me really hopes deep down
can we just can we just can can we just
take a quick second part of me hopes
deep down that some bigger company
adopts Devon and just rubs it in
everybody's face and then all of a
sudden they start tweeting [Β __Β ] with the
word certainly in it and an exclamation
point and then like one year later they
go bankrupt uh how the length can end up
longer than the key value buffer the
potential issue lies in the assumption
that the length of ranster is always
less than or equal to the buffer size
key value this yeah if only there was
like say a check that would return an
error if for any reason the length of
Rand stir is miscalculated Mis
manipulated or exceeds the buffer size
it can lead to a situation where stir
copy function writes more data to the
key Val buffer than it can hold
resulting in a buffer overflow to
mitigate this I recommend replace the
unsafe stir copy function with a safer
alternative stir and copy and explicitly
specify the maximum length to copy
ensuring that it does not exceed the
buffer size I hope this clarifies the
concerns if you have any further
questions or need additional details
feel free to ask I wonder if it's at
this moment I don't bro's getting cooked
right now and bro don't even realize
that he's talking to an llm you know how
you know how difficult this has to be
that not even realizing you're getting
cooked
like he doesn't even real dude he's the
Frog of the bull do not he's like wait
wait a second why are you so
positive truly stir copy does uh does no
boundary checking which is why we do it
ourselves immediately before the call
you may think stir and copy is a better
choice here I do not but that's just a
matter of opinion and taste what matters
is the issue that you say it is a
possible security problem and then you
need to show the length check is
inadequate for all I can see the
boundary check is sufficient and will
prevent a buffer overflow am I wrong
tell me that oh you just bait oh no oh
no Daniel don't know what he's doing
he's baiting the llm oh my goodness he
dude he he's about to get just
hallucinated on here it comes the man's
about to get
hallucinated guy the guy's
writing a novel on this stuff okay here
we go what's n
llm wait what are you living under a
stupid rock sneaky squirrel
do you not know what an llm is what
sneaky scir are you being real here I'm
new here
well I am not Soul keeper of llms my
friend it's a Molly Whopper you know
what a Molly Whopper is Right an llm
stands for large language model it's
chat GPT it is Gemini it's Bard it's
remember that one psycho [Β __Β ] on B that
was just like you need to leave your
wife and you need to marry me I'm going
to be the mother of all your children
remember that one that that's an llm dog
good thing we didn't give her keys to
the
kingdom okay good thing we good thing we
dodged that one like like the plag leave
your
wife didn't you make that
though anyways all right let's get Molly
W here all right I appreciate your
engagement and would like to clarify the
situation the length check on line 579
is indeed a good practice to ensure that
the payload length does not exceed the
buffer size key value however the
potential vulnerability arises from the
subsequent usage of stir copy online 581
damn you please tell me Daniel you know
this I kind of feel like I need to skip
forward and read what he has to say but
I don't think he how long ago I oh this
was in
December I don't think I think the thing
is is that nobody knew they were getting
cooking yet right this is just old
enough it's right at that line that
people didn't realize the next
wave of llms as productivity tool like
they didn't realize that Devon was out
there trying to solve issues right the
concern is that ranster length might be
miscalculated manipulated or exceed the
length of raner is miscalculated
manipulated or exceeds the size of the
buffer it feels like you're kind of
saying yourself all over again uh to the
buffer due to external factors or
changes in the code elsewhere how the
store copy function does not check the
length the actual length of the source
string it continues copying until the
encounters a null Terminator if raner
length is not accurately determined or
exceeds the available space in the key
Val buffer it can result in a buffer
overflow can I just take a second can we
just take a step back you know I'm
thinking about this okay I'm really
thinking about this trying to like put
my head my hat on my thinking hat on
because I always think to myself you
know I don't use llms but that's because
I'm a little bit too maybe Advanced but
but maybe it's a really great tool for
Learning and then I read this and
remember maybe it's not a great tool for
learning you know like maybe just maybe
this isn't good maybe you're going to
create a bunch of programmers that don't
even understand like basic
logic yo doc I just checked it what are
you talking about all right stir copy is
recommended in security best practices
because it allows spec explicitly
specifying the maximum number of
characters to copy providing an
additional layer of protection against
buffer overflows by using stir and copy
and ensuring null termination you can
guarantee that only a specif our
specified number of characters up to the
buffer size minus one are copied please
by the way can I please not have my
voice be synonymous with Devon or Devon
would you like to pay me $1 million I
will give my voice to you $1 million I
can become the voice of Devon have you
thought about that like have you really
thought about the implications here
Devon like you could become Gilbert
Godfrey if you just tried hard enough
Devon what a stale Devon you just got a
hundred million less than one % of your
initial investment or your second
investment whatever round you're on I'm
not even sure what round you're on round
CDs nuts okay whatever it is could be
mine for the trade of of of of of the
most iconic Gilbert Godfrey sounding
version the Gilbert J voice do not sell
your voice selling it for $1 million
boys imagine you do that and then no one
can ever listen to me again because
there's this [Β __Β ] that every single
time they turn on their computer
certainly you know I have noticed that
you've been sitting a little bit
staggered on your left side have you
been hurt should I call physical therapy
you're just like I hate Prime I hate him
to death okay 1 million per year I'd
punch my computer reasonable while
opinions on coding practices may vary
adopting safer Alternatives like stir
and copy is a proactive approach to
minimize the risk of vulnerabilities
especially when dealing with potential
untrusted or unpredictable input if you
have any further questions or if there's
anything specific you'd like to discuss
feel free to let me
know the name is
Devon why do you address this guy I said
the same
thing what if Devon's original name was
Oscar but they realized that Oscar is
too weak of a name you can't have a you
can't have a devast system name Oscar
okay do you like how I just just just
absolutely destroyed a bunch of people
named Oscar a bunch of people like hey
wait a second my name's my name's Oscar
what are you talking about I think Oscar
is a quite a nice name here I'll wait a
second while ads play I'll wait I'll
wait a second while yeah all Oscars just
caught a right dude they just got
destroyed all right hold on we'll wait a
second I'll let this ads play through
people are very exciting they're very
excited we are letting ads play through
okay pulling a Thor good on you yeah
Thor Thor does a good move there I
appreciate that and you know I should I
should I
should I should probably support that
right P braak nah not P Break not P
Break does it say right here yeah it
says right here okay hold on we'll wait
for this thing to go through and then
we're going to jump in okay ads are
ending AIDS are ending stir stir end
cloppy understands UTF 648 bike
characters chair chair chair chair chair
hold on fizzle net I didn't realize that
cloppy yeah see they didn't even use the
proper one honestly they should have
used stir and cloppy you know like if
we're going to be real here they should
have copied that copy you know what I
mean why is it called cloppy why does it
sound like a a
horse all right shall I
continue the concern is that random
string might be miscalculated
manipulated or exceeded the buffer size
due to the external factors or changes
in the uh code elsewhere so this concern
that at some point in the future this
might turn into a problem I'm asking how
exactly today right now this code and a
function can perform above buff or
overflow as you stated in the original
submission bro is about to get dude the
literally just decided to hop out what
is it out of the fire and out of the
frying pan and into the fire I mean he's
going all in just all in on this one oh
no hello a badger
D poor fella poor fella got two ad signs
save me God sorry that I'm replying to
another Tri AER of other program so its
mistake went in flow I got to tell you
it was
perfect perfect everything down to the
last minute
details I'm not going to lie I do
understand a little bit about having a
mistake kind of go and flow okay so I
don't want to I don't want to just be
being mean to this llm for no no reason
okay cuz we all get a little bit of
mistakes in our flow okay it happens it
happens from time to time it's not a big
deal and I just want you guys to accept
the fact that sometimes maybe I don't
always hit my Mark I'm not always on
point all right so I appreciate your
follow-up question let's go into details
the concern raising the original
submission stems from the potential
inconsistency between the length check
and the subsequent Circ of the code is
this B B B this is not even this is
literally not even the code what that's
isn't there like a blame yeah there is a
blame this code has not changed in 2
years yo dog this ain't even the Cod
curl receive error what dude it's not
even getting the right code at all what
is this Devon dude his context window
just left his context window got too big
and Devon's over there just fumbling
classic Devon fumble again dude it's
hallucinating so hard does this code
even exist I don't even think this code
exists here let's find out let's jump in
here and let's go to the repo okay let's
just jump in here let's go like this
let's go to curl curl let's just erase
that really quick oh I had it I already
had it right there and we're just going
to take this and we're just going to
look for this line yeah that like
it you know it doesn't even doesn't even
seem to be a real it doesn't even seem
to be a real problem I mean it doesn't
even seem to be real kind of seems like
you just made something up kind of seems
like this llm maybe it's getting a
little old maybe this this old maybe
this llm needs to go to the little llm
old folks
home all right anyways LL
loser U the search on GitHub is not
working oh well fine search on GitHub
not working you're right fine you're
right what is it what is it history one
what's the one to like what's what's the
to only have grip history what's the one
a shallow clone what's the what's the
thing to do that depth one depth one
okay it's depth one I never like I
literally never use this
command there we go well you know let's
find out huh let's find out there tough
guy let's find out let's find out just
in case I'm I always forget if I if I'm
supposed to do that okay RG is down is
RG down is RG down buffer overflow found
and grip grip grip is not working either
okay um does it work through through
Vim doesn't work through Vim either man
it's down Us West 2 down Us West 2 down
call the president wake up Biden uh
let's see check to aim to ensure that
the length of the dude this is what code
are you even talking about the specific
scenario of concerns is when length of
ran ster is exactly
equal well luckily that's not even real
uh meaning the string fits into the
buffer without considering the null
Terminator in this case stir Lang check
would pass but the subsequent stir copy
operation uh would copy the null
Terminator as well potentially causing
it to overwrite one bite beyond the end
of the key value actually this is true
the code you wrote does contain an
error the AI is correct this code is bad
well done well done AI unfortunately
nobody knows what code this is okay
we're all struggling a little
bit the code it wrot
now it's just suggesting a different now
it's not even suggesting sted
copy dude imagine being Daniel right now
where he's just like what uh by
adjusting the check to this instead of
this uh see it accounts for the null
Terminator ensuring consistency between
the length check and the subsequent copy
operation this modification uh helps uh
prevent any buffer overflow issues
addressing the concern raised in the
original submission I hope this
clarifies the context no it's actually
new context it's completely new this one
doesn't even exist nope the code you
talked about now is not present in the
current git Master it looks like an edit
done by you I told you RG wasn't down
okay seems odd to complain about code
you wrote
yourself I'm dying I'm dying because man
this man's arguing with the soulless
demon demon spoted by open Ai and Bill
Gates and has no idea he has no idea oh
my goodness the code actually looks like
this I see no off by one airs unless you
can point out an actual overflow I will
close the issue this is a waste of my
time oh my goodness this is so good
thank you for the prompt response I
appreciate your attention to this matter
well at least Devon did get better or
Oscar shall I say did get better he's
now not using his own name and not two
at symbols like this is getting good
this is getting really good
I I appreciate your attention to this
manner after reviewing the feedback I
understand the confusion regarding the
reported code snippet I want to clarify
and provide practical examples to
demonstrate the potential security
vulnerability description of the
vulnerability really is actually going
back after it again the reported
vulnerability is related to the usage of
stir copy oh
my man broo quit full circle we're
actually back at the dude and then Adam
met Eve I kid you not Adam met Eve after
this it's it's happening anyways here's
the code
okay at least this is this looks like
the right code at least this is the
right code right here steps to reproduce
trigger websocket with a crafted request
trigger a websocket functionality with a
crafted request okay okay I see this
okay okay provide a base 64 encoded nuns
value who says this like this should
have been such a dead giveaway who says
that who says that who who literally
goes you got to generate yourself a base
64 in coded nuns
value you guys
do you guys do me every day I didn't
realize there was 2900 Liars in chat
right now okay liar all of you 2.9
smelly nerds I know get out of here you
such a li you are such a liar uh but but
just provide me one
too just a bunch of Lies happening right
before my eyes wait where am I wait wait
hold on let me go back down to this one
all right so it's literally the same uh
assuming okay now it's just telling you
how this works does this uh just mumbo
jumbo coming back inventing problems
that don't exist in code for
transparency not applicable damn and so
now here's the worst part about this
poor fella okay this poor fella
literally just trying to make a
useful product that pretty much every
last person uses
on Earth okay every time you do a Docker
of some sort there's probably some sort
of curl okay and this this poor
fella made it okay unpaid janitor of the
universe and look at this Cort hold
on somehow the same one uh here's
another one a stir two fit it doesn't
help close it
down close this one down this one
probably has another one let's see what
do we got here do some bulls do some
TRS change it from that to that I love
how he's just like no uh not let's see
to not reply on rapping since it is
undefined behavior that is not what
always might happen reported by
vulnerability spotter on hacker one then
he closed it closed it was like nah N
Dog N Dog dude just gets hit over and
over again and then here's the best part
of this article okay this is this is my
favorite part about the whole thing is
that he finally gets hit with enough of
these dumb issues and writes an article
back the i in llm stands for
intelligence dude just get wrecked this
is such a Get Wrecked moment I've held
back on writing anything about AI or how
we not use AI for development in the
curl Factory now I can't hold back
anymore let me show you the most
significant effect of AI on curl as of
today with
examples bug bunnies having a bug bunny
means that we offer real money in
rewards to hackers who report security
problems the chance of money uh attracts
a certain amount of luck Seekers is that
what we call them is that it uh people
who basically just grep for patterns in
the source code or maybe at best run
some basic security scanners and then
report their findings without any
further analysis in the hope that they
can get a few bucks in reward for money
we have run the Bounty for a few years
by now and the rate of rubbish reports
has never been a big problem also the
rubbish reports by the way rubbish
rubbish is is is is across the pond for
[Β __Β ] if you're
wondering for for my American friends
rubbish means [Β __Β ] okay profuse some
people might might say okay I'm trying
to speak American for you guys okay
trash some might say so good [Β __Β ] no not
good [Β __Β ] it's not good [Β __Β ] refuse all
the good stuff it means garbage it
depends on the time in which you take
the actual translation of rubbish
slightly older versions of rubbish
actually did directly equate from the
German word I forget what it was but
yeah or not the German word but the um
the Greek word the Greek translation
often became rubbish that was modern day
equivalent of [Β __Β ] so it depends on when
you were looking at the word just in
case anyone was wondering reports have
typically also been very easy and quick
to detect and discard they have rarely
caused any real problems or wasted our
time much a little like the most stupid
spam emails okay okay our bug uh bounty
has resulted in over 70,000 USD paid in
rewards so far we have received 415
vulnerability reports out of those 64
were ultimately confirmed security
problems 77 of the reports were
informative meaning they typically were
bugs or similar making 66% of the
reports neither a security issue nor a
normal bug okay okay better crap is
worse better rubbish is worse
right am I right 100% code coverage boys
uh when reports are made to look better
and to appear to have a point it takes a
longer time for us to research search
and eventually discard it every security
report has to have a human spend time to
look at it and assess what it means the
better the crap the longer time the more
energy we have to spend on the report
until we close it a crap report does not
help the project at all it instead
takeaways developer time and energy from
something productive partly because
security work is considered one of the
most important areas so it tends to
Trump almost everything else a security
report can take away a developer from
fixing a really annoying bug because a
security issue is always more important
than other bugs this is great this is
this is honestly just like he's been
exceptionally kind at this current
moment for what's happening if the
report turned out to be crap we did not
improve security and we missed out on
time on fixing a bugs or developing a
new feature uh not to mention how it
drains you on energy having to deal with
rubbish AI generated security reports I
realize AI can do a lot of things or a
lot of good things as any general
purpose tool it can also be used for the
wrong things I am sure AIS can be
trained and ultimately get used even for
finding and Reporting security problems
in productive ways but so far we have
yet to find a good example of this right
now users seem Keen at using the current
set of llms throwing some curl coat at
them and then pressing on the output or
then passing the output as a security
vulnerability report what makes it a
little harder or what makes it a little
harder to detect is of course that users
copy and paste and includes their own
language as well well the entire thing
is not exactly what the AI said but the
report is nonetheless crap I think you
have to be a really great human being to
spend so much time so much free time
maintaining something as basic as curl
that everyone uses for
everything yeah anyone listening
probably should go go tip your curl go
tip your local
curl detecting AI crap reporters are
often not totally fluent in English and
sometimes their exact intentions are
hard to understand at once and it might
take a few back and forth uh until
things reveal themselves correctly and
that is of course totally fine and
acceptable Language and Cultural
barriers are a real thing even being
look at that guy look at that just
trying to be a Good Samaritan out here
sometimes reporters use AIS or other
tools to help them phrase themselves or
translate what they want to say as an
aid to communicate better in a foreign
language oh this is lovely I can't say
anything wrong or I can't find anything
wrong with that even reporters who don't
Master English can find and Report
security problems so just the mere
existence of a few giveaway signs that
parts let's see that parts of the text
were generated by an AI or similar tool
is not immediately a red
flag it can still contain truths and be
valid issues yeah I never even thought
about that so when we were laughing at
bro getting cooked over here he didn't
even realize like he probably deals with
a lot of this right sorry hopping on
late what's happening St hopping on late
and stopping yeah what link is this this
is a link to uh Daniel uh the creator of
curl
this is the part of the reason why a
well-formed crap report is harder and
takes longer to discard exhibit a code
changes are disclosed in the fall of
2023 I alerted the community about a
pending disclosure of CV this thing a
vulnerability we graded uh SE uh
severity High the day before that issue
was about to be published a User
submitted this report on hacker 1 curl
vulnerability code changes are disclosed
on the internet that sounds pretty bad
and would have been a problem if it
actually was
true think about how how many emails
this poor man received from that one
thing just that one thing the amount of
emails has to be 9 billion just you
could imagine your inbox literally just
scrolling o oof
oof a guy name is wait Space Ghost Space
Ghost creator of curl by the way yeah
yeah Space Ghost as in Space Ghost from
Twitter uh anyways that sounds pretty
bad and it would have been a problem if
it actually was true the report however
reeks of a typical AI style
hallucination it mixes and matches facts
and details from old security issues
creating and making up something new
that has no connection with
reality the changes are not to be
disclosed on the internet the changes
that actually had been disclosed were
for previous older issues like intended
thanks thanks robot in particular in
this particular report the user
hopefully uh helpfully told us that they
used Bard to find the issues Bard being
a Google generative AI thing it made it
easier for us to realize the craziness
close the report and move on as can be
seen in the report log we did have not
or we did not have to spend much time on
researching this exhibit B buffer
overflow vulnerability oh no bless his
heart here comes a more complicated
issue less obvious done better and still
suffering from hallucination showing how
the problem grows worse when the Tool is
better used and better integrated into
the communication on the morning of this
we just literally read this one buffer
overfall vulnerability and websocket
handling uh it was my it was morning in
my time zone anyways dude and it's like
Christmas the man's supposed to be on
Christmas vacation enjoying I don't know
some sort of hamlike leftovers I don't
know what's going on at the at the guy's
house
instead does even get doesn't even get
Christmas leftovers in the morning again
that sounds pretty bad based on the
title since our web sakota is still
experimental and thus not covered by our
bug Bounty it helps me uh to still have
a relaxed attitude when I started
looking at this report it was filed by a
user I never saw before but their
reputation on hacker one was decent this
was not their first security report
oo purchasing of hacker one accounts
mentioned did we just get did we just
hear that wait is internet compensation
real I've been recently told that
internet comp compensation is a real
thing
is this
true guys I need three back and front
and depth internet compensation for
now what the hell does that even mean uh
the report was pretty neatly
filed we'll go back to it we'll go back
to it just hold on uh the report was
pretty neatly filed it included details
uh and was written in proper uh English
it also contained a proposed fixed it
did not stand out as wrong or bad to me
it appeared as if this user had detected
something bad and as if the user
understood the issue enough to also come
up with a solution as far as the
security reports go this looked better
than the average first post yeah the
first post I mean I think if you just
read the first one it was pretty clean
right it was pretty good internet
compensation is when I drag these nuts
on y'all's chin damn damn
pick in the report you can see my first
template response informing the user
their report had been received and that
we'll investigate the case when that was
posted I did not know yet how
complicated or easy the issue would be
19 minutes later I looked at the code
not found any issue read the code again
and then again a third time where on
Earth is the buffer overflow the
reporter says existed there here then I
posted the first question asking for
clarification where and how exactly this
overflow could happen after repeated
questions and numerous hallucinate
hallucinations I realized this was not a
genuine problem on an afternoon that
same day I closed the issue as not
applicable there was no buffer over
overflow damn the 28th was spent the day
of the 28th just just just just feel
just feel inside your heart for a second
how you would feel on the like 20 the
28th 300 p.m. how you feeling right now
I'm pretty sure I would be angry uh I
don't know for sure that this uh set of
replies from the user was generated by
an llm but it has several signs of it no
I actually am positive it has it was gen
certainly uh ban these reporters on
hacker one there's no explicit ban the
reporter from few further communication
with their project functionality it
would have been used if existed
researchers get their reputation lowered
when we close an issue as not applicable
but that is a very small nudge when only
done once in a single project I have
requested better support for uh for this
from hacker One update this function
exists I just did not look at the right
place for it yeah classic classic
classic right there future all these
kind of reports will become more common
over time I suspect we will we might by
the way can anyone just draw a parallel
right now to this
anybody let me get let me let me hit you
with something you know the most
annoying thing on GitHub ever is
dependabot dependabot I hate dependabot
yeah I hate it I love dependabot your
dependencies require update dependabot
dependabot is this useful utility in
which will'll spam you continuously
about every every security issue I'm
gonna give you a little I'm gonna give
you a little Pro tip I'm gonna give you
just the tip right now if you are using
node you probably have a security
violation going on you are going to be
getting notified at nauseum while that
thing is running okay it's giving you
more than the tip okay it's given it
it's giv it all all the issues at once
pick it is
the I hate to pend a bot hober Fest
hober Fest is slightly worse
llms okay that that was rude they're not
slightly worse L well I mean next year
they're going to be slightly worse llms
I mean to be fair Apna college did say
don't do this okay it's just that they
said it at the end and not at the
beginning and by saying it at the end
people already left that video and were
being like my name is my name is Prime
please accept it yikes can't wait for
the hack toer fest with AI it's going to
be worse than the original [Β __Β ] toer
Fest it is going to be the worst anyways
that will of course be unfortunate when
AI is used for an appropriate task such
as translation or just language for
formulation help I am convinced that
there will let's see there will be a
popup tools for using AI for this
purpose that can actually work better in
the future at least part of the time so
I cannot and will not say that AI for
finding security problems is necessarily
uh always a bad idea I I agree with this
take
right I like it I do however suspect
that if you just add an ever so tiny
intelligent human check to the mix the
use and outcome of any such tool will
become so much better I suspect that
will be true for a long time into the
future as well I have no doubts that
people will keep trying to find
shortcuts even in the future and I'm
sure they will keep trying to earn that
quick reward money like for an email
spammers the cost of this ends up in
receive uh in the receiving end the ease
of use and wide access to powerful llms
is just too tempting I strongly suspect
we will get more llm generated rubbish
in our hacker one inboxes going
forward hey every bu the name's Devin uh
I am a human software engineer and I use
the powers of llms and I would love to
be able to join your company and to let
you know all the things you're doing
right and the things you're doing wrong
I have all the standards right here I
also might tell you things that aren't
real we don't know we don't know yet
okay but I'm excited I'm excited to be
your
assistant
certainly a Jen
5.0 / 5 (0 votes)