Serverless to Homeless - Case study
Summary
TLDRThe video discusses a case where a user received a $100,000 bill from Netlify for a static website, highlighting the issue of unexpected costs with cloud service providers. It explores the possibility of a Distributed Denial of Service (DDoS) attack and the lack of automatic DDoS protection, comparing bandwidth costs across different providers. The video emphasizes the importance of being aware of service tiers and potential expenses, and ends with Netlify waiving the charges after public discussion, raising questions about trust and transparency in cloud platforms.
Takeaways
- 📈 A user received a $100,000 bill from Netlify for a static website, highlighting a potential issue with cost management and unexpected expenses.
- 🚀 Static websites are typically expected to have minimal costs, with some providers offering pro plans that include unlimited bandwidth within certain tiers.
- 💸 Overshooting the included resources in a plan, such as serverless compute or bandwidth, can lead to significant charges based on the provider's rates.
- 🌐 The incident at Netlify (nlii) involved a single day with 60 terabytes of bandwidth usage, which is highly unusual and indicative of a possible DDoS attack.
- 💰 Comparing costs, Netlify charges $55 for 100 GB of bandwidth, which is significantly more expensive than cloud providers like Hetzner or AWS.
- 🛡️ Lack of automatic DDoS protection can be a vulnerability; however, some providers offer emergency DDoS modes to help mitigate such attacks.
- 🔧 The user's website was affected by a DDoS attack focused on a single file, possibly an audio clip, leading to massive data transfer.
- 🌐 Netlify's response suggested hosting music on third-party platforms to reduce bandwidth usage, which may not be ideal for all users needing to host certain assets.
- 🤝 Netlify CEO eventually responded to the situation on Hacker News, stating that the user would not be charged for the excessive bandwidth usage.
- 📉 The incident raised concerns about the trustworthiness of platform providers and the potential for being unknowingly targeted by DDoS attacks.
Q & A
What was the initial bill amount that the user received from Netlify?
-The user initially received a bill for almost $104,000 in USD from Netlify.
What type of website incurred such a high bill?
-The high bill was for a simple static website, which typically should have minimal to no hosting costs.
How did the user initially react to the bill?
-The user initially thought it was a joke or a scam, but after checking their dashboard, they realized it was a legitimate overdue bill.
What is the usual cost for a pro plan on platforms like Netlify or Vercel?
-The usual cost for a pro plan on platforms like Netlify or Vercel is around $20 a month, with an uninterrupted tier included.
What caused the user to exceed the plan's bandwidth limit?
-The user was charged for exceeding the bandwidth limit due to a spike in traffic, with 60 terabytes of bandwidth used in a single day.
How does the cost of bandwidth on SaaS platforms like Netlify compare to cloud providers?
-SaaS platforms like Netlify charge significantly more for bandwidth compared to cloud providers. For example, Netlify charges $55 for 100 GB, while cloud providers like Hetzner and AWS have much lower rates or even offer free tiers for certain amounts of traffic.
What is the issue with automatic DDoS protection on these platforms?
-Automatic DDoS protection is difficult to implement because it's challenging to differentiate between legitimate and malicious traffic during a distributed denial-of-service attack.
What did Netlify suggest as a solution to prevent such high costs in the future?
-Netlify suggested hosting music on third-party platforms like YouTube, Bandcamp, or SoundCloud to reduce bandwidth usage, regardless of the site's popularity.
How did the user feel about Netlify's response to the situation?
-The user disagreed with Netlify's response, feeling that the platform was placing blame on the user and not providing adequate solutions for such incidents.
What was the final outcome for the user who received the $100,000 bill?
-Netlify's CEO responded on Hacker News, stating that 100% of the charges were removed and that their policy is not to shut down free sites during traffic spikes that don't match attack patterns.
What concerns do users have about the transparency and fairness of billing practices on these platforms?
-Users are concerned that they cannot verify the legitimacy of the charges or protect themselves from potential fraudulent activities by the platform, as the databases and billing systems are controlled by the platform companies.
Outlines
💸 Unexpected $100,000 Bill from Netlify
The video begins with the startling revelation that the user received an invoice for $100,000 from Netlify for a simple static website. The speaker discusses the potential reasons for such a high bill, emphasizing the importance of understanding one's hosting provider's pricing structure, especially for services like serverless compute, bandwidth, and image optimization. The speaker shares a personal experience with Netlify, detailing how they were mistakenly billed for an excessive amount due to a spike in bandwidth usage, suspected to be a DDoS attack. The discussion highlights the high costs of bandwidth through SaaS providers like Netlify and WorldPress compared to traditional cloud providers, and the lack of automatic DDoS protection, which leaves users vulnerable to unexpected expenses.
🚫 Netlify's Response and Recommendations
The speaker criticizes Netlify's response to the situation, which suggested that the user's static assets, such as sound files, should be hosted on a third-party platform to reduce bandwidth usage. The speaker argues that these assets are necessary for the website's functionality and should be delivered through the edge network provided by Netlify. The video also addresses the issue of DDoS protection and the difficulty for providers to automatically detect and prevent such attacks. The speaker shares their own experience of reducing the bill by attributing 20% of the cost to the DDoS attack and further negotiating with Netlify, which ultimately led to the charges being waived. The speaker expresses concern about the lack of transparency and control users have over their data and the potential for misuse by platform companies.
📢 Conclusion and Future Considerations
In the concluding part of the video, the speaker reflects on the incident and its resolution, noting that the user was not charged for the excessive bandwidth usage. The speaker commends Netlify's policy of not shutting down free sites during traffic spikes that don't match attack patterns, but also acknowledges the stress and potential financial burden such a mistake can cause for users. The speaker raises questions about the trustworthiness of platform providers and the lack of recourse for users in such situations. The video ends with an invitation for viewers to share their thoughts in the comments and a prompt to like and subscribe for more content.
Mindmap
Keywords
💡bill
💡static website
💡serverless compute
💡bandwidth
💡DDoS attack
💡Cloudflare
💡CDN (Content Delivery Network)
💡Netlify
💡AWS (Amazon Web Services)
💡cost reduction
💡support response
Highlights
A user received a $100,000 bill from Netlify for a simple static website, raising concerns about cost management and prevention strategies.
The expected cost for hosting a static website is minimal, with most plans offering an uninterrupted tier to cover usage within limits.
Exceeding the plan's limits, such as serverless compute or bandwidth, results in additional charges based on the provider's rates.
A similar incident occurred at NLII with a user receiving an unexpectedly high bill, indicating a potential issue with cost management.
The user's bill was almost $104,000 in USD, which was initially dismissed as a joke or scam but was confirmed upon checking the dashboard.
The peak day of February 16th showed a usage of 60 terabytes of bandwidth, which is unusually high and indicative of a possible DDoS attack.
Bandwidth costs from SaaS providers like World and Netlify are significantly higher compared to cloud providers like Hetzner and AWS.
Cloudflare offers zero cost for data transfer out of the internet with their CDN service, contrasting with the high charges from other providers.
The lack of automatic DDoS protection is noted, as implementing such protection is challenging due to the difficulty in distinguishing between legitimate and malicious traffic.
Providers like Cloudflare offer emergency DDoS mode to absorb the attack and allow legitimate users through after solving a challenge.
The high bandwidth usage was attributed to ancient user agents and Google Cloud addresses, suggesting a possible DDoS attack.
Netlify's response to the incident included a recommendation to host music on third-party platforms to reduce bandwidth usage.
The user's dissatisfaction with Netlify's response highlights the issue of placing blame on the user for necessary static assets.
The incident raises concerns about the trustworthiness of platform companies and their internal systems, which are black boxes to consumers.
Netlify CEO's response on Hacker News assured that the user would not be charged for the bill, and all charges were removed.
The policy of not shutting down free sites during traffic spikes that don't match attack patterns was emphasized, but apologies were made for the lack of clarity in initial support replies.
The user's experience with the unexpected bill and the subsequent resolution process was shared, highlighting the importance of transparent communication from service providers.
Transcripts
so net lii sent a user a bill of
$100,000 for a simple static website
let's take a look at what this is what
went wrong and how you if you're using a
provider like worel or nettia or
something like that you can prevent this
Behavior or you can prevent this cost so
$100,000 is of course not a small amount
of course if you're hosting something
you know you're expecting if it is
static you're expecting it to be pretty
much zero cost at most at you know you
are probably on a pro plan on what sale
or even netlify which cost like $20 a
month but all of these plans even the
Pro Plan comes with an uninterrupted
tier right so what happens is that if
you are using the things within that
tier it'll just be included in your plan
but if you overshoot let's say for
serverless compute or bandwidth or image
optimization or anything like that then
they will just charge you based on you
know their rates so a similar incident
incident happened at nlii where somebody
received a $100,000 bill and uh it's to
say the very least it's a bit strange so
let's take a look at what happened so I
received an email from netlify last
weekend saying that I have
104,000 almost in USD in Bill overdue at
first I thought this is a joke or some
scam but after checking my dashboard it
seems like I'm truly owning them that
amount so I was like and think okay
maybe I got dos since netlify charges
$55 for 100 GB for the exceeding
bandwidth the peak day of February 16th
had this much amount which is like you
know 60 terabytes of bandwith in a
single day see so that's that's the
thing that is the thing that two things
wrong here the first one is that you
know this cost is outrageous if you take
a look at actual cost for bandwidth it
is not as expensive as $55 for 100 GB
right now why worel and why netlify and
why you know all these SAS providers
sell the bandwidth extremely expensive
worel is cheaper than this worel sells
it for 40 USD which I'm assuming like
they are reducing now I have heard some
things that Al is in talks with a lot of
people to reduce this number but let's
see what it is but netlify charges
$55 for 100 GB of bandwidth this cost is
insane right so if you take a look at
herzner for example see so with herzner
20 terab of traffic is already included
and extra traffic is € per month per TV
right so you can see that the markup
here is 55 times more 55 times is you
know its orders of magnitude more than
what a cloud provider like herzner is
costing you and even AWS doesn't charge
this month right AWS also has huge
charges and but it doesn't charge that
plus if you look at providers like Cloud
flare Cloud flare has basically zero
cost for this so Cloud flare basically
doesn't charge you anything at all if
you are transferring data out of
Internet with their CDN service or you
know with R2 I think as well which is
their S3 alternative so this number in
itself is bad but the second second
thing is that there is no automatic dos
protection which is which I agree like
implementing a Dos protection is by its
very definition extremely hard because
dos means that it's distributed denial
of service attack that means that all
across the world the computers are
pinging your IP address and your website
address and it's extremely hard to
determine that this particular visit is
a bot visit or is some malicious visit
and that particular visit is a legit
visit so what happens generally in cases
of Dos is that you anyway have to like
you know put a capture on every single
page of the website for every single
user at least for that time being so if
you go to Cloud flare and these
providers A lot of these providers
actually what they do is they offer you
a you know an emergency dos mode so when
you toggle that on everyone gets a
challenge page on top of there you know
whenever somebody's visiting your
website they'll see a challenge page
from that provider which itself absorbs
all the DS attack and then just lets
legitimate users pass through because
they have to solve a capture or
something like that so see this is what
what happened like it was dsed on a
single file which is you know a sound
file and it got a lot of terabytes in
data transfer and what they told what
netlify told is that told them that
after looking into this further it seems
like a lot of bandwith usage came from
some user agents that are quite ancient
and uses Google Cloud addresses this
would include devices such as this this
this so either you have a fan base with
a passion for older technology or this
was likely a Dos attack I mean somebody
who's getting 16 4 terab I'm assuming
that they don't have a fan base of that
particular audio clip as aggressive as
that so so this seems like a this seems
like a weird message weird email to
write to someone who is already stressed
about you know paying a $100,000 bill
but okay going forward I would recommend
hosting music on a thirdparty music
platform such as YouTube band camp or
SoundCloud and reduce your bandwidth
usage no matter how popular your site
becomes so see this is where I really
disagree with the with how netlify has
responded in this email because if I'm
hosting a website it is my website and I
need those static assets whether that's
an image whether that's a sound file
whether that's a vasm binary I need that
on the edge Network which you are
providing right that that was the whole
proposition that we will deliver files
very fast to you sure I can host my
files on S3 but the only reason I would
do that is to save money save cost not
from a point of view that somebody just
downloads my file 10 times and you know
I'll be just bankrupt I will be homeless
so I think this is slightly bad take
from netlify where it's it's putting the
blame on the user itself and especially
in the case where Theos attack happened
right so it's not like they're
complaining out of thin air that you
know something like that happened it's
especially bad it puts a bad image that
you can't rust netlify you know the next
time you're probably hosting an image
which is like 1 MB 2 MB you know you're
not running any Optimizer or let even go
of that let's just assume you know
you're hosting something like a Monaco
editor or a VSS code sort of instance
which we also do on on code Dam so if I
show you for example this instance over
here which let's say if you boot up this
playground so you're going to see that a
vs code like editor opens up now this
syntax highlighting which you see this
syntax highlighting can you see that
these keywords are of different colors
just like how it works in VSS code you
know how that is possible that is
possible through a vasam bilary called
onm so o GM if you Google this this
onm binary it's a web assembly Port of
something that is required for this
tokenization and colorful syntax which
you see so basically this vam binary or
this package is required for VSS code to
provide syntax highlighting and for
reasons which I don't want to get into
this video but you need to host this
binary under your own main domain right
so you have to have that as a static
asset on your domain it's also bundled
internally by a node you know it's
bundled by the build pipeline itself
right so how let's say if something like
this happens to code Dam or something
like this happens to a website which is
using solution like this then how would
you would you really say that you know
you can just pick your binary and just
host it somewhere else I mean that's not
the solution right we have to develop
better Solutions than just blaming it on
the user so we normally discount these
kinds of attacks to about 20% of the
cost which would make your new bill
$20,000 I've currently reduced it to
about 5% which is $5,000 I know this is
still a lot of money and I apologize for
the inconvenience I mean this is is
somebody who's imagine it from a
scenario like you are not sitting in US
you're sitting in a country like India
where 5,000 USD is probably months of
your salary or months of your savings
right for an average developer or for an
average person who's by the way like
earning relatively well in India also
this this sort of behavior is something
which is you know which which can be
heart crushing it can be very brutal for
somebody who's trying to just host a
website a static website and they're
just experimenting seeing things so yeah
it's it's actually bad so he also posted
this on Hacker News and nlii CEO
response our support team has reached
out to the user from thread to let them
know that they are not getting charged
for this so of course like 100% of the
charges charges are removed it is
currently our policy to not shut down
free sides during traffic Spike that
doesn't match attack patterns but
instead for giving any bills from
legitimate mistakes after the fact
apologies that this didn't come through
in the initial supporter reply so see
here people have also did done some
pretty wild Acquisitions on nlii that
how on Earth can I as a consumer be sure
that nlii has not paid somebody to DS me
I mean this is completely fair question
I would say but these are some of the
questions where you can't do anything
about it right so you can't do anything
at all right because these are the
platforms they are controlled by these
platform companies right so at the end
of the day their databases are black
boxes their databases and what they
report to you fundamentally are black
boxes what we can just assume is that
everyone has the best interest for
customers in their minds and nobody's
like messing around or you know just
tweaking numbers or you know doing
things like these to charge or
overcharge customers that's the best we
can think about just to answer this
question there is no way you in in the
world can tell that if netlify did that
or did not do that because it's their
own systems at the end of the day you
can't tell it from outside like watching
it watching this video like this there
is nothing I can do to tell like what
happened so yeah that's basically it
about this one I think it ended well
with the person at least not paying
anything I don't see like there is any
sort of resolution so far on this that
what exactly happened like who dsed so
that's it for this video I think it's at
least the end was well for at least the
person who got dsed I don't think there
is still any update on like they say
they have the support hasn't come back
with the IP information so that's still
work in progress yeah what do you think
about this let me know in the comments
below make sure you like And subscribe
thank you so much for watching and I'll
see you in the next video really
soon
浏览更多相关视频
DDoS Attack | DDoS Attack Explained | What Is A DDoS Attack? | Cyber Attacks Explained | Simplilearn
Cloud Computing In 6 Minutes | What Is Cloud Computing? | Cloud Computing Explained | Simplilearn
5G Network Security Threat: Performing a DDOS Attack with UERANSIM
DDoS Attack Explained | How to Perform DOS Attack | Ethical Hacking and Penetration Testing
Cara Mengamankan Website dari Serangan Hacker | IDCloudHost
Key Cloud service providers and their services
5.0 / 5 (0 votes)