Is this my Fault?
Summary
TLDRIn this gripping video, the creator recounts the worst day of their life, marked by personal loss and a catastrophic server room leak. The narrative unfolds as they investigate the cause of the leak, which leads to a perplexing case of galvanic corrosion in their custom water cooling loop. Despite the extensive damage, the majority of their systems survive, with only the son's gaming rig succumbing to the disaster. The video promises a follow-up to unravel the mystery and rebuild the affected system, engaging viewers with a blend of personal story and technical troubleshooting.
Takeaways
- 😢 The narrator experienced the worst day of their life with the loss of a family member and a significant issue with their server rack cooling system.
- 🔍 Upon discovering a water leak, the narrator embarked on a meticulous inspection of their server equipment to identify the source of the problem.
- 🛠️ The cause of the leak was a severely corroded fitting in the cooling system, which resulted in water damage to multiple servers and gaming rigs.
- 🤔 The narrator questioned the occurrence of galvanic corrosion in a system composed only of copper, brass, and plastic, as stainless steel was not in direct contact with the fluid.
- 🔬 The narrator followed a scientific method to investigate the issue, comparing fittings from unaffected parts of the system to identify discrepancies.
- 🧩 The disassembly of the affected components revealed unexpected oxidation and metal deposition, suggesting an unusual corrosion process.
- 👨🔧 Contact with the manufacturer (EK) led to a theory that the stainless steel cover might have contributed to the corrosion, though the narrator remained skeptical.
- 🔄 The investigation expanded to other systems sharing the same cooling loop to determine if the issue was isolated or widespread.
- 🛡️ Despite the extensive damage, most of the systems survived the water leak, with only the son's gaming rig being the main casualty.
- 🔍 The narrator planned to conduct further tests on the affected systems and collaborate with the manufacturer to understand the root cause of the corrosion.
- 🔨 The video ends with a commitment to rebuild and rectify the situation, highlighting the complexity of troubleshooting and the importance of understanding the underlying issues.
Q & A
What was the significant event that happened on June 14th of this year in the narrator's life?
-The narrator found out that his sister had passed away, which he describes as probably the worst day of his life so far.
What issue did the narrator encounter with his server rack's cooling system?
-The narrator discovered that the pump running the cooling system for his server rack was running dry, indicating a leak in either the reservoir or the systems.
What was the initial visible sign of the problem with the server rack?
-A watermark on the floor indicated the presence of water, which led the narrator to investigate further and find the source of the leak.
What was the cause of the leak in the server rack's cooling system?
-The leak was caused by a completely corroded fitting that dumped gallons of water onto everything underneath it.
What is galvanic corrosion, and why was the narrator surprised to find it in his cooling system?
-Galvanic corrosion is an electrochemical corrosion process that occurs when two different metals are in contact with each other and an electrolyte. The narrator was surprised because the loop in question only had Copper, Brass, and plastic, which should not cause galvanic corrosion.
What was the narrator's goal after discovering the issue with the cooling system?
-The narrator's goal was twofold: to understand what caused the corrosion and to get his son's gaming rig back up and running.
What did the narrator find when he disassembled the blocks of the cooling system?
-Upon disassembling the blocks, the narrator found dust, plastic filings, metal shavings, and organic matter, but none of these would explain the leak or the corrosion.
What theory did EK, the manufacturer, propose regarding the stainless steel cover's involvement in the corrosion process?
-EK theorized that the stainless steel cover might have been participating in the galvanic corrosion process because it was somehow bridged to the fluid, although the narrator did not see how this could be possible.
What was the outcome of the corrosion issue on the narrator's computer systems?
-The only system that was a casualty due to the corrosion was the narrator's son's gaming rig. Almost everything else survived, despite being subjected to the leaking water.
What steps did the narrator take to address the aftermath of the corrosion issue?
-The narrator planned to test the affected systems to see if they still worked, contact the manufacturer for further investigation, and rebuild the damaged system.
What preventive measures did the narrator consider to avoid future corrosion issues?
-The narrator considered increasing the iodine dosage in the loop and possibly adding a filter to prevent future corrosion issues.
Outlines
😢 Tragic Loss and Server Catastrophe
The narrator recounts the devastating day of June 14th, marked by the loss of their sister and the subsequent family turmoil. Amidst the grief, they faced a technical nightmare as their server rack's cooling system malfunctioned, causing extensive water damage. The server room contained multiple machines, including personal and work-related equipment, as well as their children's gaming rigs. The root cause was a corroded fitting that had leaked gallons of water, leading to a frantic disassembly process to assess the damage and identify the source of the leak. The situation was further complicated by the presence of galvanic corrosion, which was unexpected given the materials used in the loop.
🔍 Investigating the Mystery of Galvanic Corrosion
After reaching out to EK, the narrator explores the theory that a stainless steel cover might have contributed to the galvanic corrosion process. However, upon inspection, no direct connection to the fluid was found. The narrator then proceeds to disassemble various components, including GPU blocks, to understand the extent of the corrosion. They find pristine fittings on some components and severely corroded ones on others, suggesting a system-specific issue rather than a widespread design flaw. The complexity of the problem is highlighted by the need to check each fitting across all connected systems to prevent future failures.
🛠️ Systematic Disassembly and the Search for Answers
The video script details the process of disassembling the water cooling system to its bare parts in order to conduct a thorough corrosion check. The narrator describes the challenges of removing fittings and hoses, and the use of an air compressor to blow out the remaining water from the system. They express hope that some components, like the SSD, might have survived the ordeal due to their location away from the leak. The summary of findings from the other systems reveals that only one system suffered from corrosion, deepening the mystery of the cause. The narrator also hints at a follow-up to address the manufacturer's response and the rebuilding process.
🎮 Post-Mortem and the Unexpected Survival of Systems
In the aftermath of the water damage, the narrator tests the affected systems to assess their functionality. Surprisingly, most systems, including the desktop, survive the incident, with the exception of the narrator's son's gaming rig, which suffered direct water damage to the motherboard. The narrator contemplates the possibility of the gaming rig still functioning despite the odds and hints at a future update on the situation. They also express gratitude for the sponsor, Nexo, for their contribution to the video and encourage viewers to check out other related content.
Mindmap
Keywords
💡Galvanic Corrosion
💡Water Cooling System
💡Server Rack
💡Fitting
💡Dust and Organic Matter
💡Iodine
💡GPU
💡Corrosion Check
💡Worm Drive Clamps
💡NVMe
💡SSD
Highlights
June 14th was identified as the worst day of the narrator's life due to the unexpected death of his sister.
The narrator had to manage family drama and grief while also dealing with a difficult work show.
Upon returning home, the narrator discovered an unexplained water leak in the server room's cooling system.
The water source was traced to a pump running dry, indicating a leak in the reservoir or the systems.
A meticulous process of disassembling the server rack was initiated to identify the leak's source.
A corroded fitting was found to be the cause of the water leakage, affecting multiple servers and gaming rigs.
The narrator expressed confusion over the galvanic corrosion occurrence given the materials used in the loop.
A comparison was made with other fittings in the loop that showed no signs of corrosion.
The goal was set to understand the cause of the corrosion to prevent future incidents and restore the gaming rig.
Disassembly of the water block revealed dust, plastic filings, and metal shavings, but no clear cause for the leak.
A theory was proposed that the stainless steel cover might be involved in the galvanic corrosion process.
Further investigation showed different levels of corrosion across various systems, deepening the mystery.
The narrator considered the possibility of starting over due to the extensive work required for repair.
A method for blowing out water from the system using an air compressor was demonstrated.
Despite the extensive damage, most systems survived, with only the son's gaming rig being a casualty.
The narrator concluded that the issue was not due to block design and that organic matter was found in the loop.
A plan to test the affected systems and communicate findings with the manufacturer was outlined.
The video ended with a teaser for a follow-up on the issue and a shout-out to the sponsor for the video.
Transcripts
June the 14th of this year was probably
the worst day of my life so far the
previous evening I had found out that my
sister passed away I had spent the day
dealing with the family drama and grief
that comes with an unexpected loss like
that I made my way through one of the
hardest W shows that I've ever done um
just focusing on the routine and when I
came home there was an unexplained wine
that I couldn't really nail down the
source of I made my way downstairs
and I found out that the pump that runs
the cooling system for my server rack
here was running dry that means one of
two things the reservoir is leaking or
the systems are leaking and I'll let
this water mark on the floor answer the
question for you so instead of eating I
began the painstaking process of tearing
everything out of the rack as safely as
possible I made my way all the way
through both storage and utility servers
my main and my backup mine and yvon's
office machines and all three of my kids
gaming rigs until I found this the
culprit a completely corroded off
fitting that dumped gallons of water on
everything underneath it how could this
happen I mean that's galvanic corrosion
plain as the nose on my face but this is
a loop that has nothing but Copper Brass
plastic what the is going on here
I've had to sit and Wonder from then
until now so that I could get someone
over here to point a camera at this
because I'm sure that you guys want to
know the answer as badly as I do and if
I'm going to lose a bunch of server
equipment well I might as well at least
help pay for it with this message from
our sponsor if I had time to untangle
cables would my hair look like this why
not switch to Magnetic cables from our
sponsor mag tame
instead they pull apart effortlessly
roll up perfectly and stick to any
magnetic surface no stress no mess no
worries we'll have them Linked In the
video description now I know what you
guys are thinking ah it's lius he's
always doing janky projects and I'm sure
he overlooked something I mean galvanic
corrosion can't happen by accident
that's
science but listen to this you want to
talk science how about the scientific
method for a control system I grabbed
one of the other ones in the loop and
look at those fittings those are the
same two fittings and they're immaculate
so if it was a configuration error in
the loop these should be eaten and yet
they're pristine our goal today is
twofold then my son obviously wants his
gaming rig back up and running which is
why he's helping but before we do that
we need to know what the heck happened
here so that we can ensure that this
won't happen again the first step of
disassembling these blocks is popping
the stainless steel cover off it's
important to note that this is purely
cosmetic since steel is not a compatible
metal with the ones that are typically
used for water cooling like Copper Brass
and
nickel now let's go a layer deeper and
remove the delin top okay not the
cleanest Loop I've ever seen why is
there so much dust on it well I can tell
what some of this is these look like
little plastic filings from working with
the tubing that we installed oh there's
a little metal shaving uh that could be
left over for manufacturing of something
radiator heat exchanger there's a
surprising amount of this that looks
like hair or other organic matter but
there's iodine in the loop and what kind
of contradicts that theory is that this
tube over here which is running water to
my system right now is Crystal Clear
well maybe one of the other systems will
give us some hints as to what that is
actually we don't need to go that far
around the edges of the reservoir there
is a little bit of something not much
but I think I might need to up my iodine
dosage and maybe throw a filter in there
either way none of that would explain
the leak let's take apart the other
block here we
go
what the crap what is this ew why does
it look burned it kind of does look
burned it's definitely not it was
underwater but the kind of oxidation
that could occur when burning I guess
could also happen when it's being
corroded but the funny thing is is that
this is where the metal and you can
actually see metallic elements here this
is where the metal was being
deposited but there's no metal for it to
deposit onto normally in galvanic
corrosion you have a sacrificial anode
and then you have a a cathode that the
metal deposits onto but it's depositing
onto the delin or the space above it you
can see that's what was being corroded
away right which is why the fitting
broke off and liquid gushed out into the
system but the deposit was so close it
was right
here the mid plate that fitting crumbled
I think
who
okay well that's something that happened
now EK had a theory when I contacted
them and that was that this stainless
steel cover was somehow participating in
the galvanic corrosion process because
it was bridged to the fluid somehow or
something but I just don't see how it
could connect none of these screw holes
externally pass through to the inside of
the block and you can clearly see with
their Decades of experience designing
these things there's no connection here
either even if I screw this in all the
way not going to lie boy I was hoping it
would be a little bit more obvious at
this point can we just fix it and then
not care what happens
we got really lucky this time we only
lost the big ups at the bottom and I
lost one of the rows of nvme back plane
slots on one of the servers the other
row fully works and I don't need the
second row um also your computer is dead
we won't necessarily get that lucky next
time there were pools of water on the
motherboards of some of the systems
under this one so we need to figure out
what happened is what I'm trying to say
which means we need to start dissecting
some more victims actually wait before
we look at a different system we should
look at a second block within this
system to see if there's something
unique about this environment these fuel
line hoses are a bit of a nightmare to
get off you can see the reinforcing
string that makes even cutting them a
bit of a chore okay what do you think
we're going to find a lot more corrosion
and metal oh crud I called it cuz this
means your computer's not getting fixed
today oh still happy no that might do it
for EK theory about that top stainless
steel thing maybe this is a more
widespread issue now we really need to
start cracking open more systems and
seeing what's up one rail came out hold
on you got to come straight
out there you go oh my goodness the plot
thickens the second fitting on this
GPU is
pristine what is going on here could it
be the Worm Drive clamps they're not
touching though and this this one hasn't
been moved I unscrewed it by just
twisting up the tube this is exactly as
it was bro
nasty what this means to me is every one
of these systems has to have every
fitting popped to check if it's
corroding a couple of the screws going
into the top of this GPU block are
rusted this is not a damp environment so
the only moisture they should have
access to is from in
here I didn't know which holes they came
out of though unfortunately I don't
think your computer is going to be
working for a while hey thank goodness
it's summer and there's lots else to do
right yeah I wouldn't let my dad get
away with that but hey thanks what do we
have to do to get it working again it
relies on a cooling system that it
shares with six other
systems all six systems need to be
checked to ensure that they don't have
corrosion in them because if one of them
corrodes it takes down the whole thing
so all the systems need to be taken
right down to the disassembly of the
bare Parts okay so like 2 months I'm the
kind of dad who believes in honoring my
promises so I am not going to promise
anything okay this is shaping up to look
like it is going to be an absolutely
terrifying amount of work like better
off starting over amount of work here
then shouldn't we just start over well I
mean it'd be kind of was wasteful don't
you think and this is your gaming rig
this has seen you through many battles
well you just going to leave it behind
on the field
yeah now it's clear there are issues
with the loop not caused by anything to
do with block design uh there is some
organic matter in here you can really
tell now that it's still wet but I
reiterate that would not cause a leak so
then it is ek's Fault well we can't say
that definitively I think what we're
going to do is we're going to
investigate one or two of the other
systems send them our findings and then
we'll go from there the worst corroded
one in both cases was nearest the
acceleration or impingement plate but on
the GPU the deposition didn't take place
on the plate itself it took place here
before it you're probably wondering what
to do next what we need is a corrosion
check on this this this and this you're
going to start by blowing all the water
out of the system how do I do that I'm
glad you asked when they aren't coupled
these quick connects stop the flow of
water and air so there's no way for us
to blow out the system
unless we put a female fitting on them
now she should just kind of
drain o hey there we go but that's slow
and we can accelerate the process using
under normal circumstances I'd be giving
you guys a bunch of lectures about air
compressors and computers there are
concerns with moisture being blown onto
the components uh static buildup but in
this case we're just blowing out tubing
so we don't need to worry about any of
that what you're going to do is you're
going to take this you're going to hold
it tight up against the in one as you
can and all the water is going to blow
out the other side when you pull the
trigger
[Applause]
W now we don't need to worry about water
leaking all over our system as we
disconnect our hoses realis ically the
system is not alive anymore but there
could still be components that are good
remember that I said that your SSD might
be okay well that's because it's over
here on the other side from where the
leak was so as long as the water didn't
come up that high or as long as there
wasn't some kind of malfunction of the
motherboard that caused a cascading
malfunction to your SSD you might still
have all your games and data and all
your saves and all that kind of stuff
get this hex head and pull the block off
clean the thermal goop off the bottom of
the block while you're handling it so
you don't make a
mess okay next thing you want to do is
get these fittings off yeah you might
need to take off these worm drives first
so you need that Flathead there's
certain things that I really hope are
not dead I mean a motherboard's a
relatively simple replacement I even
have a cold spare of these just in case
you know we experienced a failure but
like these pcie Gen4 extensions and
stuff like
that H it'll just probably take a little
while to get new ones
then again I've made no commitments
about when this will be done so now
we're just going to play twisty twist
until this comes out not the world's
most fun game twisty twist is very
effective how'd you get to be so
sarcastic bad parenting I guess yeah
yeah yeah yeah yeah yeah yeah yeah
speaking of bad parenting I think I'm
just going to have to give you a really
sharp knife to cut the tubing off thank
you perfect
okay so now we just put it back well I
don't know man like this is Raising more
questions than it's
answering I think what we need to do is
have a look at the fittings on the GPU
now so this one's perfect on both
yeah maybe this is a system by System
thing then the fittings are perfect on
the GPU on the second system remember
our first control system as well got two
more to go though let's see what's
shaken in those phrase
interesting so yours is the bad one oh
man I guess this is It's kind of good
news but it also just deepens the
mystery and there you have it looking at
the last two systems the ones above the
leak flawless flawless flawless
lless only one system experienced
corrosion out of the five and it was on
two different
blocks for real chemistry Majors get out
of your armchairs or rather get in your
arm chairs what happened here so you're
probably wondering well what now then
well now I'm going to test the two
systems that were under number three and
see if they still work which means maybe
ooh caps up white light
good hey desktop better as unbelievable
as it might seem the only system
casualty is my son's gaming rig almost
everything else survived
is it
possible that this system still
functions I kind of doubt it it gushed
water over the motherboard like right
over the vrms let everything else
survive what do you think Andrew is she
just going to like post I'm I'm not I
think this one's
[Music]
dead now modern Electronics
got a lot of protections you know short
circuit over current over voltage but
dump gallons of flipping water over it
protection obviously I can't deploy it
the way it is it's got its blocks and
fittings all corroded to heck and back
so there's definitely going to be a
followup where I hear from some of you
folks I go back to the manufacturer we
figure out exactly what's going on we
rebuild this mess but I'd say this is
about the best outcome I could have
possibly hoped for just like I couldn't
hope for for a better sponsor than our
sponsor next to go have you ever hopped
on a meeting and thought your screen was
frozen because the person you're talking
to his picture is moving at 5 FPS how
embarrassing enhance your work from home
setup with the nexo hello cam Pro it
features Windows hello facial
recognition for an effortless login
experience and it also has an automated
privacy cover that closes when the
camera is not active plus you'll look
great at your next meeting thanks to the
1080p 60fps resolution and refresh rate
learn more using the link in the
description if you enjoyed this video
and you like this kind of raw
troubleshooting SL building with the
kids then maybe check out uh fixing the
gaming minivan they were helpful
تصفح المزيد من مقاطع الفيديو ذات الصلة
I broke my son's gaming PC and now I’m making him fix it
WAS EIN AUTO!😱💥 ELI reagiert auf "Der verunfallte Porsche GT3 ist FERTIG ✅🏎️" von @dave_
I will PAY you not to buy this
تحدي اول سلاحين من نفس النوع فقط لمده يوم كامل في ببجي موبايل ! واجهت هكر🔥😱
Safir 21. Bölüm @atvturkiye
The Secret of Piper Alpha Oil Rig Disaster lies at the Bottom of the Sea
5.0 / 5 (0 votes)