Is this my Fault?

Summary

In this gripping video, the creator recounts the worst day of their life, marked by personal loss and a catastrophic server room leak. The narrative unfolds as they investigate the cause of the leak, which leads to a perplexing case of galvanic corrosion in their custom water cooling loop. Despite the extensive damage, the majority of their systems survive, with only the son's gaming rig succumbing to the disaster. The video promises a follow-up to unravel the mystery and rebuild the affected system, engaging viewers with a blend of personal story and technical troubleshooting.

Takeaways

• 😢 The narrator experienced the worst day of their life with the loss of a family member and a significant issue with their server rack cooling system.
• 🔍 Upon discovering a water leak, the narrator embarked on a meticulous inspection of their server equipment to identify the source of the problem.
• 🛠️ The cause of the leak was a severely corroded fitting in the cooling system, which resulted in water damage to multiple servers and gaming rigs.
• 🤔 The narrator questioned the occurrence of galvanic corrosion in a system composed only of copper, brass, and plastic, as stainless steel was not in direct contact with the fluid.
• 🔬 The narrator followed a scientific method to investigate the issue, comparing fittings from unaffected parts of the system to identify discrepancies.
• 🧩 The disassembly of the affected components revealed unexpected oxidation and metal deposition, suggesting an unusual corrosion process.
• 👨‍🔧 Contact with the manufacturer (EK) led to a theory that the stainless steel cover might have contributed to the corrosion, though the narrator remained skeptical.
• 🔄 The investigation expanded to other systems sharing the same cooling loop to determine if the issue was isolated or widespread.
• 🛡️ Despite the extensive damage, most of the systems survived the water leak, with only the son's gaming rig being the main casualty.
• 🔍 The narrator planned to conduct further tests on the affected systems and collaborate with the manufacturer to understand the root cause of the corrosion.
• 🔨 The video ends with a commitment to rebuild and rectify the situation, highlighting the complexity of troubleshooting and the importance of understanding the underlying issues.

Q & A

What was the significant event that happened on June 14th of this year in the narrator's life?

The narrator found out that his sister had passed away, which he describes as probably the worst day of his life so far.

What issue did the narrator encounter with his server rack's cooling system?

The narrator discovered that the pump running the cooling system for his server rack was running dry, indicating a leak in either the reservoir or the systems.

What was the initial visible sign of the problem with the server rack?

A watermark on the floor indicated the presence of water, which led the narrator to investigate further and find the source of the leak.

What was the cause of the leak in the server rack's cooling system?

The leak was caused by a completely corroded fitting that dumped gallons of water onto everything underneath it.

What is galvanic corrosion, and why was the narrator surprised to find it in his cooling system?

Galvanic corrosion is an electrochemical corrosion process that occurs when two different metals are in contact with each other and an electrolyte. The narrator was surprised because the loop in question only had Copper, Brass, and plastic, which should not cause galvanic corrosion.

What was the narrator's goal after discovering the issue with the cooling system?

The narrator's goal was twofold: to understand what caused the corrosion and to get his son's gaming rig back up and running.

What did the narrator find when he disassembled the blocks of the cooling system?

Upon disassembling the blocks, the narrator found dust, plastic filings, metal shavings, and organic matter, but none of these would explain the leak or the corrosion.

What theory did EK, the manufacturer, propose regarding the stainless steel cover's involvement in the corrosion process?

EK theorized that the stainless steel cover might have been participating in the galvanic corrosion process because it was somehow bridged to the fluid, although the narrator did not see how this could be possible.

What was the outcome of the corrosion issue on the narrator's computer systems?

The only system that was a casualty due to the corrosion was the narrator's son's gaming rig. Almost everything else survived, despite being subjected to the leaking water.

What steps did the narrator take to address the aftermath of the corrosion issue?

The narrator planned to test the affected systems to see if they still worked, contact the manufacturer for further investigation, and rebuild the damaged system.

What preventive measures did the narrator consider to avoid future corrosion issues?

The narrator considered increasing the iodine dosage in the loop and possibly adding a filter to prevent future corrosion issues.

Outlines

00:00

😢 Tragic Loss and Server Catastrophe

The narrator recounts the devastating day of June 14th, marked by the loss of their sister and the subsequent family turmoil. Amidst the grief, they faced a technical nightmare as their server rack's cooling system malfunctioned, causing extensive water damage. The server room contained multiple machines, including personal and work-related equipment, as well as their children's gaming rigs. The root cause was a corroded fitting that had leaked gallons of water, leading to a frantic disassembly process to assess the damage and identify the source of the leak. The situation was further complicated by the presence of galvanic corrosion, which was unexpected given the materials used in the loop.

05:00

🔍 Investigating the Mystery of Galvanic Corrosion

After reaching out to EK, the narrator explores the theory that a stainless steel cover might have contributed to the galvanic corrosion process. However, upon inspection, no direct connection to the fluid was found. The narrator then proceeds to disassemble various components, including GPU blocks, to understand the extent of the corrosion. They find pristine fittings on some components and severely corroded ones on others, suggesting a system-specific issue rather than a widespread design flaw. The complexity of the problem is highlighted by the need to check each fitting across all connected systems to prevent future failures.

10:01

🛠️ Systematic Disassembly and the Search for Answers

The video script details the process of disassembling the water cooling system to its bare parts in order to conduct a thorough corrosion check. The narrator describes the challenges of removing fittings and hoses, and the use of an air compressor to blow out the remaining water from the system. They express hope that some components, like the SSD, might have survived the ordeal due to their location away from the leak. The summary of findings from the other systems reveals that only one system suffered from corrosion, deepening the mystery of the cause. The narrator also hints at a follow-up to address the manufacturer's response and the rebuilding process.

15:03

🎮 Post-Mortem and the Unexpected Survival of Systems

In the aftermath of the water damage, the narrator tests the affected systems to assess their functionality. Surprisingly, most systems, including the desktop, survive the incident, with the exception of the narrator's son's gaming rig, which suffered direct water damage to the motherboard. The narrator contemplates the possibility of the gaming rig still functioning despite the odds and hints at a future update on the situation. They also express gratitude for the sponsor, Nexo, for their contribution to the video and encourage viewers to check out other related content.

Keywords

💡Galvanic Corrosion

Galvanic corrosion is an electrochemical process where one metal corrodes preferentially over another when both are in electrical contact and exposed to an electrolyte. In the video, the narrator encounters this phenomenon with their water cooling system for computer components, where a fitting corroded away, leading to a leak. The script mentions the unexpected nature of this corrosion given the materials used in the loop.

💡Water Cooling System

A water cooling system is a method used to cool the components of a computer, particularly the CPU and GPU, by circulating a liquid, usually water, through a series of pipes and radiators. In the video, the failure of a component within the water cooling system results in water leakage, causing damage to the narrator's server rack and other equipment.

💡Server Rack

A server rack is a framework or enclosure used to organize and house computer servers, networking equipment, and other hardware. In the script, the narrator describes discovering a water leak in their server rack, which houses multiple servers and gaming rigs, leading to a significant issue due to the water damage.

💡Fitting

In the context of the video, a fitting refers to a part used to connect pipes or tubes, often in a plumbing or cooling system. The narrator identifies a corroded fitting as the source of the leak in their water cooling system, which dumped gallons of water onto the equipment below.

💡Dust and Organic Matter

Dust and organic matter are contaminants that can accumulate in a closed system like a water cooling loop. The narrator finds dust, plastic filings, metal shavings, and what appears to be hair within the loop, which could potentially contribute to the corrosion process, although the script suggests that they do not directly cause the leak.

💡Iodine

Iodine is a chemical element that can be used in water cooling systems to help prevent the growth of algae and bacteria. The script mentions the presence of iodine in the loop, which is part of the maintenance routine to keep the system clean, but it is not the cause of the corrosion issue.

💡GPU

A GPU, or Graphics Processing Unit, is a specialized electronic circuit designed to rapidly manipulate and alter memory to accelerate the creation of images in a frame buffer intended for output to a display device. In the video, the narrator inspects the GPU block and its fittings for signs of corrosion, finding one fitting that appears burned or heavily corroded.

💡Corrosion Check

A corrosion check involves inspecting a system or component for signs of material degradation due to a chemical reaction, often oxidation. The narrator plans to perform a corrosion check on the remaining systems connected to the water loop to ensure that they have not been similarly affected by the corrosion issue.

💡Worm Drive Clamps

Worm drive clamps are a type of hose clamp that uses a screw mechanism to tighten a band around an object, often used in automotive and plumbing applications. The script mentions these clamps in the context of the water cooling system, where they are used to secure the hoses but are not believed to be the cause of the corrosion.

💡NVMe

NVMe stands for Non-Volatile Memory Express, a protocol that allows for high-speed data transfer between a host system and a non-volatile memory subsystem. The narrator mentions NVMe back plane slots on one of the servers, which were affected by the water leak, indicating potential data loss or hardware damage.

💡SSD

An SSD, or Solid-State Drive, is a data storage device that uses solid-state memory to store persistent data. The narrator speculates that the SSD might still be functional despite the water damage, depending on the extent of the leak and the specific components affected.

Highlights

June 14th was identified as the worst day of the narrator's life due to the unexpected death of his sister.

The narrator had to manage family drama and grief while also dealing with a difficult work show.

Upon returning home, the narrator discovered an unexplained water leak in the server room's cooling system.

The water source was traced to a pump running dry, indicating a leak in the reservoir or the systems.

A meticulous process of disassembling the server rack was initiated to identify the leak's source.

A corroded fitting was found to be the cause of the water leakage, affecting multiple servers and gaming rigs.

The narrator expressed confusion over the galvanic corrosion occurrence given the materials used in the loop.

A comparison was made with other fittings in the loop that showed no signs of corrosion.

The goal was set to understand the cause of the corrosion to prevent future incidents and restore the gaming rig.

Disassembly of the water block revealed dust, plastic filings, and metal shavings, but no clear cause for the leak.

A theory was proposed that the stainless steel cover might be involved in the galvanic corrosion process.

Further investigation showed different levels of corrosion across various systems, deepening the mystery.

The narrator considered the possibility of starting over due to the extensive work required for repair.

A method for blowing out water from the system using an air compressor was demonstrated.

Despite the extensive damage, most systems survived, with only the son's gaming rig being a casualty.

The narrator concluded that the issue was not due to block design and that organic matter was found in the loop.

A plan to test the affected systems and communicate findings with the manufacturer was outlined.

The video ended with a teaser for a follow-up on the issue and a shout-out to the sponsor for the video.

Transcripts

June the 14th of this year was probably

the worst day of my life so far the

previous evening I had found out that my

sister passed away I had spent the day

dealing with the family drama and grief

that comes with an unexpected loss like

that I made my way through one of the

hardest W shows that I've ever done um

just focusing on the routine and when I

came home there was an unexplained wine

that I couldn't really nail down the

source of I made my way downstairs

and I found out that the pump that runs

the cooling system for my server rack

here was running dry that means one of

two things the reservoir is leaking or

the systems are leaking and I'll let

this water mark on the floor answer the

question for you so instead of eating I

began the painstaking process of tearing

everything out of the rack as safely as

possible I made my way all the way

through both storage and utility servers

my main and my backup mine and yvon's

office machines and all three of my kids

gaming rigs until I found this the

culprit a completely corroded off

fitting that dumped gallons of water on

everything underneath it how could this

happen I mean that's galvanic corrosion

plain as the nose on my face but this is

a loop that has nothing but Copper Brass

plastic what the is going on here

I've had to sit and Wonder from then

until now so that I could get someone

over here to point a camera at this

because I'm sure that you guys want to

know the answer as badly as I do and if

I'm going to lose a bunch of server

equipment well I might as well at least

help pay for it with this message from

our sponsor if I had time to untangle

cables would my hair look like this why

not switch to Magnetic cables from our

sponsor mag tame

instead they pull apart effortlessly

roll up perfectly and stick to any

magnetic surface no stress no mess no

worries we'll have them Linked In the

video description now I know what you

guys are thinking ah it's lius he's

always doing janky projects and I'm sure

he overlooked something I mean galvanic

corrosion can't happen by accident

that's

science but listen to this you want to

talk science how about the scientific

method for a control system I grabbed

one of the other ones in the loop and

look at those fittings those are the

same two fittings and they're immaculate

so if it was a configuration error in

the loop these should be eaten and yet

they're pristine our goal today is

twofold then my son obviously wants his

gaming rig back up and running which is

why he's helping but before we do that

we need to know what the heck happened

here so that we can ensure that this

won't happen again the first step of

disassembling these blocks is popping

the stainless steel cover off it's

important to note that this is purely

cosmetic since steel is not a compatible

metal with the ones that are typically

used for water cooling like Copper Brass

and

nickel now let's go a layer deeper and

remove the delin top okay not the

cleanest Loop I've ever seen why is

there so much dust on it well I can tell

what some of this is these look like

little plastic filings from working with

the tubing that we installed oh there's

a little metal shaving uh that could be

left over for manufacturing of something

radiator heat exchanger there's a

surprising amount of this that looks

like hair or other organic matter but

there's iodine in the loop and what kind

of contradicts that theory is that this

tube over here which is running water to

my system right now is Crystal Clear

well maybe one of the other systems will

give us some hints as to what that is

actually we don't need to go that far

around the edges of the reservoir there

is a little bit of something not much

but I think I might need to up my iodine

dosage and maybe throw a filter in there

either way none of that would explain

the leak let's take apart the other

block here we

go

what the crap what is this ew why does

it look burned it kind of does look

burned it's definitely not it was

underwater but the kind of oxidation

that could occur when burning I guess

could also happen when it's being

corroded but the funny thing is is that

this is where the metal and you can

actually see metallic elements here this

is where the metal was being

deposited but there's no metal for it to

deposit onto normally in galvanic

corrosion you have a sacrificial anode

and then you have a a cathode that the

metal deposits onto but it's depositing

onto the delin or the space above it you

can see that's what was being corroded

away right which is why the fitting

broke off and liquid gushed out into the

system but the deposit was so close it

was right

here the mid plate that fitting crumbled

I think

who

okay well that's something that happened

now EK had a theory when I contacted

them and that was that this stainless

steel cover was somehow participating in

the galvanic corrosion process because

it was bridged to the fluid somehow or

something but I just don't see how it

could connect none of these screw holes

externally pass through to the inside of

the block and you can clearly see with

their Decades of experience designing

these things there's no connection here

either even if I screw this in all the

way not going to lie boy I was hoping it

would be a little bit more obvious at

this point can we just fix it and then

not care what happens

we got really lucky this time we only

lost the big ups at the bottom and I

lost one of the rows of nvme back plane

slots on one of the servers the other

row fully works and I don't need the

second row um also your computer is dead

we won't necessarily get that lucky next

time there were pools of water on the

motherboards of some of the systems

under this one so we need to figure out

what happened is what I'm trying to say

which means we need to start dissecting

some more victims actually wait before

we look at a different system we should

look at a second block within this

system to see if there's something

unique about this environment these fuel

line hoses are a bit of a nightmare to

get off you can see the reinforcing

string that makes even cutting them a

bit of a chore okay what do you think

we're going to find a lot more corrosion

and metal oh crud I called it cuz this

means your computer's not getting fixed

today oh still happy no that might do it

for EK theory about that top stainless

steel thing maybe this is a more

widespread issue now we really need to

start cracking open more systems and

seeing what's up one rail came out hold

on you got to come straight

out there you go oh my goodness the plot

thickens the second fitting on this

GPU is

pristine what is going on here could it

be the Worm Drive clamps they're not

touching though and this this one hasn't

been moved I unscrewed it by just

twisting up the tube this is exactly as

it was bro

nasty what this means to me is every one

of these systems has to have every

fitting popped to check if it's

corroding a couple of the screws going

into the top of this GPU block are

rusted this is not a damp environment so

the only moisture they should have

access to is from in

here I didn't know which holes they came

out of though unfortunately I don't

think your computer is going to be

working for a while hey thank goodness

it's summer and there's lots else to do

right yeah I wouldn't let my dad get

away with that but hey thanks what do we

have to do to get it working again it

relies on a cooling system that it

shares with six other

systems all six systems need to be

checked to ensure that they don't have

corrosion in them because if one of them

corrodes it takes down the whole thing

so all the systems need to be taken

right down to the disassembly of the

bare Parts okay so like 2 months I'm the

kind of dad who believes in honoring my

promises so I am not going to promise

anything okay this is shaping up to look

like it is going to be an absolutely

terrifying amount of work like better

off starting over amount of work here

then shouldn't we just start over well I

mean it'd be kind of was wasteful don't

you think and this is your gaming rig

this has seen you through many battles

well you just going to leave it behind

on the field

yeah now it's clear there are issues

with the loop not caused by anything to

do with block design uh there is some

organic matter in here you can really

tell now that it's still wet but I

reiterate that would not cause a leak so

then it is ek's Fault well we can't say

that definitively I think what we're

going to do is we're going to

investigate one or two of the other

systems send them our findings and then

we'll go from there the worst corroded

one in both cases was nearest the

acceleration or impingement plate but on

the GPU the deposition didn't take place

on the plate itself it took place here

before it you're probably wondering what

to do next what we need is a corrosion

check on this this this and this you're

going to start by blowing all the water

out of the system how do I do that I'm

glad you asked when they aren't coupled

these quick connects stop the flow of

water and air so there's no way for us

to blow out the system

unless we put a female fitting on them

now she should just kind of

drain o hey there we go but that's slow

and we can accelerate the process using

under normal circumstances I'd be giving

you guys a bunch of lectures about air

compressors and computers there are

concerns with moisture being blown onto

the components uh static buildup but in

this case we're just blowing out tubing

so we don't need to worry about any of

that what you're going to do is you're

going to take this you're going to hold

it tight up against the in one as you

can and all the water is going to blow

out the other side when you pull the

trigger

[Applause]

W now we don't need to worry about water

leaking all over our system as we

disconnect our hoses realis ically the

system is not alive anymore but there

could still be components that are good

remember that I said that your SSD might

be okay well that's because it's over

here on the other side from where the

leak was so as long as the water didn't

come up that high or as long as there

wasn't some kind of malfunction of the

motherboard that caused a cascading

malfunction to your SSD you might still

have all your games and data and all

your saves and all that kind of stuff

get this hex head and pull the block off

clean the thermal goop off the bottom of

the block while you're handling it so

you don't make a

mess okay next thing you want to do is

get these fittings off yeah you might

need to take off these worm drives first

so you need that Flathead there's

certain things that I really hope are

not dead I mean a motherboard's a

relatively simple replacement I even

have a cold spare of these just in case

you know we experienced a failure but

like these pcie Gen4 extensions and

stuff like

that H it'll just probably take a little

while to get new ones

then again I've made no commitments

about when this will be done so now

we're just going to play twisty twist

until this comes out not the world's

most fun game twisty twist is very

effective how'd you get to be so

sarcastic bad parenting I guess yeah

yeah yeah yeah yeah yeah yeah yeah

speaking of bad parenting I think I'm

just going to have to give you a really

sharp knife to cut the tubing off thank

you perfect

okay so now we just put it back well I

don't know man like this is Raising more

questions than it's

answering I think what we need to do is

have a look at the fittings on the GPU

now so this one's perfect on both

yeah maybe this is a system by System

thing then the fittings are perfect on

the GPU on the second system remember

our first control system as well got two

more to go though let's see what's

shaken in those phrase

interesting so yours is the bad one oh

man I guess this is It's kind of good

news but it also just deepens the

mystery and there you have it looking at

the last two systems the ones above the

leak flawless flawless flawless

lless only one system experienced

corrosion out of the five and it was on

two different

blocks for real chemistry Majors get out

of your armchairs or rather get in your

arm chairs what happened here so you're

probably wondering well what now then

well now I'm going to test the two

systems that were under number three and

see if they still work which means maybe

ooh caps up white light

good hey desktop better as unbelievable

as it might seem the only system

casualty is my son's gaming rig almost

everything else survived

is it

possible that this system still

functions I kind of doubt it it gushed

water over the motherboard like right

over the vrms let everything else

survive what do you think Andrew is she

just going to like post I'm I'm not I

think this one's

[Music]

dead now modern Electronics

got a lot of protections you know short

circuit over current over voltage but

dump gallons of flipping water over it

protection obviously I can't deploy it

the way it is it's got its blocks and

fittings all corroded to heck and back

so there's definitely going to be a

followup where I hear from some of you

folks I go back to the manufacturer we

figure out exactly what's going on we

rebuild this mess but I'd say this is

about the best outcome I could have

possibly hoped for just like I couldn't

hope for for a better sponsor than our

sponsor next to go have you ever hopped

on a meeting and thought your screen was

frozen because the person you're talking

to his picture is moving at 5 FPS how

embarrassing enhance your work from home

setup with the nexo hello cam Pro it

features Windows hello facial

recognition for an effortless login

experience and it also has an automated

privacy cover that closes when the

camera is not active plus you'll look

great at your next meeting thanks to the

1080p 60fps resolution and refresh rate

learn more using the link in the

description if you enjoyed this video

and you like this kind of raw

troubleshooting SL building with the

kids then maybe check out uh fixing the

gaming minivan they were helpful