How He Got $600,000 Data Engineer Job

Summary

In this video, Zach Wilson shares his journey from a data analyst to a data engineer at major tech companies like Facebook, Netflix, and Airbnb. He discusses his salary progression from $80k to $600k, the unique work cultures at these companies, and the importance of data engineering skills like SQL, Python, and Apache Spark. Zach also addresses the impact of AI on data engineering roles and advocates for learning tools like Airflow and Databricks. He advises aspiring data engineers to focus on data modeling and machine learning for job security and provides resources to start a career in data engineering.

Takeaways

• 🚀 Zach Wilson's career started as a data analyst but evolved towards data engineering after realizing the rapid growth of data.
• 🐘 Zach was initially attracted to Hadoop, viewing it as a critical skill to learn early in his career.
• 👔 He found the corporate environment at Teradata too restrictive compared to the startup culture he preferred.
• 🌉 Zach's experience at Facebook (Meta) was significant for his growth as a data engineer, where he utilized tools like Hive and Dataflow.
• 📈 Netflix offered Zach a high salary upfront but also induced imposter syndrome due to the highly skilled team environment.
• 💼 Zach's salary progression from 80k to 600k is attributed to joining big tech companies and effectively negotiating his worth.
• 💼 He advises never to give the first number in a salary negotiation to avoid potentially lower offers.
• 💡 Zach emphasizes the importance of learning languages like SQL and Python, tools like Spark and Airflow, and skills in data modeling for aspiring data engineers.
• 🔮 The future of data engineering might see roles transformed or replaced by AI and low-code/no-code tools, but roles requiring nuanced expertise like machine learning will remain secure.
• 🌐 Zach misses the collaborative environment of corporate life but enjoys the autonomy and accountability of entrepreneurship.
• 🌟 For those looking to break into data engineering, Zach recommends his blog and the 'Data Engineer Handbook' GitHub repository for comprehensive resources.

Q & A

What was Zach Wilson's initial career role before becoming a data engineer?

Zach Wilson initially started his career as a data analyst, specifically a 'Tableau guy', which involved working with data visualization tools.

Why did Zach Wilson decide to transition from data analysis to data engineering?

Zach Wilson became bored with his data analyst role after mastering Tableau in about 9 months and decided to transition to data engineering after reading a statistic that 90% of the world's data had been created in the last 18 months, which made him realize the potential in the field.

Which company did Zach Wilson join after recognizing the growth in data and why?

Zach Wilson joined a startup called Think Big Analytics after recognizing the significant growth in data, as he was drawn to the idea of working with big data technologies, particularly Hadoop.

How did Zach Wilson's career progress from Teradata to Facebook?

Zach Wilson's career progressed from Teradata, where he worked for about 6-7 months in a very corporate environment, to Facebook after he left Teradata due to dissatisfaction with the corporate culture. He then moved to Washington DC for a brief period of software engineering before landing a data engineering role at Facebook in San Francisco.

What was the approximate salary progression for Zach Wilson from his first job to his role at Netflix?

Zach Wilson's salary progressed from 80k at Teradata to 185k after joining Facebook, and then to 365k when he joined Netflix, where he was able to negotiate an increase to 550k within his first year.

What was the significant mistake Zach Wilson made during his salary negotiation at Netflix?

Zach Wilson's significant mistake during his salary negotiation at Netflix was accepting the initial offer of 365k without doing proper research. He later found out that the median salary for his team was 500k, indicating he could have negotiated a higher starting salary.

What cultural differences did Zach Wilson experience between Facebook, Netflix, and Airbnb?

Zach Wilson experienced a very collaborative culture at Facebook, which he felt was sometimes too collaborative. Netflix had a more mature and less intrusive work culture, with team members having established personal lives. Airbnb was not explicitly described, but he mentioned that both Google and Airbnb were known for their good work-life balance.

How did Zach Wilson overcome his imposter syndrome at Netflix?

Zach Wilson overcame his imposter syndrome at Netflix after about a year, particularly after achieving a significant success with a database project, which made him feel that he truly belonged in the role.

What advice does Zach Wilson give for someone looking to get into data engineering?

Zach Wilson advises that to get into data engineering, one should learn critical languages like SQL and Python, become familiar with tools such as Spark and Airflow, and understand data modeling. He also recommends his blog and the 'Data Engineer Handbook' GitHub repository as resources.

What is Zach Wilson's perspective on the impact of AI on data engineering roles?

Zach Wilson believes that while AI will change some data engineering roles, especially those involving simple tasks, more nuanced roles involving machine learning or master data management are safe. He sees AI as a tool that can help solve problems faster but doesn't see it replacing the need for expert data engineers.

What is the 'Data mesh' architecture mentioned by Zach Wilson and why is it significant?

The 'Data mesh' architecture is a new approach where business owners can manage and maintain their own data pipelines, potentially reducing the need for traditional data engineering roles. Zach Wilson believes it could become more successful with the advancement of tools like LLM (Large Language Models), although he has seen it fail in companies due to lack of proper tooling.

Outlines

00:00

😀 Introduction to Data Engineering with Zach Wilson

The video begins with the host welcoming Zach Wilson, a data engineer, to the channel. Zach is introduced as someone who has been in the industry since its early days, starting as a data analyst and moving into data engineering roles. The conversation starts with a discussion about the evolution of data engineering as a field and Zach's personal journey, including his time at various companies like Teradata, Facebook, Netflix, and Airbnb. The host expresses excitement about discussing data engineering with Zach, hinting at the valuable insights he is expected to share.

05:00

💼 Zach's Career Progression and Experiences at Major Tech Companies

Zach shares his career progression, starting from an initial salary of $80k to eventually reaching $600k. He discusses his experiences working at different tech companies, highlighting the collaborative culture at Facebook (Meta), the all-cash compensation structure at Netflix, and the work-life balance at Airbnb. He also talks about his salary negotiations, emphasizing the importance of research and not accepting the first offer. Zach reflects on the differences in company cultures and how they affected his work and personal growth.

10:01

🚀 Zach's Transition from Corporate to Entrepreneurship

Zach talks about his decision to leave Netflix and his subsequent journey into entrepreneurship. He shares his initial struggles with identity and purpose after leaving the corporate world, and how the pandemic influenced his transition. He discusses his new role as a content creator and entrepreneur, focusing on data engineering education. Zach also expresses some of the things he misses about corporate life, such as working in a team and the structured support systems, while also appreciating the accountability and freedom that come with being an entrepreneur.

15:03

🛠 Advice for Aspiring Data Engineers and the Impact of AI

Zach provides advice for those looking to enter the field of data engineering, emphasizing the importance of learning key languages like SQL and Python, and tools like Spark and various workflow orchestration systems. He also stresses the significance of data modeling skills. Regarding the impact of AI, Zach believes that while some data engineering roles may evolve or be replaced due to advancements in AI and low-code/no-code tools, there will always be a need for specialized expertise in areas like machine learning and master data management. He predicts that the data engineering landscape will continue to change with the adoption of new architectures like Data Mesh.

Keywords

💡Data Engineering

Data Engineering is the practice of designing, building, and maintaining the infrastructure required for the storage and management of data. In the video, Zach Wilson discusses his career transition from a data analyst to a data engineer, highlighting the importance of this field in managing the vast amounts of data being generated. His journey illustrates the evolution of data engineering roles and the skills required to excel in this domain.

💡Tableau

Tableau is a business intelligence tool used for data visualization and analytics. Zach initially started his career as a 'Tableau guy,' indicating his role was focused on data analysis using this tool. The script suggests that after mastering Tableau, he sought new challenges, which led him to explore data engineering, reflecting the desire for professionals to expand their skillsets beyond single tools.

💡Hadoop

Hadoop is an open-source framework used for distributed storage and processing of big data. The 'yellow elephant' reference in the script symbolizes Zach's fascination and eventual specialization in Hadoop, which became a significant part of his work as a data engineer. This highlights the importance of Hadoop in handling large data sets and its influence on career paths in data engineering.

💡Collaborative Culture

The term 'collaborative culture' refers to a work environment that encourages teamwork and cooperation. Zach contrasts the collaborative nature of Facebook/Meta with his experiences at other companies. He mentions that while Facebook/Meta was very collaborative, it sometimes interfered with his productivity, indicating the balance companies strive for in creating a supportive yet productive work culture.

💡Imposter Syndrome

Imposter Syndrome is a psychological pattern where individuals doubt their accomplishments and fear being exposed as a 'fraud'. Zach shares his personal experience of feeling like an imposter at Netflix, especially being the youngest on his team. This concept is relevant to the video's theme as it discusses the personal and psychological challenges faced by professionals in the tech industry.

💡Salary Negotiation

Salary negotiation is the process of discussing and agreeing on a job's compensation. The script recounts Zach's experience with salary negotiation, particularly his regret over not researching enough before accepting an offer at Netflix, which cost him potentially $100,000 more. This underscores the importance of thorough research and negotiation in determining professional compensation.

💡Data Mesh

Data Mesh is an architectural pattern that promotes a decentralized data ownership model. Zach discusses the potential of Data Mesh to change traditional data engineering roles by allowing business owners to manage and maintain their data pipelines directly. This concept is integral to the video's exploration of the future of data engineering and the evolving roles within it.

💡LLM (Large Language Models)

Large Language Models (LLM) refer to advanced AI models capable of understanding and generating human-like text. Zach suggests that LLMs, combined with low-code/no-code tools, might replace certain data engineering tasks, indicating a shift towards more automated and simplified data pipeline creation. This reflects the video's discussion on the impact of AI on data engineering roles.

💡Spark

Apache Spark is an open-source distributed computing system that facilitates fast and general data processing. Zach emphasizes the importance of learning Spark for data engineers, as it continues to grow in adoption despite being around for over a decade. This highlights Spark's central role in big data processing and its relevance to the skills required in data engineering.

💡Data Modeling

Data Modeling is the process of creating a conceptual or logical model for organizing data in a database. Zach stresses the importance of data modeling in data engineering, noting that a pipeline's value is diminished if the produced data is difficult to use. This concept ties into the video's broader message about the critical skills needed for effective data engineering.

Highlights

Zach Wilson's career transition from data analyst to data engineer.

The rapid evolution of data engineering as a field.

Zach's experience at Facebook, Meta, Netflix, and Airbnb, highlighting the cultural differences.

The importance of collaboration and work-life balance in tech companies.

Zach's salary progression from $80k to $600k and the strategies behind it.

The significance of negotiation in salary increases, especially at Netflix.

Zach's entrepreneurial journey and the transition from corporate to content creation.

The role of AI in data engineering and the potential for job displacement.

The concept of Data Mesh and its impact on data engineering roles.

Zach's advice for aspiring data engineers on essential skills and learning paths.

The importance of data modeling in data engineering.

Zach's thoughts on the future of data engineering roles in the context of AI and automation.

The benefits and challenges of the entrepreneurial path compared to corporate roles.

Zach's recommendations for resources to learn data engineering, including his GitHub repository.

The impact of low-code and no-code tools on the future of data engineering.

Zach's final thoughts on the importance of specialized skills in data engineering.

Transcripts

median 50th percentile person on my team

was making 500 I do think that there's

definitely some data engineering roles

that are going to be uh replaced hi

everybody welcome back to another video

in today's video we're going to talk

about data engineering with none other

than Zach

Wilson Zach welcome to my YouTube

channel I'm so excited to have you thank

you I'm very happy to be here Zach is

currently in Seattle and we just hosted

a Meetup together and after this Meetup

we thought we would create some content

for all of you so thank you Zach for

being here and I'm so excited to talk to

you today yeah it's going to be great so

let's jump in cuz I am pretty sure when

you became a data engineer data

engineering did not exist yeah like I

actually started my career as like a

data analyst kind of Tableau guy that's

like what I did at my very first job and

I recognized at least for that role back

then this was like 2014 like 10 years

ago and I recognized I was like I don't

want to be doing this like for I'm done

like CU I feel like I mastered Tableau

in like 9 months and I was like this is

as much as this tool can do like you

can't go further and so that's when I

got kind of got bored of that role and I

was like I need to go somewhere else and

then like I read a stat stat was that

90% of the world's data had been created

in the last 18 months I the same Stu

yeah I'm telling you and I was like damn

I need to get in there right and like so

that's when I joined this startup called

think big analytics okay and the other

one I got obsessed with like the yellow

elephant Hadoop right that's like I was

just very drawn to that elephant I was

like I need to learn this elephant I

remember I made a post back then where I

was like this elephant is about to

become my entire life right and I did

not understand like how true that posted

end up being but like then I worked at

think big for a while uh I was not there

very long because what happened was

think big got acquired by terata and

terod is like this corporate company and

like you had to like wear like freaking

like business like like a button- down

shirt like it was like very corporate

very corporate I was there I was there

for like 6 months 7 months and I was

like I'm like I hate it here I don't

like this this is not the company I got

hired for I thought I was joining like

some hip startup and yeah and I was like

this is not it for me and left I left

and did like some software stuff for a

little bit like 6 months where I

literally like went from Utah to

Washington DC did software engineering

for 6 months and then I got a job at

Facebook doing data Engineering in San

Francisco so I moved again and yeah 2016

was a messy year for me where I was just

driving all the time I felt like that's

all I was doing that year but then once

I got in at Facebook like that's when I

felt like I actually was like doing real

data engineering and I was like getting

into it more like that's we using like a

lot of Hive back then it was a lot of

Hive and this thing called Data swarm

which is essentially airflow meta

Netflix Airbnb and then entrepreneurship

I want to talk about Netflix and Airbnb

and meta like how was the culture

different between the three oh they're

very different So like um Facebook one

of the things about is great about

facebook/ meta is that like it's very

friendly everyone is very collaborative

in some ways I actually felt like it was

too collaborative right where I was like

I'm just trying to get my work done and

then because like then people like hey

Zach I have a question hey Zach I have a

question I'm like dude I get 30 minutes

a day to work on things right like and S

familiar and so like uh and then Netflix

was not as much that way right well and

one of the other things at least for me

in my journey there was like I got hired

at Netflix and when I was really young

especially for Netflix cuz so when I

joined Netflix I was 24 and the next

youngest person on my team was 35 so

like there was an 11year gap between me

and the next youngest person on my team

and so like for me definitely at Netflix

I like the culture was still very

collaborative but also people like had

their own lives their own families they

have all their own like stuff like and

all that stuff and like I like I for me

in my relationship to work in Netflix

though I just had imposter syndrome the

whole time because it was just like I

was like I don't belong here like I'm

working with all these really talented

people like why am I here too right and

so I like and I just felt like I'm like

I have to work to like prove that I

belong here right yeah for sure did you

get to that point then like where you're

imposter and went away yeah it was like

um I felt like it was after about a year

when I like when I especially when I got

like my first big win with this like

database thing I was working on like I

was like okay okay I belong here I

belong here for I love that I love that

I feel like especially like first year

for anybody like who's just starting out

it's always like cuz I had a similar

experience first year such a big

imposter syndrome and then after

delivering some stuff I'm like no I

think I got it like I can do it so I

know you started when you got your first

job you started at 80k M how did that

progression go from 80k to meta Netflix

and then

Airbnb 600k that's like unheard of so

how did you do that how do people do

that yeah it's a journey like and it was

actually something that I even was

surprised by myself especially like when

I got that job making 80k at terod dat

back when I had a wear a button- down

shirt and I hated it like I my dream for

myself was like I'm like dude if I can

get to 200 I will be such a baller I

will be like this is going to be my life

and I thought 200 was like when I'm 35

right when I'm like 10 years deep 15

years deep that's when I'm going to hit

200 I learned a couple things about it

like a big thing about it is like you

need to get into the system in big Tech

that's a big part of it is like if you

can get that experience in especially

meta specifically I feel like they do a

great job at like investing in their

Engineers better than Amazon I feel cuz

Amazon really like holds that senior

promotion as like a you got to really

like work incredibly hard to get senior

whereas a meta like they like encourage

people more like my friend Ryan right he

did from he went from Junior to staff at

meta in three years he just got promoted

every wow you have to do that at meta

right they push you they really do like

I mean it's good and bad I feel like cuz

it's like it's good like if you want to

grow but it's bad if you want like a

life if you want to CH if you want a

coast meta is not the place for you no

that's

Google or Airbnb or Airbnb both yeah

both are really great so for me like

when I got in at Facebook so I went from

80 and then like a year later cuz I was

making adk in 2015 at terod data and

then when I got in at Facebook it was

like 185 only actually ever been

internally promoted one time in my

career one time and it was at Facebook

at Facebook cuz I got hired as a junior

engineer at Facebook and I got promoted

to L4 right that's the only promo I've

ever actually gotten so going to L4

bumped me to like 230 is right and then

I was like I'm done I want to be a

software engineer and I'm like then I go

to Netflix Netflix is crazy Netflix is a

crazy company they give you all cash

yeah they give you all cash just up

front here you go we're going to pay you

as much as like the market will give you

right and I actually made a mistake I

made a big mistake at when I got my

interview at Netflix I probably lost

$100,000 from from this mistake so I

accepted my initial offer at Netflix I

accepted was 365k cuz going from 220 to

365 I was overjoyed right like I

actually learned though I learned that

the median for my team the not the

highest paid guy the median 50th

percentile person on my team was making

500 right so and I didn't even know

about that I just was like so excited to

get in at Netflix I learned about that

about six months deep into my into my

time were they expecting you to

negotiate is that why they gave you

offer yeah I thought I was being smart

so what I said was like I won't accept

anything less than what an E5 makes at

meta right and they gave me exactly that

right you basically yeah and like but

they they would have given me more than

$100,000 more like had I freaking like

actually like done my research that's

why you never give the first number

never never never and good news though

was about that was that like uh

ultimately after about 6 seven months at

Netflix I got on to a very like senior

like borderline staff level project like

this graph database project and I was

able to negotiate up I was able to

negotiate up in my first year at Netflix

from 365 to 550 right and then I was

like damn those like I've never had a

raise like that before I was like this

is a crazy raise crazy especially if

it's all cash yeah it's all cash right

your bank account is probably freaking

out taxes

man don't talk to me about taxes on the

topic of tech salary I analyzed

entry-level data scientist salary using

this powerful AI tool that you should

know about it's called Julius AI think

of it like a data analyst or a data

scientist that is available at your

fingertips that can write python code

for you and yes if you're wondering it

is 10 times better than Chad GPT

Advanced analysis don't believe it let

me show you so the first thing we're

going to do is find a file that we're

going to use for data analysis next

thing I do is I go to Julius so here I

have an option to either upload the file

or connect it to Google Sheets next

thing I'm going to do is I'm going to

ask it to filter the data with zero or

one year of experience and the coolest

thing is that it writes code while doing

it so if you're just learning python or

R yes it can do R it has R capabilities

as well so if you want to learn this is

actually a great place to Learn Python

and R while it's doing the data analysis

for you the next thing I'm going to do

is I'm going to ask you to create

visualization of data scientist salary

by company in Seattle and it has created

the visualization for me basically

looking at this plot I can tell that

Uber pays the highest amount of

entry-level salary to data scientist

followed by Amazon then Zillow and then

Facebook so as you can see that it has

not only created a visualization for me

it has also given me additional prompts

and things that I can ask you this took

me less than 3 minutes imagine if you

had access to it actually you do you can

try Julius AI for free using the link in

description thank you Julius AI for

sponsoring this section of the video now

let's go back to Zach's salary

progression to 600k M 20 19 2020 like

they Netflix changed they changed a lot

so the big thing they did so before they

had two orgs they had data engineering

and infrastructure and then they had

data science and algorithms two orgs

right and then what they did in 2019 is

they just they fired the data

engineering leaders all of them my VP my

director my manager the whole chain just

right acts right and then they were like

oh yeah we're not hiring him we're not

going to rehire those leaders you guys

are all now part of data science and for

me I was like I don't know about that I

don't think cuz especially because the

leaders that they got they let they let

go I really believed in MH so I became

very disillusioned about working there

because I was like also just living in

fear cuz I was like okay these people

who I really these leaders who I looked

up to like they taught me a lot like I

feel like they were like some of the

best people I've ever worked with and

like they're getting let go and I didn't

really get it and I'm like and I lost

all motivation to work there so I was

like I'm done I'm not going to work here

anymore and then I left and I just like

went on some soul searching for a bit

cuz I like I realized at that point that

work was my life that was my entire life

that was everything right and so like

after I quit there was some months there

where I was like who Am I who am I as a

person who is Zach right I was like I I

really lost in touch with like my own

like person and then 2020 pandemic that

year was wild that year was so wild and

then ultimately I got in back at Airbnb

like 9 months later right when I started

uh at Airbnb that was also when I

started making content CU I knew cuz

when I quit Netflix it's the work life

balance it's for it's true for me too

yeah yeah for sure gole content yeah and

for me when I quit Netflix I had a

vision for myself that I was going to be

a Creator cuz I vividly remember when I

quit Netflix the day I quit when I was

like I we had like a going away lunch

and when I was gone I was like when I

like I gave a peace sign to all my

co-workers and I'm like you're going to

see me again everywhere that's what I

said right I had a vision for myself

then and I wish I would have overcome

some of that like depression and sadness

in 2020 to start earlier but I mean I'm

happy I started when I did cuz like I

was able to get in it and like uh just

start growing cuz it's all about like

just getting that initial momentum of

like I'm going to do this this is my new

thing right then after a while it's like

compound interest just keeps you

motivated after a while you know so 600k

at Airbnb yeah for sure and that's when

you decided to leave all together and

when was it this was about a year ago

2023 okay so for a year you have been an

entrepreneur primarily focused on data

engineering what like data engineering

teaching uh that's my big Focus yeah

okay and how is that going do you miss

corporate Ah that's a great question I

miss some things about corporate yes I

mean I actually gave an interview a

couple weeks ago I had my own doubts

like the entrepreneur journey is very

full of doubts itself when you have

employees and you need to make payroll

and you have like all of this like

pressure to like you have to make sales

it's like if you don't make sales you're

going to have to fire people right it's

like there's like a lot of like

consequences like when you're an

entrepreneur that are like when you're

an engineer you just don't think about

them you just think about solving the

problem in front of you I feel like for

me the things that I miss the most about

corporate is the big one is working on a

team actually like in an office I missed

the office actually a lot that was

actually one of the things that like I

was like kind of seeking out where I'm

like I need I need to like be around

people and like work on problems

together uh I think that's the big thing

I miss I kind of also miss just like

being able to like not think about food

and have to like having to like feed

myself three times a day it's a lot of

work when I worked at meta it was like

they would give me breakfast lunch and

dinner and it's just like automated and

I don't have to think about it right

that's also really nice so I know that's

a very Tech bro take right there but

that's like it is what it is right no

it's a fair take for F I'm there the

things I don't miss are like feeling

like uh I'm in a situation that is

outside of my control like I feel like

one of the things that's great about an

entrepreneur is it's extreme

accountability like if you are

successful it's because of things you

did if you aren't successful it's

because of things that you did or didn't

do so talking about data engineering uh

really quick if somebody wants to get

into Data engineering what should they

learn where should they start what's

your advice to them that's great uh I

think there's a couple things there like

you have to learn the languages

languages are the like critical It's

like because if you don't know the

languages you can't speak the stuff

that's necessary so you SQL and python

are going to be the most important if

you're trying to be more like in the

like Cutting Edge space then maybe learn

like Scala or rust rust in particular is

probably like in the future like give it

like five years it's going to be bigger

um and then uh then you need to know the

tools the tools are like spark airf flow

but there's like 5 million competitors

for airf flow now there's like air flow

Mage prefect Dag data bricks workflows

Azure workflows there's like a there's

there's a lot of freaking ways to

schedule a job right so and just

learning Kon and how to schedule a job

and then picking one flavor of that and

then spark is so important though I

think that spark is going to keep being

bigger it's actually getting more

adoption which I think is crazy because

it's been around for so long it's been

around for like 12 years and it's like

and it and it still feels like it's in

the you know the early part of the

adoption curve it hasn't like it hasn't

leveled off yet it's still like it's

still growing which

boggles my mind I feel so lucky that I

like caught that in 2016 where I'm like

I'm going to learn spark and it's going

to be awesome and I was

right that's awesome yeah I think and

then the last important skill though is

data modeling because you got to

remember that like just because you have

a pipeline that is productive and good

and efficient if it produces data that

is annoying to use then you you drop all

your value at The Last Mile right and

that that last mile of like what is the

contract between you and the data

scientist right like what is like what

are the name what do you name your

columns make sure you don't have stupid

column names it's like there's a lot of

these things are actually very basic but

a lot of people get them wrong you like

you'll see it in the wild even in big

Tech at companies where they pay people

500k they still get it wrong and so like

getting that like modeling stuff right

is very important and I think those are

going to be the three ones like so

orchestration spark and modeling are

going to be the big things and then like

where to start I mean my blog is pretty

good so blog. dateng engineer. a lot of

things there I have a lot of road maps

that go a lot more in detail than the

last two minutes that I've said but like

that's going to be a big one and then

there's like a big another thing the

Google Google the data engineer handbook

so I have a GitHub repository that has

over 8,000 stars that has all the

resources that you need to get into Data

engineering and yeah I built that like a

couple months ago in the description

awesome yeah so that's how I'd say to

get in for sure okay awesome cool last

thing and then we'll close the video um

AI yeah should data Engineers be worried

about AI I mean ah great question I

think less worried than the current

market makes it feel like it sometimes

it feels like you know there's like

those things like Devon that are out

there where it's like look this thing is

can just do your job and it's like no it

can't like like I mean but like there's

uh there are spaces right like I feel

you need to be able to leverage AI in a

way like for example for me like when I

uh have been building things like uh

like you can solve problems like so much

faster like if you just use AI like and

like there it it's like a I think of it

more of like as like a super Google it's

like a Google that gives you a an answer

that might just be right and you can

paste it in and it just works right but

not all the time obviously you still

need to know the nuances of things but

like I think as a tool it's great and I

think uh there are going to be roles I

do think that there are I mean I I do

think that there's definitely some data

engineering roles that are going to be

uh replaced and uh changed or like

they're going to morph right because

like especially with like the

combination of llm generated stuff and

then you have like uh low code no code

tools you know like five Tran and all

those other kind of tools there and when

those things marry and then you have

like people who can create pipelines

with like a sentence and five Tran then

like there's going to be a lot of those

roles that right now are done like by a

data engineer that don't need to be done

by a data engineer and that's why

there's this new architecture that's

coming out called Data mesh where you

have people who like cuz the data

engineering pattern itself is in some

ways kind of like a middle

right because like you as a data

engineer need to talk to the business

and talk to the analysts and understand

try to understand all the pieces of the

puzzle when like you aren't as close to

the business yourself like you are like

kind of in the middle ultimately if the

the business owners the people who are

solving those problems directly like I

don't it might be the PMS it might be

some other people if those people can

write the pipeline themselves and manage

and maintain the pipeline themselves

it's a better ownership pattern it's a

more stable ownership pattern as well

and that's what data mesh is looking to

solve and I ultimately think that data

mesh will work even even though I've

seen it fail I've seen it fail a bunch

of times in companies but I think it's

because we don't quite have the tooling

yet and llm is one of those things

that's going to make that more likely to

happen yeah but for a lot of the data

engineering roles the ones that are not

just like a select query and an

aggregation or like something that's

just a little bit more like anything

doing with like Master data or machine

learning or anything like in those areas

those roles are very safe because those

roles are just so nuanced and so

difficult to get the the model right

that like you need to have that person

with that expertise so that's where if

you want to like have a safer data

engineering role lean more into uh like

machine learning stuff lean more into

Master data management and because those

roles are very safe I do not see those

roles going away anytime soon awesome

cool where can people find you yeah I

mean I'm on YouTube data with Zach is

probably going to be the best place you

can also find me anywhere um on the

internet my username is exactly e CZ l y

and that's yeah those are going to be

the main places awesome well thank you

so much this is awesome and hopefully

people watching they learned something

watching from this video so thank for

sure glad to be here yeah awesome all

right bye so good