ML Was Hard Until I Learned These 5 Secrets!

Summary

In this video, the speaker reveals five essential 'secrets' to mastering machine learning, which are often overlooked in traditional education. These secrets include understanding the human ideas behind mathematical formulas, collecting and applying mathematical rules and patterns, recognizing that debugging is a crucial part of coding, effectively navigating large codebases, and maintaining realistic expectations about the time and effort required to master machine learning. The speaker emphasizes that these insights can transform the learning experience, making complex concepts more accessible and manageable over time.

Takeaways

• 🤔 Understanding math in machine learning is about translating human ideas into mathematical formulas.
• 🧠 Think like a scientist and not just focus on the math itself.
• 🔍 Each step in a mathematical derivation is just applying a specific rule or definition.
• 🛠️ Build a mathematical toolkit of rules and tricks to apply during derivations.
• 🐍 Coding in machine learning involves a lot of debugging, which is a normal and expected part of the process.
• 🔧 When dealing with large codebases, start by understanding key files and use debugging to step through the code.
• 📚 Minimal educational implementations of algorithms can help in understanding the main ideas.
• 💡 Mastering machine learning takes time and perseverance, not just a few weeks of study.
• 📈 Practical experience, working on projects, and reading state-of-the-art papers are essential for mastery.
• ⏳ Having realistic expectations about the time and effort required makes the learning process easier and more enjoyable.

Q & A

What is the main issue the speaker faced when starting to learn machine learning?

The speaker focused too much on the actual mathematical formulas rather than understanding the human ideas behind them.

What is the first secret the speaker reveals about understanding math in machine learning?

Think of the idea a human had, understand it, and then think of how to translate it into the language of math, rather than seeing math as something abstract.

How does the speaker suggest dealing with complex mathematical derivations?

Each step of a mathematical derivation usually applies a specific rule or definition. By recognizing and collecting these rules, you can apply them during derivations.

What realization helped the speaker with debugging code?

Understanding that debugging is an essential part of coding, not separate from it, and that spending hours debugging is normal and expected.

What strategy does the speaker recommend for understanding large codebases?

Start with key files like train.py and eval.py, set breakpoints at the beginning, and step through the code with a debugger to gain an overview.

What approach should be taken when trying to understand a complex algorithm?

Use minimal educational implementations of the algorithm and step through the main function with a debugger to understand the core ideas.

What does the speaker identify as a major reason people fail to learn machine learning?

People fail because they stop learning too early due to false expectations and not enjoying the learning process.

What is the '10,000-hour rule' mentioned by the speaker?

The rule suggests that spending 10,000 hours on a specific skill will lead to mastery, indicating that learning machine learning takes significant time and effort.

How does having realistic expectations impact the learning process according to the speaker?

Realistic expectations help you relax and make the learning process more enjoyable and successful.

What is the speaker's final piece of advice for mastering machine learning?

Really learn the theory, gather practical experience, and work on projects to encounter real-world problems and read state-of-the-art papers.

Outlines

00:00

🤯 Rethinking Math in Machine Learning

The author discusses the common struggles with understanding math in machine learning, emphasizing the importance of thinking like a scientist rather than focusing solely on mathematical formulas. They highlight that math is a formalization of human ideas and suggest understanding the underlying concepts before delving into the mathematical representations. This approach helps make sense of complex formulas by relating them to familiar programming concepts like loops and conditionals.

05:02

🔍 Breaking Down Mathematical Derivations

This section explains how to handle mathematical derivations in machine learning. The author reveals that each step in a derivation typically applies specific rules or definitions. By collecting a 'toolkit' of these rules and practicing pattern recognition, one can simplify the process of understanding and executing mathematical transformations. The emphasis is on practice and the systematic application of known rules to demystify complex derivations.

10:04

💻 Coding Challenges in Machine Learning

The author addresses the difficulties of learning to code in machine learning, particularly the frustration of debugging. They stress that debugging is an integral part of coding, often taking more time than writing the initial code. Realizing this can alleviate frustration and set realistic expectations. They also mention tools like GitHub Copilot that can assist in coding but highlight the irreplaceable value of personal experience and continuous learning.

🔄 Understanding Large Code Bases

In this section, the author offers strategies for navigating and understanding large machine learning code bases. They recommend starting with key files like 'train.py' and 'eval.py' and using a debugger to step through the code. This method provides an overview of the code structure and helps identify where to implement new features. The author also advises against diving into highly optimized implementations for understanding basic algorithms, suggesting minimal educational implementations instead.

⏳ The Long Journey to Mastering Machine Learning

The final secret emphasizes the importance of perseverance in mastering machine learning. The author argues that many people give up too soon due to false expectations and frustration with the learning process. They share their own journey, including initial failures and eventual successes, to illustrate that mastering machine learning takes time and continuous effort. The author encourages embracing the learning process, working on real-world projects, and engaging with cutting-edge research to achieve mastery.

Keywords

💡Machine Learning

Machine Learning (ML) refers to the field of study that gives computers the ability to learn from data and improve from experience without being explicitly programmed. In the video, it is described as a complex field requiring understanding of both mathematical concepts and coding skills. The speaker discusses the initial challenges faced in learning ML and the realization of key insights that make the learning process more manageable.

💡Mathematical Formulas

Mathematical formulas are expressions using mathematical symbols and operations to represent relationships or principles. The video emphasizes the common mistake of focusing too much on these formulas rather than understanding the underlying human ideas they represent. The speaker suggests thinking about the concepts behind the formulas and then translating these ideas into mathematical language.

💡Intuitive Understanding

Intuitive understanding involves grasping the underlying concepts and ideas in a natural and straightforward manner. The speaker highlights the importance of achieving an intuitive grasp of mathematical concepts in ML, rather than just memorizing formulas. This approach helps learners connect mathematical rules to real-world applications, making the learning process less daunting.

💡Derivations

Derivations in mathematics involve the process of showing how a result or formula is obtained from a set of assumptions or previously known facts. The speaker discusses the initial intimidation caused by complex derivations and explains that these can be broken down into simpler steps by applying specific rules and definitions, which can be learned and recognized through practice.

💡Coding

Coding refers to writing instructions for computers to perform specific tasks. In the context of ML, coding is used to implement algorithms and build models. The video addresses the frustration many learners feel when coding ML models, emphasizing that debugging is a normal part of the process and that persistence is key to overcoming initial challenges.

💡Debugging

Debugging is the process of identifying and fixing errors or bugs in code. The speaker redefines coding to include debugging as an essential part, noting that much of the time spent coding involves resolving issues. This perspective helps learners understand that struggling with bugs is a normal and necessary part of learning to code effectively.

💡Algorithms

Algorithms are step-by-step procedures or formulas for solving problems. In ML, algorithms are used to process data and make predictions. The speaker emphasizes the importance of understanding the human ideas behind algorithms before delving into their mathematical representations, which makes the learning process more approachable.

💡Patterns

Patterns in mathematics and coding refer to recognizable and repeatable sequences or structures. The video suggests that recognizing patterns in mathematical derivations and coding can simplify the learning process. By identifying and applying known rules and definitions, learners can more easily navigate complex problems.

💡GitHub Copilot

GitHub Copilot is an AI-powered code completion tool that helps developers write code by suggesting lines or blocks of code. The speaker mentions it as an invaluable tool for learning and implementing ML models, as it can generate and explain code, thereby assisting learners in overcoming coding challenges.

💡Persistence

Persistence refers to the continued effort to do or achieve something despite difficulties or failures. The video underscores the importance of persistence in mastering ML, as the field is complex and requires significant time and effort to learn. The speaker encourages learners to maintain realistic expectations and enjoy the gradual process of learning and improving.

Highlights

Learning machine learning can be hard, but there are five secrets to make the process easier.

The first secret is to think of math as a human idea rather than abstract formulas.

Understanding math involves translating human ideas into mathematical language.

Components of formulas represent parts of human ideas, like loops and conditionals in code.

The second secret is that mathematical derivations are simpler than they appear.

Derivations are a series of rule applications and definitions.

Collecting a mathematical toolkit helps in recognizing and applying rules during derivations.

The third secret is that coding involves a lot of debugging, not just writing code.

Debugging is an integral part of coding, and it's normal to spend more time debugging than coding.

The fourth secret is understanding large code bases by starting with key files and using a debugger.

Debuggers provide an effective way to step through code and understand complex systems.

For educational purposes, minimal implementations of algorithms are more accessible than optimized versions.

The final secret is that mastering machine learning requires persistence and time, not quick fixes.

False expectations and giving up too early are common reasons why people fail to master machine learning.

Mastery comes after learning fundamentals and gaining experience through projects and papers.

The journey to mastering machine learning is a marathon, not a sprint.

There are three main ways to learn machine learning, and finding the best approach is key.

Transcripts

look at all this math and code you need

to understand to learn machine learning

it can be very hard for me at least it

was as well until I learned these five

Secrets which honestly aren't even

Secrets but no one really teaches you

them although everyone should know them

I mean I spent the last 3 and a half

years studying machine learning and it

took me way too long to learn these

secrets on my own so let me reveal them

to you so that you don't have to

struggle for that

long

you're thinking of math the wrong way

around which already sets you up for

failure at the very beginning but isn't

your fault back when I started learning

machine learning some of my professors

would simply throw a formula on screen

and tell us this is the loss function

for a decision tree and that was it I

and most other peers were confused and

simply stared at the formula and were

waiting for a magical aha moment where

the formula made sense I was always

asking myself how those smart scientists

could understand math so well they could

develop new algorithms and just think in

the language of math until I realized I

was thinking the wrong way around I was

focusing too much on the actual

mathematical formulas in the realm of

math instead of taking a step back and

thinking like a scientist I mean I was

literally looking at a formula and try

to understand the formula as a whole

which for me now now after learning this

secret just doesn't make any sense

anymore don't think of math like

something abstract and make it then

human interpretable you need to realize

that you need to think the other way

around think of the idea a human had

understand it and then think of how to

translate it into the language of math

this perhaps sounds very confusing but

math is not a standalone language in

which people think scientists think just

like you and me in natural language they

just know how to translate their ideas

into the formalisms of math which then

allows them to be implemented and then

further developed using the rules of

math as mentioned I was always looking

at a formula as a whole but each

component of a formula is just a

component of this human idea for example

a sum or a product are literally just a

for Loop that can have some conditions

that are literally equivalent to an if

else statement in code of course this is

easier said than done and to understand

a mathematical concept using human ideas

requires someone to actually properly

teach you these human ideas and how to

translate them step by step but in my

experience there are two scenarios one

the teacher does that already but you

don't understand he does it because you

were never explicitly taught to think

that way or two the teacher does really

only look at the formulas and

derivations in that case you need to try

to figure out the original human idea

Yourself by for example looking it up

online but the good thing is you now

know it is not your fault and there is

an intuitive understanding to the math

that you can find math is just the

formalization of a human idea very few

people actually think in the language of

math it's just a tool but when it comes

to every intermediate derivation step

then you often actually think in the

language of math which is very difficult

unless you know the next

secret this secret literally changed the

way I look at scary moth derivations

like this one again jumping back in time

to when I watched a lecture at College

my professor would explain the intuitive

idea of an ml algorithm show the

translation into the language of math

and then show us where we want the

formula to end up to make it more

efficient or simply actually work as an

algorithm that can be implemented but at

some point he would go off on a

derivation spree writing out one step

after the other and expected us to

understand why he did what he did

everyone was confused and of course

scared and annoyed but these derivations

are simpler than you might think not

easy but much simpler to execute after

learning this one secret I realized each

step was simply applying one specific

rule or a definition I realized that up

to a certain degree these mathematical

derivations or Transformations just

require you to have a list of rules and

tricks you need to collect that you can

then simply apply during the lectures

for each step I see I would explicitly

look for the rule and definition they

used and write that's down on my list

when solving or reading math derivations

on my own for homework assignments or

during an exam in most cases I would

literally just do some sort of pattern

matching I would look at where I current

L am go down my list of rules and

definitions and apply what fits the

pattern of course some patterns rules

and definitions are harder to spot than

others but after doing this for long

enough you just start to memorize

certain patterns but in general for most

ml math this secret technique does work

wonders you need to collect your

mathematical toolkit and learn to

recognize when you can apply which rule

which means practice practice practice

but math is of course far from all there

is to ml coding is also a very

challenging

skill learning the basics of python and

then an ml Library like pyo is really

cool and fun you simply follow a

tutorial online and have a really steep

learning curve you follow a recipe of

steps and Implement a lot of code you

really see and feel the progress you are

making but then when you want to go

further and learn to implement actual

algorithms or ml pipelines on your own

you hit a wall all of a sudden you sit

on one annoying problem for several

hours have written perhaps five lines of

code and you think you are not making

any progress this is very very

frustrating and is the point where a lot

of people determine coding is really

hard and that they will never be able to

really learn to code writing five lines

of code in 3 hours how pathetic I mean I

always thought I was so stupid for

writing code that never worked until I

debugged it for hours this can get so

bad that you don't even want to start

coding because you know it will fail but

that is simply the wrong way to think

actually writing code for 1 hour will

very likely mean let's say 3 hours of

debugging once I learned that this is

what it really means to be coding I all

of a sudden felt so relieved and just

not stupid anymore coding ml models

didn't feel impossible and hard anymore

because I was doing exactly what was

normal and expected and nowadays there

are amazing tools that I can't live it

out like GitHub co-pilot that can

generate code for you and explain code

for you but there's so much that you

simply learn through your own experience

or the experience of others that's why I

have a completely free Weekly Newsletter

where I share my experience as a machine

learning researcher including actionable

tips AI news and more I'll just pin a

comment below with the link to sign up

but anyway you have to realize that

writing code is not actually coding

debugging is coding this realization

really helps you with implementing

things on your own step by step but when

you have to work with an existing code

base where you have everything at once

you will probably still be overwhelmed

so let's look at the next secret

tip there are two cases where you will

need to understand complex code the

first one is when build bu on top of an

existing repository I remember when I

started working on my first larger

project where I built on top of an

existing code base it was so much code I

literally had no idea where to start

just like in the previous secret I again

felt like learning to read code was as

impossible as writing code all those

tutorials and smaller personal projects

didn't prepare me for this much code I

Tried reading each source file and

started to write code as soon as

possible into places I thought made

sense but that unsurprisingly LEDs to a

lot of headaches and wasting time

writing code that was destined to fail I

had to learn the hard way that there is

a very simple strategy to understanding

large code bases I wish someone would

have simply told me once how to approach

a challenge like this with most large ml

repositories you have a train.py and an

evil. piy file those should always be

the starting point I find those files

set a breakpoint in the beginning and

start stepping through the code with the

debugger I cannot emphasize enough how

simply yet insanely effective this

technique is it's literally like

cheating you can step through the data

pre-processing training Loop the actual

model the evaluation metrics and every

other detail depending on the code base

and your experience this takes just a

few hours and you have an amazing

overview of the code base and will have

a much better feeling for where to add

the new code for your own idea that said

you might not always want to build on

top of an existing highly optimized code

base but simply want to understand an

algorithm better for example when you

want to understand poo a famous

reinforcement learning algorithm I would

not recommend to look at the optimized

implementation that's way too Overkill

and complex luckily for many important

models there are minimal educational

implementations that just implement the

main idea so that people can understand

the model and here yet again the best

way is to set a breakpoint in the

beginning of the main function and then

just start debugging finally there's one

fundamental secret to mastering machine

learning that you need to

know this final secret ties everything

we just discussed together and is the

one reason that will determine your

success or failure of mastering ml 34%

of organizations consider poor AI skills

expertise or knowledge as the top reason

blocking successful AI adoption

according to an IBM study from 2022 why

do you think people fail learning ML and

fall into the category of people with

poor AI skills is it because it is hard

yes but it was also hard for every

person who has now mastered it people

fail to master ml because they stop

learning machine learning too early and

give up and why do people give up

because they have false expectations and

don't enjoy the process of learning they

think mastering ml is hard because they

didn't learn ml in a few weeks or

because they didn't understand a video

explaining an ml concept the first time

they will never understand it I took my

first introductory AI College course

about 3 and a half years ago after that

semester I took my first real ml course

along with working on my first ml

projects after that semester I continued

with my first deep learning courses and

continued working on projects and

reading a lot of papers I definitely

didn't understand everything the first

time but I knew that was normal over

time I learned the secrets mentioned

before and that somewhat mastering ml

takes time I failed my first ml

interviews for internships at Amazon

nuro and Google deep mind but now I am

working with an ex meta professor and

collaborating with a Google deepmind

researcher it takes time period this is

not a skill you learn over a weekend the

10,000 hour rule applies here as well if

you spend 10,000 hours on a specific

skill you will master I'm absolutely not

trying to discourage you rather the

opposite I don't want to be some weird

influencer that wants to sell you a

dream of mastering ml in a few weeks I

want to encourage you to really learn

machine learning to really learn the

theory and really gather practical

experience and the Mastery of machine

learning comes after after learning the

Fundamentals by really working on

projects encountering real world

problems and reading real

state-of-the-art papers or respective

blog posts by having this expectation

that it will take time you relax way

more and the learning process becomes

much easier enjoyable and successful in

the end all these secrets are

universally true no matter how you

decide to learn machine Lear and I say

that because there are mainly three ways

to do so and if you want to find out

which one is best for you you might want

to watch this video next I there talk

about data science but everything

applies the same way to machine learning

in

general

bye-bye