How To Do Machine Learning Research Without A PhD

Summary

The video script discusses the roles and requirements in AI research labs like Google Brain, DeepMind, and OpenAI. It clarifies that while a PhD in machine learning is often expected for research scientists aiming for top-tier publications, it's not mandatory for software engineers who build infrastructure. Research engineers fall in between, contributing to research papers and requiring less specialized ML knowledge. The speaker encourages those interested in AI research to consider applying for roles even without a PhD, emphasizing on-the-job learning and familiarity with ML jargon.

Takeaways

• 🧑‍💻 There are three common roles in research labs: software engineer, research scientist, and research engineer, each with different requirements for machine learning knowledge.
• 🔧 Software engineers primarily build infrastructure for research teams and typically require less machine learning knowledge, often not needing a PhD in ML.
• 💼 Software engineers might specialize in low-level systems or distributed systems, focusing on performance and scalability across data centers.
• 📈 Research scientists are expected to publish in top-tier conferences and usually have a PhD, as their main output is research papers.
• 🔬 Research engineers sit between software engineers and researchers, contributing to research papers and often implementing novel techniques from academic papers.
• 🤝 Research engineers may work closely with software engineers and researchers on ambitious projects, utilizing more resources to tackle complex problems.
• 📚 Not all research labs have a research engineer role; for example, Google does not, while DeepMind and OpenAI do.
• 🎓 Having a PhD is not a prerequisite for all roles, and even software engineers can publish papers in top-tier venues.
• 📘 The speaker advises against pursuing a PhD solely for the purpose of working in a research lab, suggesting there are many other options available.
• 📚 It's recommended to take introductory courses in machine learning to familiarize oneself with the field and the specific research focus of the lab of interest.
• 💼 For those interested in AI research, applying directly to labs as a software engineer or research engineer without a PhD is encouraged, as skills can be learned on the job.
• 🔍 The transcript emphasizes that a PhD is not always necessary for cutting-edge work in machine learning and AI research labs.

Q & A

What are the common roles found in research labs like Google Brain, DeepMind, OpenAI, and Facebook AI Research?

The three common roles are software engineer, research scientist, and research engineer. Each role has different responsibilities and requirements for machine learning knowledge.

What is the primary responsibility of a software engineer in a research lab?

A software engineer is mainly responsible for building infrastructure for the research team, which could be specific to a single research experiment or used by multiple teams for various research papers.

Do software engineers in research labs need to have a deep understanding of machine learning?

Typically, the required knowledge of machine learning for software engineers is at the lower end, and they are less likely to solve complex machine learning problems during interviews.

Is a PhD in machine learning necessary for the role of a software engineer in a research lab?

No, a PhD in machine learning is not typically required for software engineers in research labs.

What are the expectations for a research scientist in terms of academic contributions?

A research scientist is expected to publish papers in top-tier conferences such as NeurIPS, ICML, ICLR, or other specialized venues related to their subfield.

How common is it for research scientists to have a PhD?

About 99% of research scientists have a PhD in their chosen field or a closely related field to machine learning.

What is the role of a research engineer in a research lab?

A research engineer sits between a software engineer and a researcher, helping with research papers, running experiments, and contributing to specific papers with novel techniques or ideas.

Do all companies have a research engineer role?

No, some companies like Google do not have a research engineer role, while others like DeepMind or OpenAI do.

Can software engineers also act as research scientists and publish papers?

Yes, many software engineers also engage in research activities and publish papers in top-tier venues.

What advice does the speaker give regarding pursuing a PhD solely to work in a research lab?

The speaker advises against spending five years pursuing a PhD with the sole purpose of working in a research lab, as there are many other options available.

What should someone interested in working in a research lab do to prepare?

The speaker recommends taking introductory courses in machine learning to familiarize oneself with the jargon and research topics of the lab they are interested in.

Outlines

00:00

🔬 Roles in AI Research Labs

The speaker discusses the varying roles within AI research labs such as Google Brain, DeepMind, OpenAI, and Facebook AI Research, emphasizing that a PhD in machine learning is not always required. They outline three common positions: the software engineer, who primarily builds infrastructure and may not need a PhD; the research scientist, who is expected to publish in top-tier conferences and typically holds a PhD; and the research engineer, who bridges the gap between the two, assisting with research and possibly contributing to papers. The video aims to clarify the misconception that a PhD is mandatory for all roles in AI research and encourages those interested to apply directly for positions such as software or research engineer roles even without a PhD.

05:04

📚 Preparing for AI Research Lab Roles

In this paragraph, the speaker advises viewers to take introductory machine learning courses to familiarize themselves with the field's terminology and to gain an overview of the type of research conducted by the labs they are interested in. They mention that while having a PhD is not a prerequisite for working in a research lab, understanding the basics of machine learning is beneficial. The speaker also cautions against pursuing a PhD solely for the purpose of working in a research lab, suggesting that there are other paths available. They conclude by inviting viewers to share their thoughts and questions in the comments section.

Keywords

💡Research Lab

A research lab is a dedicated space where scientists and researchers work on experiments and projects to advance knowledge in a specific field. In the context of the video, research labs like Google Brain, DeepMind, OpenAI, and Facebook AI Research are focused on cutting-edge machine learning and artificial intelligence research. The script discusses the different roles within these labs and the varying requirements for each.

💡PhD in Machine Learning

A PhD, or Doctor of Philosophy, in Machine Learning is the highest academic degree one can achieve in the field, signifying expertise and the ability to conduct original research. The video script addresses the common question of whether a PhD is necessary to work in a research lab, suggesting that while it may be preferred for some roles, it is not a strict requirement for all.

💡Software Engineer

A software engineer is a professional who designs, develops, and maintains software systems. In the video, this role is described as primarily building infrastructure for research teams, which may involve low-level programming or working with distributed systems. The script emphasizes that software engineers in research labs typically do not need a PhD in machine learning.

💡Infrastructure

Infrastructure refers to the underlying framework or set of tools that support a system or process. In the context of the video, it pertains to the systems built by software engineers to facilitate research experiments and projects in AI and machine learning. These could be specific to one paper or used across multiple teams.

💡Distributed Systems

Distributed systems are computer systems that consist of multiple machines working together to perform tasks. The script mentions that some software engineers specialize in building infrastructure for distributed systems, which can scale across a cluster of machines in data centers, supporting large-scale AI research.

💡Research Scientist

A research scientist is an individual who is expected to conduct original research and publish findings in academic journals or at conferences. The video script highlights that research scientists typically have a PhD and are the primary authors of research papers in top-tier conferences such as NeurIPS, ICML, and ICLR.

💡Publishing Papers

Publishing papers refers to the academic practice of sharing research findings in peer-reviewed journals or at conferences. The script discusses the expectation for research scientists to publish papers as a key part of their role in contributing to the field of machine learning.

💡Research Engineer

A research engineer is a hybrid role that combines elements of both research and engineering. The script describes this role as sitting between a software engineer and a researcher, helping with research papers, running experiments, and potentially contributing to the publication process. This role may or may not require a PhD, depending on the lab.

💡Cutting-Edge Research

Cutting-edge research refers to the most advanced and innovative work being done in a field. The video emphasizes the expectation for research scientists and engineers to be involved in such research, often leading to the publication of papers in prestigious venues.

💡Machine Learning Jargon

Jargon refers to the specialized terminology used within a particular field or profession. The script suggests that having familiarity with machine learning jargon is important for anyone interested in working in a research lab, as it helps in understanding and communicating about the research being conducted.

💡Reinforcement Learning

Reinforcement learning is a type of machine learning where an agent learns to make decisions by taking actions in an environment to maximize some notion of cumulative reward. The script uses OpenAI as an example of an organization that focuses on this type of research, indicating that knowledge of reinforcement learning would be beneficial for applicants.

Highlights

Working in top research labs like Google Brain, DeepMind, OpenAI, and Facebook AI Research does not always require a PhD in machine learning.

Three common roles in research labs are software engineer, research scientist, and research engineer, each with varying ML knowledge requirements.

Software engineers primarily build infrastructure for research teams, which may be specific to one experiment or used across multiple teams.

Software engineers may specialize in low-level systems programming in C++ or work on distributed systems infrastructure for scalability across data centers.

Machine learning knowledge for software engineers is typically at the lower end, with less emphasis on ML problem-solving during interviews.

A PhD in machine learning is generally not required for software engineers in research labs.

Research scientists are expected to publish papers in top-tier conferences like NeurIPS, ICML, ICLR, and are typically required to have a PhD.

Research scientists collaborate with software engineers on ambitious projects and have access to more resources for tackling complex problems.

The research engineer role sits between the software engineer and researcher, assisting with experiments, infrastructure, and contributing to research papers.

Research engineers focus on novel techniques, implementing academic papers, and may propose research ideas for publication.

Not all research labs have research engineer roles; for example, Google does not, while DeepMind and OpenAI do.

Many software engineers act as research scientists, publishing papers in top-tier venues, indicating that a PhD is not a strict requirement for publishing research.

There is no need for a PhD to work in cutting-edge machine learning and AI research; consider applying as a software or research engineer without one.

Taking introductory machine learning courses is recommended to familiarize oneself with the jargon and research of the lab to which one is applying.

Pursuing a PhD solely to work in a research lab is not recommended, as there are many other options available for gaining experience and expertise.

The big takeaway is that a PhD is not always necessary for contributing to state-of-the-art research in machine learning and AI.

Transcripts

A lot of people have asked me what it takes  to work in a research lab like Google Brain,  

DeepMind, OpenAI and Facebook AI Research. And  a very common question I get is, do I need a  

PhD in machine learning to work on cutting edge  machine learning? I have news for you that is  

maybe. So stay tuned and I'll tell you  which roles require a PhD and which ones  

don't. So let's get into it. There's three very  common roles that you find in research labs,  

and they sort of exist on a spectrum of minimal  requirement for machine learning knowledge  

to requiring very deep specialized ML knowledge.

The first most common job role exists on this  end of the spectrum, and that's going to be the  

software engineer. This software engineer  is primarily concerned with building out  

infrastructure for the rest of their research  team, and this infrastructure might be specific  

to a one research experiment for a single paper  or it might be some info that's used by multiple  

teams or even ML info that's going to be used for  multiple research papers. These software engineers  

typically need to be very strong at either low  level systems if they're doing kind of low level  

programming and C++ or doing things on  like TPUs or GPUs at the kernel level.

Alternatively, you have software engineers who  are more specialized in distributed systems, and  

they're going to be building out infrastructure  that can scale across a cluster of machines that  

are available in data centers. Now, both of these  software engineering roles are sort of specialized  

in those particular domains and it's by no means  easy work. But they're required knowledge of  

machine learning typically is going to be  at the lower end, and they're less likely  

during an interview to be required to solve the  kind of problems that other roles would require.

Oh, and one more thing about the software  engineer is they typically don't require a PhD  

in machine learning, so that's one role in  a research lab that doesn't require a PhD.  

Now on the flip side, on the opposite of the  spectrum, you have the research scientist,  

the research scientist has an expectation  typically of publishing papers into top  

tier conferences like NeurIPS, ICML, ICLR, or if  they're specialized in some subfield like computer  

vision or speech, they're going to want to publish  in those particular conferences and those venues.

Because they're expected to publish papers and do  research that is cutting edge, typically they're  

required to have a PhD. And I'd say about 99% of  research scientists have a PhD in their chosen  

field or very close field to machine learning  because their primary output is going to be  

research papers. Many times you'll have a research  scientist working with software engineers and they  

will be building out some larger projects that  is more ambitious and requires a larger team,  

but also they have the unfair advantage of  having more resources at their disposal to  

solving maybe a much harder problem than  they could if they were just one person.

And then there's a third role, which is going to  be the research engineer. The research engineer  

sort of sits in between the software engineer  and the researcher. Their expectation is to help  

with research papers. They're typically running  experiments, helping out on the infrastructure  

side, maybe applying it to a very specific  problem contributing for a very specific paper.  

And they will typically be focused on very  novel techniques or reading academic papers  

that they downloaded off of archive, or  that they saw at our research conference.

And they'll take that information and try to  implement some papers. And they might even  

be proposing ideas for avenues of research in  their particular like in a paper that they're  

trying to publish with their research scientist  peers. Now, some research labs don't actually  

have research engineers. For example, at Google,  there's no such thing as a research engineer role.  

And whereas at companies like DeepMind or OpenAI,  

that role exists and has a more clearly  defined skillset. Not everyone who has a PhD  

is a research scientist. There are many software  engineers who actually on their day to day job  

act as research scientists, and they are  publishing papers in top tier venues.

So you don't need to feel like because you're  not a research scientist, you can't publish.  

So yeah, there you have it. That's a high level  overview of the different kinds of roles you have  

in state-of-the-art research labs, doing  machine learning and artificial intelligence  

research. The big takeaway is that you don't  actually need a PhD to do a lot of this work.  

In fact, if you're very interested in getting  into that kind of work, strongly consider  

applying directly to one of those labs as maybe  a software engineer or a research engineer if  

you don't have a PhD already, because you'll be  able to learn many of those skills on the job.

Now, I would highly recommend still taking some  introductory courses in machine learning so that  

you're familiar with the jargon or at least have  an overview of the kind of research that the lab  

you're applying to does. For example, OpenAI does  a lot of reinforcement learning type research,  

and you should probably be familiar with that  kind of a subject before you're applying. But  

don't feel like you need to have a PhD before  you do that. I would not recommend spending five  

years of your life pursuing a PhD with the sole  purpose of wanting to work in a research lab.

There are many other options available to you,  and that can be the topic of the next video.  

With that, I'd like to thank you for your time  and let me know below in the comments anything