Tesla FSD v12 Rollout - FIRST THOUGHTS! (Ep. 752)

Summary

The video discusses Tesla's new FSD software version 12, which completely shifts to a neural net approach for full self-driving capabilities instead of relying on rules-based coding. Early testing results from a non-Tesla employee show impressive performance, but also times when FSD gave up control due to low confidence. The key challenge now is improving FSD through massive amounts of data, compute power, and enhancements so it rarely lacks confidence to handle complex situations. If Tesla can achieve major leaps in FSD capabilities this year, it would be an exciting advancement towards unsupervised autonomous driving.

Takeaways

• 😲 Tesla rolled out FSD version 12 software to select non-employees, marking a major milestone
• 🧠 FSD v12 shifts from rules-based to neural net end-to-end approach for entire planning process
• 🚗 This is monumental and ambitious, as driving is very complex
• 🤔 FSD v12 needs to drive as well or better than prior versions with guardrails to be safe
• 👍 Early FSD v12 results are promising enough for limited external testing
• ❓ Current weaknesses seem to be lower confidence in more complex situations
• 🌄 Goal is to improve confidence across most driving scenarios
• 📈 Rapid improvements expected as more data and compute is added
• 🔬 FSD team laser focused on data, compute, model improvements this year
• 😃 Exciting to see how fast FSD v12 abilities advance throughout 2022

Q & A

What is Tesla's FSD version 12?

FSD version 12 is Tesla's upcoming rewrite of their full self-driving software. It shifts their approach from rule-based coding to an end-to-end neural net.

Why did Tesla previously rely on rule-based coding for FSD?

The rules provided guard rails and instructions on how to drive safely. Without the rules, the car could potentially do something dangerous.

What are the challenges in moving from rule-based to neural nets?

The neural nets need to handle unguarded situations and new scenarios without thousands of hardcoded rules. They need to drive better and safer than with the rules.

When did Tesla decide to go all in on the end-to-end approach?

Around spring last year, after early testing showed promising results with the end-to-end neural net approach.

Why does FSD version 12 need massive amounts of data and compute?

To train the neural nets to handle driving safely without relying on hardcoded rules, which requires analyzing millions of video clips.

What are some initial weaknesses observed in FSD version 12?

It sometimes gives up with red hands when confidence drops below a threshold, rather than trying to push through low confidence situations.

How can Tesla improve FSD version 12?

By feeding more data to improve the neural nets, adding more compute power, and iterating to increase driving ability and confidence.

What is the goal for FSD version 12 this year?

To improve it dramatically, so it rarely gives up control and can confidently handle complex driving situations.

When could Tesla potentially reach full self-driving capabilities?

It's unclear, but rapid improvements this year could demonstrate the path and timeline to unsupervised, full self-driving.

Why is it exciting that FSD version 12 was given to non-employees?

It shows Tesla has enough confidence in the initial version to let people test it, despite still needing major improvements.

Outlines

00:00

🏎️ Tesla FSD Rewrite Overview

Paragraph 1 provides background on Tesla's full self-driving software version 12, which completely rewrites it to use neural nets instead of rules-based coding. This is a monumental shift allowing the car to figure out driving on its own. Tesla let a non-employee test drive it, showing confidence in the rewrite. The challenge is getting the neural nets to drive better without relying on hardcoded safety rules.

05:01

😕 Current Weaknesses and Path Forward

Paragraph 2 discusses version 12's current weaknesses and path to improvement. It still sometimes lacks confidence to handle complex situations. Tesla needs to train the neural nets with more data and compute power to dramatically improve driving ability over time. The goal is to increase confidence levels so it rarely needs human intervention.

Keywords

💡Full Self-Driving (FSD)

Tesla's advanced driver assist system that aims to provide full autonomous driving capabilities. A core focus of the video is the major software rewrite of FSD called 'FSD version 12', which shifts from rule-based coding to a neural net, end-to-end approach.

💡neural net

A type of machine learning model that mimics how the human brain works. Tesla is using neural nets instead of hard-coded rules to have AI figure out driving on the fly rather than dictating instructions.

💡end-to-end

Refers to training a single neural network on raw input data to produce the desired output instead of multiple networks for different steps.

💡planning

An important component of driving along with perception. Planning involves figuring out how to navigate the driving environment, which requires complex rules and instructions without neural nets.

💡rules

The countless hardcoded rules and instructions Tesla relied on previously to dictate FSD's driving and prevent accidents. Version 12 removes these 'guard rails' and replaces rules with neural nets.

💡confidence threshold

The level of certainty required for the FSD neural net to continue driving confidently. Below this threshold, FSD will hand control back to the driver as a safety precaution.

💡data

Key to improving FSD is training the neural nets on massive amounts of driving data, which requires huge compute power that Tesla is ramping up.

💡compute

Massive computing power, measured in units like petaflops, which is essential to train FSD's neural nets quickly on huge data sets to learn driving skills.

💡improvements

Refers to enhancements and upgrades made to the neural net models and architecture itself over time to expand abilities.

💡robotaxi

Fully autonomous Tesla vehicles providing ride-share transportation without drivers. The ultimate goal of the FSD rewrite to neural nets which requires full autonomy.

Highlights

Tesla rolled out its FSD version 12 software to a very small number of non-tesla employees.

FSD version 12 is Tesla's upcoming rewrite of their full self-driving software.

In version 12 Tesla shifted their approach to an end-to-end neural net approach.

Tesla started to experiment with an end-to-end full self-driving approach using only neural nets.

Early results were promising so Tesla decided to go all in on the end-to-end neural net approach.

The challenge is getting to a first version that's good enough for customers to actually use.

Version 12 needs to drive better and more safely than previous versions with hardcoded rules.

Version 12 still has some guard rails like handing control to the driver when confidence drops.

Tesla needs to improve neural nets through more data and compute to boost confidence.

Tesla needs to improve FSD performance through data, compute, and neural net enhancements.

FSD 12 handled many situations amazingly but sometimes gave up with red hands.

Tesla needs to improve FSD confidence so it rarely gives up control.

Tesla's challenge is to quickly improve FSD to handle complex situations confidently.

Hopefully we'll see fast improvements in FSD version 12 performance this year.

It will be exciting to see FSD version 12 progress and improvements this year.

Transcripts

hey it's Dave so yesterday there was big

news for Tesla FSD or full stelf driving

FSD version 12 if you don't know already

is Tesla's upcoming rewrite of their

full self-driving software previously

Tesla relied on over 300,000 lines of

code to largely put guard rails around

FSD and give instructions on how to

drive but in version 12 Tesla shifted

their approach to an endtoend neuronet

approach which means rather than

dictating how the car drives by

instructions Tesla is letting a neuronet

AI determine how to drive on the Fly

I'll explain more on this and why it's

important but last night Tesla rolled

out its FSD version 12 software to a

very small number of non-tesla employees

and one of them was Omar from hullar's

blog who covers Tesla on X in this video

I wanted to share my reaction and

thoughts to his initial drives with

fsd2 all right so first off it's a huge

accomplishment for Tesla to give a

non-employee fsc2 it means that they are

confident enough with the current

iteration to let people see it drive and

now this is a big deal because of how

Monumental the fsd2 software rewrite is

in fact it's something that's never been

done before at this scale or ambition at

least that I know of where something as

difficult as driving has shifted from

using horis sixs or rule-based coding to

a neuronet end-to-end approach this is

impressive and in order to understand

why we need to understand why the

heuristic code was there in the first

place now over the years Tesla shifted

their perception code to neuron Nets and

it's already been running almost all

neuron nets for a while perception means

the part of the code that perceives the

world or in other words analyzing mostly

video along with how the car is moving

but there's another big part of driving

and that is planning even if you see the

world around you you still need to

figure out how to navigate that world

and Tesla was trying to cautiously move

over that part of the code to more

neural Nets and they've been doing so

it's just that it's really hard to get

rid of all those rules you put in place

and it's because the rules are there for

a good reason without those rules the

car might do something really dangerous

and get into an accident and there are a

lot of rules countless and for a ton of

situations here's my speculation as

Tesla tried to move rule-based coding to

neural Nets with planning they

encountered a lot of challenges and it

was slower than what they hoped for take

away some rules but then it introduces

these unguarded situations that neuron

Nets need to handle but what if they

can't handle those situ

or there's new situations that are

created by the absence of the rules that

you took away it's all extremely

complicated to say the least but we know

from isaacson's biography of Elon Musk

that over a year ago Tesla started to

experiment with an endtoend full

self-driving approach forget coding all

the rules just use neural Nets not just

for perception but also for the complete

planning process in other words get rid

of all the guard rails and see if neuron

Nets by themselves can figure out how to

drive safely and better than any

previous FSD version as Tesla tested

this out the early results were

promising and the more they tested they

realized that this was the way to go

they decided to go all in earlier last

year my guess was probably sometime

around spring I think but then you got

this Monumental task of building and

training a massive neuron net with the

right data to make sure it could drive

safely the video data they need to do

that is just massive and the compute

required is massive too that's why Tesla

increased their compute goal last year

is so that they can go all in with

endtoend neuron nets for FSD now the Big

Challenge in my opinion is to get to a

first version that's good enough for

customers to actually use the reason why

this is so hard is because of how big

the jump is from rule-based FSD planning

to purely neuronet based planning the

jump is huge and the new neuronet

planning approach needs to be as good as

the old code-based one or else it's not

safe to roll out to customers here's

another way to look at it you've got

thousands of rules R in place to make

sure the car doesn't get into accidents

and if you take them away then the car

really needs to know how to drive by

itself without those rules all the

crutches come off and then you're left

with the true driving ability of the AI

but that's hard because why did Tesla

need all of those rules in the first

place it's because it needed those rules

to prevent dangerous mishaps so in order

to get rid of all those rules then FSD

needs to go through a huge jump in

ability probably the biggest jump in its

history it needs to somehow get to a

place where it can drive without

thousands of hardcoded rules and figure

out driving from millions of video clips

and figure it out in such a way that I

can drive better and more safe than it

did with all those rules it almost seems

like an impossible task that's why any

customer roll out of FSC version 12 is

such a big deal even though the first

version rolled out to Hol Mar's blog and

a few others is far from perfect it

shows that Tesla has figured out its

endtoend neural network approach enough

where it can handle most situations now

the question is what is version 12's

current weaknesses and how long will it

take to fix those and ultimately how

long will it take to get to unsupervised

FSD and robotaxi now on the second

question I think we need to see more of

FSD version 12 progress over the next

several months to get a better feeling

of how fast Tesla is able to improve

version 12's driving abilities my hunch

is that we're going to see sizable leaps

and abilities as Tesla brings on larger

compute clusters for training I think

they'll need all the compute training

that they're bringing online this year

and probably even more but how about

current weaknesses in Omar's tweets he

did share how FSD handled a lot of

situations amazingly however there are

times where it would just stop with red

hands and basically give up now I

interpret that as there's a confidence

level where the neuron Nets have a

certain level of confidence to handle

situations and if that confidence level

drops below a certain threshold then

it's taught to basically give up and

hand control over to the human it's a

safety feature where they don't don't

want the neural Nets to hallucinate and

just make up a path to drive when it has

low confidence in other words there are

still guard rails with FSD it's just

that they aren't thousands of rule-based

code rather it's probably some training

to the model where when confidence level

drops below a certain threshold then it

shouldn't push it and continue in that

situation and that's a reasonable rule

especially for the time being now the

challenge is going to be to increase FSC

version 12's driving ability so it

rarely ever drops below the threshold of

confidence in other words Tesla is going

to need to feed a lot more data into the

neuron Nets and also improve the neural

Nets to the point where it can drive

dramatically better than this early

version and so much so that it's night

and day Tesla version 12 needs to get to

a point where it rarely ever holds up

red hands and gives up control to the

driver rather the neuron Nets need to

improve to a point where it's confident

in practically every driving situation

even complex ones and that's the

challenge Tesla's FSD team has this year

is to radically and quickly improve

fst's abilities through data training

and improvements with their neural Nets

it's a challenging problem for sure but

one that I think has a clear path on how

to improve it they basically need a lot

of data they need a lot of compute and

they need a lot of improvements right to

their neural Nets that's what Tesla's

FSD team is laser focused on this year

and it's going to be exciting to see the

results hopefully we'll see some fast

improvements and some amazing

performance out of FSD version 12 this

year all right take care everyone we'll

see you guys my next video