Is Apple's M2 Max Good for Machine Learning?

Summary

The video discusses the release of Apple's M2 chips, focusing on whether they are worth purchasing. It recommends the M2 for Apple devotees needing mobility and large models, but suggests alternatives for others. Benchmarks show the M1 Max's performance, and the expected generational uplift for the M2 Max is compared to the Nvidia 3090. The video also explores the cost-effectiveness of Apple products versus comparable Windows and Linux options, highlighting the value of desktop systems for those not needing mobility.

Takeaways

• 🍏 The M2 chip is a powerful upgrade, but the decision to buy should be based on individual needs and preferences.
• 💼 For Apple devotees who require mobility and work with large models, the M2 chip could be a good investment.
• 🔋 The M1 Max's unified memory architecture is beneficial for those working with large models that demand substantial RAM.
• 📈 Benchmarks suggest the M1 Max offers about an eighth of the performance of a 3090 GPU, with the M2 expected to close this gap somewhat.
• 💰 The cost of a high-spec MacBook Pro with M2 Max can be significant, and there may be more cost-effective alternatives available.
• 🛠️ If mobility isn't a requirement, a desktop setup could offer better performance for the price.
• 🖥️ Non-Apple users needing mobility might consider Windows or Linux laptops with dedicated Nvidia GPUs for better performance.
• 📊 The M2 Max is expected to have a performance uplift of about 30-40% over the M1 Max, but still falls short of high-end desktop GPUs.
• 💡 The script laments the lack of unbiased benchmarks and the demise of trusted review sources, highlighting the difficulty in obtaining reliable performance data.
• 🛍️ The new Mac Mini with M2 Pro is not considered a great value outside of the Apple ecosystem, especially considering the lack of peripherals.
• 🔬 For those not tied to the Apple ecosystem, a desktop or a Windows laptop with a dedicated GPU may offer superior performance and value.

Q & A

What is the main topic of the video script?

The main topic of the video script is an analysis of Apple's new M2 chip, discussing whether it's a worthwhile purchase and comparing its performance with other options.

Who is the target audience for the M2 chip according to the script?

The target audience for the M2 chip includes Apple devotees who run large models and need mobility, and those who are convinced they want the chip regardless of the analysis.

What are the independent benchmarks mentioned in the script?

The independent benchmarks mentioned in the script are from a user on GitHub, which the author uses due to the difficulty of finding unbiased benchmarks in the popular media.

Why does the script mention the M1 Max chip in the context of the M2 chip?

The M1 Max chip is mentioned because its performance can be used to extrapolate the potential performance of the M2 Max chip, given the expected generational gains.

What is the significance of the unified memory architecture for users working with large models?

The unified memory architecture is significant for users working with large models because it provides access to large amounts of memory (up to 21 GB), which is beneficial for such models and not commonly available in mobile GPUs.

How does the script compare the performance of the M1 Max chip to the Nvidia RTX 3090?

The script compares the performance by stating that the M1 Max chip has an eighth of the performance of the RTX 3090 and consumes about the same amount of power for that speed gain.

What is the estimated performance improvement of the M2 Max over the M1 Max according to the script?

The script estimates a performance improvement of around 30 to 50 percent for the M2 Max over the M1 Max, based on Apple's benchmarks and assumptions.

Why does the script suggest that the M2 chip might not be a great buy for some users?

The script suggests that the M2 chip might not be a great buy for users who do not need the large unified memory, are not devoted to Apple, and do not require mobility, as there are more cost-effective and higher-performing alternatives available.

What alternative options are presented in the script for users who do not need the M2 chip's specific features?

The script presents alternative options such as last-gen laptops with Nvidia GPUs, Windows or Linux laptops, and high-end desktop systems as more cost-effective and higher-performing alternatives.

What is the script's stance on the value of the new Mac Mini with the M2 Pro chip?

The script's stance is that the new Mac Mini with the M2 Pro chip is not a great value for those outside the Apple ecosystem, as it lacks peripherals and offers less performance per dollar compared to building a custom system with a comparable GPU.

Outlines

00:00

🤖 M2 Chip Release and Target Audience

The video script discusses the release of Apple's new M2 chips and evaluates their potential purchase for different user types. It emphasizes that those deeply invested in Apple and requiring mobility for large models like Vision or language models will find the M2 chips a worthy investment. The script also hints at benchmark comparisons and expresses concerns about the availability of unbiased benchmarks in the media, citing the closure of trusted review sources.

05:01

💻 Performance Analysis and Alternatives to M2 Max

This paragraph delves into the performance of the M1 Max as a benchmark for the M2 Max, given the lack of direct M2 Max benchmarks. It discusses the performance of the M1 Max in handling large models and its unified memory advantage. The script compares the M1 Max's performance per watt to the Nvidia 3090 GPU, highlighting the trade-offs between performance and power efficiency. It also speculates on the M2 Max's potential performance improvements based on generational uplift and suggests that for those not needing the maximum memory capacity, other options might be more suitable.

10:02

💸 Cost Considerations and Non-Apple Alternatives

The script addresses the high cost of Apple products equipped with the M2 Max chip, particularly the 16-inch MacBook Pro, and compares it to other laptops with similar or better performance at a lower price point. It mentions a specific MSI laptop with an Nvidia GeForce 3080 GPU as a more cost-effective alternative. The speaker also touches on their personal preference for desktop computers over mobile ones, outlining a hypothetical high-end desktop build that offers superior performance for a similar price to the MacBook Pro with M2 Max.

15:03

🖥️ Desktop Superiority and the Future of GPU Technology

The final paragraph reinforces the idea that desktop computers offer better performance for the price, especially when one does not require the mobility or the ecosystem of Apple. It also mentions the imminent release of Nvidia's new Mobility chips for the 4000 series, suggesting that these will provide significant performance improvements over the M2 Max. The script concludes with a reminder that the choice of computer should be based on individual needs and preferences, and invites viewers to share their thoughts in the comments section.

Keywords

💡M2 Chip

The M2 Chip refers to the latest generation of Apple's custom-designed system on a chip (SoC), which powers their Mac computers. In the video, the M2 Chip is the central focus, with the host discussing its performance capabilities and whether it is worth purchasing. The script mentions the M2 Max, which is expected to be a more powerful variant of the M2 Chip, and compares its potential performance to existing chips like the M1 Max.

💡Performance Benchmarks

Performance benchmarks are standardized tests or measurements used to evaluate the performance of a computer system or its components. In the context of the video, the host refers to benchmarks to compare the M1 Max's capabilities with the expected performance of the M2 Max, noting the lack of unbiased benchmarks in popular media and the reliance on user-generated data from GitHub.

💡Unified Memory

Unified memory is a type of memory architecture used in Apple's M-series chips where the CPU, GPU, and other components share a single pool of memory. This is highlighted in the script as a benefit for users working with large models that require significant memory, as it allows for more RAM to be accessible compared to traditional discrete GPU setups.

💡M1 Max

The M1 Max is a previous generation of Apple's M-series chips, known for its high performance in tasks that require substantial processing power and memory. The video script uses the M1 Max as a reference point to discuss the expected improvements in the M2 Max, such as generational gains and performance per watt.

💡GPU (Graphics Processing Unit)

A GPU is a specialized electronic circuit designed to rapidly manipulate and alter memory to accelerate the creation of images in a frame buffer intended for output to a display device. The script discusses the GPU capabilities of the M1 Max and compares it with Nvidia's offerings, emphasizing the importance of GPU performance for tasks like machine learning and rendering.

💡Nvidia

Nvidia is a company known for its graphics processing units and is a major player in the GPU market. The video script mentions Nvidia GPUs, such as the 3090 and 3080, as comparisons to Apple's M-series chips, discussing their performance and value for money in different computing setups.

💡Machine Learning

Machine learning is a subset of artificial intelligence that provides systems the ability to learn and improve from experience without being explicitly programmed. The script discusses the importance of machine learning tasks in evaluating the performance of the M2 Chip, especially for users who work with large models.

💡Mobility

In the context of the video, mobility refers to the ability to use a computing device in different locations, such as carrying a laptop to a cafe or between home and work. The host discusses the relevance of the M2 Chip for users who require a portable yet powerful device for their work.

💡Apple Ecosystem

The Apple ecosystem refers to the interconnected system of hardware, software, and services designed by Apple. The script mentions the Apple ecosystem as a factor for some users who might prefer to stay within this system for their computing needs, despite potential performance or cost considerations.

💡Desktop vs. Laptop

The script contrasts desktop and laptop computers, discussing the trade-offs between performance, portability, and cost. The host expresses a preference for desktop computers for their superior performance and value, while acknowledging the benefits of laptops for those who need mobility.

💡Value for Money

Value for money is a measure of the worth or benefit of a product in relation to its cost. The video script evaluates the M2 Chip and Apple's products in terms of the value they provide compared to alternative options, such as Windows or Linux-based systems with Nvidia GPUs.

Highlights

Apple's new M2 chips are powerful, but the decision to buy depends on individual needs and preferences.

The M2 is an obvious purchase for Apple devotees who require mobility and work with large models.

For those not devoted to Apple or not needing mobility, the M2 may not be a necessary investment.

Benchmarks for the M1 Max are used to extrapolate the performance of the M2 Max, due to the lack of available M2 benchmarks.

The unified memory architecture of the M1 Max is beneficial for those working with large models requiring substantial memory.

The M1 Max offers performance comparable to an eighth of the NVIDIA 3090, with 8 times the power draw.

Apple's claimed 30-50% performance improvement in the M2 Max over the M1 Max is taken into consideration for comparison.

The M2 Max is estimated to be around 5 times the performance of the NVIDIA 3090, based on generational uplift.

For those needing 21GB of memory, the M2 Max is currently the only option in a mobile form factor.

Comparing the cost of a high-spec 16-inch MacBook Pro with the M2 Max to alternatives shows a significant price difference.

Non-Apple laptops with NVIDIA GPUs offer similar or better performance at a lower price point.

The MSI laptop with a GeForce 3080 and 16GB VRAM is suggested as a cost-effective alternative to the MacBook Pro.

Desktop computers offer superior performance and value for money compared to mobile options.

A high-end desktop setup is detailed, providing a comparison to the MacBook Pro in terms of specs and price.

NVIDIA's new Mobility chips for the 4000 series are mentioned as a significant upgrade over the M2 Max.

The Mac Mini with the M2 Pro is criticized for its lack of peripherals and poor performance-to-price ratio.

The video concludes with a recommendation for those not in the Apple ecosystem to consider Windows or Linux laptops for better performance.

The presenter emphasizes the importance of making an informed decision based on individual needs rather than brand loyalty.

Transcripts

so Apple has just released their new M2

chips and they appear to be quite

powerful now the question is would I buy

one and what do I think I should

recommend to you

first of all let's talk about who this

is an obvious purchase for so if you are

convinced you want this by all means buy

it I'm not here to talk you out of how

to spend your money that is up to you uh

second if you are an apple devotee who

runs large models large Vision or

language models and need Mobility as in

you have to pick up your work machine

and go to like a cafe or between home

and a workplace then by all means this

is actually a really good uh really good

investment for you it's not a bad

product by any means however if you are

someone who is not devoted to Apple or

you do not need Mobility or you do need

mobility and are not devoted to Apple I

don't think it's necessarily a great buy

I'm going to show you why in this video

so the first thing I want to take a look

at are some benchmarks for the M1 now we

don't have benchmarks for the M2 Max as

of yet but we do have some independent

benchmarks for the M1 Max now this is

from a user on GitHub and the reason I'm

using this you can take uh you can take

you can have some issues with this

that's valid but the reason I'm picking

this is because it's very difficult to

get unbiased benchmarks in the popular

media there were really only two sources

that I trusted for GPU benchmarks and

they were hard ocp and the tech report

both of which are now defunct hard ocp

no longer produces reviews and the tech

report is now a AI generated site that

was purchased by some internet

entrepreneur to make money with clicks

just total garbage total garbage is

quite a shame they shouldn't just shut

it down

but I digress it's very difficult to get

truly unbiased reviews uh from people

that are receiving Hardware uh due to

their relationships with the underlying

businesses so that's not you know where

I choose to draw benchmarks from this is

a benchmark by a normal person someone

like you or I that has access to both

the 3090 and the M1 Max

now the reason the M1 Max is relevant is

because we can extrapolate based upon

generational gains the performance of

the m to Max

so taking a look at some performance on

some basic models and I believe pytorch

on say resnet 50 they're getting a

performance a throughput of 140 images

per second consuming 21 gigabytes of RAM

now this is why of memory excuse me this

is why I said those that need access to

those that are working with large models

should consider this because the unified

memory is a benefit if you are working

with large models that require large

amounts of memory as of yet there are no

comparable gpus that are going to give

you access to more than 16 gigabytes of

vram in a mobile form factor so if you

need that then this is the only game in

town and the performance is pretty good

performance per watt is what I would

expect it to be so I don't think it's a

terrible purchase in that case but when

you compare it to the 3090 down here you

can see that it's 1100 images per second

now that's not surprising it's something

like what is that seven or eight times

eight times the performance and also

about eight times the power draw

slightly more so it's a little bit less

efficient for that speed gain so we know

that the M1 Max is about an eighth you

know the performance of a 30 90 and we

know that the generational uplift is

going to be something like between 30 to

40 percent so if you take a look at

um this article from Mac World it says

in Apple's benchmarks again I take those

with a grain of salt but for the

purposes of argumentation let's just

take them as is and give them a bonus

say they're sandbagging so it says M2

Max tells a 31 improvement over the M1

Max and DaVinci Resolve again that's not

that's not machine learning uh so I

don't know and uh says it's faster than

I9 MacBook Pro and some other tasks and

with rendering in Cinema 4D it's 30

faster with the M1 Max so 30 let's

assume for the sake argument round

numbers it's 50 faster in machine

learning tasks I don't know where that's

going to come from but let's just assume

it's you know 30 excuse me 50 faster

taking a look here this uh eight to one

then goes down to about five to one ish

approximately comparing the 39e to the

M2 Max in the same Benchmark so about 5x

of performance and I'm assuming

performance per watt is going to be

better so it's going to have the same

Peak power so it's going to get better

with respect to Performance per watt

okay all well and good so if you have a

use case for this if you need that 21

gigabytes of memory this is the only

game in town cool go buy it I don't have

a problem with that I'm not a zealot I'm

not you know I don't like mac products

for myself but I don't care if you buy

them

however where things get a little dicey

is if you take a look at other types of

models and I know that for what I do

here on this channel deep reinforcement

learning you don't need huge amounts of

vram you need like a go a gigabyte or so

for the models that we use uh maybe two

at most if you're running something

really really complex so about a

gigabyte of vram so pretty much any

modern GPU can handle that

but then you see stuff like 16 and

excuse me 13 and 14 gigabytes so

something that would comfortably fit in

the envelope of a 16 gigabyte GPU and

the performance you know varies from you

know seven to eight X or whatever so

it's significantly slower than the 3090

as you would expect even accounting for

the generational uplift with the M2 Max

Okay so

then that raises the question suppose

you don't need these huge models you

have a slightly less stringent

requirement on your vram on total memory

uh what should you get then now if

Mobility is a concern if you really need

Mobility but you're not devoted to Apple

and you don't need the really large

models then I would actually consider

something like

um a last gen laptop so looking here oh

and the other thing to consider is price

so

if you take a look at a reasonably

specked 16-inch MacBook Pro with the M2

Max

with 32 gigs of unified memory so enough

to comfortably fit the resnet model and

a terabyte of SSD storage you're looking

at a price of thirty five hundred

dollars now that is a significant chunk

of change if it makes you money it's a

solid investment again if you need it

you need it buy it and move on with your

life don't even lose sleep over it but

uh what you can do then is take a look

at some comparable products that are not

in the Mac ecosystem that will offer

similar performance so or better

performance because they're based on

Nvidia gpus so

a last gen laptop here so this MSI from

Newegg runs thirty four hundred dollars

uh it's got a one terabyte pcie so it's

a slightly faster SSD drive does that

really matter not so much it's not so

critical but it is faster and faster is

always better you do get the same amount

of RAM and this particular processor uh

is a couple Generations old but offers

16 threads as opposed to 12 cores now as

far as I can tell the apple chips don't

have simultaneous multi-threading so

when it says 12 core here it really

means 12 core it's not 12 core you know

times two for 24 threads it's just 12

core so you get 16 threads versus 12.

okay cool but then scrolling down to the

GPU this might be a little bit hard to

read

um the where did the GPU go it is down

here so it's a Radeon or excuse me a

GeForce 3080 with 16 gigabytes of vram

so it'll comfortably fit everything

except the resnet models and the 3080

laptop GPU is going to be around my by

my guest room is twice as fast as the M2

Max maybe two to three times because

it's going to be around 60 or so of the

uh 30 90. so it should be sufficiently

fast to be faster than the M2 Max uh

for the same sort of price envelope now

of course power efficiency will not be

as great on this because it's a

dedicated GPU so if power efficiency

really matters to you which I I don't

honestly think it does but if you want

to lie and say that it matters to you

then by all means buy the Apple product

um so for a few hundred dollars less you

get something that's going to be

nominally faster albeit based on Windows

I'm not really a fan of Windows I would

personally wipe the hard drive and put

Linux on it but that's just me

so then that raises the question okay so

if you are in the Apple fan base uh you

are dedicated to the Apple ecosystem you

love it and you need it by all means buy

it if you are not dedicated to the Apple

uh ecosystem uh but you do need Mobility

then perhaps a Windows laptop is

something to consider or better yet

Perhaps even a Linux laptop from a

company like system 76 or even Dell

would be something that would appeal to

you now if you don't need Mobility which

I don't you can see here I use a desktop

I'm in Camp desktop firmly I don't

really like mobile form factors I don't

think they do anything any better other

than be moved around and I don't want to

move a Computing device around you know

if I really want to do something I have

a phone otherwise I don't need mobility

in my Computing device however

what is the better option so I have

specked out a high-end desktop on NewEgg

this is using current prices as of

February 1st 2023 these may be lower or

higher in the future of course we have

just come out the other side of the

mining boom and GPU prices have finally

started normalized and lo and behold

I looked this morning could actually see

a an RTX 4090 for sale on NewEgg and in

stock

now probably by the time this video goes

live it'll be gone but I did find other

options for 50 to 100 more on other

websites so if you look around you can

purchase a 4019 if you live near a Micro

Center uh then you can just go pick one

up most of the time

but taking a look at what I have specked

out here you have a motherboard GPU CPU

Ram monitor fan case

um power supply and one terabyte nvme

SSD and it all comes out to thirty six

hundred dollars excluding tax of course

um so for about a hundred dollars more

you get a significantly better desktop

system than you do with the Apple now

you can install any operating system on

this

um you could if you were Intrepid enough

try to do a Hackintosh with it I don't

know I don't actually think that would

work for you but I've never tried it so

I don't know but it may be something to

consider uh I would run Linux or BSD or

if you really must then you can run

um you can run Windows on it of course

that's always an option just factor in a

little bit of money for the operating

system but the point here being you get

an AMD high-end CPU 16 core 32 threads

32 is oftentimes greater than 12. you

get 32 gigs of RAM a 24 gigabyte GP

which will run those resnet models you

get a high-end motherboard you get

um a decent monitor I didn't price out

the monitor too much that's you know up

to you what you want to spend on a

monitor but you get a high-end air

cooler and a very high-end case and

power supply and a significantly faster

SSD to boot so

foreign

you know for 3 600 bucks you get the top

of the line desktop that's going to last

for several years and that's the camp

I'm in personally oh one other copy it

is I don't necessarily endorse any one

of these specific Parts I've just thrown

these in here for pricing comparison uh

so you know if uh if you buy it and you

say hey this board sucks if the

motherboard sucks I didn't say go buy

this motherboard I just said it for 300

you get a you get an am5 motherboard or

your 7950x CPU uh

so other thing to consider though is

that

Nvidia is dropping their new Mobility

chips for the 4090s for the 4000 series

so the 40 uh 40 90 is on sale on Amazon

it's currently sold out

um but the price is 38.99 and this uh

will come with comparable specs 32

gigabytes of ddr5 RAM a two terabyte

nvme drive so pretty fast uh 17.3 inch

chassis so same size as the MacBook

basically

for a little bit more money and the 4090

Mobility is going to be significantly

faster than the M2 Max and pretty much

any task it will consume more power it's

a few hundred dollars more expensive but

if you can afford 3 500 you can probably

afford 3 900. now that's if you're

agnostic to which platform you want to

run on you know Mac versus Windows uh

and you need Mobility this is a great

option or you can save a few hundred

dollars get something that's last gen in

terms of GPU and a couple gens behind in

CPU which isn't as important given the

slow progression and Hardware

requirements for on the CPU front but

you still get more threads than you do

in the Apple case so I hope this was

helpful that's how I take a look oh

there is one other thing and this all

came about because someone in my slack

group so for my students on the neural

net Academy I run a slack group they

said hey what about the uh the new Mac

Mini with the M2 Pro and my gut reaction

was no it's not really that great and I

still I haven't really warmed up to it

after looking at any of this and the

reason is that again if you're not

within the Apple ecosystem you know you

don't have to have that for the price

it's really not that great of a bargain

right because you can buy

you know a 40 70 TI or you could buy a

3070 TI an old 3080 something like that

for seven eight hundred bucks and build

a system around that of comparable

quality and this is kind of the worst of

all worlds right because it's it doesn't

have any any peripherals you don't have

a monitor you don't have a mouse or

keyboard so it's not mobile unless

you're going to Lug around a monitor and

a mouse and keyboard in which case I I

don't know what you're doing that's not

really mobile

so it's not really mobile and if you

don't need something that isn't mobile

um then really the only reason to get it

is if you're in firmly in the Mac camp

in which case hey it's not a bad product

you know I'm not here to hate on Mac I

just don't think they provide very good

performance per dollar I'm not firmly

against Apple it's not something I

choose it's not an ecosystem I choose to

buy into it's it's antithetical to my

personal values you know the whole

Walled Garden stuff isn't for me but if

it's for you and you have done your

research and believe this to be right

for you then by all means buy it I'm not

going to sit here and Bash Apple Fans

because well that's a stupid thing to do

so I don't think the performance per

dollar is very good I think if you don't

need Mobility if you don't need large

models and you are not in the Apple

ecosystem then a desktop is going to be

a far better option if you do need

Mobility but don't need uh very large

models and are not married to the Apple

ecosystem then a Windows based laptop

with a dedicated Nvidia GPU is going to

be a significantly better performing

option in terms of absolute performance

though not performance per watt of

course that is all I have to say on that

in the coming days I'm going to make

another video on my thoughts around the

current state of what type of system to

buy this isn't like a system buying Guy

this is just my thoughts on the M2 Chip

uh any you know disagreements leave them

down below I suspect there will probably

be some I've done my best to not step on

anyone's toes here's so disagreements

comments questions suggestions leave

them in the comments down below and to

see more stuff like this hit that

subscribe button I'll see you in the

next video