AMD's Hidden $100 Stable Diffusion Beast!

Summary

This video explores the rapid advancements in machine learning and the potential emergence of General Artificial Intelligence within five years. It discusses the use of AMD's Instinct MI-25 GPUs for machine learning tasks, highlighting their cost-effectiveness and compatibility with PyTorch. The script also covers the process of repurposing these GPUs for stable diffusion models and the challenges of cooling and software support. The presenter shares insights on building powerful systems for AI, emphasizing the progress and capabilities of current hardware in facilitating AI advancements.

Takeaways

• 🚀 The rapid advancement in machine learning could lead to the emergence of General Artificial Intelligence (AGI) within the next five years.
• 🎮 Gamer GPUs can be utilized for machine learning tasks, although they may lack the VRAM compared to professional GPUs.
• 🔍 AMD is making significant strides in the supercomputer space, challenging Nvidia's dominance with competitive offerings.
• 🛍️ Instinct MI-25 GPUs can be found for around a hundred dollars on eBay, offering a cost-effective entry point for those looking to experiment with machine learning.
• 🔧 With some effort, Instinct MI-25s can be flashed with a V BIOS to become WX 9100s, enhancing their capabilities for machine learning tasks.
• 🔄 AMD has been continuously updating its software to support its Instinct line of GPUs, which is crucial for running modern machine learning models.
• 🛠️ Cooling solutions are a significant challenge when repurposing older GPUs like the MI-25 for machine learning applications.
• 💻 The script mentions a DIY approach to cooling and modifying hardware for better performance in machine learning tasks.
• 🌐 AMD's partnership with PyTorch facilitates easy setup for Python-based machine learning projects on their GPUs.
• 📈 The potential of older hardware like the MI-25 is highlighted, showing that it can still perform competently for tasks like stable diffusion.
• 🔮 The script speculates on the future of AI, suggesting that we may soon have personal AI assistants that are indistinguishable from AGI.

Q & A

What is the current pace of development in the machine learning space according to the transcript?

The transcript suggests that the pace of development in the machine learning space is very fast, with the possibility of seeing General Artificial Intelligence (AI) or something resembling it within the next five years.

Why does the speaker believe that we might see General AI sooner than expected?

The speaker believes that the current advancements and the rate of progress in hardware and software development, as well as the increasing support for AI, indicate that General AI might be achieved sooner than previously anticipated.

What are the challenges of using gamer GPUs for machine learning?

Gamers GPUs can be used for machine learning, but they may lack sufficient VRAM, which is necessary for handling large datasets and complex models in machine learning.

Why is AMD catching up fast in the supercomputer space according to the transcript?

AMD is catching up fast due to their competitive offerings and their presence in significant projects like the Oak Ridge supercomputer, which uses the AMD stack.

What is the significance of the Instinct MI-25s in the context of the discussion?

The Instinct MI-25s are significant because they offer a cost-effective option for machine learning with 16GB of VRAM, and with some modifications, they can be repurposed to work with newer systems.

What modifications can be done to an Instinct MI-25 to enhance its capabilities?

An Instinct MI-25 can have its V BIOS flashed to become a WX 9100, which can almost double the power limit of the card, provided it can be kept cool with appropriate cooling solutions.

Why is the AMD partnership with PyTorch important for machine learning?

The partnership with PyTorch is important because it allows for seamless integration of AMD's hardware with Python-based machine learning frameworks, making it easier for developers to start working on machine learning projects.

What is the role of the 3D printable shroud in the context of the script?

The 3D printable shroud is used to create a cooling solution for the Instinct MI-25 GPU. It allows for the attachment of a brushless blower motor to help dissipate heat more effectively.

What is the significance of the mi-25's dual 8-pin power connectors?

The dual 8-pin power connectors on the mi-25 are significant because they are standard GPU style connectors, making it easier to integrate the card into existing systems.

What is the current status of AMD's support for AI and machine learning?

AMD is actively supporting AI and machine learning by partnering with frameworks like PyTorch, developing new software and features for their Instinct line of products, and working on proper ROCm support for their 7000 series GPUs and beyond.

What is the potential future application of AI as mentioned in the transcript?

The transcript suggests a future where AI can be used to create personalized content, such as substituting actors and characters in movies or creating custom mashups and memes.

Outlines

00:00

🚀 Advancements in AI and GPU Technology

The script discusses the rapid progress in machine learning and the potential for General Artificial Intelligence (AI) to emerge within the next five years. It touches on the use of gamer GPUs for AI experiments due to their limitations in VRAM and compares the performance and market presence of Nvidia and AMD in the supercomputer space. AMD's Instinct MI-25 GPUs are highlighted as a cost-effective option for those willing to invest time in setup, with the potential to be repurposed for AI tasks. The script also mentions the partnership between AMD and PyTorch for machine learning, and the capabilities of older hardware like the Instinct MI-25 for running AI models, despite being on the edge of software support.

05:01

🎨 AI's Creative Potential and Hardware Considerations

This paragraph delves into the creative applications of AI, such as generating images of characters like Danny DeVito in various scenarios, showcasing AI's ability to perform tasks once thought to be years away. It emphasizes the support AMD provides for PyTorch and AI development, and the potential for AI to serve as a personal assistant indistinguishable from General AI. The script also discusses the challenges of using older hardware like the Radeon Pro V540 for AI tasks and the progress being made in GPU pass-through technology. It concludes with a look at the Instinct MI-25's capabilities for running AI models like Stable Diffusion, and the importance of cooling solutions for maintaining performance.

Keywords

💡General Artificial Intelligence (AGI)

General Artificial Intelligence, often referred to as AGI, is the hypothetical ability of a machine to understand, learn, and apply knowledge across a broad range of tasks at a level equal to or beyond that of a human. In the video, the speaker suggests that AGI might be closer than previously thought, possibly within the next five years, indicating a significant advancement in the field of AI.

💡Machine Learning

Machine learning is a subset of AI that enables computers to learn from data and improve their performance on a specific task over time without being explicitly programmed. The script discusses the rapid pace of advancements in machine learning, hinting at the potential for AGI to emerge from these developments.

💡GPUs (Graphics Processing Units)

GPUs are specialized electronic hardware designed to accelerate the creation of images in a frame buffer intended for output to a display device. In the context of the video, GPUs are discussed as essential components for machine learning tasks, with the speaker mentioning both gamer GPUs and enterprise GPUs for different purposes.

💡VRAM (Video Random Access Memory)

VRAM is a type of memory used in GPUs for storing image data. The script mentions the limitations of VRAM in certain GPUs and how it can affect the machine learning process, especially when dealing with large models that require substantial memory for processing.

💡Nvidia and AMD

Nvidia and AMD are two leading companies in the production of GPUs. The video script discusses how both companies are significant in the machine learning space, with Nvidia often receiving more attention, but AMD making strides and being a key player in supercomputer technologies.

💡Instinct MI-25

The Instinct MI-25 is a specific model of GPU by AMD, mentioned in the script as an affordable option for those looking to experiment with machine learning. The video describes how it can be repurposed and used for tasks that would typically require more expensive hardware.

💡PyTorch

PyTorch is an open-source machine learning library based on the Torch library. It is widely used for applications such as computer vision and natural language processing. The script mentions AMD's partnership with PyTorch, making it easier for users to implement machine learning tasks on AMD hardware.

💡Stable Diffusion

Stable Diffusion is a term used in the script to describe a machine learning model capable of generating images from text descriptions. It is an example of how AI can create content, and the video discusses its capabilities when run on different types of hardware.

💡HBM2 (High Bandwidth Memory 2)

HBM2 is a type of memory technology that offers higher bandwidth than traditional DDR memory, making it suitable for high-performance computing applications like machine learning. The script highlights the advantage of having 16 gigabytes of HBM2 in the Instinct MI-25 for handling machine learning tasks.

💡VFIO (Virtual Function I/O)

VFIO is a framework in the Linux kernel that allows for the assignment of hardware devices to virtual machines or user space applications. The script briefly mentions VFIO in the context of GPU pass-through, which is a technique used to give direct access to a GPU to a virtual machine or container for improved performance in machine learning tasks.

💡CDNA (Compute DNA)

CDNA is AMD's architecture for its data center GPUs, designed specifically for compute tasks rather than gaming. The script distinguishes CDNA from AMD's RDNA architecture used in gaming GPUs, highlighting the different use cases and optimizations for each.

Highlights

The rapid advancement in machine learning could lead to the emergence of General Artificial Intelligence within the next five years.

Experimentation with hardware for AI can be tricky due to limitations like VRAM, with options like using gamer GPUs or piecing other components together.

AMD is catching up fast in the supercomputer space, with Oak Ridge using the AMD stack for their operations.

Instinct MI-25s, once used by the one percent, can now be found on eBay for around a hundred dollars and repurposed for AI tasks.

AMD has partnered with PyTorch for ease of use in machine learning with Python.

With some effort, an Instinct MI-25 can be flashed with a V BIOS to become a WX 9100, almost doubling its power limit.

The MI-25, despite being older, still offers 16 gigabytes of VRAM and can handle machine learning tasks effectively.

Stable diffusion models can run on the MI-25, providing high-fidelity previews in a reasonable time frame.

The MI-25 has dual 8-pin power connectors and uses a standard GPU style connector, making it compatible with existing systems.

Cooling is a significant challenge when repurposing enterprise cards like the MI-25 for AI tasks.

A 3D printable shroud and brushless blower motor can be used to cool the MI-25 effectively in standard cases.

Stable diffusion is capable of running 768x768 models on the MI-25, demonstrating the card's surprising competence.

The MI-25's performance is impressive for its price, especially when considering its 16GB HBM2 VRAM.

AMD's focus on supporting PyTorch and AI in general is driving the development of new software and features for their Instinct line.

The potential for AI to replace characters in movies with AI-generated images, like Danny DeVito, is closer than expected.

The hardware available today is capable of running the software that will enable AI agents to perform complex tasks.

The Radeon Pro V540, while not ideal for machine learning, represents an opportunity for experimentation with VFIO GPU pass-through.

AMD's CDNA and RDNA are separate lines, with CDNA being more focused on data centers and compute tasks.

The project demonstrates the intersection of hardware experimentation, 3D printing, and machine learning for creative AI applications.

Transcripts

[Music]

foreign

[Music]

things are moving so fast in the machine

learning space that we could actually

see General artificial intelligence or

at least something that resembles

General artificial intelligence within

the next five years like I know we've

been saying that since the 80s or

certain people have been saying that

since the 80s but maybe it's actually

really happening this time I don't know

if you want to experiment with this no

it gets a little tricky you can use

gamer gpus but you don't have a lot of

vram or you can try to piece other

things together Nvidia gets all the

attention but AMD is actually catching

up fast but make no mistake they've

always been there in the super computer

space I mean there's a reason that Oak

Ridge is using the AMD stack for all of

their stuff but those are the smartest

guys in the room and sometimes it's

exhausting being the smartest guys in

the room so what do you do well the

Instinct mi-25s are about a hundred

dollars on eBay because the one

percenters don't want those anymore they

don't want those in the data center

they're busy buying forty thousand

dollar gpus or twenty five thousand

dollar gpus systems like the mi2 10. I

took a part with Gamers Nexus and we did

some builds our super micro big twin

system six mi210s and 2u that is an

absolutely ridiculous system an AMD for

their part they partnered with pi torch

so if you use Python for machine

learning or anything like that you can

drop in and you're ready to go it's a

little bit more of an uphill battle

getting an instinct mi-25 to work with

that setup but if you're willing to put

in the work a hundred dollars for an

instinct mi-25 you can Flash the V bios

on it to be a WX 9100 and it does

actually have a single Mini DisplayPort

out which will work with that bios you

can almost double the power limit of the

card and as long as you can keep it cool

with whatever Madness you happen to be

running

it will actually be pretty stable now

gigabuster on our forum is the one that

put this together and figured it out and

the dependencies and all of the software

see the mi-25s are so old they're right

on the edge of software support and AMD

has been adding new software and new

features and new everything for their

Instinct line for you know like the

mi-100 and the mi-200 and now the Mi 300

were on the precipice of that and so

those are the cards that are getting the

most attention the mi-25 is based around

the Vega 10 so that's gcn 5.0 but it has

16 gigabytes of vram 16 gigabytes of

vram yes the membrane bandwidth is 462

gigabytes per second you can do a lot

with that with machine learning even

though 16 gigabytes I mean some of these

models take like 40 gigabytes of vram

but you can still do a lot of stable

diffusion stable diffusion automatic 111

running in your local browser doing your

own stuff you can get a bunch of

previews I mean it takes like 20 minutes

to get 16 previews at very high fidelity

768 I'll show you it's it's worth it I

promise and the Mi 25

has dual 8-pin power connectors and is

fortunately a standard GPU style

connector sometimes the CPU 8-pin and

the GPU 8-pin the Enterprise cards a lot

of the time will have a CPU style 8-pin

connector which is a different wiring

than a GPU style 8-pin connector but

these two have the seat the GPU style

eight pin connector so it's pretty easy

to hook up in an existing system the

biggest problem is cooling so we've got

the NZXT bracket here that we've

modified a little bit getting the GPU

mounting pressure just right when you do

this is a little tricky definitely not

recommended not for the faint of heart

and probably not your first project in

an Ideal World the more accessible

solution is to download this 3D

printable shroud and uh bum somebody's

3D printer if you don't have one it'll

Mount here on the end of your card and

then you can pick up a standard you know

this is a

bfb1012h brushless blower motor that's

three pin so it's wired for your

motherboard and then boom look at that

now this is the longest GPU ever but

this will work in cases such as the

fractal meshify the big one and as long

as this fan is running at Full Tilt you

can run

170 Watts through this card without too

much issue now stable diffusion is a lot

of fun and you can run 768 by 768 models

with this it's it's actually

surprisingly competent so at floating

Point 32 512 by 512 with the Euler 20

step it's about uh it's 2.56 to 2.57

iterations per second at 768 by 768 it's

more like 27 seconds so not bad but it

is only using 12 gigabytes of vram so

you're staying well under the 16

gigabyte limit for comparison for how

far we've come that super micro big twin

system if you didn't see those videos be

sure to check out those videos floating

Point 32 20 Step 2 seconds for 512 by

512 and 6 seconds for 768 by 768. that's

pretty fast and so once you follow the

guide and get everything up and running

it works really well now if you're using

newer Hardware you don't really have to

worry about the versions as much again

because AMD is supporting the pi torch

foundation and because they're

supporting AI in general and because you

know it's it's it's it's amd's coming we

did this fun clip of The Shining for a

video that we released on Halloween last

year and it's even more like that today

I've got this thing running generating

you know fun interesting Danny DeVito

images because if you've watched level

one for a long time you know our

Benchmark for AI is when we get to an AI

agent that's just here is the Lord of

the Rings movies from Peter Jackson I

would like for you to replace every

character in this movie with Danny

DeVito and we're basically at the point

where AI can do that

a lot sooner than I expected so that's

why I say artificial general

intelligence probably coming a lot

sooner than I expected and probably on

the order of five years or so or at

least you can have a personal assistant

that is indistinguishable from General

artificial intelligence maybe I don't

know we'll see because you can do this

on an mi-25 it's the software that's

catching up the hardware that we have

today is what's going to run that that's

probably why people are buying these

gpus for 25 35 45

000 even on eBay well the newer ones not

the mi-25s these are these are you know

a hundred dollars oh actually this is

the Radeon Pro v540 Amazon is getting

rid of these right now this is not the

kind of GPU that you will want to do

this stuff on but this is a dual GPU

solution Amazon used to have these you

can get your hands on these as well this

is going to be a different video though

these are maybe not for machine learning

and it's a little tricky to get the

drivers for that if you can help with

the windows drivers for this because

they're in the Amazon Cloud and that's

pretty much it they're not on the AMD

website because this is a you know v540

which is a dual

version of another AMD GPU so it's a

little weird but maybe is a good

candidate for our vfio GPU pass-through

stuff which by the way is making a lot

of progress gonna look out for a video

on that soon yeah the Instinct Mi 25 for

100 that you're able to do this at a

reasonable speed genuinely very

impressive and yeah you can do it on a

gamer GPU but 16 gigabytes of hbm2 for a

hundred dollars again that's a really

good deal I don't know that I would pay

a lot more than 100 because you will put

in a lot of work in order to get it

actually working and follow the level

one guide again thanks gigabuster but uh

yeah you can you can build kind of a

beastly machine assuming that you can

keep them cool stable diffusion on AMD

Hardware both old and new

basically ready

shockingly good and it's a preview for

what's next I've also written a little

guide on getting open Assistant working

with one of their open source models

yeah there's a model that's really good

but it's sort of encumbered by some

licensing issues for commercial and

other use but they do have fully open

models so you can download one of the 12

billion parameter models there and be

able to run it but you will need a beefy

GPU it out of memories uh even with 16

gig vram

gpus and AMD is working on proper Rock M

support for 7000 series gpus and Beyond

so 20 gigs 24 gigs but just understand

the AMD has their cdna and their rdna

and those are separate lines these are

cdna cards compute DNA and that's what

they still have in the data center

that's what our mi210 is that's what the

Mi 300s are and eventually those roads

may come back together but fundamentally

your cdna and your uh your your gaming

cards are a different things and so it's

a little it's there for experimentation

but it's it's a little different this

has been a project I mean where else can

you play with the angle grinder and 3D

printed parts and also machine learning

toward our ultimate goal of being able

to just ask an AI agent to substitute in

your favorite actors and characters into

whatever movie and genre you want to

create any kind of mashup or meme that

you want much to the horror of literally

everybody that's not a normal human

being I'm one of those level one this

has been some fun

with the AMD Instinct mi-25 and to show

that you know if you're just going to

run pytorch you're basically good to go

on an AMD cdna cards at this point and

it's very good there's a reason that Oak

Ridge is using this

and whether this level one I'm signing

out you find me in the level one forms

foreign

[Music]