#黃仁勳 驚喜為 #美超微 #梁見後 #COMPUTEX 主題演講站台｜完整精華

Summary

In this engaging keynote, Nvidia CEO Jensen Huang discusses the transformative impact of AI and accelerated computing on data centers. Highlighting the convergence of efficiency and performance, he introduces the concept of 'Green Computing' and the emergence of generative AI. Huang emphasizes the need for modernizing data centers to harness the potential of these technologies, which are projected to grow to a $3 trillion industry by 2030. He also unveils new products, including advanced liquid cooling systems, designed to optimize energy consumption and computational throughput, ultimately driving revenue in what he refers to as 'AI factories.' The talk underscores the importance of safety, technology, and policy advancements in the AI domain.

Takeaways

• 🧠 AI is revolutionizing computing with the advent of accelerated computing and Green Computing, focusing on energy efficiency.
• 📈 The demand for AI is soaring, with data centers needing to be modernized to handle the transition to generative AI, which is expected to impact every data center globally.
• 💡 Green Computing is not just an environmentally friendly approach but also a cost-efficient one, aiming to reduce waste and energy consumption in data centers.
• 🚀 Nvidia is at the forefront of this change with new products and technologies, including a significant number of new products aimed at accelerating data centers.
• 💧 Nvidia is also innovating in cooling technologies, such as data center liquid cooling (DLC), to lower power consumption and enable more AI chips to be manufactured.
• 🔢 The scale of operations is massive, with Nvidia shipping thousands of units per month and aiming to increase this number significantly in the near future.
• 🔑 The importance of software cannot be overstated, as it plays a crucial role in the performance and efficiency of high-performance computing systems.
• 🌐 Networking is evolving into a computing fabric, facilitating distributed computing and the efficient distribution of workloads across networks.
• 🔄 Checkpoint restart is a vital feature for high uptime and utilization in AI systems, and new technologies like Grace CPU are designed with this in mind.
• 🛡️ Safety is a paramount concern in AI, with the need for guardrails, monitoring systems, and good practices to ensure the responsible advancement of AI technology.
• 🌟 The future of AI is bright, with the potential for significant revenue generation through the creation and utilization of intelligent tokens across various industries.

Q & A

What is the significance of the term 'Green Computing' as mentioned in the transcript?

In the context of the transcript, 'Green Computing' refers to energy-efficient computing. It's about making data centers more efficient and reducing wasted energy and costs, which is a key focus for Nvidia and the future of computing.

What is the current state of CPU scaling according to the transcript?

The transcript mentions that CPU scaling has slowed for many years, leading to an enormous amount of wasted energy and cost trapped inside data centers.

What is the role of accelerated computing in the context of data centers?

Accelerated computing is crucial for making data centers more efficient. It helps to release the trapped waste and use that energy for new purposes, such as accelerating every application and data center.

What does the term 'generative AI' refer to in the transcript?

Generative AI in the transcript refers to the process of AI generating new content such as text, images, and videos. It is a significant shift in computing that will affect every data center globally.

How does the speaker describe the impact of generative AI on data centers?

The speaker suggests that the transition to generative AI will impact every single data center in the world, necessitating the modernization of the trillion-dollar worth of data centers that are established.

What is the significance of the number '1,000 R' mentioned in the transcript?

The number '1,000 R' refers to the shipping of 1,000 units of a certain product per month, which is part of Nvidia's efforts to lower power consumption and enable the manufacturing of more AI chips.

How does liquid cooling (DLC) contribute to AI chip manufacturing according to the transcript?

Liquid cooling (DLC) is being shipped in production to lower power consumption, allowing for the manufacturing of more AI chips and contributing to the efficiency and performance of data centers.

What is the goal for the year mentioned by the speaker in relation to shipping?

The goal for the year, as mentioned in the transcript, is to ship more than 10,000 units, indicating a significant increase in production and shipping targets.

What does the speaker mean by 'AI factories'?

The term 'AI factories' refers to data centers that are directly generating revenues for factories by utilizing AI technologies to create value through processes like token generation.

How does the speaker view the future of computing throughput and its relation to revenue?

The speaker views computing throughput as directly tied to revenue generation. The faster the generation of tokens (which embed intelligence), the higher the throughput, utilization, and consequently, the revenues.

What is the importance of software compatibility in high-performance computing as discussed in the transcript?

Software compatibility is crucial in high-performance computing because it allows for the seamless integration of systems and ensures that all components work together efficiently, which is vital for achieving high performance and efficiency in data centers.

What are the three important software stacks mentioned in the transcript?

The three important software stacks mentioned in the transcript are CUDA, which is famous for parallel computing, a networking stack for creating a computing fabric, and DOA, which is used for distributing workload across the network efficiently.

How does the speaker emphasize the importance of safety in AI?

The speaker emphasizes the importance of safety in AI by comparing it to autopilot in airplanes, stating that just as many technologies and practices were needed to keep autopilot safe, similar measures will be necessary for AI, including guardrails, monitoring systems, and good policies.

Outlines

00:00

🤖 AI and Green Computing Revolution

The speaker, Jensen Huang, CEO of Nvidia, discusses the transformative impact of AI and the concept of Green Computing. He emphasizes the arrival of accelerated computing to address the exponential growth in data processing and the inefficiency of CPU scaling. The talk highlights the potential for significant energy and cost savings in data centers through the adoption of accelerated computing. Huang also introduces the idea of generative AI, which involves creating new content such as text, images, and videos, and predicts its widespread influence on data centers globally. The conversation touches on the preparation of Supermicro to modernize these data centers with new systems and products, reflecting the urgent need for such advancements due to the growing demand and the potential for energy efficiency and cost savings.

05:01

🚀 Advancements in AI Chip Technology and Liquid Cooling

The discussion shifts to the specifics of AI chip production and the innovative steps taken to enhance power efficiency. The speaker mentions the shipping of data center liquid cooling (DLC) systems to reduce power consumption, allowing for the manufacturing of more AI chips. The conversation includes playful banter about the听不懂 (bu dong ting) between Chinese and American colleagues, highlighting cultural and language barriers. The speaker also provides insights into the massive scale of operations, with plans to ship thousands of units per month, and the technological marvel of creating the most advanced computers in the world. The summary underscores the importance of energy efficiency in data centers and the financial implications of these advancements, introducing the concept of token generation as a new commodity with monetary value.

10:02

🏭 Transforming Data Centers into AI Factories

The speaker elaborates on the transition of traditional data centers into AI factories, which are revenue-generating entities. He explains that these AI factories are not merely for file retrieval or email exchanges but are directly involved in generating income through AI computations. The speaker discusses the significance of factors like reliability, throughput, and startup time in maximizing factory output and revenue. The conversation also covers the integration of systems into a rack scale for efficient operation and the importance of software compatibility. The speaker assures that all systems are ready to serve customers, emphasizing the readiness of Supermicro to meet the demands of the AI factory era.

15:03

🛠️ The Importance of Software Stacks in AI Development

The speaker delves into the crucial role of software stacks in the development and operation of AI systems. He mentions CUDA as a renowned software stack and discusses the importance of networking as a computing fabric, especially in the context of high-performance computing. The speaker highlights the advancements in networking speeds, moving from gigahertz to terahertz, and the significance of software that operates on top of these networks for distributed computing. The conversation also touches on the integration of various chips and systems, such as Grace CPU and Blackwell GPUs, and the energy efficiency of high-speed interconnects. The speaker concludes by emphasizing the importance of energy efficiency in translating to higher performance and the concept of Green Computing.

20:05

🛡️ Prioritizing Safety and Advancement in AI

In the final paragraph, the speaker addresses the importance of safety and continuous advancement in AI technology. He draws a parallel between the safety measures implemented in aviation, such as autopilot systems, air traffic control, and pilot monitoring, to the safety measures needed for AI. The speaker stresses the need for guardrails, monitoring systems, and good practices to ensure the safe operation of AI. He also underscores the importance of good policies and the collective responsibility to advance good science, engineering, business, and industrial practices. The speaker concludes with a humorous note, suggesting that buying more products equates to increased safety, highlighting the company's commitment to providing safe and advanced AI solutions.

Keywords

💡AI

AI, or Artificial Intelligence, refers to the simulation of human intelligence in machines that are programmed to think like humans and mimic their actions. In the video, AI is central to discussions about the future of computing and data centers, as well as generative AI, which involves AI creating content such as text, images, and videos.

💡Accelerated Computing

Accelerated Computing is a concept where computational tasks are performed faster than they would be on traditional CPUs. The script mentions that accelerated computing has arrived at a time when CPU scaling has slowed down, and it is crucial for handling the exponential increase in data processing with more efficiency and less energy waste.

💡Green Computing

Green Computing is the practice of designing, manufacturing, using, and disposing of computers and computing resources efficiently and effectively to minimize environmental impact. In the script, green computing is associated with energy-efficient computing, which Nvidia is focusing on to reduce waste and save energy in data centers.

💡Generative AI

Generative AI is a subset of AI that focuses on creating new content rather than just recognizing or classifying existing content. The script discusses generative AI's potential to transform every data center by generating text, images, and videos, which is a significant shift from traditional inference tasks.

💡Data Centers

Data Centers are large facilities that house numerous servers used to store, process, and manage large amounts of data. The script highlights the need to modernize existing data centers, which are estimated to be worth a trillion dollars, with new technologies to improve efficiency and performance.

💡Supermicro

Supermicro is a company mentioned in the script that specializes in high-performance server technology, storage solutions, and computing. The CEO of Nvidia, Jensen Huang, discusses Supermicro's readiness to provide products and services for the modernization of data centers.

💡Liquid Cooling

Liquid Cooling is a method of cooling that uses a liquid coolant to absorb and dissipate heat from computer components. The script mentions that Supermicro is now shipping data center liquid cooling (DLC) products to reduce power consumption and enable more AI chips to be manufactured.

💡GPU

A GPU, or Graphics Processing Unit, is a specialized electronic circuit designed to rapidly manipulate and alter memory to accelerate the creation of images in a frame buffer intended for output to a display device. In the script, GPUs are discussed as a critical component in the complex systems that make up modern data centers and AI factories.

💡Cuda

CUDA, which stands for Compute Unified Device Architecture, is a parallel computing platform and application programming interface (API) model created by Nvidia. It allows software developers to use Nvidia GPUs for general purpose processing (GPGPU). The script refers to Cuda as a foundational software stack for building applications on top of Nvidia's technology.

💡Networking

In the context of the script, networking refers to the infrastructure that allows computers to communicate with each other within a data center or across the internet. The script discusses the evolution of networking from simple communication to a computing fabric that is essential for distributed computing and high-performance computing.

💡Checkpoint Restart

Checkpoint Restart is a technique used in computing to save the state of a process or computation at certain points (checkpoints) so that it can be resumed if it is interrupted. The script mentions the importance of checkpoint restart for high utilization and uptime in data centers, particularly during the training phase of AI models.

Highlights

Nvidia CEO Jensen Huang shares his vision on AI and its impact on computing.

Accelerated Computing and Green Computing are two concurrent trends shaping the future of data centers.

Data processing needs have grown exponentially, while CPU scaling has slowed, leading to energy and cost inefficiencies.

Accelerating data centers can lead to significant savings by reducing waste.

Introduction of 220 new products by Nvidia, emphasizing their commitment to innovation.

Generative AI is emerging as a new form of computing, distinct from inference.

Generative AI will transform every data center, with a projected $3 trillion worth by 2030.

The demand for modernizing data centers with advanced systems is immense.

Supermicro is ready to support the modernization with their products and services.

Nvidia is shipping data center liquid cooling systems to reduce power consumption.

The significance of energy efficiency in AI chip manufacturing and its impact on the industry.

Nvidia's goal to ship more than 10,000 units, highlighting their growth and ambition.

The complexity and technological marvel of Nvidia's GPUs and the systems they are part of.

The importance of software in running advanced computer systems and its role in Nvidia's offerings.

Nvidia's focus on three key software stacks: CUDA, networking as a computing fabric, and distributed computing.

The innovative use of liquid cooling systems to eliminate costs and improve data center efficiency.

The concept of token generation as a new commodity in the AI industry, with direct revenue implications.

The transformation of data centers into AI factories, emphasizing their role in generating revenue.

The integration of systems into rack scale for efficient startup, utilization, and throughput.

Nvidia's commitment to advancing AI technology, safety, and policy to ensure responsible AI development.

Transcripts

I know only

some fortunately we are very lucky again

to invite the AI

genius our common

friend our common friend is very busy

huh Invidia found CEO Jensen to share

his great vision with us

[Applause]

[Music]

[Applause]

[Music]

thank

you hi

everybody now

what that AI is changing minium because

of you what's new

today I have to admit just now when I

was coming to your keynote in the car I

fell asleep

and so right now right now I'm a little

bit groggy so if I say nonsense things

please I let me apologize first no well

let's see um uh Charles we've gone back

a very long ways yeah and and um uh what

are we doing oh I needed some water I

need to spe up okay right my energy

yeah they said I was on this side and

you keep going on my

side this is what happens when we don't

practice you don't need to and you are

no time you you don't need and so so um

I uh what what were we saying um this is

a very important time because we have a

new agent Computing coming there are two

things that are happening at the same

time the first is accelerated Computing

accelerated Computing has arrived at a

time

oh Green Computing

yeah Green computer yeah okay

Computing I think I think when you say

Green Computing you mean energy

efficient Computing right yes Nvidia is

energy efficient Computing yes we have S

we follow you all

right look Green Computing and Green

Computing all right so so um uh

accelerated computing's time has come

because for a very long time the amount

of data processing has been increasing

exponentially yeah and yet CPU scaling

has slowed for many many years so we've

been we have now an enormous amount of

waste wasted energy and wasted cost

trapped inside the data centers so when

we accelerate the data centers the

savings

incredible because it has been sold long

of waste

trapped and so now we can release the

waste and use that energy for a new

purpose number one accelerate every

application accelerate every data center

these

amazing servers here right so many new

products so many new products you have

220 new products unbelievable did he

tell you that already no I W very high

I came to announce super micros products

and so that's the first thing the second

thing is because the Energy Efficiency

and the performance efficiency and the

cost efficiency is so incredibly great

with accelerated Computing a new way of

doing Computing has emerged and it's

called generative AI generative AI is an

incredible thing people say generative

AI inference it's related not the same

inference

recognizing C dog speech inference

generation text Generation image

Generation video generation that's what

we call a generative AI the pressure of

generative AI to not the pressure but

the the transition to generative AI will

affect every single data center in the

world we have a trillion dat a trillion

dollars worth of data centers in the

world that's established $3 trillion

probably by 2030 in another 6 years we

have to to modernize all of them with

these amazing systems yeah that's the

reason why the demand is so great

because all of these data centers has to

be modernized and Charles and the super

micro team is ready to take your

order Json I'm your I'm your best sales

guy thank you I work on commission no

commission we buy more cheaper from you

don't buy more

[Laughter]

chips so

that's Jon sh Michael is now shipping

data center uh liqu cooling DLC R inum

production now to lower the power

consumption so you can manufacture more

AI chip yeah yeah thousand of how here

you see

[Applause]

[Laughter]

I have many American colleagues they

don't understand my Chinese I have many

Chinese colleagues they don't understand

my

Chinese hi

y we are shipping up to 1,000 R per

month now 1,000 R like it is multiply by

ASP yeah you're going to be a gigantic

company yeah thank

you that's why I need a more

CH did you guys all do the

math Millions

times thousands time 52 no no no you

charging me $2 million more than $2

million for d

[Laughter]

are we allowed to do this on TV are we

on

TV I I guess the well is

this so we are shipping about 1,000

that's incredible now this this uh

600,000 Parts this is probably more than

600,000 parts how many pounds oh I don't

know can I move three I think it's 3,000

lb more than 3,000 lb

yeah it's incredible so yeah our goal

this year is to ship more than 10,000

record you know the Charles this is the

thing that's really amazing uh people

think that we're building

gpus you know GP is a

chip there are 72 chips in here and then

there are six

600,000 other

parts

it's 72 chips probably weighs one

pound this is 3, 2,999 other

pounds so the amount of Technology

that's inside one of these RS is really

quite extraordinary this is a technology

Marvel the most most most complex most

advanced computer the world's ever made

yeah exactly the p in the world now yeah

absolutely incredible and the software

that it takes to run this is

unbelievable yeah unbelievable isn't

that right and so I think that that

people now are starting to realize that

when we say GPU server of course the

brain is the GPU yeah but the system is

much much more complex than that and

super micro does amazing engineering

thank

[Laughter]

you huh what I

okay then we there's some Americans this

year we are going to ship hopefully make

when we're together sometimes we speak

Taiwanese sometimes we speak Mandarin

and then when we disagree we speak

[Laughter]

English we try to make a thlc mar share

from 1% to 15%

this year wow Save lot of power for your

TB yeah yeah the Energy Efficiency is so

much better the cost to the data center

is cheaper cheaper that's right people

don't realize this liquid cooled systems

eliminates an enormous amount of cost in

the data center yeah so that you can use

that waste capture that waste and put it

into Computing in the future in the

future Computing throughput is

revenues because it's token generation

and token generation is dollars per

million tokens just like

energy dollars per kilowatt hour we have

now invented a new commodity this is a

very important idea for all of you this

is a new commodity it has value and the

faster you can generate it the higher

throughput the greater utilization the

higher your revenues it is absolutely

true and it's directly measurable that's

why this is a factory not a data center

that's why this is a factory not a file

server it's not a retrieval of files

it's not used for exchanging emails this

is directly generating revenues for

factories that's why we call it AI

factories and so

powerful and only s million

[Laughter]

dollars a

such okay so $3 million and you can

generate who knows how much revenue per

year right uh 3 million 1,000 and every

year have how many

months

12 the the return on the return on large

language model generation token

generation is going to be very very good

yeah be huge and the reason for that is

because the token embeds intelligence

yeah and the int could be used in so

many different Industries and so the

future is very important it's time to

Startup yeah time to

Startup throughput yeah

utilization all matter so

reliability has Revenue implication

throughput has Revenue implication

startup has Revenue implication yeah

that's why it's so important that we

integrate the whole s whole system into

a rack scale get all the software

working connected to all the all the

networking so that and we build all of

our own data centers we build our own

supercomputers so that we know when you

install this when you install super

micro in your factories the startup time

will be extremely fast your utilization

will be extremely high and your

throughput will be extremely high

because your revenues depends on it

Factory output is measured by all of

those factors very complicated yeah and

all of those R are Invidia sofware

license all certified so the sound of

that parking the cable and they can run

and it runs that's right and all of the

Nvidia Nims all of the large language

models it just runs on all these systems

yeah

[Laughter]

[Applause]

we are shipping thousand R

very

yes very

beautiful Charles Charles said that this

is

everything everything in here is NVIDIA

for all the American citizens

there

from to H AI everything all Nvidia sare

all all Green Computing all Green

Computing all green computer all all

support that's

fantastic

good let go through something detail

okay okay

okay H1 H2 B1 for you cooling wow

shipping in B wow and this one your p200

uhhuh fully ready beautiful for your

chip beautiful beautiful this will be

how many time faster than this so we

have we have we have uh uh for Blackwell

Blackwell has air cool

liquor

cooled

x86

Grace MV link 8 MV link 2 MV link 36 MV

link 72 yeah so many different

configurations yeah so that depending on

the type of type of utilization type of

use case you have the type of data

center that you have uh Charles is ready

to serve you immediately right

immediately doesn't need to acheve yeah

one hand we got to acheve second hand we

Shi to C W thank goodness we only need

two hands in two weeks in two

weeks that's incredible and all of it

software compatible this is really this

is really the amazing thing certifi

literally everything here is software

compatible one% yeah and software as we

know is the most complex part of high

performance Computing yeah thank you for

those great offering they are all ready

to service our customer there are three

very important software Stacks that we

have in our company that everything is

built on top of the first of course is

Cuda very famous the second for all of

the networking because networking is

just not networking networking today

networking today is a Computing

fabric networking today is a Computing

fabric not just for sending email to

each

other

4 Mez a gigahertz megahertz this is not

1980s

be

Mez

kilohertz gahz gigahertz yes 400

gigabits per second 800 gigabits per

second and and then of course Next

Generation coming 1600 but the important

thing is all of the software that we

have that runs on the networking for

distributed computing is on top of two

software Stacks one is called DOA for

the nick nickel for the fabric yeah and

it enables us to distribute the workload

across the network very very efficiently

because ethernet was was not designed

for hyperform computing you make our job

easier but still very py because you

have so many

great my job is to help give you

[Laughter]

job we

and because because you do such a good

job it becomes gives me job oh don't

forget that your another

baby yeah yeah yeah

yeah inside

here this this is an incredible

incredible system in fact in

fact in fact um these chips are all

connected together using high-speed

interconnect the world's fastest CIS the

CIS is incredibly fast and very energy

efficient and so we can connect this

great CPU to dual Blackwell gpus and

that's very important because in the

training stage the memory system of

Grace could be used for checkpoint

restart checkpoint and restarting is

very important for high utilization and

high uptime and so checkpoint restart uh

could be stored in the system memory

that system memory is very low energy

very low power and the link between

Blackwell and Grace is very very high

second during inference time as you know

there's a concept called

prompts context in context training

prompting that prompt memory that

context memory is right here this is the

memory memory the thinking memory the

working memory of AI and so this memory

needs to be very high performance very

low energy and so during training we

have good use for gray gray CPU during

inference we have excellent use for gray

CPU and the interconnect is very very

high speed very low power F optimiz and

so the re the benefit is because we

compress so many in one system yeah if

we

save 20 watts 50 Watts on the

interconnect you multiply by the whole

rack then we can take the energy and use

it for computing y so Energy Efficiency

translates to higher performance to

that's right Green

[Applause]

Computing

huh I am a super micro employee

super micro

employee where AI Control

us of course not um we we have to we

have to uh the most important thing of

course at the moment is we have to make

AI work

well right now ai is of course uh

working extremely well and in many

applications AI has become good enough

to good enough to become useful it has

achieved the plateau of good enough very

useful however we want it to be

incredibly good we want it to be very

functional everything from Guard railing

for uh fine-tuning skill learning there

are many different things that we still

have to improve okay so we know that AI

is AI still has long ways to go that's

job number one is Advance the technology

at the same time we have to advanced

Safety technology as you know uh our the

planes that we all flew on to come here

has autopilot and autopilot is automatic

technology in order for planes to be

safe a great deal of Technology had to

be invented to keep the plane safe yeah

also practices to monitor the planes air

traffic control other planes monitor the

planes Pilots monitoring each other many

different ways to keep uh AI uh keep

autopilot safe in the future we'll do

the same thing with AI there will be AIS

that watch AIS there are people that

watch AIS there's gu right guard rails

that keep AI guard rail and so there's

going to be a whole lot of different

Technologies we need to create for

safety technology for safety and then

third of course we need to have good

policies for safety good practices and

good policies for safety talking about

it is very important so that we can all

remind each other that we have to do

good science good engineering good

business practice good policy practice

good industrial practice all of those

things has to advance so perfect

strategy so the conclusion is one the

more you buy the more

safe the more you buy the more you safe

the more you buy the more you safe

yeah thank you Jas thank you so much

good job thank you

everybody thank you okay thank you thank

you thank you thank you all right have a

great

thank you

[Music]