#黃仁勳 驚喜為 #美超微 #梁見後 #COMPUTEX 主題演講站台|完整精華
Summary
TLDRIn this engaging keynote, Nvidia CEO Jensen Huang discusses the transformative impact of AI and accelerated computing on data centers. Highlighting the convergence of efficiency and performance, he introduces the concept of 'Green Computing' and the emergence of generative AI. Huang emphasizes the need for modernizing data centers to harness the potential of these technologies, which are projected to grow to a $3 trillion industry by 2030. He also unveils new products, including advanced liquid cooling systems, designed to optimize energy consumption and computational throughput, ultimately driving revenue in what he refers to as 'AI factories.' The talk underscores the importance of safety, technology, and policy advancements in the AI domain.
Takeaways
- 🧠 AI is revolutionizing computing with the advent of accelerated computing and Green Computing, focusing on energy efficiency.
- 📈 The demand for AI is soaring, with data centers needing to be modernized to handle the transition to generative AI, which is expected to impact every data center globally.
- 💡 Green Computing is not just an environmentally friendly approach but also a cost-efficient one, aiming to reduce waste and energy consumption in data centers.
- 🚀 Nvidia is at the forefront of this change with new products and technologies, including a significant number of new products aimed at accelerating data centers.
- 💧 Nvidia is also innovating in cooling technologies, such as data center liquid cooling (DLC), to lower power consumption and enable more AI chips to be manufactured.
- 🔢 The scale of operations is massive, with Nvidia shipping thousands of units per month and aiming to increase this number significantly in the near future.
- 🔑 The importance of software cannot be overstated, as it plays a crucial role in the performance and efficiency of high-performance computing systems.
- 🌐 Networking is evolving into a computing fabric, facilitating distributed computing and the efficient distribution of workloads across networks.
- 🔄 Checkpoint restart is a vital feature for high uptime and utilization in AI systems, and new technologies like Grace CPU are designed with this in mind.
- 🛡️ Safety is a paramount concern in AI, with the need for guardrails, monitoring systems, and good practices to ensure the responsible advancement of AI technology.
- 🌟 The future of AI is bright, with the potential for significant revenue generation through the creation and utilization of intelligent tokens across various industries.
Q & A
What is the significance of the term 'Green Computing' as mentioned in the transcript?
-In the context of the transcript, 'Green Computing' refers to energy-efficient computing. It's about making data centers more efficient and reducing wasted energy and costs, which is a key focus for Nvidia and the future of computing.
What is the current state of CPU scaling according to the transcript?
-The transcript mentions that CPU scaling has slowed for many years, leading to an enormous amount of wasted energy and cost trapped inside data centers.
What is the role of accelerated computing in the context of data centers?
-Accelerated computing is crucial for making data centers more efficient. It helps to release the trapped waste and use that energy for new purposes, such as accelerating every application and data center.
What does the term 'generative AI' refer to in the transcript?
-Generative AI in the transcript refers to the process of AI generating new content such as text, images, and videos. It is a significant shift in computing that will affect every data center globally.
How does the speaker describe the impact of generative AI on data centers?
-The speaker suggests that the transition to generative AI will impact every single data center in the world, necessitating the modernization of the trillion-dollar worth of data centers that are established.
What is the significance of the number '1,000 R' mentioned in the transcript?
-The number '1,000 R' refers to the shipping of 1,000 units of a certain product per month, which is part of Nvidia's efforts to lower power consumption and enable the manufacturing of more AI chips.
How does liquid cooling (DLC) contribute to AI chip manufacturing according to the transcript?
-Liquid cooling (DLC) is being shipped in production to lower power consumption, allowing for the manufacturing of more AI chips and contributing to the efficiency and performance of data centers.
What is the goal for the year mentioned by the speaker in relation to shipping?
-The goal for the year, as mentioned in the transcript, is to ship more than 10,000 units, indicating a significant increase in production and shipping targets.
What does the speaker mean by 'AI factories'?
-The term 'AI factories' refers to data centers that are directly generating revenues for factories by utilizing AI technologies to create value through processes like token generation.
How does the speaker view the future of computing throughput and its relation to revenue?
-The speaker views computing throughput as directly tied to revenue generation. The faster the generation of tokens (which embed intelligence), the higher the throughput, utilization, and consequently, the revenues.
What is the importance of software compatibility in high-performance computing as discussed in the transcript?
-Software compatibility is crucial in high-performance computing because it allows for the seamless integration of systems and ensures that all components work together efficiently, which is vital for achieving high performance and efficiency in data centers.
What are the three important software stacks mentioned in the transcript?
-The three important software stacks mentioned in the transcript are CUDA, which is famous for parallel computing, a networking stack for creating a computing fabric, and DOA, which is used for distributing workload across the network efficiently.
How does the speaker emphasize the importance of safety in AI?
-The speaker emphasizes the importance of safety in AI by comparing it to autopilot in airplanes, stating that just as many technologies and practices were needed to keep autopilot safe, similar measures will be necessary for AI, including guardrails, monitoring systems, and good policies.
Outlines
🤖 AI and Green Computing Revolution
The speaker, Jensen Huang, CEO of Nvidia, discusses the transformative impact of AI and the concept of Green Computing. He emphasizes the arrival of accelerated computing to address the exponential growth in data processing and the inefficiency of CPU scaling. The talk highlights the potential for significant energy and cost savings in data centers through the adoption of accelerated computing. Huang also introduces the idea of generative AI, which involves creating new content such as text, images, and videos, and predicts its widespread influence on data centers globally. The conversation touches on the preparation of Supermicro to modernize these data centers with new systems and products, reflecting the urgent need for such advancements due to the growing demand and the potential for energy efficiency and cost savings.
🚀 Advancements in AI Chip Technology and Liquid Cooling
The discussion shifts to the specifics of AI chip production and the innovative steps taken to enhance power efficiency. The speaker mentions the shipping of data center liquid cooling (DLC) systems to reduce power consumption, allowing for the manufacturing of more AI chips. The conversation includes playful banter about the听不懂 (bu dong ting) between Chinese and American colleagues, highlighting cultural and language barriers. The speaker also provides insights into the massive scale of operations, with plans to ship thousands of units per month, and the technological marvel of creating the most advanced computers in the world. The summary underscores the importance of energy efficiency in data centers and the financial implications of these advancements, introducing the concept of token generation as a new commodity with monetary value.
🏭 Transforming Data Centers into AI Factories
The speaker elaborates on the transition of traditional data centers into AI factories, which are revenue-generating entities. He explains that these AI factories are not merely for file retrieval or email exchanges but are directly involved in generating income through AI computations. The speaker discusses the significance of factors like reliability, throughput, and startup time in maximizing factory output and revenue. The conversation also covers the integration of systems into a rack scale for efficient operation and the importance of software compatibility. The speaker assures that all systems are ready to serve customers, emphasizing the readiness of Supermicro to meet the demands of the AI factory era.
🛠️ The Importance of Software Stacks in AI Development
The speaker delves into the crucial role of software stacks in the development and operation of AI systems. He mentions CUDA as a renowned software stack and discusses the importance of networking as a computing fabric, especially in the context of high-performance computing. The speaker highlights the advancements in networking speeds, moving from gigahertz to terahertz, and the significance of software that operates on top of these networks for distributed computing. The conversation also touches on the integration of various chips and systems, such as Grace CPU and Blackwell GPUs, and the energy efficiency of high-speed interconnects. The speaker concludes by emphasizing the importance of energy efficiency in translating to higher performance and the concept of Green Computing.
🛡️ Prioritizing Safety and Advancement in AI
In the final paragraph, the speaker addresses the importance of safety and continuous advancement in AI technology. He draws a parallel between the safety measures implemented in aviation, such as autopilot systems, air traffic control, and pilot monitoring, to the safety measures needed for AI. The speaker stresses the need for guardrails, monitoring systems, and good practices to ensure the safe operation of AI. He also underscores the importance of good policies and the collective responsibility to advance good science, engineering, business, and industrial practices. The speaker concludes with a humorous note, suggesting that buying more products equates to increased safety, highlighting the company's commitment to providing safe and advanced AI solutions.
Mindmap
Keywords
💡AI
💡Accelerated Computing
💡Green Computing
💡Generative AI
💡Data Centers
💡Supermicro
💡Liquid Cooling
💡GPU
💡Cuda
💡Networking
💡Checkpoint Restart
Highlights
Nvidia CEO Jensen Huang shares his vision on AI and its impact on computing.
Accelerated Computing and Green Computing are two concurrent trends shaping the future of data centers.
Data processing needs have grown exponentially, while CPU scaling has slowed, leading to energy and cost inefficiencies.
Accelerating data centers can lead to significant savings by reducing waste.
Introduction of 220 new products by Nvidia, emphasizing their commitment to innovation.
Generative AI is emerging as a new form of computing, distinct from inference.
Generative AI will transform every data center, with a projected $3 trillion worth by 2030.
The demand for modernizing data centers with advanced systems is immense.
Supermicro is ready to support the modernization with their products and services.
Nvidia is shipping data center liquid cooling systems to reduce power consumption.
The significance of energy efficiency in AI chip manufacturing and its impact on the industry.
Nvidia's goal to ship more than 10,000 units, highlighting their growth and ambition.
The complexity and technological marvel of Nvidia's GPUs and the systems they are part of.
The importance of software in running advanced computer systems and its role in Nvidia's offerings.
Nvidia's focus on three key software stacks: CUDA, networking as a computing fabric, and distributed computing.
The innovative use of liquid cooling systems to eliminate costs and improve data center efficiency.
The concept of token generation as a new commodity in the AI industry, with direct revenue implications.
The transformation of data centers into AI factories, emphasizing their role in generating revenue.
The integration of systems into rack scale for efficient startup, utilization, and throughput.
Nvidia's commitment to advancing AI technology, safety, and policy to ensure responsible AI development.
Transcripts
I know only
some fortunately we are very lucky again
to invite the AI
genius our common
friend our common friend is very busy
huh Invidia found CEO Jensen to share
his great vision with us
[Applause]
[Music]
[Applause]
[Music]
thank
you hi
everybody now
what that AI is changing minium because
of you what's new
today I have to admit just now when I
was coming to your keynote in the car I
fell asleep
and so right now right now I'm a little
bit groggy so if I say nonsense things
please I let me apologize first no well
let's see um uh Charles we've gone back
a very long ways yeah and and um uh what
are we doing oh I needed some water I
need to spe up okay right my energy
yeah they said I was on this side and
you keep going on my
side this is what happens when we don't
practice you don't need to and you are
no time you you don't need and so so um
I uh what what were we saying um this is
a very important time because we have a
new agent Computing coming there are two
things that are happening at the same
time the first is accelerated Computing
accelerated Computing has arrived at a
time
oh Green Computing
yeah Green computer yeah okay
Computing I think I think when you say
Green Computing you mean energy
efficient Computing right yes Nvidia is
energy efficient Computing yes we have S
we follow you all
right look Green Computing and Green
Computing all right so so um uh
accelerated computing's time has come
because for a very long time the amount
of data processing has been increasing
exponentially yeah and yet CPU scaling
has slowed for many many years so we've
been we have now an enormous amount of
waste wasted energy and wasted cost
trapped inside the data centers so when
we accelerate the data centers the
savings
incredible because it has been sold long
of waste
trapped and so now we can release the
waste and use that energy for a new
purpose number one accelerate every
application accelerate every data center
these
amazing servers here right so many new
products so many new products you have
220 new products unbelievable did he
tell you that already no I W very high
I came to announce super micros products
and so that's the first thing the second
thing is because the Energy Efficiency
and the performance efficiency and the
cost efficiency is so incredibly great
with accelerated Computing a new way of
doing Computing has emerged and it's
called generative AI generative AI is an
incredible thing people say generative
AI inference it's related not the same
inference
recognizing C dog speech inference
generation text Generation image
Generation video generation that's what
we call a generative AI the pressure of
generative AI to not the pressure but
the the transition to generative AI will
affect every single data center in the
world we have a trillion dat a trillion
dollars worth of data centers in the
world that's established $3 trillion
probably by 2030 in another 6 years we
have to to modernize all of them with
these amazing systems yeah that's the
reason why the demand is so great
because all of these data centers has to
be modernized and Charles and the super
micro team is ready to take your
order Json I'm your I'm your best sales
guy thank you I work on commission no
commission we buy more cheaper from you
don't buy more
[Laughter]
chips so
that's Jon sh Michael is now shipping
data center uh liqu cooling DLC R inum
production now to lower the power
consumption so you can manufacture more
AI chip yeah yeah thousand of how here
you see
[Applause]
[Laughter]
I have many American colleagues they
don't understand my Chinese I have many
Chinese colleagues they don't understand
my
Chinese hi
y we are shipping up to 1,000 R per
month now 1,000 R like it is multiply by
ASP yeah you're going to be a gigantic
company yeah thank
you that's why I need a more
CH did you guys all do the
math Millions
times thousands time 52 no no no you
charging me $2 million more than $2
million for d
[Laughter]
are we allowed to do this on TV are we
on
TV I I guess the well is
this so we are shipping about 1,000
that's incredible now this this uh
600,000 Parts this is probably more than
600,000 parts how many pounds oh I don't
know can I move three I think it's 3,000
lb more than 3,000 lb
yeah it's incredible so yeah our goal
this year is to ship more than 10,000
record you know the Charles this is the
thing that's really amazing uh people
think that we're building
gpus you know GP is a
chip there are 72 chips in here and then
there are six
600,000 other
parts
it's 72 chips probably weighs one
pound this is 3, 2,999 other
pounds so the amount of Technology
that's inside one of these RS is really
quite extraordinary this is a technology
Marvel the most most most complex most
advanced computer the world's ever made
yeah exactly the p in the world now yeah
absolutely incredible and the software
that it takes to run this is
unbelievable yeah unbelievable isn't
that right and so I think that that
people now are starting to realize that
when we say GPU server of course the
brain is the GPU yeah but the system is
much much more complex than that and
super micro does amazing engineering
thank
[Laughter]
you huh what I
okay then we there's some Americans this
year we are going to ship hopefully make
when we're together sometimes we speak
Taiwanese sometimes we speak Mandarin
and then when we disagree we speak
[Laughter]
English we try to make a thlc mar share
from 1% to 15%
this year wow Save lot of power for your
TB yeah yeah the Energy Efficiency is so
much better the cost to the data center
is cheaper cheaper that's right people
don't realize this liquid cooled systems
eliminates an enormous amount of cost in
the data center yeah so that you can use
that waste capture that waste and put it
into Computing in the future in the
future Computing throughput is
revenues because it's token generation
and token generation is dollars per
million tokens just like
energy dollars per kilowatt hour we have
now invented a new commodity this is a
very important idea for all of you this
is a new commodity it has value and the
faster you can generate it the higher
throughput the greater utilization the
higher your revenues it is absolutely
true and it's directly measurable that's
why this is a factory not a data center
that's why this is a factory not a file
server it's not a retrieval of files
it's not used for exchanging emails this
is directly generating revenues for
factories that's why we call it AI
factories and so
powerful and only s million
[Laughter]
dollars a
such okay so $3 million and you can
generate who knows how much revenue per
year right uh 3 million 1,000 and every
year have how many
months
12 the the return on the return on large
language model generation token
generation is going to be very very good
yeah be huge and the reason for that is
because the token embeds intelligence
yeah and the int could be used in so
many different Industries and so the
future is very important it's time to
Startup yeah time to
Startup throughput yeah
utilization all matter so
reliability has Revenue implication
throughput has Revenue implication
startup has Revenue implication yeah
that's why it's so important that we
integrate the whole s whole system into
a rack scale get all the software
working connected to all the all the
networking so that and we build all of
our own data centers we build our own
supercomputers so that we know when you
install this when you install super
micro in your factories the startup time
will be extremely fast your utilization
will be extremely high and your
throughput will be extremely high
because your revenues depends on it
Factory output is measured by all of
those factors very complicated yeah and
all of those R are Invidia sofware
license all certified so the sound of
that parking the cable and they can run
and it runs that's right and all of the
Nvidia Nims all of the large language
models it just runs on all these systems
yeah
[Laughter]
[Applause]
we are shipping thousand R
very
yes very
beautiful Charles Charles said that this
is
everything everything in here is NVIDIA
for all the American citizens
there
from to H AI everything all Nvidia sare
all all Green Computing all Green
Computing all green computer all all
support that's
fantastic
good let go through something detail
okay okay
okay H1 H2 B1 for you cooling wow
shipping in B wow and this one your p200
uhhuh fully ready beautiful for your
chip beautiful beautiful this will be
how many time faster than this so we
have we have we have uh uh for Blackwell
Blackwell has air cool
liquor
cooled
x86
Grace MV link 8 MV link 2 MV link 36 MV
link 72 yeah so many different
configurations yeah so that depending on
the type of type of utilization type of
use case you have the type of data
center that you have uh Charles is ready
to serve you immediately right
immediately doesn't need to acheve yeah
one hand we got to acheve second hand we
Shi to C W thank goodness we only need
two hands in two weeks in two
weeks that's incredible and all of it
software compatible this is really this
is really the amazing thing certifi
literally everything here is software
compatible one% yeah and software as we
know is the most complex part of high
performance Computing yeah thank you for
those great offering they are all ready
to service our customer there are three
very important software Stacks that we
have in our company that everything is
built on top of the first of course is
Cuda very famous the second for all of
the networking because networking is
just not networking networking today
networking today is a Computing
fabric networking today is a Computing
fabric not just for sending email to
each
other
4 Mez a gigahertz megahertz this is not
1980s
be
Mez
kilohertz gahz gigahertz yes 400
gigabits per second 800 gigabits per
second and and then of course Next
Generation coming 1600 but the important
thing is all of the software that we
have that runs on the networking for
distributed computing is on top of two
software Stacks one is called DOA for
the nick nickel for the fabric yeah and
it enables us to distribute the workload
across the network very very efficiently
because ethernet was was not designed
for hyperform computing you make our job
easier but still very py because you
have so many
great my job is to help give you
[Laughter]
job we
and because because you do such a good
job it becomes gives me job oh don't
forget that your another
baby yeah yeah yeah
yeah inside
here this this is an incredible
incredible system in fact in
fact in fact um these chips are all
connected together using high-speed
interconnect the world's fastest CIS the
CIS is incredibly fast and very energy
efficient and so we can connect this
great CPU to dual Blackwell gpus and
that's very important because in the
training stage the memory system of
Grace could be used for checkpoint
restart checkpoint and restarting is
very important for high utilization and
high uptime and so checkpoint restart uh
could be stored in the system memory
that system memory is very low energy
very low power and the link between
Blackwell and Grace is very very high
second during inference time as you know
there's a concept called
prompts context in context training
prompting that prompt memory that
context memory is right here this is the
memory memory the thinking memory the
working memory of AI and so this memory
needs to be very high performance very
low energy and so during training we
have good use for gray gray CPU during
inference we have excellent use for gray
CPU and the interconnect is very very
high speed very low power F optimiz and
so the re the benefit is because we
compress so many in one system yeah if
we
save 20 watts 50 Watts on the
interconnect you multiply by the whole
rack then we can take the energy and use
it for computing y so Energy Efficiency
translates to higher performance to
that's right Green
[Applause]
Computing
huh I am a super micro employee
super micro
employee where AI Control
us of course not um we we have to we
have to uh the most important thing of
course at the moment is we have to make
AI work
well right now ai is of course uh
working extremely well and in many
applications AI has become good enough
to good enough to become useful it has
achieved the plateau of good enough very
useful however we want it to be
incredibly good we want it to be very
functional everything from Guard railing
for uh fine-tuning skill learning there
are many different things that we still
have to improve okay so we know that AI
is AI still has long ways to go that's
job number one is Advance the technology
at the same time we have to advanced
Safety technology as you know uh our the
planes that we all flew on to come here
has autopilot and autopilot is automatic
technology in order for planes to be
safe a great deal of Technology had to
be invented to keep the plane safe yeah
also practices to monitor the planes air
traffic control other planes monitor the
planes Pilots monitoring each other many
different ways to keep uh AI uh keep
autopilot safe in the future we'll do
the same thing with AI there will be AIS
that watch AIS there are people that
watch AIS there's gu right guard rails
that keep AI guard rail and so there's
going to be a whole lot of different
Technologies we need to create for
safety technology for safety and then
third of course we need to have good
policies for safety good practices and
good policies for safety talking about
it is very important so that we can all
remind each other that we have to do
good science good engineering good
business practice good policy practice
good industrial practice all of those
things has to advance so perfect
strategy so the conclusion is one the
more you buy the more
safe the more you buy the more you safe
the more you buy the more you safe
yeah thank you Jas thank you so much
good job thank you
everybody thank you okay thank you thank
you thank you thank you all right have a
great
thank you
[Music]
تصفح المزيد من مقاطع الفيديو ذات الصلة
NVIDIA CEO Jensen Huang Reveals AI Future: "NIMS" Digital Humans, World Simulations & AI Factories.
NVIDIA'S HUGE AI Chip Breakthroughs Change Everything (Supercut)
A Conversation with the Jensen Huang of Nvidia: Who Will Shape the Future of AI? (Full Interview)
A Conversation with the Founder of NVIDIA: Who Will Shape the Future of AI?
Nvidia Stock Has Soared 24,000% in 10 Years | NVDA Stock Analysis
Nvidia's meteoric rise to $3 trillion | About That
5.0 / 5 (0 votes)