NVidia is launching a NEW type of Accelerator... and it could end AMD and Intel

Summary

Nvidia's recent developments in accelerator technology, as discussed in a video script, highlight their strategic move towards more efficient AI processing. The script delves into Nvidia's patent dump, revealing an upcoming accelerator designed to enhance inference performance through techniques like vector scale quantization, pruning, and clipping. This could potentially disrupt the market, offering significant speed improvements over current GPUs like Blackwell, and may be integrated into future products in various forms, from discrete cards to integrated systems in laptops and workstations.

Takeaways

• 📅 Nvidia's Computex 2024 keynote provided updates on upcoming products, including the successor to Blackwell and Vera, a new CPU.
• 🛠️ Nvidia discussed an accelerator that was previously mentioned in 2022, which is crucial for their strategic direction and is expected to be disruptive.
• 🔢 The script delves into the technical aspects of number representation and its evolution in Nvidia GPUs, from FP32 to FP16, and the introduction of complex instructions like Tensor Cores and HMMA.
• 🚀 Nvidia's approach to balancing operation cost and data movement has been key to their success, allowing them to maintain a performance lead in AI workloads.
• 💡 The accelerator mentioned is designed to improve inference performance, using techniques like vector scale quantization, pruning, and clipping to achieve high efficiency.
• 📈 Nvidia's CUDA platform plays a significant role in enabling software to take advantage of hardware capabilities, including the new accelerator's features.
• 🔑 The accelerator is expected to be much faster at inference tasks compared to current GPUs, potentially offering up to six times the performance.
• 💻 The script suggests that this new accelerator could be implemented in various ways, including as a discrete PCIe card, an integrated part of an SoC, or part of a larger system like a superchip.
• 🔍 Nvidia has patented an API that allows for seamless integration of the accelerator with existing systems, handling both GPU and accelerator tasks from a single call.
• 🏢 The implications of this technology extend beyond consumer devices to enterprise applications, potentially influencing the future of AI inference in both edge servers and client devices.
• 🔮 The video script hints at a follow-up video that will explore the broader applications and impact of this accelerator on the market and existing players like Intel and AMD.

Q & A

What was the main topic of Nvidia's Computex 2024 keynote?

The main topic of Nvidia's Computex 2024 keynote was the introduction of Ruben, the successor to Blackwell, and Vera, a new CPU to succeed Grace. The keynote also discussed the strategy and future of Nvidia's accelerators.

What is the significance of the number representation changes in Nvidia's GPUs over the years?

The number representation changes in Nvidia's GPUs, such as the shift from 32-bit to 16-bit and the introduction of 8-bit and 4-bit integer data types, have been significant for improving performance in AI workloads. These changes allow for reduced precision, which in turn reduces the amount of data needed, leading to lower bandwidth usage and energy efficiency.

What is the purpose of the 'Tensal' instruction in Nvidia's GPUs?

The 'Tensal' instruction, which stands for Tensor Matrix Multiply and Accumulate, is a complex instruction that performs multiple operations at once. It reduces the need for frequent data fetching from memory, thus improving energy efficiency and performance in AI workloads.

How does the introduction of the 'Imma' instruction benefit Nvidia's GPUs?

The 'Imma' instruction, or Integer Matrix Multiply and Accumulate, allows for the use of 8-bit and 4-bit integer data types in matrix operations. This further reduces the precision and energy cost of operations, making Nvidia's GPUs more efficient for AI inference tasks.

What is the role of the new accelerator discussed in the script?

The new accelerator discussed in the script is designed to perform inference tasks more efficiently than traditional GPUs. It uses techniques like vector scale quantization, pruning, and clipping to achieve high performance with reduced precision, making it suitable for AI services and edge devices.

How does the accelerator improve inference speed and efficiency?

The accelerator improves inference speed and efficiency by performing operations in a single cycle that would take multiple cycles on a traditional GPU. It also optimizes memory usage for specific data structures and operations, leading to high bandwidth and low energy consumption.

What are the potential implementations for the new Nvidia accelerator?

The potential implementations for the new accelerator include a discrete PCIe card for inference acceleration, an integrated accelerator in a system-on-chip (SoC), and as part of a board or platform similar to the Grace Blackwell superchip but potentially scaled down for use in laptops or other devices.

How does the accelerator handle API calls in a heterogeneous system?

The accelerator handles API calls in a heterogeneous system by automatically splitting the call along the pipeline into the GPU and the accelerator. This allows both components to share the same memory pool and ensures that programmers don't have to code specifically for the accelerator.

What is the potential impact of the new accelerator on the client PC market?

The new accelerator could significantly impact the client PC market by enabling more efficient and faster AI inference on edge devices. This could lead to a shift in control of the market, as companies that can effectively implement inference acceleration may dominate both the edge server market and client devices.

What are some of the applications where the new accelerator could be used?

The new accelerator could be used in a wide range of applications, including AI services like chatbots and virtual assistants, gaming for AI-driven features, and in professional fields such as data analysis and scientific research, where fast and efficient AI inference is crucial.

Outlines

00:00

🚀 Nvidia's Upcoming Accelerator and CPU Reveals

The video script discusses Nvidia's recent keynote at Computex 2024, which included details about the successor to Blackwell, named Ruben, and a new CPU called Vera. The script also mentions an accelerator that was not revealed during the keynote but is believed to be a significant part of Nvidia's strategy. The video is sponsored by UCDkeys.com, offering discounts on Windows 11 and Office 2019 keys. The script delves into technical aspects of Nvidia's GPUs, highlighting the evolution of number representation and the introduction of complex instructions like HMMA and IMMA, which have improved energy efficiency and performance in AI workloads. The accelerator discussed is expected to be a disruptive technology, aligning with the themes of Jensen's presentation at Computex.

05:01

🛠️ Deep Dive into Nvidia's AI Performance Enhancements

This paragraph provides a detailed analysis of Nvidia's advancements in AI performance, focusing on the evolution of GPU architecture and the introduction of complex instructions that reduce data movement and increase energy efficiency. The discussion covers the shift from 32-bit to 16-bit precision, the introduction of HMMA in the Pascal microarchitecture, and the further reduction to 8-bit and 4-bit integer data types with the Hopper architecture. The summary explains how these changes have led to significant performance improvements, with number representation being a key factor in achieving greater performance in AI workloads. The paragraph also introduces the concept of dedicated accelerators for inference, which could potentially outperform current GPUs like Blackwell in terms of speed and energy efficiency.

10:04

🔍 Nvidia's New Accelerator: A Game Changer for Inference

The script outlines Nvidia's development of a new accelerator designed to optimize inference operations for AI models, particularly large language models. The accelerator incorporates techniques such as vector scale quantization, pruning, and clipping to achieve high performance with reduced precision, allowing for faster and more efficient inference on resource-constrained devices. The potential impact of this technology is discussed, suggesting that it could redefine the client PC market and pose a significant challenge to competitors like Intel and AMD. The accelerator's performance is compared to existing GPUs, highlighting its potential to be multiple times faster and more energy-efficient for inference tasks.

15:05

🔧 Exploring the Potential Implementations of Nvidia's Accelerator

The script speculates on the possible ways Nvidia could bring the new accelerator to market, including as a discrete PCIe card, an integrated accelerator in a system-on-chip (SoC), or as part of a larger board or platform similar to the Grace Blackwell superchip. It discusses patents that detail how the accelerator could work within a heterogeneous system, sharing memory pools and being automatically integrated with API calls from CUDA or other programming environments. The potential applications of the accelerator in various devices, from laptops to data centers, are also considered, highlighting its versatility and the broad impact it could have on the industry.

20:06

🎮 The Broad Impact of Nvidia's Accelerator on Future Applications

The final paragraph of the script hints at the wide-ranging applications of Nvidia's new accelerator, suggesting that it could be used in various fields, including gaming. The script invites viewers to subscribe for the continuation of the discussion and mentions upcoming coverage of Computex. It also encourages support through Patreon for the in-depth analysis provided, including the examination of patents and technical presentations related to Nvidia's technology.

Keywords

💡Nvidia

Nvidia is a multinational technology company known for its graphics processing units (GPUs). In the video, Nvidia's developments and innovations in AI and GPU technology are a central theme, showcasing their advancements and the impact on computing performance.

💡Accelerator

An accelerator is a specialized hardware designed to enhance the performance of specific tasks, particularly in AI and machine learning. The video discusses Nvidia's new accelerator, which aims to improve efficiency and speed in AI workloads by reducing data movement and increasing operational complexity.

💡Blackwell

Blackwell is a code name for Nvidia's upcoming GPU architecture. The video mentions Blackwell as the successor to previous architectures, highlighting its significance in Nvidia's ongoing advancements in AI and GPU technology.

💡Precision

Precision refers to the accuracy of numerical representations in computing. The video explains how Nvidia has improved AI performance by reducing precision, from 32-bit to 16-bit, and even lower, which helps in optimizing data processing and energy efficiency in AI models.

💡Tensor

Tensor is a term used by Nvidia to describe complex instructions that perform multiple operations simultaneously. The video highlights the importance of Tensor operations in AI workloads, which reduce the need for frequent data fetching from memory, thus improving energy efficiency.

💡Inference

Inference in AI refers to the process of making predictions or decisions based on a trained model. The video discusses Nvidia's focus on developing accelerators that enhance inference performance, which is crucial for applications like AI assistants and real-time data processing.

💡Quantization

Quantization is a technique used to reduce the precision of data in neural networks, improving computational efficiency without significantly affecting accuracy. The video explains how Nvidia's accelerators utilize quantization to enhance performance in AI inference tasks.

💡CUDA

CUDA is Nvidia's parallel computing platform and programming model. It allows developers to leverage the power of Nvidia GPUs for general-purpose computing. The video mentions CUDA in the context of how Nvidia's software ecosystem supports its hardware innovations.

💡AI Workloads

AI workloads refer to the computational tasks involved in training and running AI models. The video explores how Nvidia's hardware advancements, particularly in GPUs and accelerators, are designed to handle these workloads more efficiently, driving significant performance improvements.

💡Energy Efficiency

Energy efficiency in computing refers to the effective use of power to perform computational tasks. The video discusses how Nvidia's innovations, such as reducing data movement and increasing instruction complexity, have led to significant improvements in the energy efficiency of their AI accelerators and GPUs.

Highlights

Nvidia's 2024 Computex keynote revealed information on Ruben, the successor to Blackwell, and Vera, a new CPU to succeed Grace.

The video is sponsored by UCDKeys.com, offering OEM keys for Windows 11 and Office 2019 at discounted prices.

Nvidia's patent dump includes an accelerator discussed in 2022, which is key to their strategy going forward.

The accelerator is designed to be one of the most disruptive technologies ever launched by Nvidia.

Nvidia's GPUs have seen a gradual change in number representation, starting from native support for FP32 to FP16, and introducing complex instructions like Tensor Cores.

The introduction of complex instructions like HMMA has significantly improved energy efficiency in Nvidia's GPUs.

Nvidia's strategy involves reducing the precision of operations in AI workloads to increase performance without sacrificing accuracy.

The new accelerator relies on techniques like vector scale quantization and clipping to achieve greater performance in inference.

The accelerator can perform operations in one cycle that would take tens or hundreds of cycles on Nvidia's current GPUs.

Nvidia's accelerator is designed to be a drop-in solution for various applications, potentially dominating the edge server and client device markets.

The accelerator could be implemented as a discrete PCIe card, an integrated accelerator in an SoC, or part of a board or platform.

Nvidia patented an API that automatically handles calls between the GPU and the new inference accelerator, sharing the same memory pool.

The implementation of the accelerator is kept open to interpretation, covering various bases without revealing too much detail.

The accelerator could redefine control of the client PC market in the next decade, posing a significant challenge to Intel and AMD.

Nvidia's approach to inference acceleration could make it easier to implement and potentially dominate both the edge server and client device markets.

The upcoming video will cover the broad scope of applications for the accelerator, including its potential impact on gaming.

The video also discusses the rest of Computex and Nvidia's position in the training hardware market.

Transcripts

Nvidia had the coputex 2024 keynote a

few hours ago so I delayed this video

that I had planned to release earlier to

add any information Nvidia revealed that

was relevant while Nvidia did not reveal

the accelerator that I'm going to

discuss today they did reveal some

information on Ruben the successor to

Blackwell and Vera a new CPU to succeed

Grace and that is where this accelerator

will fit in there's a lot to discuss so

I'll have to split this video into two

parts so let's Dive In this video is

sponsored by Ur cdkeys.com if you pay

full price for a Windows 11 key you are

wasting your money instead you can get a

Windows 11 OEM key from Ur cdkeys.com

for just over $29 or even lower at

$21.86 if you use my coupon code c25 at

checkout all Ur CD Keys product keys are

on sale right now so don't miss the

super spring sale follow the link below

to the windows 11 page that urc.com

click purchase enter your c25 discount

code you can pay with a credit card or

PayPal and then just add the Windows key

to your Windows activation settings and

you're done the code is sent to you

within minutes by the way you can also

use my c25 code on other products so if

you need office 2019 make sure to get it

from U cdkeys.com for a much lower price

than retail check the exclusive links in

the video description to get your cheap

OEM windows or office keys today from U

cdkeys.com

Nvidia had a big patent dump this week

and as I looked through the drawings and

descriptions I remembered an accelerator

that Nvidia talked about back in 2022 at

the vsi circuits conference before I

explain what this accelerator does we

have to get a little bit technical for a

couple of minutes just so you understand

the context for such a processor to make

sense and why I believe Nvidia is

launching it next year this accelerator

is key to strategy going forward in fact

underlying Jensen's presentation at

comput X were all the elements that will

make this accelerator one of the most

disruptive technologies that Nvidia has

ever launched if we look back at the

gpus Nvidia has launched in the last 12

years or so from kler to Blackwell which

will launch later this year one of the

Paradigm changes that were gradual but

constant was the change in number

representation native support So in

kapler operations were Scala at fp32 2

so that's 32-bit and this was around

2012 which was when Nvidia started

porting models to their gpus to see if

the parallelism would benefit Ai

workloads and obviously not only it did

but it has made advv one of the biggest

companies in the world in just over a

decade now kler was already out and the

next gen was already taped out so it

took a while until we started to see

number representation changes and fp16

was only natively supported in 2016 when

the Pascal microarchitecture launched in

the form of the p100 GPU so Nvidia

reduced Precision from 32bit to 16 bit

why does that matter well in AI

workloads one of the lwh hanging fruits

for achieving greater performance was

reducing Precision I won't get into how

that works but on an algorithm and API

level you can reduce the Precision of

the input and achieve the same results

with less data which means you don't get

choked on bandwidth but more importantly

you don't waste as much energy moving

data around reducing Precision works

wonderfully in AI workloads where you

can prune the network of unnecessary

data and focus only on the things that

are likely to have the results you need

so after Pascal Nvidia launched in 2017

the v00 and introduced its now famous

tensal a tensal is just a fancy

marketing name for an instruction a

complex instruction that is called half

Matrix multiply and accumulate or hmma

before you had adds and multiplies and

now you have a complex instruction that

does a bunch of stuff all at once what

that means is that you don't have to

keep fetching stuff from memory in other

words if the instruction is simple but

you have to keep fetching data from

memory you end up using a ton of energy

in data movement whereas the operation

itself uses very little energy I've

explained this in past videos so it

makes sense From nvidia's perspective to

have the instruction do more operations

in one go and fetch the data less times

and that's what a Tens SC does it's an

instruction that does a lot of things at

once instead of just an ad or a multiply

to give you an idea back in the days of

fused multiply ad operations in the

generations prior to Pascal to do a

thatch and decode so that state of

movement that costs 30 pjws it was 40

times more costly to do that data

movement than to actually do a multiply

ad operation once you had a data which

cost a mere 1.5 PS so that's a 2,000%

overhead because of moving that data

around it's only 1.5 PJs to do the

operation in logic but it costs 30 PJs

to get the data from memory to do that

operation but with the introduction of

this complex instruction hmma or what

nvidia's marketing callus the tensal the

operation cost went up significantly

because the logic is doing a lot more

operations in one go but the cost of

moving the data was reduced massively so

now the same 30 PJ F and decode cost 110

p jeel but since the data didn't have to

go back and forth for the rest of the

operations to conclude the overhead was

only 22% instead of 2,000% it's a

massive Improvement in Energy Efficiency

so you have this balance of how complex

you make the instruction which will

result in the logic being active longer

and using more energy per operation and

how much data do you move around Nvidia

has so far figured that the cost of

moving data around in a 2.5d package at

least is much higher than the actual

operation ations and has therefore added

more and more complex instructions to

their gpus over the years always giving

it a fancy marketing name from VTA to

Turing to anir and then to Ada nothing

much changed on the instruction level

what changed were the matrices sizes the

next real big step was when another

complex operation was introduced in the

form of imma which stands for integer

Matrix multiply and accumulate when

Harper launched so with harer Nvidia

introduced 8bit and 4bit integer data

types as inputs for those matrices

nvidia's marketing calls this the

Transformer engine but like I said that

name is just marketing for another

complex instruction now notice what

happens to the energy cost of this new

complex instruction in Hopper the same

30 P je Fetch and decode now only

represents a 16% overhead even though

the operation itself is more costly to

execute at 160 PJs and this is the very

reason why dedicated acceler Racers from

other companies haven't been able to

catch up with Nvidia because of this

Balancing Act between the operation cost

and data movement Nvidia has gotten this

right this approach also means that

Nvidia can circumvent bandwidth

limitations to some extent like I said

earlier because you are doing less calls

to memory so this reduction in Precision

is achieved by natively supporting these

different types of number representation

now down to int 8 and int four if we

take a bir eye view of where per

performance jumps have come from since

kapler up until hoer in workloads where

Precision can be reduced number

representation accounted for the

greatest leap at

32x when you take into account

parasitics then the introduction of

complex instructions resulted in a

performance jump of around 12.5 x the

process node is hard to say but at least

3x and sparity another 2 a so that's why

you hear Jensen on stage saying they

achieve the 1,000x performance uplift in

AI in 8 years the cumulative effect of

all these changes are what led to that

and with Blackwell well it's two Hoppers

fused together so you'll get another 2x

so you see that number representation

has been the loow hanging fruit of

achieving greater performance in this

last decade of course for all of this to

work you need the software to take

advantage of it and that's where

nvidia's Cuda mode plays its part but

now that we're down to four Bits And if

Blackwell is just two Hoppers fused

together pretty much where is NVIDIA

headed next at cutx a few hours before

this video goes live Jensen hammered on

the fact that acceleration will be key

both in future BCS but also data centers

but key to what key to

inference well back to that accelerator

from 2022 as far as number

representation The Next Step will be

support for log 8 which is log 4.3 along

with a plus or minus symbol I won't go

into that here but if if you are on the

Discord server I'll be happy to explain

there how this format works and why it's

more performant than in4 in Blackwell

where it comes out later this year

anyway more importantly in addition to

the work being done for training Nvidia

came up with this dedicated accelerator

back in 2022 that relies on two

techniques to achieve greater

performance in inference these

techniques are scaling and optimal

clipping to do large language models for

instance without any loss of accuracy as

you can see in this graph from Nvidia

with quantization you can map High

Precision formats like flow 32 to lower

Precision formats like int 8 the reason

you would do this is to achieve greater

inference speed on resource constrained

devices you know devices like laptops

the particular technique that Nvidia

shown in recent papers is vsq or vector

scale quantization so this is the first

technique the second one is clipping

like I said which is another form of

reducing Precision without losing

accuracy so anyway what does this

technical jargon mean in Practical terms

the accelerator that Nvidia created does

precisely these optimizations on the

hardware level an accelerator takes

special data types and th Special

Operations that are not natively

supported on nvidia's gpus which means

you can do in one cycle what would take

tens or hundreds of Cycles to do in

Blackwell and accelerator can also do

massive parallelism but with locality so

it's massively performant compared to to

a GPU with a complex memory system you

also get optimized memory with high

bandwidth and low energy for those

specific data structures and operations

and the key ingredient is that you can

do software and Hardware code design so

the algorithm can be specifically

tailored for the hardware as you can

imagine this is Jensen's wet dream yet

another mode to force people into using

nothing but Nvidia this time for

inference nvidia's gpus are great for AI

currently because they are programed

able they are flexible they can adapt to

new models and algorithms but for micro

operations that are to some degree

agnostic to the developments in models

Nvidia can build accelerators to improve

performance particularly in inference

which is where the majority of

computation will move to in the next 5

years while training will decelerate so

this is the layout of the test

accelerator that Nvidia taped out H us

per area of one of their Network

switches on 5 NM and served as a

prototype for what Nvidia will be

bringing to Market and which I believe

could truly put Intel and AMD in a tough

spot so this accelerator does Vector

scaled quantization pruning and clipping

in order to support in4 operations so it

can do large language models without

losing any accuracy at five times the

speed of Blackwell so almost 100 ter Ops

per watt versus 20 ter Ops per watt on

the b00 GPU that's coming out this year

now note that this was on five n so when

Reuben comes out so that will be the

r100 either late next year or early 2026

presumably this accelerator will be on 3

nmet so possibly six times as fast as

Blackwell in inference workloads it's

also possible that Nvidia will include

it in the b200 next year instead so when

you use an AI service like a tax prompt

to do search you enter your prompt that

text is then tokenized so the whole text

or some of the words or part of the word

is split into smaller units called

tokens which are a unit of data that the

AI model can understand these tokens are

converted into a numerical format so

fp32 int 8 in 16 int 4 Etc so these are

the number representations that gpus

work on to perform computation in the

context of inference so when you are

doing queries locally on your computer

or phone you don't have the compute

resources to do complex operations so

they have to be simplified and acceler

ated by fixed function units like this

accelerator that NV has prototyped now

you might be saying but don't queries

just get sent to the cloud and process

there why do I need them done locally

for one there's sensitive security

issues with personal data or company

data that you don't want to be sent off

premises secondly you can think of

service providers as AI clients

themselves and these accelerators will

also be deployed there sort of an

intermediary level between yourself and

the data center in order to speed things

things up massively and thirdly it's

going to be much faster to do most of

the computation locally and get an

almost immediate response than sending

all the tokens to a data center having

the processing done there and then

fetching the results you want AI agents

you know chat GPT and virtual assistance

and whatnot to respond immediately to

your requests so you need the operations

to be accelerated locally without

resources to do full Precision you need

this accelerator to use the techniques I

mentioned to reduce the number of

representation down to something like

int 4 so that even a phone or a laptop

can perform operations that would

normally require a bunch of powerful

gpus like I said one of these

accelerators locally on your PC is

around six times faster at inference

than a Blackwell GPU would be at doing

the same operation it's funny that in a

way we're coming full circle if you were

around in a days of CPUs doing the 3D

rendering in games and then saw the

massive jump in performance that GPU

accelerators brought to that render you

can think of gpus now having the role of

CPUs back then and this new accelerator

being as disruptive as gpus were back

then except instead of accelerating 3D

rendering in games the workload is no

inference and you can bet that whoever

gets inference right and makes it easy

to implement will dominate both the edge

server Market but also client devices

remember how over the past year I've

been saying that Nvidia will come to the

PC market as a disruptor and leave the

incumbent in a tough spot well that's

how important this accelerator is it

could Define who will control the client

PC market in the next 10 years Intel and

AMD are not prepared for this even if

they have the hardware to compete Nvidia

has the verticals so that's the software

that will make this accelerator a dropin

solution for a bunch of applications to

wrap part one up how exactly is NVIDIA

bringing this accelerated Market there

are three options as a discrete pcie

card similar to a discrete GPU that you

slot into your system to accelerate

Graphics but in this case it would

accelerate inference secondly as an

integrated accelerator in an S so so

this would be a hetrogeneous chips

similar to an APU or similar to the

Apple amp chips which feature a bunch of

accelerators also and finally as part of

a board or platform similar to The Grace

Blackwell super chip but scaled down

possibly down to a laptop platform

looking at the recently published

patterns on this accelerator the first

one titled application programming

interface to transfer information

between accelerator memory we see the

first embodiment of this accelerator

which is in a hetrogeneous processor in

this 110 Page Long patent basically

Nvidia is describing how different

accelerators can access the same memory

pool in a hetrogeneous system in essence

an API call from either Cuda or ROM or

one API is automatically split along the

pipeline into the GPU and the

accelerator so that programmers don't

have to code specifically for this

accelerator so in other words Nvidia

patented a way for an API call to be

automatically handled by the GPU with

the accelerator getting specific code

generated for it with both sharing the

same memory Nvidia goes on to detail

several possible variations of this and

states that this can be both implemented

in a single device so that could be for

example a laptop or in a distributed

computer system the second relevant

patent published on the same date is

complimentary and handles any API call

errors so it essentially handles where

in memory any errors will be stored in

such a system the third and final

Associated patent describes how the

processing occurs in such a system like

I was saying the API call is put into a

stream that has all the instructions and

then some parts of the stream are

allocated to the GPU while other parts

of the stream to the accelerator note

that the pattern states that the

instructions in this stream that are

meant to go into the accelerator can be

distributed to multiple accelerators so

a possible implementation is having more

than one of these accelerators to speed

inference even more we could see this

being useful in asch servers for

instance but possibly also in client

devices now obviously I'm simplifying

all of the possible Hardware

implementations here out of the almost

400 pages of patents that I went through

but you can see that Nvidia has kept the

accelerator implementation wide enough

or rather open to interpretation so as

not to reveal too much but still ensure

they have all bases covered personally I

think it's unlikely that Nvidia would

release a pcie type card for inference

even though that would be awesome while

not outside the realm of possibility I

think it's more likely that future

generation of gaming gpus will include

this accelerator in some form at least a

highend gpus so just like today you can

use Cuda to accelerate say video

rendering in Adobe Premiere with your

RTX 4070 or 4090 you could be able to

use this inference accelerator in the

future to speed up some of the verticals

that we will look at in the second part

of this video I do hope Nvidia releases

a PCI type card for the consumer Market

how awesome would that be another piece

of Hardware we can lust over then spend

an insulting amount of money on and then

never use it PC Master race style so the

other two implementations I think are

the most likely approaches so that's

having a board wide system similar to

the Blackwell Superchip where Nvidia

could have an arm CPU either developed

intern or licensed a GPU chip and then

the inference accelerator all sharing

memory and this could be used for

laptops or workstations for instance and

finally the most traditional and I guess

the most likely application would be an

S so in the vein of a large Apu with arm

cores Nvidia GPU Cordes and this

inference accelerator to do the heavy

lifting of AI inference locally

depending on how large and power hungry

this is there could be a variant that

would go into smaller devices and not

just laptops perhaps even even a phone

or a new Shield tablet or a virtual

reality headset similar to the one apple

made and that everyone has already

forgotten about or a Mini PC especially

with Dell hinting this week they will be

selling AI workstations and AI thin

clients or BCS next year with Nvidia

Hardware included so in part two we will

look at why Nvidia is wasting time with

this when they are already banking

massively on training Hardware selling

millions of gpus at massive margins and

also what applic we will see this used

in I don't think people realize how

broad a scope such an accelerator will

have and the sorts of applications it

can make viable and that includes games

so subscribe to the channel right now so

you don't miss part two and this coming

week I'll also be covering the rest of

computex so stay tuned for that consider

joining my patreon to support all this

work of reading 400 Pages worth of

patents and going through Nvidia obscure

presentations in circuits conferences

and for just $2 a month you will also

get access to the cortex Discord server

thanks for watching and until the next

one