Really Low Power AI-enabled Microcontroller is SPOT On
Summary
TLDRThe video discusses the importance of edge AI and power efficiency, introducing Ambiq's SPOT technology that optimizes low-voltage operation for significant energy savings. It highlights the Apollo5 chip, based on the Arm Cortex M55, which supports AI with enhanced instruction sets and efficient neural network processing. The chip's features include a 2.5D GPU for rich displays, nonvolatile MRAM, and robust security measures, emphasizing a balance between performance and power consumption.
Takeaways
- 🔋 SPOT technology by Ambiq is designed to maximize intelligence within a minimal power budget.
- 🔌 Subthreshold Power Optimized Technology (SPOT) operates at low voltages, achieving quadratic energy reductions.
- 🛠️ SPOT technology requires advanced techniques to ensure circuit robustness under low-voltage conditions.
- 🤖 Ambiq applies SPOT to MCU cores like Arm’s Cortex M55 and M4, offering familiar development environments with reduced power consumption.
- 🔄 A mix of subthreshold, near threshold, and standard operating voltages is used within Ambiq's chip design to balance power and performance.
- 💡 The Apollo5 chip incorporates the Arm Cortex M55 with enhanced instruction sets for efficient AI and signal processing tasks.
- 🛠️ neuralSpot software suite bridges the gap between TensorFlow and the chip, simplifying neural network development.
- 📚 AI Development Kits (ADKs) provide customers with in-house developed and trained neural networks as a starting point for their projects.
- 🔍 Profiling capabilities of the M55 allow developers to benchmark and understand the internal workings of neural networks.
- 🖼️ Apollo5 includes a 2.5D GPU for high-performance graphics processing, suitable for rich displays.
- 🔒 The chip is designed with security in mind, featuring cryptographic acceleration, secure lifecycle management, and TrustZone support.
Q & A
What is the main challenge addressed by Ambiq's SPOT technology?
-The main challenge addressed by SPOT (Subthreshold Power Optimized Technology) is to put maximum intelligence into a tiny power budget by running at extremely low voltages, which results in quadratic energy reductions as voltage drops.
How does SPOT technology achieve energy efficiency?
-SPOT technology achieves energy efficiency by running at low voltages, where the energy is proportional to the square of the voltage, allowing for significant energy savings as the voltage decreases.
What is the significance of the M55 core in Ambiq's Apollo5 chip?
-The M55 core is significant because it is an efficient compute engine capable of performing signal processing tasks with great efficiency, which is beneficial for neural network development.
How does neuralSpot software complement the Apollo5 chip's AI capabilities?
-NeuralSpot is a software suite that sits between TensorFlow and the bare silicon, providing utilities that make neural network development easy and complements the AI capabilities of the Apollo5 chip.
What is the purpose of the AI development kits (ADKs) provided by Ambiq?
-The AI development kits (ADKs) are pre-developed and trained neural networks provided to customers as a starting point for their own neural network development efforts.
Why is the profiling capability of the M55 important for developers?
-The profiling capability of the M55 is important because it allows developers to benchmark and understand the internal operations of the chip, helping them identify inefficiencies and optimize their neural network development.
What role does the GPU play in the Apollo5 chip?
-The GPU in the Apollo5 chip, a 2.5D GPU running up to 250MHz, is designed to handle rich displays and graphics development, offering nearly four times better performance than the previous Apollo generation.
How does Apollo5 ensure data security for neural network applications?
-Apollo5 ensures data security by incorporating cryptographic acceleration, a complete secure lifecycle management protocol, and support for TrustZone, making it a secure chip for neural network applications.
What is the significance of having on-chip memory in the Apollo5 chip?
-On-chip memory in Apollo5 is significant as it allows for the storage of large frame buffers, graphics assets, or large neural networks locally, providing hyper-efficient memory access compared to package memory.
What are some of the key features of the Apollo5 chip aside from its AI capabilities?
-Aside from its AI capabilities, the Apollo5 chip features a 2.5D GPU for graphics, nonvolatile MRAM for data storage, and a focus on security with cryptographic acceleration and TrustZone support.
How does the use of near-threshold operation in digital circuits benefit power efficiency?
-Near-threshold operation, where circuits run just above the turn-on voltage of the transistor, offers a good balance between power and performance, contributing to overall power efficiency.
Outlines
🔋 SPOT Technology and AI on the Edge
The first paragraph discusses the importance of AI on the edge and the power requirements associated with it. Scott introduces Ambiq's SPOT (Subthreshold Power Optimized Technology), which is designed to maximize intelligence within a minimal power budget by operating at extremely low voltages. This approach results in significant energy savings due to the quadratic relationship between energy and voltage. The technology is applied to MCU cores like Arm’s Cortex M55 and M4, offering familiar software development environments with reduced power consumption. The design of the chip incorporates a mix of subthreshold, near-threshold, and standard operating voltages to balance power efficiency with performance and user experience. The paragraph also touches on the support for AI development with the M55's enhanced instruction set features and the neuralSpot software suite, which facilitates neural network development by providing utilities and AI development kits (ADKs) as a starting point for customers.
🚀 Advanced Features of Apollo5 Chip
The second paragraph highlights additional features of the Apollo5 chip, including its nonvolatile MRAM, GPU, and security aspects. The chip includes a 2.5D GPU capable of running at 250MHz, providing nearly four times the performance of the previous Apollo generation, which is beneficial for customers with rich display requirements. The GPU is supported by large on-chip memory for efficient storage of graphics assets and neural networks. On the security front, Apollo5 is designed with a focus on data protection, featuring cryptographic acceleration, a secure lifecycle management protocol, and TrustZone support, making it a secure, low-power, and high-performance chip. The paragraph concludes by emphasizing the chip's capabilities and encouraging developers to explore neural network development with the provided tools and profiling capabilities.
Mindmap
Keywords
💡AI on the edge
💡Power Budget
💡SPOT Technology
💡Subthreshold Voltage
💡MCU Cores
💡Threshold Voltage
💡Deep Subthreshold Mode
💡Apoll5
💡NeuralSpot
💡AI Development Kits (ADKs)
💡Nonvolatile MRAM
💡GPU
💡Cryptographic Acceleration
💡TrustZone
Highlights
AI on the edge requires a balance between intelligence and power consumption.
Ambiq's SPOT technology aims to maximize intelligence within a minimal power budget.
SPOT stands for Subthreshold Power Optimized Technology, leveraging very low voltage operation for significant energy savings.
Running circuits at low voltages requires robust design techniques to ensure functionality.
SPOT can be applied to MCU cores like Arm's Cortex M55 or M4, offering familiar development environments with reduced power.
The chip design utilizes a mix of subthreshold, near threshold, and standard operating voltages for optimal balance.
Apoll5 chip integrates the Arm Cortex M55 with enhanced instruction sets for AI.
The M55 is efficient for signal processing tasks like FFTs and spectrogram building in neural networks.
Ambiq provides neuralSpot software suite for easy neural network development, compatible with TensorFlow.
AI development kits (ADKs) offer pre-trained neural networks as a starting point for customers.
Developers are encouraged to use profiling tools to identify inefficiencies in neural network development.
Apoll5 features nonvolatile MRAM, a 2.5D GPU, and large on-chip memory for graphics and neural networks.
The GPU in Apoll5 operates at up to 250MHz, providing significant performance improvements.
Security is integral to Apoll5, with cryptographic acceleration, secure lifecycle management, and TrustZone support.
Apoll5 is designed to be extremely low power, secure, and high performing for AI applications.
Transcripts
AI on the edge is very important these days.
But one of the key factors is the amount of power that's required.
We'd like to have a lot of intelligence,
but it doesn't necessarily come cheap.
So, Scott,
could you tell us a little bit about the SPOT technology
that Ambiq has developed?
Yeah.
So we developed our core SPOT
technology to address exactly this problem, right?
We want to put maximum intelligence
into a tiny little power budget
and the way we do that is something we call
Subthreshold Power Optimized Technology
and it's based on an old old approach to low power,
which involves running an extremely low voltages.
In fact, those voltages are so low that
that you may not turn a transistor fully on.
Now, the benefit of running at such low voltages is
energy is proportional to the square of the voltage.
So we get quadratic energy reductions as voltage drops.
That requires
a whole bunch of, techniques to be deployed
to make circuits robust under such conditions.
but we've been able to do that.
And in fact, by applying SPOT to conventional MCU cores like Arm’s
Cortex M55 or Arm's Cortex M4, we can offer a very familiar
software development environment at a fraction of the power budget
that our customers are used to.
Now, in terms of the design of the chip itself,
how do you utilize SPOT within the various peripherals, processors?
Is it the same architecture all the way through?
Yeah. So we we use a mixture of approaches.
So, for most of our digital circuits we’ll apply what is called near
threshold operation.
And that is where we're, we're running at just above the,
the turn-on voltage or the threshold voltage of the transistor
and that offers a really good
balance between power and performance
In the analog domain, we'll run often at the deep sub threshold mode.
That offers extreme low power,
especially when you're talking about sleep modes,
and then for certain circuits,
we'll even run at standard operating voltages.
GPIOs, for example,
I want my customers to have an experience
when they use our chip that matches what they get from other chips,
which means that the
the pad ring surrounding the chip needs to run at standard voltages.
Certain memories need to run at standard voltages.
So we have a mix of subthreshold,
near threshold, and standard operating voltages inside our chip.
And it takes a lot of technology to make all of that possible.
Now, your latest chip,
the Apoll5, is based on the Arm Cortex M55 that you mentioned.
One of the features of that are some enhanced instruction set
features for artificial intelligence.
Could you tell us a little bit about the support that you have
and some of the development tools that you have to complement this?
Well, so, first of all, the M55 is an amazing compute engine.
It's, it is capable of doing signal processing with great efficiency.
So if you think about building a typical
neural network, you've got to do pre-processing
before you bring the data in.
So you might be running FFts and building a spectrogram.
It's really good at the at those.
It's got a big wide vector unit
and and is well tuned for that, for the neural network itself.
Similarly, it has a big vector unit
and can run those efficiently,
but none of that is any good if you don't have a whole software,
suite paired with it.
And so what we offer to our customers
is, a suite of software that we call neuralSpot,
and the way to think about neuralSPOT
is it's that layer of software that sits between TensorFlow,
which is the Google's training framework, and the bare silicon itself.
It provides a bunch of utilities
that makes neural network development easy. And we couple that with
software that we call our AI development kits,
Our ADKs
And these are, neural networks
that we developed and trained in-house.
and then we provide them to our customers in reference form.
So they include,
the network design itself,
everything you need to train that, including the training data,
including weights.
And then we hand that off to the customer.
And the goal is not to give them a perfectly sealed
and ready to go production worthy neural network.
it's to give them a seed to start their own effort.
So if there's if there's one message I have
for for developers out there, it's that
AI development is is very, attainable today, right?
We we're providing the tools that make it relatively straightforward.
What I'll also say is you have to be careful with it.
There's a lot of talk about
wonderful NPUs and great hardware out there
to accelerate neural network development,
But in reality, a lot of the inefficiencies
lie to software level today, so so we provide tools
allow you to benchmark where you're at.
The M55 has some amazing
profiling capabilities
that allow you to get a sense of what's really going on inside there.
So, don't be afraid of neural network development on Apollo5.
And if you
if you do it, make sure make sure that you're
paying close attention to what that profiling data looks like.
Well, briefly,
there are a lot of other features that the Apollo 510 has.
for example, it has nonvolatile MRAM.
Can you tell us a couple of the other features,
maybe the GPU and the security aspect?
Yeah,
so a lot of our customers have big, rich displays,
and so we have included a GPU, a 2.5D GPU
on board, that GPU runs up to 250MHz and is, achieves performance
that's nearly four times
better than the previous generation of Apollo,
which was already, really nice chip for graphics development.
And that's coupled with some really big memories,
that, that allow you to, store
big frame
buffers, big graphics assets, or back to a previous
discussion, have big neural networks in locally.
And that's all on-chip memory.
It's not in package memory. It's all on-chip memory.
That's that's, hyper efficient.
And then on the security front,
look, security is a really important
part of the neural network story.
We're,
if we're going to entrust our data to these devices
and let them learn from us,
we have to make sure that all of our data is secure.
And so,
Apollo5 was designed from the ground up with security in mind.
So we have,
we have, of course, cryptographic acceleration,
but we have a complete secure lifecycle management
protocol built into the chip,
and we also have support for TrustZone built into the chip.
So, really secure chip, extremely low power. And, very performant.
Great. Well, thanks for filling us in on SPOT and the Apollo5 family.
Browse More Related Video
Snapdragon just obsoleted Intel and AMD
Nvidia's Breakthrough AI Chip Defies Physics
Energy Efficient AI Hardware: neuromorphic circuits and tools
Sakya Dasgupta: A Journey from Gaming To Enabling Embodied Intelligence
[자막뉴스] 엔비디아 능가한 '초저전력 AI 반도체'...한국 세계 최초로 개발 / YTN
Microsoft vs. Apple: Satya Nadella Says AI-Focused Copilot+ PCs Beat Macs | WSJ
5.0 / 5 (0 votes)