Really Low Power AI-enabled Microcontroller is SPOT On

Summary

The video discusses the importance of edge AI and power efficiency, introducing Ambiq's SPOT technology that optimizes low-voltage operation for significant energy savings. It highlights the Apollo5 chip, based on the Arm Cortex M55, which supports AI with enhanced instruction sets and efficient neural network processing. The chip's features include a 2.5D GPU for rich displays, nonvolatile MRAM, and robust security measures, emphasizing a balance between performance and power consumption.

Takeaways

• 🔋 SPOT technology by Ambiq is designed to maximize intelligence within a minimal power budget.
• 🔌 Subthreshold Power Optimized Technology (SPOT) operates at low voltages, achieving quadratic energy reductions.
• 🛠️ SPOT technology requires advanced techniques to ensure circuit robustness under low-voltage conditions.
• 🤖 Ambiq applies SPOT to MCU cores like Arm’s Cortex M55 and M4, offering familiar development environments with reduced power consumption.
• 🔄 A mix of subthreshold, near threshold, and standard operating voltages is used within Ambiq's chip design to balance power and performance.
• 💡 The Apollo5 chip incorporates the Arm Cortex M55 with enhanced instruction sets for efficient AI and signal processing tasks.
• 🛠️ neuralSpot software suite bridges the gap between TensorFlow and the chip, simplifying neural network development.
• 📚 AI Development Kits (ADKs) provide customers with in-house developed and trained neural networks as a starting point for their projects.
• 🔍 Profiling capabilities of the M55 allow developers to benchmark and understand the internal workings of neural networks.
• 🖼️ Apollo5 includes a 2.5D GPU for high-performance graphics processing, suitable for rich displays.
• 🔒 The chip is designed with security in mind, featuring cryptographic acceleration, secure lifecycle management, and TrustZone support.

Q & A

What is the main challenge addressed by Ambiq's SPOT technology?

The main challenge addressed by SPOT (Subthreshold Power Optimized Technology) is to put maximum intelligence into a tiny power budget by running at extremely low voltages, which results in quadratic energy reductions as voltage drops.

How does SPOT technology achieve energy efficiency?

SPOT technology achieves energy efficiency by running at low voltages, where the energy is proportional to the square of the voltage, allowing for significant energy savings as the voltage decreases.

What is the significance of the M55 core in Ambiq's Apollo5 chip?

The M55 core is significant because it is an efficient compute engine capable of performing signal processing tasks with great efficiency, which is beneficial for neural network development.

How does neuralSpot software complement the Apollo5 chip's AI capabilities?

NeuralSpot is a software suite that sits between TensorFlow and the bare silicon, providing utilities that make neural network development easy and complements the AI capabilities of the Apollo5 chip.

What is the purpose of the AI development kits (ADKs) provided by Ambiq?

The AI development kits (ADKs) are pre-developed and trained neural networks provided to customers as a starting point for their own neural network development efforts.

Why is the profiling capability of the M55 important for developers?

The profiling capability of the M55 is important because it allows developers to benchmark and understand the internal operations of the chip, helping them identify inefficiencies and optimize their neural network development.

What role does the GPU play in the Apollo5 chip?

The GPU in the Apollo5 chip, a 2.5D GPU running up to 250MHz, is designed to handle rich displays and graphics development, offering nearly four times better performance than the previous Apollo generation.

How does Apollo5 ensure data security for neural network applications?

Apollo5 ensures data security by incorporating cryptographic acceleration, a complete secure lifecycle management protocol, and support for TrustZone, making it a secure chip for neural network applications.

What is the significance of having on-chip memory in the Apollo5 chip?

On-chip memory in Apollo5 is significant as it allows for the storage of large frame buffers, graphics assets, or large neural networks locally, providing hyper-efficient memory access compared to package memory.

What are some of the key features of the Apollo5 chip aside from its AI capabilities?

Aside from its AI capabilities, the Apollo5 chip features a 2.5D GPU for graphics, nonvolatile MRAM for data storage, and a focus on security with cryptographic acceleration and TrustZone support.

How does the use of near-threshold operation in digital circuits benefit power efficiency?

Near-threshold operation, where circuits run just above the turn-on voltage of the transistor, offers a good balance between power and performance, contributing to overall power efficiency.

Outlines

00:00

🔋 SPOT Technology and AI on the Edge

The first paragraph discusses the importance of AI on the edge and the power requirements associated with it. Scott introduces Ambiq's SPOT (Subthreshold Power Optimized Technology), which is designed to maximize intelligence within a minimal power budget by operating at extremely low voltages. This approach results in significant energy savings due to the quadratic relationship between energy and voltage. The technology is applied to MCU cores like Arm’s Cortex M55 and M4, offering familiar software development environments with reduced power consumption. The design of the chip incorporates a mix of subthreshold, near-threshold, and standard operating voltages to balance power efficiency with performance and user experience. The paragraph also touches on the support for AI development with the M55's enhanced instruction set features and the neuralSpot software suite, which facilitates neural network development by providing utilities and AI development kits (ADKs) as a starting point for customers.

05:00

🚀 Advanced Features of Apollo5 Chip

The second paragraph highlights additional features of the Apollo5 chip, including its nonvolatile MRAM, GPU, and security aspects. The chip includes a 2.5D GPU capable of running at 250MHz, providing nearly four times the performance of the previous Apollo generation, which is beneficial for customers with rich display requirements. The GPU is supported by large on-chip memory for efficient storage of graphics assets and neural networks. On the security front, Apollo5 is designed with a focus on data protection, featuring cryptographic acceleration, a secure lifecycle management protocol, and TrustZone support, making it a secure, low-power, and high-performance chip. The paragraph concludes by emphasizing the chip's capabilities and encouraging developers to explore neural network development with the provided tools and profiling capabilities.

Keywords

💡AI on the edge

AI on the edge refers to the deployment of artificial intelligence capabilities at the edge of a network, close to the source of data generation. This allows for faster processing and decision-making without relying on centralized data centers. In the video, the importance of AI on the edge is highlighted in the context of power requirements and the development of energy-efficient technologies like SPOT.

💡Power Budget

A power budget is the maximum amount of power that a device or system can consume. It is a critical factor in designing electronic devices, especially when aiming for energy efficiency. The script discusses the challenge of incorporating high levels of intelligence within a limited power budget, which is addressed by Ambiq's SPOT technology.

💡SPOT Technology

SPOT, which stands for Subthreshold Power Optimized Technology, is a core technology developed by Ambiq to maximize intelligence within a minimal power consumption. It is based on running circuits at extremely low voltages, which results in significant energy savings. The script explains how SPOT is applied to MCU cores to offer familiar software development environments with reduced power usage.

💡Subthreshold Voltage

Subthreshold voltage refers to the voltage levels below the threshold voltage of a transistor, where the transistor is not fully turned on. In the context of the video, Ambiq's SPOT technology operates at these low voltages to achieve quadratic energy reductions, which is a key aspect of their low-power design approach.

💡MCU Cores

MCU cores, or Microcontroller Units, are the central processing units used in microcontrollers. They are integral to embedded systems and IoT devices. The script mentions that SPOT technology is applied to conventional MCU cores like Arm's Cortex M55 or M4, providing a familiar development environment with reduced power consumption.

💡Threshold Voltage

Threshold voltage is the minimum voltage required to turn a transistor fully on. In the script, near-threshold operation is discussed as a technique used in SPOT technology, where circuits operate just above the threshold voltage to balance power and performance.

💡Deep Subthreshold Mode

Deep subthreshold mode refers to operating a circuit at voltages significantly below the threshold voltage, which results in extremely low power consumption. The script mentions that Ambiq uses this mode in the analog domain for sleep modes to achieve ultra-low power operation.

💡Apoll5

Apoll5 is a chip developed by Ambiq, based on the Arm Cortex M55, and is designed for AI and machine learning applications. It features enhanced instruction sets for AI and is discussed in the script as having support for neural network development with the neuralSpot software suite.

💡NeuralSpot

NeuralSpot is a suite of software provided by Ambiq that facilitates neural network development on their chips. It acts as an interface between TensorFlow, a training framework, and the chip itself, offering utilities to simplify the development process. The script describes it as a layer of software that makes AI development more accessible.

💡AI Development Kits (ADKs)

AI Development Kits (ADKs) are a set of tools, neural networks, and resources developed and trained by Ambiq to assist customers in their AI development efforts. The script explains that these kits include network designs, training data, and weights, serving as a starting point for customers to develop their own AI solutions.

💡Nonvolatile MRAM

Nonvolatile MRAM, or magnetoresistive random-access memory, is a type of memory that retains data even when power is not supplied. It is highlighted in the script as a feature of the Apollo5 chip, which allows for efficient storage of large graphics assets and neural networks on-chip.

💡GPU

GPU stands for Graphics Processing Unit, a specialized electronic circuit designed to rapidly manipulate and alter memory to accelerate the creation of images in a frame buffer intended for output to a display device. The script mentions that Apollo5 includes a 2.5D GPU capable of running at high frequencies for enhanced graphics performance.

💡Cryptographic Acceleration

Cryptographic acceleration refers to the hardware or software that speeds up cryptographic operations, which are essential for security. In the context of the video, Apollo5 is designed with security in mind, including cryptographic acceleration to ensure data protection.

💡TrustZone

TrustZone is a technology by Arm that provides a secure environment for applications by isolating secure tasks from general tasks in a system. The script mentions that Apollo5 supports TrustZone, indicating a focus on security for AI and neural network applications.

Highlights

AI on the edge requires a balance between intelligence and power consumption.

Ambiq's SPOT technology aims to maximize intelligence within a minimal power budget.

SPOT stands for Subthreshold Power Optimized Technology, leveraging very low voltage operation for significant energy savings.

Running circuits at low voltages requires robust design techniques to ensure functionality.

SPOT can be applied to MCU cores like Arm's Cortex M55 or M4, offering familiar development environments with reduced power.

The chip design utilizes a mix of subthreshold, near threshold, and standard operating voltages for optimal balance.

Apoll5 chip integrates the Arm Cortex M55 with enhanced instruction sets for AI.

The M55 is efficient for signal processing tasks like FFTs and spectrogram building in neural networks.

Ambiq provides neuralSpot software suite for easy neural network development, compatible with TensorFlow.

AI development kits (ADKs) offer pre-trained neural networks as a starting point for customers.

Developers are encouraged to use profiling tools to identify inefficiencies in neural network development.

Apoll5 features nonvolatile MRAM, a 2.5D GPU, and large on-chip memory for graphics and neural networks.

The GPU in Apoll5 operates at up to 250MHz, providing significant performance improvements.

Security is integral to Apoll5, with cryptographic acceleration, secure lifecycle management, and TrustZone support.

Apoll5 is designed to be extremely low power, secure, and high performing for AI applications.

Transcripts

AI on the edge is very important these days.

But one of the key factors is the amount of power that's required.

We'd like to have a lot of intelligence,

but it doesn't necessarily come cheap.

So, Scott,

could you tell us a little bit about the SPOT technology

that Ambiq has developed?

Yeah.

So we developed our core SPOT

technology to address exactly this problem, right?

We want to put maximum intelligence

into a tiny little power budget

and the way we do that is something we call

Subthreshold Power Optimized Technology

and it's based on an old old approach to low power,

which involves running an extremely low voltages.

In fact, those voltages are so low that

that you may not turn a transistor fully on.

Now, the benefit of running at such low voltages is

energy is proportional to the square of the voltage.

So we get quadratic energy reductions as voltage drops.

That requires

a whole bunch of, techniques to be deployed

to make circuits robust under such conditions.

but we've been able to do that.

And in fact, by applying SPOT to conventional MCU cores like Arm’s

Cortex M55 or Arm's Cortex M4, we can offer a very familiar

software development environment at a fraction of the power budget

that our customers are used to.

Now, in terms of the design of the chip itself,

how do you utilize SPOT within the various peripherals, processors?

Is it the same architecture all the way through?

Yeah. So we we use a mixture of approaches.

So, for most of our digital circuits we’ll apply what is called near

threshold operation.

And that is where we're, we're running at just above the,

the turn-on voltage or the threshold voltage of the transistor

and that offers a really good

balance between power and performance

In the analog domain, we'll run often at the deep sub threshold mode.

That offers extreme low power,

especially when you're talking about sleep modes,

and then for certain circuits,

we'll even run at standard operating voltages.

GPIOs, for example,

I want my customers to have an experience

when they use our chip that matches what they get from other chips,

which means that the

the pad ring surrounding the chip needs to run at standard voltages.

Certain memories need to run at standard voltages.

So we have a mix of subthreshold,

near threshold, and standard operating voltages inside our chip.

And it takes a lot of technology to make all of that possible.

Now, your latest chip,

the Apoll5, is based on the Arm Cortex M55 that you mentioned.

One of the features of that are some enhanced instruction set

features for artificial intelligence.

Could you tell us a little bit about the support that you have

and some of the development tools that you have to complement this?

Well, so, first of all, the M55 is an amazing compute engine.

It's, it is capable of doing signal processing with great efficiency.

So if you think about building a typical

neural network, you've got to do pre-processing

before you bring the data in.

So you might be running FFts and building a spectrogram.

It's really good at the at those.

It's got a big wide vector unit

and and is well tuned for that, for the neural network itself.

Similarly, it has a big vector unit

and can run those efficiently,

but none of that is any good if you don't have a whole software,

suite paired with it.

And so what we offer to our customers

is, a suite of software that we call neuralSpot,

and the way to think about neuralSPOT

is it's that layer of software that sits between TensorFlow,

which is the Google's training framework, and the bare silicon itself.

It provides a bunch of utilities

that makes neural network development easy. And we couple that with

software that we call our AI development kits,

Our ADKs

And these are, neural networks

that we developed and trained in-house.

and then we provide them to our customers in reference form.

So they include,

the network design itself,

everything you need to train that, including the training data,

including weights.

And then we hand that off to the customer.

And the goal is not to give them a perfectly sealed

and ready to go production worthy neural network.

it's to give them a seed to start their own effort.

So if there's if there's one message I have

for for developers out there, it's that

AI development is is very, attainable today, right?

We we're providing the tools that make it relatively straightforward.

What I'll also say is you have to be careful with it.

There's a lot of talk about

wonderful NPUs and great hardware out there

to accelerate neural network development,

But in reality, a lot of the inefficiencies

lie to software level today, so so we provide tools

allow you to benchmark where you're at.

The M55 has some amazing

profiling capabilities

that allow you to get a sense of what's really going on inside there.

So, don't be afraid of neural network development on Apollo5.

And if you

if you do it, make sure make sure that you're

paying close attention to what that profiling data looks like.

Well, briefly,

there are a lot of other features that the Apollo 510 has.

for example, it has nonvolatile MRAM.

Can you tell us a couple of the other features,

maybe the GPU and the security aspect?

Yeah,

so a lot of our customers have big, rich displays,

and so we have included a GPU, a 2.5D GPU

on board, that GPU runs up to 250MHz and is, achieves performance

that's nearly four times

better than the previous generation of Apollo,

which was already, really nice chip for graphics development.

And that's coupled with some really big memories,

that, that allow you to, store

big frame

buffers, big graphics assets, or back to a previous

discussion, have big neural networks in locally.

And that's all on-chip memory.

It's not in package memory. It's all on-chip memory.

That's that's, hyper efficient.

And then on the security front,

look, security is a really important

part of the neural network story.

We're,

if we're going to entrust our data to these devices

and let them learn from us,

we have to make sure that all of our data is secure.

And so,

Apollo5 was designed from the ground up with security in mind.

So we have,

we have, of course, cryptographic acceleration,

but we have a complete secure lifecycle management

protocol built into the chip,

and we also have support for TrustZone built into the chip.

So, really secure chip, extremely low power. And, very performant.

Great. Well, thanks for filling us in on SPOT and the Apollo5 family.