Architecture All Access: Neuromorphic Computing Part 2

Summary

In this episode of Architecture All Access, Mike Davies from Intel's Neuromorphic Computing Lab explores designing silicon chips that mimic the brain's efficiency. Discussing the challenges of replicating biological neural networks in CMOS circuits, Davies highlights the advantages of CMOS technology, such as speed and reliability. He outlines key neuromorphic principles like asynchronous communication and packetized spike routing, which Intel's Loihi 2 chip exemplifies. Davies emphasizes the progress in neuromorphic computing, particularly in solving complex optimization problems, and the ongoing challenge of programming these chips for practical applications.

Takeaways

• 🧠 The goal of neuromorphic engineering is to design chips that mimic the brain's computational efficiency and structure.
• 🔄 Despite decades of research, directly copying biological neural structures into CMOS circuits is impractical due to differences in design tools and manufacturing technology.
• 🏎️ CMOS technology offers significant advantages in speed, with the ability to build circuits that operate at nanoseconds and gigahertz, compared to the brain's milliseconds and kilohertz.
• 🔗 Neuromorphic chips focus on principles like sparse, distributed, and asynchronous communication to process sensory input quickly and minimize energy consumption.
• 💡 The concept of asynchronous activation of neurons, where only the most activated neurons communicate, is crucial for efficient neuromorphic computing.
• 🌐 Unlike the brain's 3D routing, neuromorphic chips use packetized spikes sent over shared time-multiplexed wiring channels to simulate more layers.
• 🚀 Intel's Loihi 2 is a fully digital neuromorphic chip that uses an asynchronous design style, aligning with the brain's massive asynchronous circuitry.
• 🔍 The neuromorphic core of Loihi 2 includes an asynchronous signal processing pipeline and internal memory, which is a critical resource in these chips.
• 📉 Neuromorphic chips like Loihi 2 use time multiplexing to reduce the effective area per neuron, allowing for higher neural density within a small footprint.
• 🤖 Implementing spike-based neuron models in neuromorphic chips results in lower activity and communication levels, leading to reduced power consumption.
• 📚 Programming neuromorphic chips to perform useful computations remains a significant challenge, but progress is being made, with applications in optimization and pattern recognition.

Q & A

What is the main challenge in designing neuromorphic chips that mimic the brain's functionality?

The main challenge is the significant difference in design tools and manufacturing technology between silicon-based chips and biological systems. Specifically, the transistor area and wiring resources in silicon are much less efficient compared to the DNA-based molecular 3D self-assembly techniques used in biological systems.

How do neuromorphic chips differ from traditional chips in terms of speed?

Neuromorphic chips can operate at nanoseconds and gigahertz, which is significantly faster than biological neurocircuits that operate at milliseconds and kilohertz scales.

What is the advantage of CMOS semiconductor manufacturing technology in neuromorphic computing?

CMOS technology allows for the construction of fast and reliable circuits that operate precisely and deterministically, unlike the brain where precision and reliability come at the expense of redundancy or more neuro-resources.

What principle of brain computation is adapted in neuromorphic chip design to minimize energy consumption and manufacturing cost?

The principle of sparse, distributed, asynchronous communication is adapted, which allows for the processing of the most important inputs quickly with minimal latency.

How do neuromorphic chips handle the routing of spikes compared to the brain's 3D routing?

Neuromorphic chips use packetized spikes sent over shared time multiplexed horizontal wiring channels, effectively creating more vertical layers and allowing for faster distribution over a routed mesh network.

What is the significance of the speed advantage in neuromorphic chips for spike routing?

The speed advantage allows neuromorphic chips to distribute thousands of spikes over the same wires in biological time scales without interference, simulating the effect of each neuron having its own dedicated axon wiring network.

How does the neuromorphic core in chips like Loihi 2 reduce its effective area per neuron?

The neuromorphic core in Loihi 2 continues to time multiplex its circuitry, which allows for a significant reduction in the effective area per neuron, enabling the implementation of several thousand neurons in a small area.

What is the role of memory in neuromorphic chips, and how is it optimized in Loihi 2?

Memory is a critical resource in neuromorphic chips, and its effective area cannot be reduced using time multiplexing. In Loihi 2, the core circuitry is optimized to use this fixed pool of memory efficiently, with features like convolutional features and a granular array of memory banks.

How does the learning process in neuromorphic chips like Loihi 2 work?

The learning process in Loihi 2 operates in the background, updating the parameters of the core's neural network, particularly the synaptic state variables, based on a combination of input and output side activity at each synapse.

What are some of the applications where neuromorphic chips like Loihi have shown significant gains over conventional solvers?

Neuromorphic chips like Loihi have shown significant gains in solving optimization problems such as railway scheduling and QUBO from quantum computing, compared to the best conventional solvers.

What is the current challenge in the field of neuromorphic computing regarding the chips' programming?

The current challenge is understanding how to program neuromorphic chips to perform useful computations effectively, as it is still a formidable task despite the progress being made in the field.

Outlines

00:00

🧠 Introduction to Neuromorphic Computing

The first paragraph introduces the concept of neuromorphic computing and the challenges of designing silicon-based chips that mimic the brain's functionality. It discusses the limitations of directly copying biological neural structures due to differences in design tools and manufacturing technology. The narrator, Mike Davies, highlights the advantages of CMOS technology, such as speed and reliability, compared to biological systems. The principle of sparse, distributed, asynchronous communication is introduced as a fundamental concept in neuromorphic design, emphasizing the importance of processing important inputs quickly and efficiently. The paragraph concludes with a discussion on the need to adapt brain computation principles to the design tools available in chip manufacturing.

05:01

🔍 Deep Dive into Neuromorphic Chip Design

The second paragraph delves into the specifics of neuromorphic chip design, focusing on the Loihi 2 chip developed by Intel's Neuromorphic Computing Lab. It explains how the chip uses a fully digital approach and asynchronous design style to mimic the brain's massive asynchronous circuit. The paragraph discusses the importance of memory in neuromorphic chips and how the core circuitry is optimized to use memory efficiently. It details the process of spike routing, neuron activation, and the learning process that operates in the background to update the neural network's parameters. The summary also touches on the challenges of programming neuromorphic chips for useful computation and the progress being made in the field, with examples of applications in optimization problems.

10:03

🚀 Conclusion and Future of Neuromorphic Computing

The final paragraph concludes the discussion on neuromorphic computing by emphasizing the progress made in the field and the challenges that still lie ahead. It acknowledges that while building neuromorphic chips is achievable, programming them for practical applications remains a significant challenge. The paragraph highlights some of the successes, such as solving complex optimization problems more efficiently than traditional methods. It encourages viewers to follow the progress of neuromorphic computing as it moves from the lab to real-world applications, suggesting that this technology has the potential to revolutionize various industries.

Keywords

💡Neuromorphic Computing

Neuromorphic Computing refers to the development of computer systems that mimic the neural structure and functioning of the human brain. In the context of the video, it is the central theme, with the discussion revolving around designing chips that behave more like a brain than traditional silicon chips. The video explains how neuromorphic chips can replicate some of the form and functions of biological neural networks, aiming to process sensory input quickly while minimizing energy consumption and manufacturing cost.

💡CMOS Circuits

CMOS, which stands for Complementary Metal-Oxide-Semiconductor, is a technology used for constructing integrated circuits. The video mentions that CMOS circuits can replicate some aspects of biological neural networks, but there are practical challenges in directly copying the structure and dynamics of biological neurons due to differences in design tools and manufacturing technology.

💡Sparse, Distributed, Asynchronous Communication

This concept refers to a communication method where only the most strongly activated neurons transmit their signals asynchronously, rather than all neurons sending signals simultaneously. The video explains that this principle can lead to quicker processing of important inputs with minimal latency, which is a key aspect of neuromorphic chip design aimed at emulating the efficiency of the brain's communication.

💡Spike-based Neuron Models

In the video, spike-based neuron models are discussed as a way to implement biological neurons' behavior in neuromorphic chips. These models use spikes, or brief electrical signals, to transmit information, similar to how biological neurons communicate. The video emphasizes that these models allow for lower activity and communication levels, leading to more efficient routing and lower power consumption.

💡Time Multiplexing

Time multiplexing is a technique where multiple signals are transmitted over a single communication channel by dividing the signal into time slots. The video explains that neuromorphic cores use time multiplexing to reduce their effective area per neuron, thus increasing the density of neurons that can be implemented on a chip.

💡Routing

Routing in the context of neuromorphic chips refers to the way spikes are directed through the chip's architecture. The video discusses how neuromorphic chips cannot directly copy the three-dimensional routing of the brain but instead use packetized spikes sent over shared time multiplexed channels to achieve efficient routing.

💡Loihi 2

Loihi 2 is a neuromorphic chip developed by Intel, as mentioned in the video. It is a fully digital chip implemented in a standard CMOS process and is designed to support many of the computational principles found in brains. The video provides an overview of Loihi 2's architecture, highlighting its asynchronous design style and internal memory structures.

💡Asynchronous Circuits

Asynchronous circuits are electronic circuits that do not rely on a global clock signal to synchronize operations. The video explains that neuromorphic chips, including Loihi, use asynchronous design styles, which is well-suited for spiking neural networks that are only sparsely active, leading to power efficiency.

💡Learning Rules

In the context of neuromorphic computing, learning rules refer to the algorithms or equations that govern how the synaptic state variables of a neural network evolve over time. The video discusses the importance of getting these rules right to enable the network to adapt and learn new patterns, which is a challenging and active area of research.

💡Optimization

Optimization in the video refers to the application of neuromorphic chips like Loihi in solving complex problems more efficiently than traditional methods. The video gives examples such as railway scheduling and QUBO problems from quantum computing, where neuromorphic chips can provide significant gains in performance.

Highlights

Mapping neuromorphic concepts into silicon chips is a challenge that requires adapting brain-like computation to CMOS technology.

CMOS circuits can replicate some form and functions of biological neural networks, but direct copying is impractical due to differences in design tools and manufacturing.

Transistor area and wiring resources in chip design are significantly different from biological systems, leading to a need for innovative approaches.

CMOS technology offers advantages in speed, with circuits operating at nanoseconds and gigahertz, compared to biological systems' milliseconds and kilohertz.

The principle of sparse, distributed, asynchronous communication is fundamental in neuromorphic chip design, mimicking the efficiency of biological neural networks.

Neuromorphic chips aim to implement neurons that represent their activations as events in time, similar to biological spikes.

Routing spikes in neuromorphic chips requires innovative solutions due to the limitation in vertical wiring compared to the brain's three-dimensional routing.

Packetizing spikes and sending them over shared time multiplexed channels can overcome the wiring disadvantage in chip design.

The speed advantage of CMOS allows for efficient spike distribution, simulating the effect of dedicated axon wiring networks.

Modern semiconductor process technology can pack metal in a one-micron strip, approaching the efficiency and density of the brain's myelinated axons.

Neuromorphic cores need to time multiplex their circuitry to reduce the effective area per neuron, achieving high neural density.

Spike-based neuron models in neuromorphic chips result in lower activity and communication levels, leading to lower power consumption.

Loihi 2, a neuromorphic chip developed by Intel, is a fully digital chip implemented in a standard CMOS process, optimized for asynchronous operation.

Asynchronous circuits in neuromorphic chips consume power only when computing, matching the sparse activity of spiking neural networks.

Loihi 2's neuromorphic core features an asynchronous signal processing pipeline with internal memory, optimized for efficient memory usage.

The learning process in Loihi operates in the background, updating the neural network's parameters based on input and output activity.

Programming neuromorphic chips for useful computation remains a challenge, but the field is making progress with exciting results in optimization problems.

Loihi has demonstrated significant gains in solving complex problems like railway scheduling and QUBO from quantum computing, compared to conventional solvers.

Transcripts

(bright music)

- [Narrator] How do we map neuromorphic concepts

into chips built from silicon?

Here's Mike Davies, Senior Principal Engineer

and Director of Intel's Neuromorphic Computing Lab.

This is Architecture All Access.

- Let's now talk about how to architect a chip

that behaves more like a brain

than the chips we're familiar with today.

If we step back and look at the state of modern neuroscience

and chip design,

we see that we can clearly build CMOS circuits

that replicate some of the form and functions

of biological neural networks.

People have been experimenting with this

for over three decades,

but an approach of directly copying the structure

and circuit dynamics of biological neurons

quickly runs into practical trouble.

The design tools and manufacturing technology

we're working with are simply too different from biology,

specifically in terms of transistor area

and wiring resources.

We are many orders of magnitude

behind what evolution has achieved

with its DNA-based molecular 3D self-assembly techniques.

As a chip designer,

I'd love to get my hands on nature's circuit design toolbox,

but those capabilities

are going to be a very long time to come.

On the other hand,

today's CMOS semiconductor manufacturing technology

does give us some important advantages.

In speed, for example,

we can construct extraordinarily fast circuits

compared to biological neurocircuits.

While biology operates at milliseconds and kilohertz scales,

our CMOS circuits operate at nanoseconds and gigahertz,

and we can construct reliable circuits

that operate precisely and deterministically.

Whereas in the brain, precision and reliability

come at the expense of redundancy or more neuro-resources.

So this leads us away

from the idea of literally copying

or exactly replicating the neural structures

we find in the brain.

Instead, we have to understand

and adapt the underlying principles of brain computation

to the design toolbox we're working with.

But despite the different design tools,

the same principles surely apply

since we are targeting the same objectives,

intelligently process sensory input quickly

while minimizing both energy consumption

and manufacturing cost.

So let's consider some of these principles.

One of the most fundamental is the virtue

of sparse, distributed, asynchronous communication.

Consider the problem of communicating the activations

of a set of neurons to some downstream neuron.

If somehow only the most strongly activated neurons

could asynchronously announce themselves in time,

then the most important inputs can be processed quickly

with minimal latency.

On the other hand, if the entire set of input neurons

has to be processed as one dense matrix,

which is the standard way

in today's mainstream architectures,

then the important inputs will get congested

and delayed behind the herd of less important inputs.

This may seem obvious, but to truly exploit this principle,

a hardware implementation needs to respond

to unpredictable neuron activations immediately,

which implies extremely fine grain parallelism

and neurons that asynchronously represent their activations

as events in time rather than as numbers

in a synchronously processed vector.

This is what biological neurons do using spikes,

and it's what we aim to implement in neuromorphic chips.

Now, if we turn to the problem

of routing those spikes in a chip we can build,

we're gonna find a clear example

where we can't just copy biology.

Brains use truly three-dimensional routing

with tens of thousands of overlapping wires

through any cross section.

In contrast, our chips, vertically,

we can only stack about 20 metal layers.

This is a huge wiring disadvantage.

But remember, we have an enormous speed advantage.

So what we can do is packetize the spikes and send them

over shared time multiplexed horizontal wiring channels,

giving us effectively many more vertical layers.

The packets can then be distributed

over a routed mesh network

at speeds of up to a million times faster

than what biology uses for its spikes.

At those speeds,

the routers can distribute thousands of spikes

over the same wires

in biological time scales without interference,

as if each neuron had its own dedicated axon wiring network

as in the brain.

Now, you may think this looks incredibly inefficient.

Spikes sent as packets with 32 or more bits

sent over bundles of wires.

How could this possibly reach the level

of efficiency and density in the brain?

But actually, we're not that far off.

The brain's long distance wires are myelinated axons,

which have a diameter of about one micron.

In modern semiconductor process technology,

we can easily pack all the metal we need

in a one micron strip of planar wires.

So our brain-inspired spiking interconnect is

in the ballpark of biology.

And once we have these routers,

we can now connect them

to neuromorphic cores that produce and consume these spikes

by modeling the essential behaviors

of biology's temporal spike-based neurons.

Each core needs to continue to time multiplex its circuitry

so it can reduce its effective area per neuron

by orders of magnitude.

This way, each core with a fraction of a square millimeter

can implement several thousand neurons,

which actually approaches the neural density

of the neocortex.

By implementing spike-based neuron models,

the activity and communication levels are far lower

than what you'd have on a conventional chip.

The cores send their spikes

as sparse peer-to-peer distributions,

leaving the routing network relatively idle

compared to the buses in a conventional chip.

In this architecture,

there's no off-chip memory interface

that has to stay saturated

with predictable vectorized memory patterns.

This all translates to lower power.

Now moving on to a real neuromorphic chip.

Here we have Loihi 2, developed by my group,

and one of the most recently developed neuromorphic chips.

The chip plot frankly looks rather boring

because, like a memory chip,

it's mostly a single repeated core instantiated in a mesh

and the core is itself dominated

by internal memory structures.

Loihi 2 is a fully digital chip

implemented in a standard CMOS process.

Historically, many neuromorphic research chips

have used analog circuits to replicate the membrane dynamics

of biological neurons, but for many reasons

relating to circuit density, precision, reliability,

the field has generally moved to fully digital designs,

at least for now.

The brain is one massive asynchronous circuit,

and a lot of neuromorphic chips, including our Loihi chips,

are implemented using an asynchronous design style.

Asynchronous circuits don't use free running clocks

and automatically stop consuming power

when there's nothing to compute.

Of course, this is a great match

for spiking neural networks that are only sparsely active.

If we now dive a bit deeper

into the details of Loihi 2's neuromorphic core,

you'll find an asynchronous signal processing pipeline

with a lot of internal memory.

Memory is the scariest resource in a neuromorphic chip,

and unlike compute circuitry,

its effective area cannot be reduced

using time multiplexing.

So the rest of the core circuitry is highly optimized

to use this fixed pool of memory as efficiently as possible.

For example, convolutional features allow a

repeated pattern of synaptic weights to be stored only once

and then reused across multiple neurons.

The core's memory is a granular array

of many banks with varying sizes.

Different processes in the core

require different levels of access parallelism.

The memory can be allocated

in a flexible way across the core's pipeline stages,

although the performance and storage density are optimized

for the typical configurations.

As spikes arrive at the core,

table lookups determine what neurons in the core

the spikes connect to.

The lookups return data structures that specify weights

and propagation delays for each destination neuron.

As each spike is applied to its fanout neurons,

the associated weights are absorbed

by a dendritic accumulation process

which maintains the total received synaptic stimulation

for each of the core's neurons.

In a completely decoupled asynchronous process,

the dendrite stage reads

each neuron's accumulated synaptic input

and then updates the neuron's state variables

for the current time

following the neuron's programmed model.

Typically, synaptic input causes a neuron state variable

to jump in a positive or negative direction,

while the absence of input causes a gradual decay

of the variable to zero.

If a neuron becomes sufficiently activated

and crosses a threshold, then it generates an output spike.

Further table lookups in an egress process

determine which core is in the network

the spiking neuron connects to,

and then spike packets are sent

through the routing infrastructure

to those destination cores.

In Loihi, we also have a learning process that operates

in the background,

updating the parameters of the core's neural network,

particularly the synaptic state variables.

Based on a combination of the input side activity

and the output side activity at each synapse,

its state variables, such as its weight,

can evolve over time according to programmed equations.

Getting these so-called learning rules right

so that the large scale behavior of the network

is to adapt online and usefully learn new patterns

is one of the trickiest and most actively researched areas

of neuromorphic computing.

And that concludes our quick tour

of neuromorphic architecture.

Hopefully, I provided an understandable glimpse

into the world of neuromorphic computing

and you're leaving with a greater appreciation

for this rather different and exotic architecture.

The architecture is actually the relatively easy part.

We can build these chips today

that support many of the computational principles

we find in brains,

but understanding how to program them

to perform useful computation,

that's still admittedly a formidable challenge.

The field is making a lot of progress on that though

with some exciting results.

For example, in the domain of optimization,

Loihi can solve problems like railway scheduling,

and QUBO from quantum computing

with orders of magnitude gains

compared to the best conventional solvers.

So please check out some of our recent publications

and follow our progress as we get this technology

out of the lab

and into game-changing products.

This has been

Architecture All Access, Neuromorphic Computing.

Thanks for joining us.

(bright music)