Architecture All Access: Neuromorphic Computing Part 2
Summary
TLDRIn this episode of Architecture All Access, Mike Davies from Intel's Neuromorphic Computing Lab explores designing silicon chips that mimic the brain's efficiency. Discussing the challenges of replicating biological neural networks in CMOS circuits, Davies highlights the advantages of CMOS technology, such as speed and reliability. He outlines key neuromorphic principles like asynchronous communication and packetized spike routing, which Intel's Loihi 2 chip exemplifies. Davies emphasizes the progress in neuromorphic computing, particularly in solving complex optimization problems, and the ongoing challenge of programming these chips for practical applications.
Takeaways
- 🧠 The goal of neuromorphic engineering is to design chips that mimic the brain's computational efficiency and structure.
- 🔄 Despite decades of research, directly copying biological neural structures into CMOS circuits is impractical due to differences in design tools and manufacturing technology.
- 🏎️ CMOS technology offers significant advantages in speed, with the ability to build circuits that operate at nanoseconds and gigahertz, compared to the brain's milliseconds and kilohertz.
- 🔗 Neuromorphic chips focus on principles like sparse, distributed, and asynchronous communication to process sensory input quickly and minimize energy consumption.
- 💡 The concept of asynchronous activation of neurons, where only the most activated neurons communicate, is crucial for efficient neuromorphic computing.
- 🌐 Unlike the brain's 3D routing, neuromorphic chips use packetized spikes sent over shared time-multiplexed wiring channels to simulate more layers.
- 🚀 Intel's Loihi 2 is a fully digital neuromorphic chip that uses an asynchronous design style, aligning with the brain's massive asynchronous circuitry.
- 🔍 The neuromorphic core of Loihi 2 includes an asynchronous signal processing pipeline and internal memory, which is a critical resource in these chips.
- 📉 Neuromorphic chips like Loihi 2 use time multiplexing to reduce the effective area per neuron, allowing for higher neural density within a small footprint.
- 🤖 Implementing spike-based neuron models in neuromorphic chips results in lower activity and communication levels, leading to reduced power consumption.
- 📚 Programming neuromorphic chips to perform useful computations remains a significant challenge, but progress is being made, with applications in optimization and pattern recognition.
Q & A
What is the main challenge in designing neuromorphic chips that mimic the brain's functionality?
-The main challenge is the significant difference in design tools and manufacturing technology between silicon-based chips and biological systems. Specifically, the transistor area and wiring resources in silicon are much less efficient compared to the DNA-based molecular 3D self-assembly techniques used in biological systems.
How do neuromorphic chips differ from traditional chips in terms of speed?
-Neuromorphic chips can operate at nanoseconds and gigahertz, which is significantly faster than biological neurocircuits that operate at milliseconds and kilohertz scales.
What is the advantage of CMOS semiconductor manufacturing technology in neuromorphic computing?
-CMOS technology allows for the construction of fast and reliable circuits that operate precisely and deterministically, unlike the brain where precision and reliability come at the expense of redundancy or more neuro-resources.
What principle of brain computation is adapted in neuromorphic chip design to minimize energy consumption and manufacturing cost?
-The principle of sparse, distributed, asynchronous communication is adapted, which allows for the processing of the most important inputs quickly with minimal latency.
How do neuromorphic chips handle the routing of spikes compared to the brain's 3D routing?
-Neuromorphic chips use packetized spikes sent over shared time multiplexed horizontal wiring channels, effectively creating more vertical layers and allowing for faster distribution over a routed mesh network.
What is the significance of the speed advantage in neuromorphic chips for spike routing?
-The speed advantage allows neuromorphic chips to distribute thousands of spikes over the same wires in biological time scales without interference, simulating the effect of each neuron having its own dedicated axon wiring network.
How does the neuromorphic core in chips like Loihi 2 reduce its effective area per neuron?
-The neuromorphic core in Loihi 2 continues to time multiplex its circuitry, which allows for a significant reduction in the effective area per neuron, enabling the implementation of several thousand neurons in a small area.
What is the role of memory in neuromorphic chips, and how is it optimized in Loihi 2?
-Memory is a critical resource in neuromorphic chips, and its effective area cannot be reduced using time multiplexing. In Loihi 2, the core circuitry is optimized to use this fixed pool of memory efficiently, with features like convolutional features and a granular array of memory banks.
How does the learning process in neuromorphic chips like Loihi 2 work?
-The learning process in Loihi 2 operates in the background, updating the parameters of the core's neural network, particularly the synaptic state variables, based on a combination of input and output side activity at each synapse.
What are some of the applications where neuromorphic chips like Loihi have shown significant gains over conventional solvers?
-Neuromorphic chips like Loihi have shown significant gains in solving optimization problems such as railway scheduling and QUBO from quantum computing, compared to the best conventional solvers.
What is the current challenge in the field of neuromorphic computing regarding the chips' programming?
-The current challenge is understanding how to program neuromorphic chips to perform useful computations effectively, as it is still a formidable task despite the progress being made in the field.
Outlines
🧠 Introduction to Neuromorphic Computing
The first paragraph introduces the concept of neuromorphic computing and the challenges of designing silicon-based chips that mimic the brain's functionality. It discusses the limitations of directly copying biological neural structures due to differences in design tools and manufacturing technology. The narrator, Mike Davies, highlights the advantages of CMOS technology, such as speed and reliability, compared to biological systems. The principle of sparse, distributed, asynchronous communication is introduced as a fundamental concept in neuromorphic design, emphasizing the importance of processing important inputs quickly and efficiently. The paragraph concludes with a discussion on the need to adapt brain computation principles to the design tools available in chip manufacturing.
🔍 Deep Dive into Neuromorphic Chip Design
The second paragraph delves into the specifics of neuromorphic chip design, focusing on the Loihi 2 chip developed by Intel's Neuromorphic Computing Lab. It explains how the chip uses a fully digital approach and asynchronous design style to mimic the brain's massive asynchronous circuit. The paragraph discusses the importance of memory in neuromorphic chips and how the core circuitry is optimized to use memory efficiently. It details the process of spike routing, neuron activation, and the learning process that operates in the background to update the neural network's parameters. The summary also touches on the challenges of programming neuromorphic chips for useful computation and the progress being made in the field, with examples of applications in optimization problems.
🚀 Conclusion and Future of Neuromorphic Computing
The final paragraph concludes the discussion on neuromorphic computing by emphasizing the progress made in the field and the challenges that still lie ahead. It acknowledges that while building neuromorphic chips is achievable, programming them for practical applications remains a significant challenge. The paragraph highlights some of the successes, such as solving complex optimization problems more efficiently than traditional methods. It encourages viewers to follow the progress of neuromorphic computing as it moves from the lab to real-world applications, suggesting that this technology has the potential to revolutionize various industries.
Mindmap
Keywords
💡Neuromorphic Computing
💡CMOS Circuits
💡Sparse, Distributed, Asynchronous Communication
💡Spike-based Neuron Models
💡Time Multiplexing
💡Routing
💡Loihi 2
💡Asynchronous Circuits
💡Learning Rules
💡Optimization
Highlights
Mapping neuromorphic concepts into silicon chips is a challenge that requires adapting brain-like computation to CMOS technology.
CMOS circuits can replicate some form and functions of biological neural networks, but direct copying is impractical due to differences in design tools and manufacturing.
Transistor area and wiring resources in chip design are significantly different from biological systems, leading to a need for innovative approaches.
CMOS technology offers advantages in speed, with circuits operating at nanoseconds and gigahertz, compared to biological systems' milliseconds and kilohertz.
The principle of sparse, distributed, asynchronous communication is fundamental in neuromorphic chip design, mimicking the efficiency of biological neural networks.
Neuromorphic chips aim to implement neurons that represent their activations as events in time, similar to biological spikes.
Routing spikes in neuromorphic chips requires innovative solutions due to the limitation in vertical wiring compared to the brain's three-dimensional routing.
Packetizing spikes and sending them over shared time multiplexed channels can overcome the wiring disadvantage in chip design.
The speed advantage of CMOS allows for efficient spike distribution, simulating the effect of dedicated axon wiring networks.
Modern semiconductor process technology can pack metal in a one-micron strip, approaching the efficiency and density of the brain's myelinated axons.
Neuromorphic cores need to time multiplex their circuitry to reduce the effective area per neuron, achieving high neural density.
Spike-based neuron models in neuromorphic chips result in lower activity and communication levels, leading to lower power consumption.
Loihi 2, a neuromorphic chip developed by Intel, is a fully digital chip implemented in a standard CMOS process, optimized for asynchronous operation.
Asynchronous circuits in neuromorphic chips consume power only when computing, matching the sparse activity of spiking neural networks.
Loihi 2's neuromorphic core features an asynchronous signal processing pipeline with internal memory, optimized for efficient memory usage.
The learning process in Loihi operates in the background, updating the neural network's parameters based on input and output activity.
Programming neuromorphic chips for useful computation remains a challenge, but the field is making progress with exciting results in optimization problems.
Loihi has demonstrated significant gains in solving complex problems like railway scheduling and QUBO from quantum computing, compared to conventional solvers.
Transcripts
(bright music)
- [Narrator] How do we map neuromorphic concepts
into chips built from silicon?
Here's Mike Davies, Senior Principal Engineer
and Director of Intel's Neuromorphic Computing Lab.
This is Architecture All Access.
- Let's now talk about how to architect a chip
that behaves more like a brain
than the chips we're familiar with today.
If we step back and look at the state of modern neuroscience
and chip design,
we see that we can clearly build CMOS circuits
that replicate some of the form and functions
of biological neural networks.
People have been experimenting with this
for over three decades,
but an approach of directly copying the structure
and circuit dynamics of biological neurons
quickly runs into practical trouble.
The design tools and manufacturing technology
we're working with are simply too different from biology,
specifically in terms of transistor area
and wiring resources.
We are many orders of magnitude
behind what evolution has achieved
with its DNA-based molecular 3D self-assembly techniques.
As a chip designer,
I'd love to get my hands on nature's circuit design toolbox,
but those capabilities
are going to be a very long time to come.
On the other hand,
today's CMOS semiconductor manufacturing technology
does give us some important advantages.
In speed, for example,
we can construct extraordinarily fast circuits
compared to biological neurocircuits.
While biology operates at milliseconds and kilohertz scales,
our CMOS circuits operate at nanoseconds and gigahertz,
and we can construct reliable circuits
that operate precisely and deterministically.
Whereas in the brain, precision and reliability
come at the expense of redundancy or more neuro-resources.
So this leads us away
from the idea of literally copying
or exactly replicating the neural structures
we find in the brain.
Instead, we have to understand
and adapt the underlying principles of brain computation
to the design toolbox we're working with.
But despite the different design tools,
the same principles surely apply
since we are targeting the same objectives,
intelligently process sensory input quickly
while minimizing both energy consumption
and manufacturing cost.
So let's consider some of these principles.
One of the most fundamental is the virtue
of sparse, distributed, asynchronous communication.
Consider the problem of communicating the activations
of a set of neurons to some downstream neuron.
If somehow only the most strongly activated neurons
could asynchronously announce themselves in time,
then the most important inputs can be processed quickly
with minimal latency.
On the other hand, if the entire set of input neurons
has to be processed as one dense matrix,
which is the standard way
in today's mainstream architectures,
then the important inputs will get congested
and delayed behind the herd of less important inputs.
This may seem obvious, but to truly exploit this principle,
a hardware implementation needs to respond
to unpredictable neuron activations immediately,
which implies extremely fine grain parallelism
and neurons that asynchronously represent their activations
as events in time rather than as numbers
in a synchronously processed vector.
This is what biological neurons do using spikes,
and it's what we aim to implement in neuromorphic chips.
Now, if we turn to the problem
of routing those spikes in a chip we can build,
we're gonna find a clear example
where we can't just copy biology.
Brains use truly three-dimensional routing
with tens of thousands of overlapping wires
through any cross section.
In contrast, our chips, vertically,
we can only stack about 20 metal layers.
This is a huge wiring disadvantage.
But remember, we have an enormous speed advantage.
So what we can do is packetize the spikes and send them
over shared time multiplexed horizontal wiring channels,
giving us effectively many more vertical layers.
The packets can then be distributed
over a routed mesh network
at speeds of up to a million times faster
than what biology uses for its spikes.
At those speeds,
the routers can distribute thousands of spikes
over the same wires
in biological time scales without interference,
as if each neuron had its own dedicated axon wiring network
as in the brain.
Now, you may think this looks incredibly inefficient.
Spikes sent as packets with 32 or more bits
sent over bundles of wires.
How could this possibly reach the level
of efficiency and density in the brain?
But actually, we're not that far off.
The brain's long distance wires are myelinated axons,
which have a diameter of about one micron.
In modern semiconductor process technology,
we can easily pack all the metal we need
in a one micron strip of planar wires.
So our brain-inspired spiking interconnect is
in the ballpark of biology.
And once we have these routers,
we can now connect them
to neuromorphic cores that produce and consume these spikes
by modeling the essential behaviors
of biology's temporal spike-based neurons.
Each core needs to continue to time multiplex its circuitry
so it can reduce its effective area per neuron
by orders of magnitude.
This way, each core with a fraction of a square millimeter
can implement several thousand neurons,
which actually approaches the neural density
of the neocortex.
By implementing spike-based neuron models,
the activity and communication levels are far lower
than what you'd have on a conventional chip.
The cores send their spikes
as sparse peer-to-peer distributions,
leaving the routing network relatively idle
compared to the buses in a conventional chip.
In this architecture,
there's no off-chip memory interface
that has to stay saturated
with predictable vectorized memory patterns.
This all translates to lower power.
Now moving on to a real neuromorphic chip.
Here we have Loihi 2, developed by my group,
and one of the most recently developed neuromorphic chips.
The chip plot frankly looks rather boring
because, like a memory chip,
it's mostly a single repeated core instantiated in a mesh
and the core is itself dominated
by internal memory structures.
Loihi 2 is a fully digital chip
implemented in a standard CMOS process.
Historically, many neuromorphic research chips
have used analog circuits to replicate the membrane dynamics
of biological neurons, but for many reasons
relating to circuit density, precision, reliability,
the field has generally moved to fully digital designs,
at least for now.
The brain is one massive asynchronous circuit,
and a lot of neuromorphic chips, including our Loihi chips,
are implemented using an asynchronous design style.
Asynchronous circuits don't use free running clocks
and automatically stop consuming power
when there's nothing to compute.
Of course, this is a great match
for spiking neural networks that are only sparsely active.
If we now dive a bit deeper
into the details of Loihi 2's neuromorphic core,
you'll find an asynchronous signal processing pipeline
with a lot of internal memory.
Memory is the scariest resource in a neuromorphic chip,
and unlike compute circuitry,
its effective area cannot be reduced
using time multiplexing.
So the rest of the core circuitry is highly optimized
to use this fixed pool of memory as efficiently as possible.
For example, convolutional features allow a
repeated pattern of synaptic weights to be stored only once
and then reused across multiple neurons.
The core's memory is a granular array
of many banks with varying sizes.
Different processes in the core
require different levels of access parallelism.
The memory can be allocated
in a flexible way across the core's pipeline stages,
although the performance and storage density are optimized
for the typical configurations.
As spikes arrive at the core,
table lookups determine what neurons in the core
the spikes connect to.
The lookups return data structures that specify weights
and propagation delays for each destination neuron.
As each spike is applied to its fanout neurons,
the associated weights are absorbed
by a dendritic accumulation process
which maintains the total received synaptic stimulation
for each of the core's neurons.
In a completely decoupled asynchronous process,
the dendrite stage reads
each neuron's accumulated synaptic input
and then updates the neuron's state variables
for the current time
following the neuron's programmed model.
Typically, synaptic input causes a neuron state variable
to jump in a positive or negative direction,
while the absence of input causes a gradual decay
of the variable to zero.
If a neuron becomes sufficiently activated
and crosses a threshold, then it generates an output spike.
Further table lookups in an egress process
determine which core is in the network
the spiking neuron connects to,
and then spike packets are sent
through the routing infrastructure
to those destination cores.
In Loihi, we also have a learning process that operates
in the background,
updating the parameters of the core's neural network,
particularly the synaptic state variables.
Based on a combination of the input side activity
and the output side activity at each synapse,
its state variables, such as its weight,
can evolve over time according to programmed equations.
Getting these so-called learning rules right
so that the large scale behavior of the network
is to adapt online and usefully learn new patterns
is one of the trickiest and most actively researched areas
of neuromorphic computing.
And that concludes our quick tour
of neuromorphic architecture.
Hopefully, I provided an understandable glimpse
into the world of neuromorphic computing
and you're leaving with a greater appreciation
for this rather different and exotic architecture.
The architecture is actually the relatively easy part.
We can build these chips today
that support many of the computational principles
we find in brains,
but understanding how to program them
to perform useful computation,
that's still admittedly a formidable challenge.
The field is making a lot of progress on that though
with some exciting results.
For example, in the domain of optimization,
Loihi can solve problems like railway scheduling,
and QUBO from quantum computing
with orders of magnitude gains
compared to the best conventional solvers.
So please check out some of our recent publications
and follow our progress as we get this technology
out of the lab
and into game-changing products.
This has been
Architecture All Access, Neuromorphic Computing.
Thanks for joining us.
(bright music)
浏览更多相关视频
The Basics of Neuromorphic Computing
Energy Efficient AI Hardware: neuromorphic circuits and tools
You don't understand AI until you watch this
Neuromorphic Intelligence: Brain-inspired Strategies for AI Computing Systems
The Next Generation Of Brain Mimicking AI
[자막뉴스] 엔비디아 능가한 '초저전력 AI 반도체'...한국 세계 최초로 개발 / YTN
5.0 / 5 (0 votes)