The Next Era of Computing | Extropic

Summary

The speaker, a physicist with a deep interest in theoretical physics and information theory, shares his journey of eight years leading to a breakthrough in computing. He discusses the evolution from quantum computing to a new paradigm that utilizes noise in thermodynamics to perform AI as a physical process. This approach is more energy-efficient and faster than current hardware, as it leverages the natural computation methods akin to how our brains work. The speaker introduces the concept of 'thermodynamic computing' and explains how it uses Josephson junctions, a key component in superconducting chips, to achieve superconductivity and exploit the Josephson effect. He envisions this technology as a significant step towards expanding the scope and scale of intelligence throughout the universe, enabling us to solve complex problems and ultimately reach for the stars.

Takeaways

• 🚀 The speaker's journey began 8 years ago with the concept that has now materialized into a new paradigm of computing, which they believe will change the world.
• 🌌 The speaker has always been interested in understanding the universe to build technologies with a maximal impact on scaling civilization to the stars.
• 🧠 The idea of viewing the universe as a simulation and embedding AI into the physics of the world came naturally from a background in theoretical physics and information theory.
• ⚙️ The challenge in quantum computing was noise, which is also a problem in low-power digital computers, but the new devices use this noise as an asset.
• 🌟 The speaker was inspired by the power of machine learning and the concept of a physical system performing machine learning as a physical process.
• 📊 Digital computers use bits to represent values, but the new paradigm is a different way of programming, closer to how nature and our brains compute.
• 🔌 As circuits get smaller, voltage noise becomes a problem for classical computation in CPUs, but the new thermodynamic computing aims to use this noise to drive computation.
• 💡 The new chips don't use transistors and instead use continuous variables and fuzzy values, which can be thought of as a point cloud shaped by data.
• 🔧 The process of machine learning is likened to a physical process where a parameterized physical device optimizes itself to accomplish a given task.
• 🌟 The speaker refers to the differentiable programming mindset, where gradient descent is considered a better programmer, as Software 2.0.
• 🏭 The speaker is in Sherbrook, Canada, at a fabrication facility where they manufacture a key component called Josephson junctions for superconducting chips.
• ❄️ The chips are cooled to 10 millikelvin to achieve superconductivity, which is necessary for exploiting the Josephson effect.
• 🔩 The technology is seen as a big step towards more energy-efficient and faster computers, which could fundamentally change how AI algorithms are run.
• 🌠 The ultimate goal is the expansion of intelligence throughout the universe, and the technology being developed is a significant step in that direction.

Q & A

What was the initial inspiration for the development of this new computing paradigm?

The idea for this new computing paradigm came to the speaker about 8 years ago, stemming from an interest in understanding the universe and building technologies with maximal impact on scaling civilization to the stars.

How does the speaker's background in theoretical physics and information theory influence their approach to AI?

The speaker's background led them to view the universe as a large simulation and naturally embed AI into the physics of the world, which is a different approach from traditional digital computation.

What is the main challenge faced in quantum computing that this new technology aims to address?

Noise is identified as a significant issue in quantum computing, which also becomes a problem when trying to create low-power digital computers. The new technology uses this noise as an asset rather than a liability.

How does the speaker describe the process of machine learning with a physical system?

The speaker describes it as a powerful concept where a physical system performs machine learning as a part of physics, which they initially tried to discredit but found to be compelling and feasible.

What is the fundamental difference between digital computers and the new thermodynamic computing approach?

Digital computers represent values in bits, while the new approach uses continuous variables and fuzzy values over a continuous variable, which is closer to how nature computes and how our brains compute.

How does the concept of 'differentiable programming' or 'software 2.0' fit into the speaker's vision for AI?

Differentiable programming, as termed by Andre Karpathy, is central to the speaker's vision where gradient descent is considered a better programmer than humans, optimizing the physical device parameters to accomplish tasks.

What are Josephson junctions, and why are they important to the new computing technology?

Josephson junctions are a key component in superconducting chips, acting like transistors for superconductivity. They are crucial for exploiting the Josephson effect and enabling AI as a physical process.

Why is superconductivity essential for the operation of these new chips?

Superconductivity is necessary to achieve the desired effects, such as low dissipation and exploiting the Josephson effect, which allows for the programmable physics of electrons in the metals.

What is the significance of the speaker's work in Sherbrook, Canada?

The speaker's work in Sherbrook involves the manufacturing of the first chips, which are superconducting and serve as a proof of concept for the thermodynamic computing paradigm, potentially revolutionizing the field.

What are the next steps for the development and adoption of this new computing technology?

The next steps include demonstrating similar principles in silicon, open sourcing the software, and encouraging global collaboration to program simulated and eventually real thermodynamic computers.

How does the speaker view the ultimate goal of their technology in the context of humanity's future?

The speaker sees the technology as a significant step towards expanding the scope and scale of intelligence throughout the universe, enabling humanity to solve complex problems and advance towards the stars.

What is the speaker's perspective on the importance of energy efficiency in the development of AI?

The speaker emphasizes that creating more energy-efficient computers that operate faster by fundamentally running AI algorithms differently is a game-changer, allowing for the application of intelligence to solve a wide range of problems.

Outlines

00:00

🚀 Quantum Computing to Thermodynamic Computing

The speaker reflects on the journey from the inception of an idea 8 years ago to the current state of a new computing paradigm. The journey began with quantum computing and evolved into the use of noise from heat as an asset in thermodynamic computing. The speaker's childhood fascination with the universe and a desire to build impactful technologies led to an exploration of theoretical physics and information theory. The realization that AI could be embedded into the physics of the world inspired the development of AI on quantum computers. The speaker and Trevor, presumably a colleague, recognized the issue of noise in quantum computing and digital computers and leveraged it for thermodynamic computing. This new approach to computing is believed to be more power-efficient and faster than current hardware. The speaker also discusses the shift from traditional binary representation in digital computers to a continuous variable model, which is closer to natural computation and human brain function. The concept of differentiable programming and the idea that gradient descent can be a superior 'programmer' is also mentioned. The speaker concludes by emphasizing the significance of this technology in expanding the scope and scale of intelligence in the universe.

05:02

🧊 Superconductivity and the Josephson Effect

The speaker discusses the process of manufacturing a superconducting chip, which is a key component in thermodynamic computing. The chip is compared to Tesla's first car, the Roadster, in terms of its pioneering role. The manufacturing process involves the creation of Josephson junctions, which are akin to transistors in superconducting technology. The speaker explains the necessity of superconductivity to achieve the desired effects, citing dissipation and the exploitation of the Josephson effect as reasons. The process requires cooling the sample to 10 Kelvin to achieve superconductivity. The speaker also highlights the need for extreme temperature control and isolation to prevent light from heating up the colder stages. The speaker expresses excitement about the potential of this technology and the future milestones, which include demonstrating similar principles in silicon and open-sourcing the software. The ultimate goal is to expand the scope and scale of intelligence and apply it to solve problems that prevent humanity from scaling to the stars.

Keywords

💡Quantum Computing

Quantum Computing is a type of computation that leverages the principles of quantum mechanics to process information. It has the potential to solve certain problems much faster than classical computers. In the video, the speaker mentions a journey through quantum computing, indicating it as a foundational step towards the new paradigm of computing they are discussing.

💡Thermodynamics

Thermodynamics is a branch of physics that deals with the relationships between heat and other forms of energy in a system. The speaker references the equations of thermodynamics to explain the universe's bias towards growth, which is a central theme in the video, suggesting a drive towards expansion and development of civilization.

💡AI on Quantum Computers

AI on Quantum Computers refers to the integration of artificial intelligence algorithms with quantum computing systems. The speaker and Trevor pioneered software and some hardware aspects of this field, which is crucial for the development of the new computing paradigm discussed in the video.

💡Noise in Quantum Computing

Noise in Quantum Computing refers to the random fluctuations that can disrupt quantum states and lead to errors in computations. The speaker identifies noise as a significant challenge in quantum computing, which they have to address by using it as an asset in their new computing approach.

💡Differentiable Programming

Differentiable Programming is a programming paradigm where the derivatives (gradients) of mathematical functions are used to automatically optimize these functions. The speaker mentions 'differentiable programming mindset' and 'software 2.0', which implies using gradient descent as a tool for programming, allowing the system to optimize itself for a given task.

💡Superconducting Chip

A Superconducting Chip is a type of electronic component that operates in a state of superconductivity, where it has zero electrical resistance and can carry electrical current without energy loss. The video discusses manufacturing such a chip, which is a significant step towards achieving the new computing paradigm.

💡Josephson Junctions

Josephson Junctions are components in superconducting circuits that can be considered the 'transistors' for superconducting technology. They are crucial for the operation of superconducting chips and are highlighted in the video as a key component in the manufacturing process.

💡Thermodynamic Computing

Thermodynamic Computing is a concept introduced in the video where the noise from heat and the jitteriness of electrons are used as assets to perform computations. This approach contrasts with traditional digital computing and is positioned as a more power-efficient and faster method for AI computations.

💡Continuous Variables

Continuous Variables in the context of the video refer to the way data is represented in the new computing paradigm, as opposed to the discrete binary system (bits) used in traditional digital computers. The speaker describes these variables as fuzzy values over a continuous spectrum, akin to a point cloud that can be shaped to represent data.

💡Optimization

Optimization in the video is related to the process of adjusting the parameters of a physical device to accomplish a specific task. It is central to the concept of differentiable programming and the self-optimizing nature of the new computing devices, which can program themselves to perform tasks more efficiently.

💡Expansion of Intelligence

The Expansion of Intelligence is a philosophical concept discussed by the speaker, which involves increasing the scope and scale of intelligence throughout the universe. The technology being developed in the video is seen as a step towards enabling this expansion, with the ultimate goal of solving complex problems and advancing civilization.

Highlights

The concept of a new paradigm of computing using quantum principles was conceived 8 years ago.

The speaker's lifelong aspiration has been to understand the universe to build impactful technologies for civilization's expansion.

The bias of the universe as described by thermodynamics pushes towards growth, which is seen as Nature's way of urging expansion.

At the age of 24, the speaker was deeply immersed in theoretical physics and information theory, viewing the universe as a large simulation.

The natural progression from embedding physics into computation led to the integration of AI into the world's physics.

The speaker and Trevor, coming from a quantum computing background, pioneered AI on quantum computers in software and hardware.

Noise in quantum computing and low-power digital computers was identified as a significant problem.

The innovation of using noise from heat and electron jitteriness as an asset in computing was a breakthrough.

Machine learning with a physics background led to the powerful idea of a physical system performing machine learning as physics.

The speaker spent 6 years trying to discredit the idea but couldn't, leading to the belief in its potential.

Digital computers represent values in bits, but this new approach is a fundamentally different way of programming, more akin to natural computation.

The new hardware is designed to perform AI more power-efficiently and faster than current hardware.

As circuits get smaller, voltage noise becomes problematic, but this new approach uses noise to drive computation, termed 'thermodynamic computing'.

The chips use continuous variables and fuzzy values over a continuous variable, likened to a point cloud shaped by data.

Differentiable programming, or 'software 2.0', allows a parameterized physical device to optimize itself for a given task.

The speaker discusses the manufacturing process of a key component called Josephson junctions, akin to transistors for superconductivity.

The manufacturing process involves a sophisticated setup, including cooling the sample to 10 millikelvin to achieve superconductivity.

The technology aims to significantly expand the scope and scale of intelligence throughout the universe by creating more energy-efficient and faster computers.

The speaker expresses excitement about the potential of this technology to fundamentally change the game in computing and AI.

The next milestones include demonstrating similar principles in silicon and open-sourcing the software for global programming.

The ultimate goal is to solve for the ultimate embedding of intelligence into the physical world, operating at the limits of physics.

Transcripts

[Music]

is it like surreal to be holding this

yeah I mean this idea came to me like 8

years ago was a whole journey through

Quantum Computing to get to this point

it's really the first step for this new

paradigm of computing that we think is

going to change the

[Music]

world ever since I was a child I wanted

to understand the universe around us

because I wanted to figure out what

technologies I could build that were

going to have maximal impact in our

ability to scale civilization to the

stars and if you study the equations of

thermodynamics there's actually a bias

of the universe that pushes us to want

to grow and I think it's kind of

Nature's Way of like urging us to

expand I was I think 24 or so I was just

immersed in theoretical physics

information Theory trying to

view the whole universe as a big

simulation if you're trying to embed

physics into computation it's very

natural for you to embed AI into the

physics of the

world so Trevor and I came from Quantum

Computing we pioneered how to do AI on

quantum computers in terms of software

and a bit of Hardware um and what we

realized was that noise was really the

bane of our existence in quantum

Computing and it's actually also a

problem if you try to go low power with

digital computers and so what these

devices do they use the noise from heat

the jitteriness of the electrons as an

asset rather than a

liability when I first learned machine

learning with a background in physics I

just inhaled all this content and it

felt like an idea was being being from

space and I was just like the conduit

for this idea it was so powerful you

know to have a physical system that does

machine learning as physics I thought

I'd lost my mind and I tried to

discredit the idea take it down for 6

years but I couldn't right I couldn't

convince myself that it wouldn't work

digital computers usually represent

values in bits and so it's completely

different way of programming and we

think it's really how nature computes

it's closer to how our brains compute

and it's a way to do AI much more power

efficiently and actually much faster

than than current

Hardware as circuits get smaller voltage

noise start to become un present and

affect the classical computation in

normal CPUs we're trying to use this

noise to drive the computation that's

what we call thermodynamic Computing

these chips in particular they don't

even use transistors and they're not

encoding data in zeros and ones they're

actually continuous variables and they

fuzzy values over this continuous

variable so you could think of like a

point cloud and I try to shape it in the

shape of my data and that process is

basically machine learning uh as a

physical process if you have a

parameterized physical device you can

just optimize over the space of those

parameters and let that Optimizer

program for you in a sense you just give

it a task and it optimizes itself to

accomplish that task so it's the

differentiable programming mindset Andre

karpathy calls it software 2.0 right the

thesis is that gradi and descent is a

better programmer than you so we're here

uh in sherbrook Canada this is our Fab

this first chip is actually a super

conducting chip which is hard to

manufacture at scale but for us it's

kind of a Tesla Roadster in the sense

that it it's the most efficient neural

information Processing Unit you could

create and it's going to open people's

minds about the performance

thermodynamic Computing can unlock and

so what we're going to see today is how

we manufacture a very key component of

these devices called uh Joseph's

injunctions which are kind of like our

transistor for super condu they're

they're really important component and

they're the hardest one to manufacture

uh this is where we do our fabrication

and we're going to do a deposition today

of Joseph's injunctions so we did the

exposure of Electron Beam resist

recently developed it uh and now we're

ready to do the deposition so this is

our uh deposition uh Tool uh it's from

angstrom engineering and we're doing a

deposition in the evaporation chamber so

what will happen is the the robotic arm

will move the the chip into the

evaporation chamber followed by

oxidation in the oxidation chamber and

then we're uh evaporating again and this

makes sure that we have this super

conducting layer uh insulating layer and

then Super connecting layer on

top and here our very first prototype

chips use a super connecting metals and

have programmable physics of electrons

in these metals that allow us to do

essentially AI as a physical process

it's going to be a lot of screws we're

using this refrigerator to cool down or

sample uh to 10 Melvin so that it

becomes super connecting in order to get

the effects that we want we need Super

connectivity this is for two reasons one

is dissipation and the other is because

we exploit a physical Effect called the

Josephson effect and for that to happen

the aluminum needs to be super

connecting we're at 273 3° C below 0 so

we're very close to absolute zero and

these cans are necessary to literally

isolate stages from the other to prevent

light from heating up these stages that

are colder and colder so just light on

its own uh will will impact the material

in the fridge it's kind of a bit of a a

Russian Dole system

[Music]

here I think the physics of all of these

circuits whether it's Quantum or

thermodynamic I mean it's both in the

end I just find physics very interesting

and that's why I'm a

physicist yeah look at this

[Music]

G beautiful that's

beautiful we just started getting our

first data finally got this whole setup

uh going as you can see it's quite

involved it's quite epic for us you know

this is an evolutionary Fork away from

uh previous Paradigm uh of quantum

Computing and and and towards uh Thermo

and so couldn't be more excited today to

be holding this really everything we

learn on this platform will carry over

to our next

ones it's going to open people's eyes

about you know this potential Paradigm

of computing that's totally different

you know the next big Milestones are

going to be demonstrate similar

principles uh in Silicon and of course

open sourcing our software and and

getting people from throughout the world

to start programming uh simulated

theramic computers at first and

eventually you know our real devices and

for them to get to try them uh and

hopefully at that point we check back

in to me I think the greatest Pursuit is

the expansion of scope and scale of

intelligence throughout the Universe

this technology we're building here here

is really a big step towards enabling

that expansion right if you can get far

more energy efficient computers that

operate much faster by fundamentally

running the AI algorithms differently in

the devices using exotic physics then

you've completely changed the game you

can apply intelligence to really solve

all the problems that perturb us and

prevent us from being able to scale to

the Stars I think it's really important

that we don't just fight for our place

in the dirt but seek to grow in scope

and scale and seek to the Stars solving

intelligence is solving a problem that

will solve problems for us right and

what we're seeking here is to solve for

the ultimate embedding of intelligence

into the physical world we're operating

at the very limits of physics here this

is what these devices demonstrate

they're as energy efficient as you can

make intelligence for

all we know

[Music]