eBPF’s Abilities and Limitations: The Truth - Liz Rice & John Fastabend, Isovalent
Summary
TLDRIn this engaging session, Liz Rice and John Fastabend introduce and explore the capabilities of eBPF, a technology allowing custom programs to run within the Linux kernel. They debunk myths about eBPF's limitations, demonstrating its power by implementing the Turing-complete 'Game of Life' directly in eBPF. The discussion covers the evolution of eBPF, its current state, and its potential for complex tasks, emphasizing the vibrant community driving its continuous improvement.
Takeaways
- 😀 Liz Rice and John Fastabend are presenting on BPF (Berkeley Packet Filter), emphasizing its power and potential for custom kernel behavior modification.
- 🔧 BPF allows for running custom programs in the kernel, which can change the kernel's behavior by attaching to events, making it a versatile tool for system operations.
- 🖥️ The kernel is central to operating systems, involved in hardware interaction, file access, network communication, and process management, making BPF's impact extensive.
- 🛠️ There are numerous infrastructure tools built using eBPF, such as Cilium and Tetragon, which focus on networking, security, and observability.
- 🚫 Common misconceptions about BPF's limitations, such as its inability to handle complex tasks like layer seven packet parsing, are being challenged in the presentation.
- 🔄 The concept of 'Turing completeness' is discussed, highlighting that while BPF is not Turing complete, it is still capable of processing complex tasks within certain bounds.
- 🔒 The BPF verifier is crucial for ensuring safety when BPF programs are loaded into the kernel, checking memory access, control flow, and preventing indefinite loops.
- 🔄 BPF has evolved to include features like loops and timers, which enable more complex and long-running programs to be safely executed within the kernel.
- 💡 The demonstration of running Conway's Game of Life within BPF showcases the capability of BPF to handle complex, ongoing computations.
- 📈 The BPF community and ecosystem are continuously growing, with ongoing work to improve the verifier, compiler, and overall usability of BPF.
- 📚 Resources for learning more about BPF, including books and labs, are available, and the presenters encourage engagement with the community for advancing BPF capabilities.
Q & A
What is eBPF and why is it significant in the context of the kernel?
-eBPF, or Extended Berkeley Packet Filter, allows custom programs to run in the kernel, enabling changes to the kernel's behavior by attaching these programs to events. It's significant because the kernel is involved in all hardware-related operations of an operating system, such as file access, network communication, and memory allocation, as well as managing system processes and permissions.
What is the role of the kernel verifier in eBPF programs?
-The kernel verifier checks eBPF programs to ensure they are safe to run. It verifies that the program can only read and write to allowed memory, has valid control flow, does not get stuck on the CPU, and properly manages locks and references.
Why was the concept of Turing completeness brought up in the discussion about eBPF?
-Turing completeness was discussed to address misconceptions about the capabilities of eBPF. It was used to illustrate that eBPF can, in theory, process any task that a Turing machine can, given it has the ability to process an arbitrary amount of data and time and store states.
What is the Game of Life and how does it relate to eBPF?
-The Game of Life is a cellular automaton that evolves based on a set of rules determined by the state of neighboring cells. It was used in the script to demonstrate that eBPF is capable of running complex tasks, as it was successfully implemented within an eBPF program.
How has the eBPF verifier evolved to accommodate more complex programs?
-The verifier has evolved to support loops, allowing programs to repeat operations without the risk of running indefinitely. It also supports callbacks and iterations, enabling eBPF programs to perform tasks over time without locking up the CPU.
What are the practical limitations of eBPF programs in terms of instructions and loops?
-While eBPF programs were initially limited to 4,000 instructions, this limit has been increased to 1 million instructions in newer kernel versions. Loops are allowed as long as they can be verified to terminate, ensuring the program does not run indefinitely.
How can eBPF programs handle memory allocation?
-eBPF programs can handle memory allocation through the use of array maps, which are memory blocks allocated by the user space and given to the eBPF program. The size of these maps is limited by the system's available memory and the user space's allocation limits.
What are some of the practical benefits of not being fully Turing complete in certain eBPF applications?
-Not being fully Turing complete can be beneficial in scenarios where bounded execution is desired, such as in parsers where an upper limit on execution time can prevent infinite loops, or in systems monitoring where it's important to avoid long-running processes that could impact system performance.
What is the relationship between eBPF and infrastructure tools like Cilium and Tetragon?
-eBPF is used in infrastructure tools like Cilium and Tetragon for networking, security, and observability tasks. These tools leverage the capabilities of eBPF to perform complex operations within the kernel, improving efficiency and performance.
What is the future outlook for eBPF in terms of development and community growth?
-The future outlook for eBPF is positive, with continuous improvements and evolution of the technology. The community is growing, and there is a vision of pushing more processing into the kernel, which could lead to even more powerful and efficient applications in areas like networking and system monitoring.
Outlines
📘 Introduction to BPF and Session Overview
The session begins with an introduction by Liz Rice and John Fastabend, who express excitement about discussing BPF (Berkeley Packet Filter). John, an expert in eBPF with nearly a decade of experience, clarifies that eBPF allows custom programs to run within the kernel, influencing its behavior by attaching to events. The kernel's role in operating systems is highlighted, emphasizing its involvement in hardware interactions, file access, network communication, and process management. The speakers aim to debunk misconceptions about eBPF's capabilities, particularly its alleged limitations in handling complex tasks like layer seven packet inspection and its supposed incompleteness compared to general-purpose programming languages.
🔍 Exploring eBPF's Potential and Limitations
The speakers delve into the concept of Turing completeness, explaining it as the ability to process any arbitrarily complex task, data, or time without bounds. They contrast this with eBPF's design, which includes intentional limitations to ensure safety and performance, such as bounded execution times. Examples of both useful and less useful Turing complete systems are given, including the Game of Life, which is Turing complete but not necessarily practical for certain applications. The benefits of bounded properties in eBPF are discussed, such as preventing infinite loops in parsers or kernel functions, which can be crucial for maintaining system stability.
🛠 The Role of the eBPF Verifier in Safeguarding Kernel Programs
John Fastabend explains the role of the eBPF verifier, a component that ensures the safety of programs running in the kernel. The verifier checks for memory access permissions, control flow integrity, and ensures that programs do not get stuck on the CPU indefinitely. It also verifies that locks and references are properly accounted for. The discussion highlights the evolution of eBPF, including the addition of features that allow programs to sleep, handle callbacks, and iterate, which have expanded the capabilities of eBPF beyond the original model.
🎲 Demonstrating Game of Life in eBPF
The speakers present a live demonstration of Conway's Game of Life implemented in eBPF, showing that complex, Turing complete tasks can be executed within the kernel. The demonstration involves a sensor added to Tetragon, an eBPF project, which generates and evolves a pattern of 'cells' according to the rules of the Game of Life. The core logic of the game is briefly explained, including the copying of cell states, iteration of rules, and updates sent to user space for visualization.
🔧 Evolution of eBPF and Overcoming Instruction Limits
The discussion addresses the evolution of eBPF, particularly the increase in the number of allowable instructions from 4,000 to 1 million, to accommodate more complex programs. The importance of loops in eBPF is highlighted, as they allow for more sophisticated program logic while still ensuring termination. The speakers also touch on the challenges of writing eBPF programs that are verified by the kernel, emphasizing the need for synchronization between human programmers, compilers, and the verifier.
🌐 Future Prospects and Encouraging Community Engagement
The session concludes with a look towards the future of eBPF, emphasizing its continuous improvement and expansion. The speakers encourage the audience to engage with the eBPF community, suggesting that with enough interest and collaboration, even more complex and diverse applications can be developed using eBPF. They highlight the potential for eBPF to handle more networking-related processing in the kernel and mention resources for learning more about eBPF, including books, labs, and community projects.
Mindmap
Keywords
💡eBPF
💡Kernel
💡Turing Completeness
💡Verifier
💡Game of Life
💡Instruction Limit
💡Looping
💡Cilium
💡Tetragon
💡Memory Management
Highlights
Introduction to BPF (Berkeley Packet Filter) and its evolution to eBPF (extended BPF), allowing custom programs to run in the kernel and modify its behavior.
John's 10-year experience with eBPF and his role as an engineer in night surveillance, emphasizing his expertise in the field.
Explanation of the kernel's role in operating systems, particularly its involvement in hardware interactions, process management, and permissions.
Discussion on the capabilities of eBPF to change kernel behavior and its use in infrastructure tools for networking, security, and observability.
Misleading statements about the limitations of eBPF, such as its inability to implement complex layer seven packet inspection, debunked by the speakers.
Introduction of the concept of 'Turing completeness' and its relevance to eBPF's ability to process complex tasks and data.
Examples of Turing complete systems, including general-purpose programming languages and the Game of Life, to illustrate the concept.
Benefits of not being fully Turing complete, such as bounded processing for safety and performance in certain applications.
Live demonstration of running the Game of Life within eBPF, showcasing the capabilities of eBPF beyond traditional expectations.
Details on eBPF verifier's role in ensuring safety, memory access control, and preventing indefinite loops in eBPF programs.
Evolution of eBPF to support loops and iterations, overcoming early limitations and allowing more complex programs to run.
Comparison of eBPF instruction limits with real-world programs like Doom, Envoy, and Clang, highlighting the practicality of eBPF for large-scale applications.
Explanation of subprograms in eBPF and how they relate to function calls, enabling large programs to be executed within the kernel.
Discussion on the future improvements in eBPF, including better support for loops and increased ease of programming.
Practical applications of eBPF in cloud providers and distributions, indicating the growing adoption and integration of eBPF in the industry.
Resources for learning more about eBPF, including books, labs, and community projects, encouraging further exploration and contribution.
Final thoughts on the potential of eBPF, emphasizing its current capabilities and the ongoing efforts to expand its utility and accessibility.
Transcripts
good morning thank you so much for
coming to our session this morning just
about still morning right uh I hope
everybody's able to find a seat if you
have got a seat next to you maybe
shuffle along and and let people in uh
yes so my name's Liz rice uh myself and
John here do you want to say hi yeah hi
my name is John fman I'm engineer night
surveillance and excited to talk about
BPF today I've been you know working on
it for almost 10 years now so hopefully
you guys get as excited as um as I am
yeah
so I mean as John says he's he works in
the Kel right he is a proper expert in
ebpf let's just quickly recap to make
sure everybody understands what we're
talking about with ebpf it allows us to
run custom programs in the
kernel so we can change the way the
kernel behaves by loading ebpf programs
there and attaching them to events why
is the kernel interesting well the
kernel is the part of the operating
system that is involved whenever we're
doing anything with Hardware so if
you're accessing a file or sending
receiving over a network or uh even
allocating memory the Kernel's going to
be involved the Kernel's also um
managing the processes running on a
system so it's looking after things like
permissions and privileges so the kernel
is involved in pretty much everything
interesting with ef we can change the
way that it
behaves so already there are lots of
really great infrastructure tools built
using ebpf I mean we're both very
involved in psyllium and uh cium sub
project tetragon doing things like
networking security observability there
are also loads of other great projects
out there and great products out there
that are doing things in these kind of
infrastructure tooling spaces
but sometimes you'll hear statements
that give the impression that there are
limits to what you can do with ebpf so
for example we've heard people saying
yeah you can't really Implement like
layer seven paing that's just too
complex for ebpf or another thing that
you'll quite often hear people saying is
yeah but there are limits to what you
can do with ebpf because it's not cheer
and complete right so do we think those
statements are true well a hint here is
these things were not said by people who
are working on
ebpf we're going to explore these kind
of statements
today so I've said the term cheering
completeness this is not going to be a
computer science class it's not going to
be like super pedantic about
terminology but really what true
incompleteness about is about is can we
process some on arbitrarily complex task
can we process an arbitrary amount of
data can we continue processing for an
unbounded amount of time do we have the
ability to store States so that we can
continue sort of process state so this
is broadly speaking what true
incompleteness is about
right so I I think it's interesting um
just to talk a little bit about some
concrete examples right I mean we could
talk about the like mathematical
formulation of turn complete but that's
perhaps not interesting to everyone here
so like typically when we think about
turning completeness people really
talking about like your general purpose
languages C C++ Java python go all the
things that we um programmers love um
but I I also want to point out that
there's a lot of like really also
interesting but maybe less interesting
for programmers you know nobody really
wants to write um your uh application in
the one instruction set computer so
that's a totally valid turing machine it
is Turing complete but you know try to
write a proxy with just the move
instruction and it might not be quite as
fun as you want um another example is
The Game of Life that will go back to
but there's just loads of these things
and you know the the point is try not to
equate I think turn completeness um with
sort of usefulness as a programming
language to um you know developers right
um I also want to note that there's some
like actual benefits of not being fully
tur and complete sometimes um you might
be interested for example in having a a
parser um that is bounded and why would
you want that you might want that
because you don't want to receive a
packet and then have your parser Loop
forever really would like the thing to
stop at some point um you know it's the
same way like in BPF space where we do
like CIS call and tetron we'll watch CIS
calls or watch kernel functions right we
really like to have an upper bound on
the time that we're going to kind of
impact the application or the operating
system and when you do that you want to
say like it's really nice to say you
know this will not run longer than a
millisecond or you know 500 NCS or
something like this right and that
really is not touring Complete because
we're putting a bound on that
environment but um there you know
there's really concrete advantages to
that as well so just want to you know
kind of lay that out there and and kind
of give you some examples of of turning
complete things that are both useful and
not useful and an example of maybe why
you you know would want some bounded
properties as well
and true and completeness talks about an
infinite tape right and there is no
infinite tape in the world so in reality
we're always dealing with just large
rather than infinite yep
exactly so that previous slide mentioned
Conway's Game of Life and this is an
example of something that's cheering
complete so this is something that uh
every step it it goes through
generations every step and each of those
steps the state of one cell is
determined by it the state of the
previous state of the cells around it so
it's a little bit like an actual living
cell if the cell has no uh living cells
around it it's going to die of
loneliness if it has too many cells
around it that are alive it's going to
dive overcrowding but if it's got the
right number of cells uh around an empty
space the cell will actually uh come to
life so this is an example of something
that can kind of go on for ever
generating these
patterns very kind of common example of
something that's uh used as an
equivalent of cheing completeness yeah
so if this is an example of something
that's cheing complete is it something
that we can write in
ebpf and one reason why you might
think I don't think it can is because of
this thing the ebpf
verifier so when you load a program into
the kernel the verifier is going to run
o over this program and analyze the
program and establish whether it's safe
to run and one of the things I would
normally say when I talk about whether
it's safe to run I'll say it's not going
to crash the kernel but I would also say
that it's going to run to completion we
might pause on that
later you want to dive intoit more what
what it's really doing yeah yeah so I I
think um users of ebpf or or even people
that are in the stack you know they
might think about the verifier as this
kind of thing that is there stopping you
from writing interesting programs
perhaps um as a konel developer though I
want to kind of lay out what do we as
developers of the Kel that we want to
ensure that the Colonel's safe to run
and all this kind what do we think the
verifier is doing right and and
specifically this is kind of my criteria
um you know the first thing we always
want to do when we load a BPF program in
to the colonels we want to make sure
that we can only read memory that is
allowed by that program and by your
permissions that you have when you load
that program and and BPF has a whole
series of places you can hook in the
kernel you can hook kernel functions you
can hook the networking stack um you can
even hook user space and and depending
on where you hook you'll have different
permissions on what kind of memory you
can read right so we want to make sure
that your program is only reading memory
it's supposed to and we definitely 100%
want to make sure that you're only
writing to memory that you're allowed to
you know want to make sure that you're
not writing to random spots in memory
you're only writing to you know valid
locations in your vpf program um the
next one is we want to make sure that
the control flow is well formed and that
it's valid and in bounds we don't want
you to jump your BPF program off into
the kernel somewhere and run an
arbitrary code and crash the kernel
right don't want to make sure that
happens and I think the fourth Point
here is really interesting because we'
we've actually evolved this over time
it's like what does it mean for ebfp
programs to be bounded in the kernel and
if you think where the original BPF that
meant the program ran to completion but
over time we've sort of evolved BPF now
BPF programs know how to sleep we know
how to do callbacks and iterations we
know how to keep a program running and
we'll go into that in a little bit um
and really what we want to verify in the
verifier is that we don't get a Program
stuck on the CPU so long that it looks
and feels like the system is hung right
and for kernel developers or or people
that are really system level programmers
I think this is a sort of intuitive
concept because you've been writing in
this low level if you're if you're a
higher up in the stack and maybe an
application programmer or um kind of a
distrib systems programmer somewhere in
there it may be foreign because um you
don't have to worry about this stuff
because the OS is doing all the context
switching right you you never in never
write an application in user space C++ C
go whatever where you go like at this
point I need to make sure that I release
the CPU right because the the OS just
deals with that for you right once you
get yourself into the kernel into you
kind of the deep system stuff you really
need to um tell the OS when it's okay to
um to let your program off the CP view
right the reason we do this is really if
you're in an interrupt context if you're
uh trying to write some kind of locking
and stuff like this we can't just you
know release it um at any arbitrary
Point um and then the last thing we want
to do with the verifier is we want to
make sure like locks and references are
all accounted for and that that's sort
of a an accounting detail but it's
important if you're going to like
reference a socket or reference a file
um walk a path you want to make sure
that those things exist when you're
doing that and that's just a um kind of
a correctness property so th those are
the things um you know that
that I think about when I'm working on
the verifier or improving the verifier
um over time
so and I think as we'll see later this
idea that ebpf continues to evolve and
continues to improve over time because
of the work that people like John are
doing this is why we're able to perhaps
do more with ebpf than you might imagine
because it continues to improve yeah so
all right John can we run Game of Life
in ebpf all right we're gonna give it a
try here because we like to do demos so
what you see here is um this is just
basically tetragon um but we added a new
sensor to tetragon called Game of Life
all right and so we're going to kick it
off um what it did here is it just
iterate created the very similar to what
Liz just showed you with that um web app
is it created a a
map filled in some cells with some
random pattern and we could load it from
a BPF map or something if we wanted to
but for the for the demo it's just a
random map
and now it's running and it's there it
goes all right so while that's running
Let's uh dig into how you were able to
get this to to work
yeah so this is just a brief I'm not
going to go into all the details of
everything here but this is basically
the core um code that is running right
now in the background and we'll get back
to it but the important pieces I do want
to call out is that very far left for
you guys is do game this is our core
game logic for life here and you can see
kind of three things we copy the cell
map it's basically taking the Old State
copying it into the new state so that we
have a good State for the next iteration
we do the next iteration which is all of
those rules are run that Liz was talking
about you check to see if your um if
your neighbors are alive Al or you
create a new cell or you remove a new
cell so that's the sort of the business
logic of this program and then we send
an update all right so what the update
it does is it sends that map State up to
user space you know unfortunately you
know I don't have a a BPF Graphics
Library yet so you know if somebody's
interested and they have like a GPU
Shader from BPF or something to come
talk to me let's make it happen but I
couldn't do it from BPF so I had to send
the map up to user space slightly
disappointing but doesn't really impact
you know the the logic of the game right
so the user space is actually taking
that BPF map and just drawing it on the
screen there is what you were seeing and
then the last piece is this run next
which just reruns the business log but
goes back to the beginning of du game
and runs it again every two seconds all
right and we'll talk about how that
happens other slides you can get to in
in your in your spare time later if
you're
interested so you may have heard that
ebpf programs are limited to only a
small number of instructions so maybe
let's dive into that yeah um this is you
know what what's really interesting here
is you know when we first landed BPF and
414 um back in like 2015 kind of the
first
iterations um we really started at 4,000
instructions and like why did we pick
4,000 instructions because it seemed
like a good number it's you know Colonel
developers like you know like numbers
with k at the end of them 496 sounds
good right
um that was interesting but we very
quickly realized that you know 496 is
kind of small right you might want more
instructions um and the the other thing
I think is important
why do we have to have a limit was not
primarily because um we need the program
to terminate in some finite time and we
decided 4K instructions was a good
amount of time but I think it's really
if you think about it we have to verify
that program and we really want to make
sure that when you load a program into
the kernel it takes some reasonable
amount of time you know less than a
second right if you loaded a 20 million
um line code program perhaps maybe it
would take 20 seconds and you know we
kind of thought well a long time right
so keep it bounded but at one some point
we decided that 4K was just too limiting
people want to write things that are
bigger than 4K and so in 54 Colonels we
went to 1 million um and and kind of
beefed up that allow allowable
limit um just kind of follow up you
might wonder like that's interesting
John but give me some kind of reference
you know like how many instructions
really is a million instructions and so
what I did is just this is kind of a
napkin approach you know know go look at
a few programs I had on my laptop of
course I had Doom right got to have Doom
there so you know I took a look it's you
know less than a,k instructions for a
for a basic kind of Doom clone if you
think of the like the original Dune and
then I had Envoy on my system because um
you know we work on celium and tetragon
and that uses Envoy as a kind of one one
option for doing proxies I took a look
15 million instructions and then clang
uh you know compiler stuff 26 million
instructions so that's kind of where
we're at
um you know you might come back and say
okay Doom is only maybe less than a
thousand 100,000 instructions but John
it might have some Loops how are you
going to deal with that right and then
stay tuned we'll talk about how to get
these kind of much larger cases going
forward so there's also this idea that
and it was true in the early days that
you couldn't do loops in BPF because you
were only able to jump forwards in a
program yeah so this was something that
we put in and it not because necessarily
PF like um folks that were working on
BPF thought hey Loops are somehow bad it
was purely like the verifier needs to
somehow ensure that this isn't going to
run forever right um and that was the
early days and so we simplest thing to
do is look at your program and ensure
that you don't have any jumps to code
that you've already been to right we
want to make sure that the program
control flow is a d basically a graph
and no Loops in that graph um so early
PR programming what we did is we did
something called called an unroll
basically you tell the compiler I'm not
allowed to do loops please just cut and
paste this code right it'd be very
similar to a developer just doing like
cut cut paste their Loop many many times
you kind of do that but you can see very
quickly that you're not going to be able
to get things to to run very long right
especially in 4k instructions but even
with a million instructions you know
you're kind of only so much can be
done yeah so clearly not a not an
approximation of forever right when
you're kind of in this space um the next
piece here is we
have we as we were evolving BPF this
became a very obvious sort of limitation
right the fact that the limitation in
two way two ways I'd say is one is it
makes it hard to write programs and two
it makes it hard for the compiler to
compile things you know optimize your
code basically so we added Loops this is
this allows the verifier to look at a
loop and verify that that Loop is going
to terminate um and then we added an
actually helper call for cases where you
want the BPF program to do a loop but
the compiler human and verifier are
having trouble all agreeing on how that
Loop is going to look like actually and
if you think about the assmbly code and
object code when it's loaded in the
compiler and once you have this you can
write statements like that at the bottom
where you have a four Loop H here is a
um is a you know kind of a pound Define
or a global variable that tells you the
max loop size and and the verifier is
perfectly happy with this um today so
that's we're allowed to Loop but we're
not allowed to Loop indefinitely at this
point right correct yep that's why we
have that Max there and so if you just
kind of take a layout of what kernels
these were added in you know you 414 and
419 we had our kind of basic cut and
paste model 54 we added Loop support so
the colonel could figure out how if
these Loops are going to terminate 515
made a bunch of technical improvements
to that so the verifier got much better
at finding these things and then 61 we
added this um call that you can do to
basically you as the human can tell the
verifier through the compiler that hey
this is going to be a loop please take a
look at it and make sure that it
terminates yeah and I just wanted to
call this out because um you know we
have people working on this the the BPF
site is just evolving continuously so
there's another type of loop coming
we're not going to talk about it here
find me afterwards if you want but even
even uh more improved version of looping
is is on the way
so Okay so we've
got a limit to how many uh instructions
but it's quite a big limit we can Loop
yeah yeah
what about these big program what about
these big programs can we do Envoy or
Clan could you write in theory a $15
million 15 million instruction ppf
program uh that would be great right um
the thing that is interesting and I I
think it's it's a choice we made very
early on in the BPF um kind of verifier
is we call these things subprograms and
we said subprograms must terminate
roughly equivalent a subprogram is is
really roughly equivalent to a function
and so the limits are actually on the
functions not on the entire program and
as sort of a back a napkin you can do
like um 31 function calls or 31
subprograms is what a BPF programmer
would say and really that gets you up to
you know 15 million without much trouble
31 million actually with 31 calls right
um You can actually do more in the
kernel depending on where you're
actually being where you're hooking but
sort of as a rough estimate we can
pretty easily get to 31 million
instructions which is a really big
program right you can all of clay is not
even 31 million instructions so give you
some words of warning you know this uh
the verifier is getting better and
better and better and people are working
on this and making it usable for humans
um to write code right um but really the
trick right now is the human's going to
write the code the compiler's going to
compile it and the verifier is going to
verify it and they all need to kind of
be in sync and understand what's going
on right um to get the kind of something
into the kernel right and this is why
it's just not as easy as it should be
you know the team is working on this um
I fully predict that in 5 years 10 years
something like that BPF will be easier
and easier to write and and kind of open
up that that uh number of people that
are you know willing to dive in and
really get into the BPF
space um and then the last thing you
know if you do go this way and you maybe
have the right mentality it might be
addictive you might try to write things
like Tetris and doom and Quake and Game
of Life in the kernel um you know the
utility of that is um kind of great for
cubec con talks right I'm not sure you
know like is surveillant Enterprise will
have a or you know yeah we we're doing
the I surveillant Enterprise launch of
Game of Life yeah exactly you heard it
here first yeah yep um so Game of Life
you know it goes on forever so what
about this idea of not
terminating yeah so this is an
interesting one right so like if you
want something to run forever I've told
you there's a maximum instructions
you're like well certainly you're going
to run out of those instructions at some
point right and this really goes back to
this kind of carefully worded um point
in the verifier where we said well it's
not just that we want the program to
terminate it's that we want to release
the CPU that we're on we don't want to
lock a CPU down and like we said early
on early kernel versions that meant the
program terminated there was no
exceptions to that the program was done
um but when we evolved the um kind of
Kernel and BPF and the verifier and the
compiler as well like all these pieces
are moving moving forward we developed
these kind of other ideas of kind of Rel
other ways to release the CPU um one of
them was an iterator this is a program
that can like run over every file in a
directory right it doesn't matter how
many files are in there and it'll
basically ensure that the CPU doesn't
get locked up right it's been iterating
for too long it'll release let something
else run the scheduler will come back
and say okay BPF program you can run
again um we've allowed programs to sleep
now so this is really really useful if
like you're trying to read some memory
and it gets the page fold so the memory
is not in in the page cache so you can't
just read it because it's not there um
and so you have to do something called a
page fault which requires sleeping
anyways it's it's in the details but
super important if you're be program
tries to read some memory and it you
know you don't want to get a fault we
added timer call backs this is basically
a way to say hey I've done something
useful um I'm going to release my
program let the OS do something and then
just call me back at some time you can
even set this to zero which just looks
like hey I'm done um but you know I'm
safe to release but call me as soon as
you can again
so and I guess the last thing is you
know how can an ebpf program allocate
some amount of memory yeah so the um
there's really a bunch of different ways
we can do this but the most obvious one
is that we have these array maps and the
maps are basically memory blocks that
the program can all ate or user space
can allocate and then give to the
program and um these can be actually
quite large the basic limit of these is
you know like whatever the user space is
allowed to allocate based on its cgroup
if it's not in a cgroup they can be more
or less unbounded and kind of related to
system memory and all this kind of stuff
we have some limits but those are tied
to like how much RAM you have in your
system right same type of thing that a
user space application has you can't
allocate more memory than you know your
system has available right so so we
could pot entally have a pretty big
absolutely uh sort of field of play for
our game yeah yeah and you know even you
know four gigabytes we've sort of
optimized this case in newer kernels to
allow applications up to four gigabytes
of virtual memory um amazing stuff so
with all of these component pieces put
together we've really avoided the
limitations right we don't have uh we
well we don't really have an effective
limit on how many instructions we don't
have an effective limit on how many time
times we can call a loop effectively
because we can use these timer call
backs shall we see if it's still running
what do you think is it still
running yeah so see how good I I am at
programming right if it's still running
or if it's still going so here you go
looks like it's still up and running um
you can see it's still doing its thing
um and if you go back to kind of some of
the details um I think we can talk for a
couple seconds here basically if you
think about that first program that was
doing in that due game you know what we
see here is it does a copy of the last
state into the new state runs the logic
that's what moves all these kind of
fields around on the screen right and
then um sends it up to user space and
then this is the user space just kind of
printing the dots you know one's on zero
is off printing the dots on the screen
and you can see it kind of it keeps
evolving over time every every two
seconds is the limit um just so that you
all can see it
um so there it goes and it's it's still
going you told me a pretty interesting
thing about like the fact that you can't
predict the future State yeah yeah one
of the neat things about like game and
life and and a lot of terrain machines
in general is that this thing will um
you you if you want to know what the
thousandth iteration is or the 10,000th
iteration of this there's no shortcut to
that from a from a kind of a
mathematical side you can't say tell me
I'm not going to actually run this thing
10,000 steps let me just run it two
times and then tell me what the answer
is is so like it's interesting in that
sense is you have to you really have to
run this thing to figure it out right um
and uh we could rerun it and see what
pattern we get you know sometimes you'll
get interesting patterns this one is
still running um since it's a random
pattern we could have been unlucky right
and had like the screen be black when we
turned this on but um you know but it
didn't it work Round of Applause for a
live demo please
great perfect yep go back
amazing so we have Game of Life in ebpf
which is really a good demonstration
that you can today do arbitrarily
complex tasks with
ebpf so those statements about things
like can you do layer sing protocol
passing in
ebpf absolutely it's possible doesn't
necessarily mean that all these things
have been implemented yet but it is
possible to do you know pretty much any
processing maybe not necessarily to do
the display part yet but the actual
computation part the processing part and
and I would say the key is there not yet
I mean I I think the really interesting
thing or one takeaway I think from this
talk is like you know there's a bunch of
folks in the kernel in the compiler
community and you know we're working to
make BPF better and make BPF improve BPF
so if you have a use case and if your
use case is graphics on GPU for your
cubec con talk come talk talk to me it
would be interesting you know like so
these things can be we can make them
happen right you know in 10 years ago
when we got started we had like this
very small scope and you know it just
keeps growing and growing the community
is getting bigger you know it's kind of
its own sub Community with ebpf but um
you know I I think definitely if you
have a use case and you think there's
value in it you going come talk to us
and we can we can figure out what it
what it takes to make it happen and I
think when we think about selum you know
selum is using ebpf for lot of network
based processing and you know our vision
is to push more and more of that
processing into the kernel so yeah
there's tons of networking related
processing happening in user space today
where I'm not going to tell you that you
know we're writing it and it's going to
be released next week that's not what
I'm trying to say but I'm trying to say
the vision is that more and more of this
processing will be happening in the
kernel yeah and I I mean I I would also
just say absolutely the same for I I
work on tetron mostly these days it's
the same thing early we started on
tetron a couple years back you know
there was missing features we added the
features and then you you figure out how
to bridge the gap in the interum but if
you look forward in two three years you
can see like all the cloud providers
will have that kernel all the uh
distributions will have that kernel and
you can start to use that functionality
right and that's I think um a really
powerful kind of kind of point of the
open source effort
here so I hope that's convinced you that
ebpf is well probably more powerful than
you
thought if you want to learn more about
ebpf we have uh some books that you can
download from is surveillent.com there's
also a ton of really great Labs on the
website there is the ebpf doio site
which has loads of information about
ebpf itself and lots of projects that
are being uh built using it uh we're
also doing uh well I'm going to be
signing some books at the I surveillant
booth at 1:00 today so if you want to
come around and pick up a copy of the
book then uh please say hello and with
that I hope you have a wonderful wrap po
to the end of your
keepon
Посмотреть больше похожих видео
Looking Ahead: the eBPF Innovation Roadmap - Thomas Graf
We write our applications in ebpf: A Tale From a Telekom Operator - Nick Zavaritsky
AI for Embedded Systems | Embedded systems podcast, in Pyjama
Introduction to Creative Non-Fiction || Creative Non-Fiction || Quarter 1 Week 1
OSFC 2018 - coreboot rompayload | Ron Minnich
SAP RE-FX Overview (Flexible Real estate Management)
5.0 / 5 (0 votes)