eBPF’s Abilities and Limitations: The Truth - Liz Rice & John Fastabend, Isovalent

Summary

In this engaging session, Liz Rice and John Fastabend introduce and explore the capabilities of eBPF, a technology allowing custom programs to run within the Linux kernel. They debunk myths about eBPF's limitations, demonstrating its power by implementing the Turing-complete 'Game of Life' directly in eBPF. The discussion covers the evolution of eBPF, its current state, and its potential for complex tasks, emphasizing the vibrant community driving its continuous improvement.

Takeaways

• 😀 Liz Rice and John Fastabend are presenting on BPF (Berkeley Packet Filter), emphasizing its power and potential for custom kernel behavior modification.
• 🔧 BPF allows for running custom programs in the kernel, which can change the kernel's behavior by attaching to events, making it a versatile tool for system operations.
• 🖥️ The kernel is central to operating systems, involved in hardware interaction, file access, network communication, and process management, making BPF's impact extensive.
• 🛠️ There are numerous infrastructure tools built using eBPF, such as Cilium and Tetragon, which focus on networking, security, and observability.
• 🚫 Common misconceptions about BPF's limitations, such as its inability to handle complex tasks like layer seven packet parsing, are being challenged in the presentation.
• 🔄 The concept of 'Turing completeness' is discussed, highlighting that while BPF is not Turing complete, it is still capable of processing complex tasks within certain bounds.
• 🔒 The BPF verifier is crucial for ensuring safety when BPF programs are loaded into the kernel, checking memory access, control flow, and preventing indefinite loops.
• 🔄 BPF has evolved to include features like loops and timers, which enable more complex and long-running programs to be safely executed within the kernel.
• 💡 The demonstration of running Conway's Game of Life within BPF showcases the capability of BPF to handle complex, ongoing computations.
• 📈 The BPF community and ecosystem are continuously growing, with ongoing work to improve the verifier, compiler, and overall usability of BPF.
• 📚 Resources for learning more about BPF, including books and labs, are available, and the presenters encourage engagement with the community for advancing BPF capabilities.

Q & A

What is eBPF and why is it significant in the context of the kernel?

eBPF, or Extended Berkeley Packet Filter, allows custom programs to run in the kernel, enabling changes to the kernel's behavior by attaching these programs to events. It's significant because the kernel is involved in all hardware-related operations of an operating system, such as file access, network communication, and memory allocation, as well as managing system processes and permissions.

What is the role of the kernel verifier in eBPF programs?

The kernel verifier checks eBPF programs to ensure they are safe to run. It verifies that the program can only read and write to allowed memory, has valid control flow, does not get stuck on the CPU, and properly manages locks and references.

Why was the concept of Turing completeness brought up in the discussion about eBPF?

Turing completeness was discussed to address misconceptions about the capabilities of eBPF. It was used to illustrate that eBPF can, in theory, process any task that a Turing machine can, given it has the ability to process an arbitrary amount of data and time and store states.

What is the Game of Life and how does it relate to eBPF?

The Game of Life is a cellular automaton that evolves based on a set of rules determined by the state of neighboring cells. It was used in the script to demonstrate that eBPF is capable of running complex tasks, as it was successfully implemented within an eBPF program.

How has the eBPF verifier evolved to accommodate more complex programs?

The verifier has evolved to support loops, allowing programs to repeat operations without the risk of running indefinitely. It also supports callbacks and iterations, enabling eBPF programs to perform tasks over time without locking up the CPU.

What are the practical limitations of eBPF programs in terms of instructions and loops?

While eBPF programs were initially limited to 4,000 instructions, this limit has been increased to 1 million instructions in newer kernel versions. Loops are allowed as long as they can be verified to terminate, ensuring the program does not run indefinitely.

How can eBPF programs handle memory allocation?

eBPF programs can handle memory allocation through the use of array maps, which are memory blocks allocated by the user space and given to the eBPF program. The size of these maps is limited by the system's available memory and the user space's allocation limits.

What are some of the practical benefits of not being fully Turing complete in certain eBPF applications?

Not being fully Turing complete can be beneficial in scenarios where bounded execution is desired, such as in parsers where an upper limit on execution time can prevent infinite loops, or in systems monitoring where it's important to avoid long-running processes that could impact system performance.

What is the relationship between eBPF and infrastructure tools like Cilium and Tetragon?

eBPF is used in infrastructure tools like Cilium and Tetragon for networking, security, and observability tasks. These tools leverage the capabilities of eBPF to perform complex operations within the kernel, improving efficiency and performance.

What is the future outlook for eBPF in terms of development and community growth?

The future outlook for eBPF is positive, with continuous improvements and evolution of the technology. The community is growing, and there is a vision of pushing more processing into the kernel, which could lead to even more powerful and efficient applications in areas like networking and system monitoring.