We write our applications in ebpf: A Tale From a Telekom Operator - Nick Zavaritsky

Summary

Nick from mnii explains the company's use of eBPF for efficient data packet processing in smart devices, such as smart mouse traps. They transitioned from using GPDK to eBPF, which aggregates GTP traffic into a single GRE tunnel, enhancing performance and handling complex scenarios. eBPF's ability to run multiple packet processing apps simultaneously is a game-changer, offering high throughput and flexibility. Challenges with state management and Kubernetes integration are addressed, showcasing eBPF's potential in modern network applications.

Takeaways

• 🐭 Nick's company, mnii, uses smart mouse traps that send push notifications when a mouse is caught, highlighting the importance of always-online connectivity.
• 📡 The smart traps are independent of Wi-Fi and use mui cement side technology for data shipping, emphasizing the shift from gpdk to ebpf for efficiency.
• 🚀 Easier data path is achieved by integrating AWS, allowing device traffic to enter the customer VPC directly without public internet exposure.
• 🔄 Transitioning from GTP to GRE tunnels is simplified using TC programs with BPF SKB get tunnel key helper, showcasing the power of BPF for packet processing.
• 🌐 The script discusses the complexity of handling multiple sessions and tunnels, where BPF's ability to aggregate traffic and manage state is crucial.
• 🛠️ BPF's flexibility is highlighted by its ability to run multiple packet processing applications side by side, a significant advantage over other methods.
• 📚 The script acknowledges the complexity of signaling in cell networks but chooses to focus on the data path for simplicity.
• 🔄 BPF maps are praised for their powerful API, complete type information, and the ability to build generic tools for state management.
• 💾 The script suggests that BPF can be used to populate maps from database tables, indicating its versatility in data handling.
• 🔒 Security is implied through the use of AWS integration and the handling of traffic within VPCs, ensuring data is kept private and secure.
• 🛠️ The script concludes that while DPDK is faster, BPF's ability to run multiple applications concurrently and its integration with Kubernetes is a game-changer.

Q & A

What is the primary function of the smart mouse traps mentioned in the script?

The smart mouse traps are designed to trap mice, and once a mouse is trapped, they send a push notification.

Why does the smart mouse trap not depend on Wi-Fi for connectivity?

The smart mouse traps are always online and do not rely on Wi-Fi because they have built-in cellular connectivity.

What is the significance of using eBPF (Extended Berkeley Packet Filter) in the context of the script?

eBPF is seen as a game-changer because it allows for running multiple packet processing applications side by side, which is crucial for handling the complexity of the service described.

How does the data path work for the smart mouse traps once a device is online and authenticated?

The data path involves a GTP tunnel that is terminated on the right-hand side, with traffic entering the customer's VPC through AWS integration without traveling over the public internet.

What is the role of AWS Transit Gateway in the data path described?

AWS Transit Gateway understands GRE (Generic Routing Encapsulation) and can match the tunnel to the customer VPC, facilitating the transfer of traffic.

Why is there a need to convert between GTP tunnels and GRE tunnels?

The conversion is necessary because the system has over 1 million GTP tunnels but uses a single GRE tunnel in collect mode to aggregate all GTP traffic, which is more efficient.

How does the TC (Traffic Control) program use eBPF to process the tunnels?

The TC program uses eBPF to aggregate GTP traffic into a single tunnel, match a tunnel to the customer, and redirect packets into a GRE device, all while handling complex corner cases like packet reassembly and fragmentation.

What is the advantage of using XDP (eXpress Data Path) in the packet processing?

XDP allows for fast packet processing at the earliest point in the network stack, matching a tunnel to the customer and finding the target GRE tunnel using shared eBPF maps.

Why is eBPF considered more flexible than DPDK (Data Plane Development Kit) in the context of the script?

eBPF is more flexible because it allows running multiple packet processing applications side by side, whereas DPDK is limited to a single task per CPU core.

How does the script's author propose to handle the challenges of state management and software updates in the context of eBPF?

The author suggests that eBPF maps, with their powerful APIs, can be used to build generic tools for dumping and restoring state, allowing for state extraction from a running instance and injection into a new one.

What challenges does the author face when trying to run eBPF-based applications in Kubernetes?

The challenges include the incompatibility of eBPF with virtual interfaces and the need for a secondary interface with a custom CNI (Container Network Interface) to ensure packets are processed as needed by the multiple packet processing applications.