Avoiding Overload in Distributed Systems | David Yanacek

Summary

In the 'System Under Skill 2021 Summary Edition', Facebook's Engineering Director, Yoon, hosts a virtual event focusing on system overload and resilience. David Dianichuk from AWS discusses strategies to avoid system overload, emphasizing load shedding and the challenges of queue-based systems. He introduces AWS Lambda's execution environments as a solution for predictable performance under load. Advanced techniques like priority queuing and shuffle sharding are presented to manage backlogs and ensure consistent customer experience, with resources for further reading available in the Amazon Builders Library.

Takeaways

• 😀 The event 'System Under Skill 2021 Summary Edition' was hosted by Yoon, an Engineering Director at Facebook, who highlighted the importance of virtual engagement due to the pandemic.
• 🏠 Yoon expressed a personal longing for face-to-face interactions, which have been limited due to working from home for over a year.
• 🌐 The online event aimed to facilitate deeper engagement through technical talks and Q&A sessions, indicating a positive trend in pandemic control.
• 🔍 The session focused on 'reliability and overload resilience', with David Janicek from AWS discussing techniques to avoid overload in distributed systems.
• 💡 David introduced the concept of 'load shedding', which is about rejecting excess work to maintain system performance and prevent wasted effort.
• 📈 He explained Amdahl's Law, illustrating how system throughput can be limited by contention, leading to increased latency and potential client timeouts.
• 🛠️ AWS SDKs implement a 'circuit breaker' pattern to prevent retries from exacerbating overload conditions, a technique that can be applied in other systems.
• 📉 David discussed the challenges of queue-based systems, where backlogs can lead to modal behavior and affect customer experience negatively.
• 🔁 He proposed using multiple queues and prioritization to manage queue backlogs effectively, allowing for better real-time processing.
• 🚀 AWS Lambda was highlighted as a service that simplifies overload management by providing each function invocation with a fixed amount of resources, ensuring consistent performance.
• 🔄 The concept of 'Shuffle Sharding' was introduced as a technique to isolate workloads and prevent a single customer's backlog from affecting others in a multi-tenant environment.

Q & A

Who is the host of the 'System Under' event in the provided transcript?

The host of the 'System Under' event is Yoon, an Engineering Director at Facebook.

What is the main topic of the event Yoon is hosting?

The main topic of the event is reliability and overload resilience, focusing on techniques to avoid overload in distributed systems.

Who is David Dianichuk and what is his role at AWS?

David Dianichuk is a Senior Principal Engineer at AWS, and he has been working there for about 15 years. He is currently working on AWS Lambda.

What does David discuss in terms of techniques to avoid overload?

David discusses load shedding and the susceptibility of queue-based systems to overload. He also covers the concept of Amdahl's Law and how it relates to system scalability.

What is Amdahl's Law as mentioned in the transcript?

Amdahl's Law, or the Universal Scalability Law, states that you can parallelize a system up to the point where contention becomes the bottleneck, after which no more throughput can be gained.

What is the significance of the bank teller example used by David?

The bank teller example is used to illustrate the concept of overload in a real-world scenario, where taking on too much work at once leads to slower service and potential customer dissatisfaction.

What is load shedding as discussed in the context of system overload?

Load shedding is the practice of rejecting excess work once a server reaches a point where it's starting to slow down, to maintain consistent latency for the requests that are served.

How does AWS Lambda handle overload according to David's explanation?

AWS Lambda handles overload by providing each function invocation with its own execution environment, which has a fixed amount of resources, ensuring predictable performance even under load.

What is the issue with queue-based systems during overload as described in the transcript?

The issue with queue-based systems during overload is that even after the load subsides, there can be a significant backlog in the queue that slows down message processing, leading to modal behavior and customer dissatisfaction.

What is the 'Shuffle Sharding' technique mentioned by David?

Shuffle Sharding is a technique used to isolate workloads by mapping each customer to a subset of queues, ensuring that one customer's backlog does not impact others.

What is the purpose of the Amazon Builders Library mentioned in the transcript?

The Amazon Builders Library is a resource where detailed articles on various AWS-related topics, including the techniques discussed in the event, are published for further reading and understanding.