API Design in System Design Interviews w/ Meta Staff Engineer

Summary

In this video, Evan, a former staff engineer at Meta and co-founder of Hello Interview, explores API design within the context of system design interviews. He covers essential API concepts like REST, GraphQL, and RPC, explaining how they work, their differences, and their use cases. The focus is on client-server communication, efficient service-to-service APIs, and practical tips for interviews. Evan also highlights pagination, security considerations, and how to balance between speed and simplicity in real-world applications. The video aims to provide clear guidance on API design for system design interviews, offering key insights for candidates.

Takeaways

• 😀 REST APIs are the default choice for most system design interviews and are based on HTTP methods (GET, POST, PUT, DELETE) with resources represented as plural nouns in the URL.
• 😀 GraphQL offers flexibility by allowing clients to request only the data they need, reducing over-fetching and under-fetching of data. It’s ideal for mobile applications with slow networks.
• 😀 RPC (Remote Procedure Calls) are efficient for inter-service communication in microservices, using binary protocols like gRPC, which eliminates overheads like HTTP headers and JSON parsing.
• 😀 For external APIs, REST is almost always the right choice, but for internal communication between microservices, RPC is preferred due to its speed and reduced network traffic.
• 😀 In REST APIs, inputs come in three forms: path parameters (required), query parameters (optional), and request bodies (for sending data). Each has a specific use case.
• 😀 Always use plural nouns for resources in REST APIs (e.g., `/events`, `/venues`) and avoid verbs in the URLs. The HTTP method (GET, POST, etc.) conveys the action.
• 😀 Pagination is critical for handling large datasets in APIs. Two common types are page-based (offset-based) and cursor-based pagination, with cursor-based pagination being more reliable in the case of frequent data updates.
• 😀 In API responses, use status codes like 200 (success), 201 (created), 400 (bad request), 404 (not found), and 500 (server error). It’s best to group errors by these broad categories in interviews.
• 😀 GraphQL helps solve inefficiencies in REST by reducing the number of API calls needed to retrieve related data. However, it can suffer from the 'N+1 problem' in certain use cases, which can be mitigated with tools like data loaders.
• 😀 Security in APIs is handled via authentication (verifying the user) and authorization (checking if they can perform the requested action). Use JWT or session tokens for authentication, and always ensure sensitive data is not in the request body.
• 😀 For effective API design, focus on simplicity and avoid over-complicating things. Stick to clear, concise entity definitions and prioritize correct use of HTTP methods and parameters to streamline your design process.

Q & A

What is the main purpose of an API in system design interviews?

The main purpose of an API in system design interviews is to allow two software components (like a client and a server) to communicate with each other using a defined set of rules and protocols. The interview focuses on designing APIs for communication between these components, especially in the context of system and product architecture.

What are the key differences between REST and GraphQL in API design?

REST is built around standard HTTP methods (GET, POST, DELETE, etc.) and resources represented as URLs. It typically uses multiple endpoints to retrieve different pieces of data. GraphQL, on the other hand, allows clients to request exactly the data they need in a single query, reducing overfetching and underfetching. REST is often the go-to solution, but GraphQL is more flexible and efficient for complex queries, especially when dealing with multiple types of data.

What are the key HTTP methods used in REST API design, and what do they do?

The key HTTP methods in REST API design are: GET (retrieve data), POST (create new data), PUT (update or replace existing data), PATCH (partially update data), and DELETE (remove data). These methods are used to interact with resources represented by URLs in a RESTful system.

Why are resources in REST APIs represented as plural nouns?

In REST APIs, resources are represented as plural nouns because they define the data being acted upon. For example, 'events' represents all events. Using plural nouns ensures that the API reflects collections of resources and not actions or individual items. It also helps maintain clarity and consistency when designing the API.

What is the difference between path parameters, query parameters, and request bodies in REST APIs?

Path parameters are used to identify specific resources in the URL (e.g., `/events/123` for a specific event). Query parameters are used to filter or refine requests (e.g., `/events?location=LA&date=2023-01-01`). Request bodies are used to send data when creating or updating a resource, typically with POST, PUT, or PATCH requests (e.g., sending JSON data to create an event).

How does GraphQL solve the inefficiencies of REST, particularly for mobile clients?

GraphQL solves the inefficiencies of REST by allowing clients to specify exactly what data they need in a single request. This eliminates the need for multiple API calls in REST, which was problematic for mobile clients with slow networks. By asking for only the required fields, GraphQL reduces data transfer, improving performance and efficiency.

What is RPC (Remote Procedure Call), and why is it preferred for microservices communication?

RPC is a protocol that allows one service to call methods on another service directly, similar to calling a local function but over a network. It is faster and more efficient than REST, especially when using binary protocols like gRPC, which reduce overhead by sending compact binary data. RPC is preferred for internal microservice communication due to its speed and efficiency.

Why does REST use HTTP and JSON, while RPC uses binary protocols like protocol buffers?

REST uses HTTP and JSON because they are widely understood by clients and pass through firewalls easily, making it suitable for public-facing APIs. On the other hand, RPC uses binary protocols like protocol buffers for internal communication to reduce data size and increase speed, as it avoids the overhead of HTTP headers and JSON parsing.

What is the N+1 problem in GraphQL, and how is it typically resolved?

The N+1 problem in GraphQL occurs when fetching a list of items (e.g., events) and then making separate database queries for each related item (e.g., venues or tickets), leading to multiple additional queries. This can be inefficient. The solution is to use tools like data loaders to batch requests, reducing the number of database queries to just one per related item.

What is pagination in API design, and why is it important?

Pagination is used to break down large datasets into smaller chunks to prevent overwhelming clients and servers with huge payloads. It is especially important when an endpoint might return large amounts of data. There are two main types of pagination: page-based (offset) pagination, which uses page numbers, and cursor-based pagination, which uses a unique identifier (cursor) to retrieve the next set of data in a consistent order.

How do JWT (JSON Web Tokens) and session tokens differ in API authentication?

JWT is a compact, self-contained token used for authentication, where the user's data (e.g., role, expiration) is embedded in the token and signed to ensure its integrity. A session token, on the other hand, is a simple identifier stored on the client side that refers to session data stored on the server. While both methods are used to authenticate API requests, JWTs are often preferred for stateless authentication, whereas session tokens are used in more traditional stateful session management.