I Failed This .NET Interview Question... So You Don't Have To

Summary

In this video, the speaker walks through a developer interview challenge involving performance optimization, concurrency, and race conditions in API requests. The task is to improve the performance of a currency conversion API by caching exchange rates to avoid redundant API calls. Through various iterations, the speaker refines the solution by introducing caching mechanisms, managing concurrency with semaphores, and addressing potential race conditions. While the solution works for single-instance applications, the speaker highlights the challenges of implementing such systems in distributed environments and the benefits of using external caching libraries like Redis.

Takeaways

• 😀 You can improve performance in API calls by implementing caching, especially for external APIs that are slow or rate-limited.
• 😀 A simple `ConcurrentDictionary` can be used for caching exchange rates but needs to be handled carefully with concurrency to avoid race conditions.
• 😀 Cache expiration is essential for preventing stale data. A timestamp like `createdAt` can be used to track when data was cached.
• 😀 To handle concurrent requests efficiently, using a `SemaphoreSlim` for locking critical sections ensures only one API call happens at a time for each currency code.
• 😀 Using a simple lock or semaphore can prevent multiple threads from calling an external API simultaneously, which is critical in high-traffic systems.
• 😀 A race condition problem known as the 'cache stampede' can occur when multiple requests trigger a new API call at the same time. This can be prevented with proper locking mechanisms.
• 😀 Instead of locking on a shared object for all requests, use a `ConcurrentDictionary<string, SemaphoreSlim>` to have locks specific to each currency code, reducing contention.
• 😀 You should avoid indefinite blocking when acquiring a lock; use a timeout to ensure that your program doesn't hang forever if it can't get a lock.
• 😀 With multiple instances of your application, in-memory caches like `ConcurrentDictionary` won’t work as they are instance-specific. A distributed cache like Redis is often used in production systems.
• 😀 Implementing caching from scratch is a useful exercise to understand the complexities involved in performance optimization and concurrency control in real-world applications.

Q & A

What is the main challenge the developer is trying to solve in this tutorial?

The developer is trying to solve performance and concurrency issues in a currency conversion API by introducing a caching mechanism. The goal is to optimize performance and handle concurrent requests effectively while ensuring the cache doesn't return stale data.

Why does the developer mention race conditions and locking in the context of this problem?

Race conditions and locking are mentioned because multiple concurrent requests could attempt to access and modify the same resource (exchange rate) at the same time. This could lead to inconsistent data or multiple API calls when only one is needed, which can be inefficient.

What kind of caching mechanism is the developer initially using?

Initially, the developer uses a `ConcurrentDictionary` to cache exchange rates. This ensures thread-safety for concurrent access, but it lacks an expiration time for cache entries.

What is the main limitation of the `ConcurrentDictionary` approach in caching?

The main limitation of using `ConcurrentDictionary` is that it doesn't handle cache expiration. So, exchange rates might remain in the cache even after they become outdated, leading to stale data being returned.

What solution does the developer propose to address cache expiration?

The developer introduces a `CacheEntry` record with two properties: the exchange rate and a `CreatedAtUtc` timestamp. This allows the program to check the age of the cache entry and determine whether it should be considered fresh or expired.

How does the developer handle concurrent requests when the cache is not yet populated?

The developer uses a `SemaphoreSlim` to prevent multiple concurrent requests from calling the external API at the same time. This ensures that only one request at a time can refresh the cache, avoiding redundant API calls.

What is the 'cache stampede' problem, and how is it addressed in the solution?

The 'cache stampede' problem occurs when multiple concurrent requests attempt to refresh the cache simultaneously, leading to multiple API calls for the same exchange rate. The developer uses a `SemaphoreSlim` to ensure that only one request updates the cache at a time, preventing this issue.

Why does the developer switch from using a lock to using a `SemaphoreSlim`?

The developer switches to using a `SemaphoreSlim` because it allows asynchronous operations (`await`), unlike `lock`, which can't be used with asynchronous code. This makes the solution more efficient and non-blocking for other requests.

What are the limitations of using `SemaphoreSlim` for concurrency control in this scenario?

One limitation is that `SemaphoreSlim` blocks all concurrent requests for the same currency code, even if they are unrelated. This means that requests for different currencies would still have to wait for the lock, leading to potential inefficiencies in high-traffic scenarios.

How does the developer propose to improve the lock contention for different currencies?

The developer proposes using a separate `SemaphoreSlim` for each currency code by storing them in a `ConcurrentDictionary`. This reduces contention because each currency has its own lock, allowing concurrent requests for different currencies to proceed without blocking each other.