My Project that got me into CRED and a $100k remote offer

Summary

In this video, the speaker discusses a personal project that played a significant role in securing both an off-campus 100k USD remote offer and an on-campus backend engineering role. The project involves creating a multithreaded proxy server with caching, implemented in C. The video covers key technical aspects such as memory management, threading, and implementing an LRU cache using linked lists. The speaker also shares insights on interview questions, challenges faced, and ideas for bonus projects like order management systems and stock market algorithms. This project showcases in-depth knowledge of computer systems, networking, and operating systems.

Takeaways

• 😀 The speaker discusses a personal project that helped secure multiple job offers, including a 100K USD CTC remote offer and a backend engineer role at a financial tech company.
• 😀 The project featured in the video is a multi-threaded proxy server with caching, implemented in C language. The server intercepts client requests and caches responses to improve performance.
• 😀 The proxy server checks if a request has been previously made by any client. If so, it serves the cached response rather than sending a new request to the server.
• 😀 The project utilizes key concepts such as memory management, Least Recently Used (LRU) caching, threading, and synchronization using semaphores and locks.
• 😀 The proxy server project involves using a linked list for LRU cache implementation, allowing for efficient storage and retrieval of responses based on client requests.
• 😀 The speaker emphasizes the importance of applying operating systems concepts, data structures, and threading to build a quality project that can be showcased in a resume.
• 😀 Interview questions related to this project included topics such as cache limitations, memory management in C (since there is no garbage collection), and differences between arrays and vectors.
• 😀 The speaker also provides bonus project ideas such as building an order management system using locks and semaphores, creating a stock market buy/sell order matching algorithm, and writing a custom VPN setup.
• 😀 Additional project ideas mentioned include creating a priority task scheduler based on CPU task scheduling algorithms used in operating systems.
• 😀 The speaker encourages viewers to experiment with these project ideas, adding their own unique twists to develop high-quality projects that can be added to resumes.

Q & A

What is the core idea behind the proxy server project described in the video?

The core idea of the project is to implement a multithreaded proxy server that intercepts client requests, caches the responses, and serves the cached responses for future identical requests. This improves performance by reducing redundant server requests and utilizing memory efficiently.

Why was C language chosen for implementing this project?

C was chosen for this project because it allows low-level memory management and provides the ability to implement concepts like LRU caching, memory allocation, and threading from scratch, which gives a deeper understanding of system-level concepts.

What is the Least Recently Used (LRU) cache and how is it used in the project?

LRU (Least Recently Used) cache is a data structure that stores a fixed number of items, evicting the least recently accessed item when the cache exceeds its limit. In this project, LRU is used to cache HTTP responses, ensuring that frequently accessed responses are quickly served while older or less accessed ones are removed when needed.

What is the role of semaphores in the proxy server project?

Semaphores are used to manage concurrent requests in the proxy server. They control access to shared resources, ensuring that only a limited number of clients can be served simultaneously. This helps in preventing race conditions and maintaining synchronization among multiple threads.

What are some challenges with cache synchronization in a distributed system, as discussed in the video?

One challenge with cache synchronization in a distributed system is ensuring that all containers in the system are in sync with each other. Since the cache is in memory and not stored in a database, if a container fails, its cache will be lost. This makes it difficult to maintain consistency across the system.

What interview question did the speaker struggle with regarding memory management?

The speaker was asked about garbage collection and how memory is cleared in C. The speaker had to clarify that in C, memory management is done manually, unlike languages like Java that have automatic garbage collection.

What is the difference between an array and a vector, as discussed in the video?

An array is a fixed-size data structure, while a vector is dynamic and can grow or shrink as needed. The speaker implemented a dynamic array-like behavior in C by resizing the buffer when it becomes full, simulating the behavior of a vector.

What are some bonus project ideas suggested in the video?

The bonus project ideas include an order management system using locks and semaphores, a stock market buy/sell order matching algorithm, setting up a Virtual Private Network (VPN), and implementing a priority-based task scheduler for CPU task management.

How does the proxy server handle incoming client requests?

When a client sends a request, the proxy server intercepts it, checks if the response is cached, and returns the cached response if available. If not, the server forwards the request to the actual server, caches the response, and sends it to the client.

What are some of the specific features of the proxy server's implementation discussed in the video?

The proxy server is implemented with multithreading, allowing it to handle multiple client requests simultaneously. It uses an LRU cache for efficient data storage and retrieval, handles HTTP GET requests, and employs synchronization mechanisms like semaphores and locks for concurrent request handling.