From earning 10 LPA as SDE to 50LPA as Data Engineer | She switched from Software Engineer to Data

Summary

In this insightful podcast, Isha shares her journey from a Software Developer to a Data Engineer. She discusses the key skills required for success, such as DSA, SQL, big data concepts like Spark and Kafka, and system design for data pipelines. Isha outlines her structured approach to interview preparation, emphasizing certifications, hands-on practice, and consistent learning. She provides valuable advice for anyone looking to transition into data engineering, highlighting the importance of persistence, continuous improvement, and a passion for data-driven solutions.

Takeaways

• 😀 Transitioning from SDE (Software Development Engineer) to Data Engineer can be fulfilling with no regrets if you're passionate about working with data and its potential impact.
• 😀 Expressing interest in the data field and proactively upskilling with certifications (like DP-200 and DP-201) can help you make a successful transition, even from a fresher role.
• 😀 Gaining experience in different technologies, like backend development (Java, SQL, etc.), while building core expertise in data-related tools (Power BI, SQL, Azure) is crucial for moving into a data engineering role.
• 😀 Continuous learning and gaining hands-on experience, such as working on data pipelines and using Azure Data Factory, is essential for credibility and career growth as a Data Engineer.
• 😀 Certifications like Azure Fundamentals, DP-200, and DP-201 can provide foundational knowledge, but real-world application and consistency in learning are key to standing out in the data field.
• 😀 Salary hikes and role transitions often come with promotions in the data field, as seen with the transition from an SDE role with a CTC of 10 to a Data Engineer role with a CTC of 18.
• 😀 The learning curve in data engineering can flatten over time, making it important to look for new challenges or projects to keep progressing and avoid monotony.
• 😀 Preparing for technical interviews in the data field often involves a combination of DSA (Data Structures & Algorithms), SQL, system design, and big data concepts, all of which are essential for landing top roles.
• 😀 DSA is a significant filtering criterion in interviews, and preparing for it through platforms like LeetCode is crucial. Even if it feels unrelated to daily work, DSA is highly valued in the interview process.
• 😀 Time management during job transitions is key. Balancing office work with self-study (e.g., dedicating after-office hours to learning DSA, SQL, system design, and big data concepts) can lead to successful career transitions.
• 😀 Core skills required for Data Engineer interviews include advanced SQL (window functions, subqueries), big data concepts (Spark, MapReduce), real-time streaming (Kafka), and system design (building data pipelines).

Q & A

What was Isha's preparation strategy for data engineering interviews?

Isha followed a structured approach, dedicating her time to learning DSA, SQL, and system design over the course of 2 months. She initially focused on DSA, moved on to SQL, and later practiced system design once she felt confident in her foundational knowledge.

How did Isha balance her learning schedule?

Isha dedicated 4 hours of study every day, initially splitting it between reading theoretical concepts and practicing DSA. After a break, she focused on SQL and DSA in the evening sessions. Once proficient, she shifted to system design.

What are the key topics Isha recommends for data engineering interviews?

Isha suggests focusing on DSA (for filtering criteria), SQL (including advanced concepts like windowing functions), Big Data concepts (like Spark and MapReduce), real-time streaming (such as Kafka), and system design, particularly in building data-intensive applications.

Why is DSA important in data engineering interviews?

DSA is crucial because many companies use it as a filtering criterion to assess a candidate's problem-solving ability, algorithmic thinking, and efficiency in handling complex tasks.

What are some advanced SQL topics Isha mentions for product-based companies?

Isha highlights that advanced SQL topics such as windowing functions may be asked by product companies, which require deeper understanding and application of SQL concepts in large-scale data handling.

What are the major concepts related to Big Data that companies might inquire about?

Companies may ask about the journey of Spark, its optimization techniques, and MapReduce, including its historical significance and how it can be applied in data processing using languages like Scala.

How does Isha describe the importance of real-time streaming knowledge in interviews?

Isha emphasizes that companies might ask questions on real-time streaming technologies, particularly Kafka, if it's mentioned on your resume. This is important for roles involving continuous data processing and real-time analytics.

What system design topics are likely to be covered in a data engineering interview?

System design interviews for data engineers usually focus on designing scalable data pipelines and data-intensive applications that can handle large volumes of data efficiently.

How did Isha build confidence in her technical skills for interviews?

Isha built confidence by thoroughly practicing each topic: starting with DSA, progressing through SQL, and finally tackling system design problems. She consistently reviewed and tested her knowledge over a two-month period.

What are some common real-world tools or technologies that might be discussed in data engineering interviews?

Interviewers may inquire about tools like Kafka for real-time data streaming, Spark for big data processing, and other related technologies such as MapReduce, Scala, and various data pipeline frameworks.