Why you should get out of QA / Testing

Summary

In this video, John Miller, a former software engineer at Microsoft, shares insights from his 30 years in the industry, including time as a test developer and manager. He discusses the evolution of software testing, from its critical role during the 90s with large teams, to the current trend where testing teams are smaller or nonexistent. Miller explains why companies like Amazon now rely on quick deployment and rollback mechanisms rather than extensive testing, and advises software testers to transition into product development roles for better career stability and growth.

Takeaways

• 🛠️ Software testing and QA used to be a significant part of software development at companies like Microsoft, with large test teams.
• 📉 Over time, many test teams were dissolved, and some products no longer have formal QA teams.
• 🖥️ In the 90s, products like Windows NT had long development cycles and required extensive testing before shipping because there were no easy ways to patch software after release.
• 📦 Software was shipped as physical media (floppy disks, CDs), so it was crucial to get it right the first time.
• 📊 QA involved a detailed process with scoping, design specifications, and writing test cases to ensure the product met its design goals.
• 🛑 Stress tests and real-world deployments were key methods for finding bugs that affected customer use, providing the most value in QA.
• 🔄 As software development evolved, agile and continuous integration practices allowed teams to quickly release updates and roll back changes, reducing the need for large QA teams.
• 👩‍💻 Developers were increasingly responsible for writing their own tests and ensuring code quality.
• 💼 The shift away from QA roles reflects a broader industry trend where product developers are prioritized over QA engineers.
• 📈 QA is still important in certain industries, but for most software companies, faster, cheaper development processes that rely on developers for quality are becoming the norm.

Q & A

What was the role of software testing and quality assurance at Microsoft in the 1990s?

In the 1990s at Microsoft, software testing and quality assurance (QA) were crucial parts of the software engineering process. The test team was often as large as, or even larger than, the product development team, with more test developers and engineers than product developers.

Why did test teams in software development shrink over time?

Test teams shrank over time because the software industry moved towards faster release cycles, automated testing, and more reliance on developers to test their own code. The emphasis shifted from rigorous upfront testing to fixing issues post-release, enabled by the ability to quickly roll out updates and patches.

How did software development practices differ in the 1990s compared to today?

In the 1990s, software development was much more methodical, with long cycles (3-5 years for major products like Windows NT). Testing was extensive, involving multiple stages like scoping, design, and stabilization, as software had to be perfect when shipped since patches were not common. Today, software is developed and released much faster, with an emphasis on fixing issues post-release rather than exhaustive pre-release testing.

What challenges did the long development cycles pose during the Windows NT era?

The long development cycles for products like Windows NT meant that a significant amount of time was spent stabilizing the product and ensuring it was thoroughly tested. This extended the overall timeline, with stabilization and bug fixing sometimes taking more than a year after the code was considered complete.

What were stress tests, and why were they important in the testing process?

Stress tests were designed to push the system to its limits by running components repeatedly under heavy load. These tests helped discover timing issues and other system problems that might not appear under normal conditions. They were crucial for finding critical bugs that could affect system stability.

What were deployments or 'dogfooding' in the context of testing?

Deployments, also known as 'dogfooding,' involved using the software in real-world scenarios, as customers would. This helped developers understand how the product performed in practical settings and allowed them to identify usability and functional issues.

Why did the speaker, John Miller, move from test development to production development?

John Miller moved from test development to production development after realizing that test roles, while important, were often seen as less critical in the long term compared to product development. HR feedback and the career trajectory of senior test engineers indicated that developers producing code had more secure and rewarding career prospects.

What impact did research on bug resolution have on the perception of testing's value?

Research showed that most high-impact bugs were found through stress testing and deployments rather than detailed internal testing. This led to the realization that many bugs discovered through traditional testing were low priority, and fixing them caused more churn than benefit. This diminished the perceived value of extensive test development.

How did companies like Microsoft and Amazon handle testing in large, distributed systems after moving away from test teams?

After dissolving test teams, companies like Microsoft (Cosmos team) and Amazon relied on developers to ensure the quality of their own code. They focused on creating robust deployment and rollback mechanisms, which allowed them to ship products quickly and fix issues in production rather than relying on large test teams for pre-release testing.

What advice does John Miller offer to those currently working in QA roles?

John Miller advises those in QA roles to consider transitioning into product development, as QA jobs may not offer the same long-term career security and opportunities. He argues that coding in production is more valued across companies and makes employees less vulnerable during downsizing.