10 Most Important Engineering Lessons Learned from 10 Years of Petabridge

Summary

In this video, Aaron Stanard, founder and CEO of PBridge, celebrates the company's 10th anniversary by sharing the top 10 engineering lessons he's learned over the past decade. He emphasizes the importance of avoiding complex, 'magical' solutions, the value of performance metrics, and the need for strong versioning practices. Key lessons include the significance of curiosity in developers, the inevitability of mistakes even in Microsoft's software, and the benefits of immutability and item-potent designs. Through practical advice, Aaron highlights how paying attention to small details can prevent software decay and ensure long-term success.

Takeaways

• 😀 Don't get cute with metaprogramming or reflection – it introduces complex, untraceable issues that can break applications, especially in serialization.
• 😀 If you can't measure it, you can't control it – performance metrics are essential to understanding and optimizing your application’s performance.
• 😀 Locality is crucial in distributed systems – reducing cross-machine latency can significantly boost performance, especially in real-time applications.
• 😀 Focus on improving multi-threading and async programming – this often yields greater performance gains than optimizing single-threaded execution.
• 😀 Version tolerance is an important skill – properly managing versioning of code, data, and APIs prevents your system from collapsing when changes occur.
• 😀 Developers should be curious about how things work – understanding the underlying systems helps solve complex problems and prevents creating technical debt.
• 😀 Even Microsoft makes mistakes – treat dependencies like Microsoft’s components with respect, report bugs, and expect fixes.
• 😀 Good code eventually becomes more functional – adopting functional programming principles (immutable data, pure functions) helps improve code maintainability.
• 😀 Idempotent designs simplify application behavior – designing operations to be idempotent (able to safely retry without side effects) leads to cleaner, more reliable code.
• 😀 Technical debt begins with small details – avoid neglecting seemingly insignificant issues in code, build, or documentation, as they can accumulate into bigger problems.

Q & A

What is the first lesson Aaron Stanard shares about engineering?

The first lesson is 'Don't get cute.' It emphasizes avoiding overly complex solutions like metaprogramming or reflection, as these introduce invisible moving parts that can lead to hard-to-trace issues, especially in areas like serialization.

What does Aaron mean by 'if you can't measure it, you can't control it'?

This lesson stresses the importance of benchmarking and performance measurement. If performance is crucial for your application, you need to quantify metrics such as latency, throughput, and memory consumption to effectively manage and optimize them.

How does Aaron describe locality in distributed systems performance?

Locality, in distributed systems, refers to minimizing latency. The closer your system components are (intra-process rather than cross-machine), the faster the interactions. Aaron suggests that to optimize latency, you need to control resource allocation efficiently within a process.

What is Aaron's advice regarding threading and performance improvements?

Aaron suggests that improving multi-threading and asynchronous programming (async/await) can yield more significant performance gains than focusing solely on single-threaded execution. Efficient task handling and eliminating flow control problems will result in better performance.

What is 'version tolerance' and why is it important?

Version tolerance refers to the ability to handle changes in software dependencies (like APIs or data formats) without breaking functionality. It’s crucial because it ensures compatibility as software evolves, allowing gradual changes without disrupting services or users.

Why does Aaron recommend developers to be curious about how things work?

Aaron advises developers to understand the fundamental concepts behind technologies, such as IPv4 vs. IPv6 or TLS handshakes. This curiosity helps when troubleshooting or solving problems, and makes a developer more valuable to the team.

How does Aaron view Microsoft's role in introducing bugs into software?

Aaron acknowledges that even Microsoft is not immune to bugs in their products like the .NET runtime or garbage collection system. He encourages developers to treat components from Microsoft (and others) as dependencies, report bugs respectfully, and work with maintainers to resolve issues.

What does Aaron mean by 'good code eventually becomes functional in style'?

Aaron believes that while object-oriented programming (OOP) is essential in learning, software development naturally evolves toward functional programming (FP) principles, like immutability and pure functions. These make code easier to reason about, test, and maintain.

What is an idempotent design, and why is it important?

An idempotent design ensures that performing the same operation multiple times has the same result as doing it once. This reduces complexity by eliminating the need for special handling of retries or failures. Aaron recommends this as a design goal for simplifying applications.

What does Aaron say about decay in software development?

Decay in software, or technical debt, begins with small details. Aaron advises developers to be diligent about details, such as treating warnings as errors and ensuring the quality of documentation. Small issues, if ignored, can accumulate and lead to significant problems over time.