The Value of Source Code

Philomatics

30 Sept 202417:46

Summary

TLDRThe video challenges the traditional view of source code as a valuable asset, arguing that its true worth is often much lower than expected. Drawing from real-world examples, the speaker explores the concept of 'Theory Building' in programming, suggesting that the real value lies in the understanding and mapping of the code to the real world, rather than the code itself. It emphasizes that documentation and source code can't fully capture this theory, and personal interaction is key to transferring this knowledge between developers.

Takeaways

💻 Source code often has little standalone value, especially in evolving projects where the code changes frequently.
🚀 Even well-written, clean code with good documentation is often less valuable than the thought process and real-world mapping behind it.
🔑 The true value in programming lies in the theory and mapping of the code to real-world problems, rather than the code itself.
🧠 Programming involves building a mental model or theory of how the real world maps to code, which is difficult to communicate through documentation alone.
📉 Code, even if stolen, quickly becomes outdated and is often more easily rewritten from scratch than modified by an outsider.
👷 Large companies are not concerned with using others’ code because it’s faster for them to build their own versions based on their needs.
📚 Documenting every decision or theory behind a program is impractical and can't fully convey the programmer’s experience and intuition.
🎯 Programming decisions are often based on intuition, experience, and context, which are not easily translatable to new developers through code or documents.
🤝 Effective knowledge transfer in programming requires close personal interaction, not just reading code or documentation.
🔧 There’s a need for better tools and environments that focus on the quick transfer of the mental models (theories) between programmers, especially for large or complex projects.

Q & A

What was the main argument presented in the video regarding the value of source code?
-The main argument was that the actual value of source code is often much lower than people think, even in large projects. The speaker suggests that the true value lies not in the code itself but in the theory or understanding of how the code maps to real-world problems.
Why did the speaker consider Microsoft using their code to be unlikely?
-The speaker found it unlikely that Microsoft or any other large company would use their code because the code was poorly written, changed frequently, and was difficult to understand. It would have been faster for a large company to build the software from scratch rather than decipher their code.
What is the 'Theory Building View' of programming as explained in the video?
-The 'Theory Building View' of programming suggests that the true output of programming is not the code or technical artifacts, but the understanding (or theory) of how the code maps to real-world situations. This theory is built up in the programmer's mind and cannot be fully captured in documentation or code.
Why does the speaker argue that beautifully written code still has limited value?
-The speaker argues that even well-written code with complete test coverage and documentation has limited value because it does not capture the full context and understanding (theory) of why the code is structured in a certain way. This understanding is tied to the programmer's experience and intuition.
What example does the speaker give to support the idea that source code itself is of limited value?
-The speaker mentions Panic, a company whose source code was stolen by hackers. Panic did not worry about competitors using the stolen code, as they believed it would quickly become outdated and irrelevant without the ongoing context and improvements made by their team.
What are the three intangible aspects of programming theory that cannot be fully expressed in code or documentation, according to the speaker?
-The three intangible aspects are: (1) the mapping of each piece of code to the real world, (2) the justification for why certain technical decisions were made, and (3) the ability to adapt the code to new requirements based on changes in the real world.
Why does the speaker believe it's often faster to rewrite code from scratch rather than modify existing code?
-The speaker argues that it's often faster to write new code from scratch because understanding and modifying existing code requires rebuilding the entire theory of the program in one's mind. This can be more difficult than starting fresh with a new theory that fits the programmer's own experience and understanding.
What challenges do companies face when onboarding new developers, according to the video?
-Companies face challenges in onboarding new developers because much of the value in a program lies in the theory that exists only in the original developer's mind. This theory is difficult to transfer through documentation alone and requires close personal interaction to fully understand.
How does the speaker compare learning to program to other skills like playing a musical instrument?
-The speaker compares programming to learning a skill like playing a musical instrument, arguing that it requires personal interaction with someone who already possesses the necessary understanding. Just as you cannot learn to play an instrument by reading instructions alone, you cannot fully understand code without guidance from someone familiar with its theory.
What suggestions does the speaker offer for improving the transfer of programming knowledge?
-The speaker suggests that programming tools like languages, IDEs, and version control systems could do a better job of helping transfer the theory behind code. For example, tools could provide better overviews or interactive sessions to help new developers quickly understand and modify existing code.

Outlines

00:00

💻 The Value of Source Code in Modern Projects

The speaker begins by challenging the viewer to consider the value of their source code, suggesting that it might be much lower than expected. He reflects on his experience working on a large-scale project that involved complex tech stacks like augmented reality (AR) and computer vision. Despite the advanced nature of the project, a new manager raised concerns about the security of storing source code on GitHub, which led the speaker to question the true value of source code, even for significant projects.

05:02

🔓 The Impact of Open-Sourcing Code

The speaker delves deeper into the implications of open-sourcing a project. Reflecting on his startup experience, he concludes that even if they had made their code public, it likely wouldn’t have impacted their business significantly. Codebases in startups change frequently, making them hard to maintain and understand, especially for outsiders. Even companies like Microsoft would find it faster to rebuild from scratch than to attempt to use his team’s evolving, complex, and often broken code.

10:03

💡 Programming as Theory Building vs. Production

The speaker introduces the concept of 'programming as theory building' by citing an article by Turing Award winner Peter Naur. This perspective contrasts with the more common 'production view,' which sees programmers as producers of technical artifacts like code and documentation. Instead, Naur’s view focuses on the theory or the mapping between the real world and the code in the programmer’s mind, which cannot be fully expressed through written documentation. This theory-building process is essential to understanding and maintaining software.

15:04

🤯 The Intangible Nature of Programming Knowledge

The speaker argues that certain aspects of programming knowledge are intangible and hard to capture in code or documentation. One key aspect is understanding how the real world maps onto the codebase. The speaker highlights that documenting every real-world factor or mapping decision would be impractical, especially in complex environments like a factory setting, where business processes are constantly evolving and hard to predict.

⚖️ The Trade-Offs in Programming Decisions

Expanding on Naur’s theory, the speaker explains that programming often involves making trade-offs based on experience and intuition. Documenting every technical decision or alternative path is not practical, as there are infinite possible ways to implement a solution. This highlights the challenge of onboarding new developers and the impossibility of fully capturing the reasoning behind every design decision in a written document.

🔄 Adapting Code to the Changing Real World

The speaker’s strongest argument is that the real world is constantly changing, and software must adapt to new requirements. He uses the example of a factory app where workers’ behavior changed after production, leading to a need for quick updates. Only developers who understand the underlying theory of the code can effectively implement these changes, as documenting every potential real-world scenario would be impossible.

🛠️ The Importance of Close Collaboration

Naur’s final point emphasizes that programming knowledge can only be transferred through close personal interaction. The speaker likens this to learning a skill like playing an instrument or writing, where written instructions alone are insufficient. New developers must work closely with those who understand the program’s theory to learn how it fits into the larger real-world context and how to manage unexpected modifications.

🚀 The Future of Programming: Theory-Driven Tools

The speaker concludes by addressing the original question: source code holds little value on its own because it lacks the context of the theory behind it. This explains why it’s often faster to rebuild software from scratch rather than modify an existing project. He suggests that programming tools, languages, and environments should evolve to facilitate the transfer of this theory between developers, and he teases his own ongoing research into creating such a language.

Mindmap

Keywords

💡Source Code

Source code refers to the human-readable instructions and statements written by programmers to create software. In the video, the speaker argues that while source code may seem valuable, its true worth is often overestimated because it quickly becomes outdated or difficult to reuse, as seen in the case of stolen code from the company Panic.

💡Theory Building

Theory building in programming, as introduced by Peta Nawa, suggests that programming is more about constructing a conceptual understanding (or 'theory') of how the code maps to the real world, rather than merely producing technical artifacts like source code. The video emphasizes that this theory is not easily transferable through written documentation but is critical for understanding and maintaining software.

💡Production View of Programming

The production view of programming is the traditional perspective that treats programming as a process of creating technical outputs (code and documentation) in response to requirements. The speaker contrasts this view with the theory building view, arguing that the former oversimplifies programming and ignores the deeper conceptual mapping required to make meaningful changes to software.

💡Code Obsolescence

Code obsolescence refers to the idea that code becomes outdated quickly as requirements change and improvements are made. In the video, the speaker illustrates this by explaining that even high-quality code, such as the stolen source code from Panic, becomes obsolete within weeks as new fixes and features are added.

💡Domain Knowledge

Domain knowledge is the deep understanding of the specific real-world context that the software is intended to address. The video argues that this knowledge, stored in the minds of programmers, is what makes their contributions valuable and difficult to replace, as it is intricately tied to the theory of the program, not just the code itself.

💡Technical Debt

Technical debt refers to the future cost of maintaining and fixing poorly written or hastily developed code. The video highlights how quickly-changing requirements and a lack of documentation often lead to technical debt, as was the case with the speaker's startup project, where code quality was poor and often broken.

💡AR (Augmented Reality)

Augmented reality (AR) refers to technology that superimposes digital information onto the real world, enhancing the user's experience. The video mentions AR as part of a complex software project that displayed virtual signs and information in real-world environments, showcasing the advanced nature of the system the speaker worked on.

💡Mapping

Mapping in programming refers to the process of associating pieces of code with specific elements of the real-world problem being solved. According to the theory building view, this mapping is essential to understanding and modifying the software. The speaker notes that without this understanding, new developers may struggle to maintain or extend the system effectively.

💡Software Onboarding

Software onboarding is the process of introducing a new developer to an existing codebase. The video argues that onboarding is often slow because much of the project's 'theory'—the conceptual understanding of how the code relates to the real world—cannot be captured in documentation, making it challenging for new developers to grasp the full picture.

💡Trade-offs

Trade-offs in programming refer to the decisions developers must make between competing solutions, each with its pros and cons. The video points out that documenting every decision and trade-off made during development would be impractical, as many are based on intuition and experience, contributing to why source code alone is not enough to understand a project.

Highlights

The value of source code, even for larger projects, may be much closer to zero than expected.

A personal experience working on an augmented reality project for mapping large buildings like malls and airports, which required custom-built solutions before modern AR kits.

A former manager's concern that storing code on GitHub could allow Microsoft to take the code and build a competing service was deemed unrealistic.

The realization that open-sourcing the project would have had minimal impact because the code was constantly changing, poorly documented, and complex to set up.

Panic, a well-known software company, had its source code stolen, but they weren't too concerned about competitors using it due to how quickly it became outdated.

Source code on its own is often not the most valuable asset, as it quickly becomes outdated and difficult to understand or maintain without the original programmer's knowledge.

The Theory-Building view of programming by Turing Award winner Peter Naur suggests that the true value in programming lies in the mental model or 'theory' a programmer builds to map the code to real-world problems.

The production view of programming, where programmers are seen as replaceable, is contrasted with the theory-building view, where programmers create valuable mental models that are difficult to transfer or document.

Theory-building emphasizes that the code is just a side product; the real output is the mental mapping between the code and the real world, which is often incommunicable through documentation alone.

The limitations of documentation: key design ideas, justifications for technical decisions, and adaptations to real-world changes are difficult or impossible to document fully.

New programmers need personal interaction with the original developers to acquire the theory behind the code, similar to learning a skill like playing a musical instrument.

The faster solution is often to write new code from scratch rather than modify existing code, as building the original theory in a new programmer's head is time-consuming.

The high turnover in the software industry is problematic because much of the value in a programmer's knowledge is tied up in their individual understanding of the code, which isn't easily replaced.

The example of a factory app demonstrates the complexities in business processes that cannot easily be documented, such as secret smoke breaks affecting data entry consistency.

Future programming environments could improve by optimizing for faster theory transfer between developers, potentially through more interactive or live onboarding methods.