Testing Is Bad For Developer Productivity

Summary

The video discusses how testing can negatively impact developer productivity if not treated as a first-class activity. Key issues include writing tests for complex legacy code, struggling to determine what/how to test, and long test runtimes. Solutions involve testing early, thinking deeply about requirements when writing tests, optimizing for test speed, and recognizing that testing is integral to development. The overall message is to make testing a priority, not an afterthought, to improve productivity.

Takeaways

• 🛠️ Writing tests for existing code is challenging due to lack of clarity on the code's functionality and the trade-offs made during its development.
• 📝 Writing tests as close as possible to the time of writing the production code, ideally beforehand, ensures better testability and understanding.
• 🔥 Tests that take a long time to run disrupt developers' flow and productivity, emphasizing the need for fast, efficient testing practices.
• 🤖 Thinking critically about what to test and how to test it is a crucial part of a developer's job, enhancing code quality and reliability.
• 📈 Treating testing as a first-class citizen in the development process is essential for maintaining high standards of quality and performance.
• 📱 Utilizing techniques like running tests in parallel and predictive test selection can significantly reduce testing time and improve efficiency.
• 👍 Prioritizing the development of tests that can run quickly and independently, without the need for complex setups or dependencies, enhances agility.
• 📚 Documenting assumptions and expected behaviors through automated tests serves as valuable, continuously validated documentation for developers.
• 🚀 Embracing a mindset of continuous integration and delivery requires integrating testing deeply into the development workflow for constant feedback.
• 💻 Making tests a core part of the development process ensures that changes do not inadvertently break existing functionalities.

Q & A

What is the main concern addressed regarding testing in software development?

The main concern addressed is that testing is often viewed as detrimental to developer productivity, leading to frustration and the feeling of not completing work efficiently.

Why is writing tests for existing code challenging?

Writing tests for existing code is challenging because developers may not remember the specifics of the code, including its intended functionality and the trade-offs made during its development, making it hard to know which tests to write.

What is the recommended approach to writing tests according to the video?

The recommended approach is to write tests as close as possible to the time of writing the production code, ideally beforehand, to ensure the code is easier to test and to avoid jeopardizing existing features.

What makes it hard to write tests, even when developers are ready and willing?

It's difficult to know what the tests should be, how to test the code effectively, and under which circumstances to test to ensure the code works correctly, requiring significant thought and planning.

How does writing tests benefit the development process?

Writing tests documents the expected behavior of the code under various conditions, assists in thinking through the code's functionality, and provides a continuously verified documentation through automated tests.

What issue do long-running tests pose to developer productivity?

Long-running tests interrupt the development flow by causing delays in feedback, leading to context switching and reduced efficiency.

What strategies can be employed to make tests run faster?

Strategies include pushing tests down the testing pyramid, making unit tests run quickly without external dependencies, running tests in parallel, and using predictive test selection to run a subset of tests.

Why is treating testing as a first-class citizen important in software development?

Treating testing as a first-class citizen ensures that tests are given priority, leading to fast-performing tests that enhance productivity and ensure that assumptions about code functionality are continuously validated.

What is the significance of thinking about what to test in the development process?

The thinking process involved in deciding what to test is crucial as it involves thorough consideration of the code's behavior and functionality, which is a fundamental aspect of a developer's job.

How does the video suggest supporting the continuous delivery channel?

The video suggests supporting the continuous delivery channel through Patreon, implying the value of financial contributions in sustaining the content and efforts of the channel.

Outlines

00:00

🤔 Writing Tests for Existing Code is Challenging

Paragraph 1 discusses the challenges of writing tests for existing code, whether written recently or long ago. Key points are that the code is often difficult to test because it wasn't written with testing in mind, and that it's hard to remember why decisions were made. Recommends writing tests as close as possible to writing the production code so the logic is still fresh.

05:01

🧐 Techniques to Speed Up Slow Test Execution

Paragraph 2 explores issues caused by slow test execution, which interrupts developer flow and productivity. Recommends techniques like moving tests down the testing pyramid, running tests in parallel, and using predictive test selection to reduce feedback loop time. Concludes that fast test execution should be treated as a first-class concern alongside test coverage.

Keywords

💡Developer Productivity

Developer productivity refers to the efficiency and effectiveness with which developers can complete their tasks. In the video, it is highlighted as being negatively impacted by certain testing practices. The script argues that while testing is essential, some approaches to testing can lead to frustration and a feeling of stagnation, thereby reducing the overall productivity of developers. Examples include writing tests for existing code without a clear understanding of its functionality, which can be time-consuming and counterproductive.

💡Writing Tests for Existing Code

This concept involves creating tests for code that has already been written, which can be challenging due to potential lack of documentation or understanding of the code's purpose. The video points out that this is particularly difficult when dealing with legacy code or code that was not designed with testability in mind. It emphasizes the importance of writing tests as close as possible to the time of writing the production code to avoid these issues.

💡Testability

Testability refers to the ease with which software can be tested. The video suggests that code should ideally be written with testing in mind to ensure it is easy to test. This involves considering how to isolate components for testing, how to input test data, and how to check outputs effectively. Improving testability can significantly enhance developer productivity by making the testing process more straightforward and less time-consuming.

💡Legacy Code

Legacy code is referred to in the video as code that has been inherited from someone else or written in the past, which now needs to be tested or modified. It poses a particular challenge because it may not have been written with modern testing practices in mind, making it difficult to test and maintain. The video highlights the importance of approaching legacy code with a strategy to improve its testability and documentation through testing.

💡Integration Tests

Integration tests are a type of testing that checks the interaction between different parts of a software system. The video mentions these tests as potentially time-consuming because they may require a specific environment or dependencies to run. It suggests minimizing reliance on integration tests by using unit tests where possible to improve the speed and efficiency of the testing process.

💡Testing Pyramid

The testing pyramid is a concept mentioned in the video that describes the ideal distribution of different types of tests within a project. It recommends having a large base of fast, isolated unit tests, fewer integration tests, and even fewer high-level end-to-end tests. This structure aims to maximize test coverage and efficiency while minimizing the time required to run tests.

💡Continuous Integration

Continuous Integration (CI) is a development practice where developers frequently integrate code into a shared repository, where it is automatically tested. The video highlights the role of CI in running tests regularly to ensure that the codebase remains in a good state. CI servers can validate assumptions and notify developers if a change has introduced errors, thereby supporting developer productivity by reducing the time spent on identifying and fixing regressions.

💡Test-Driven Development

Although not explicitly mentioned, the principle of Test-Driven Development (TDD) underlies the video's advocacy for writing tests before writing production code. TDD involves writing a failing test before writing the code to make the test pass, ensuring that code is designed to be testable from the start. This practice can lead to better designed, more reliable software and is implied in the recommendation to write tests as close as possible to the production code.

💡Automated Testing

Automated testing involves using software tools to run tests automatically, rather than manually checking all aspects of an application. The video underscores its importance by suggesting that documenting assumptions and expected behaviors in automated tests can serve as a form of live documentation that is verified by CI. This approach improves reliability and developer productivity by ensuring that tests are run consistently and that their results are immediately known.

💡Performance of Tests

The performance of tests, in terms of their execution speed, is highlighted in the video as critical to maintaining developer flow and productivity. Slow-running tests can interrupt the development process, leading to context switching and reduced efficiency. The video advocates for optimizing test performance through practices such as reducing dependencies, simplifying tests, and running tests in parallel, to ensure that they provide rapid feedback without breaking the developer's workflow.

Highlights

Introduction to the impact of testing on developer productivity.

Exploring the challenge of writing tests for existing code.

The difficulties in testing code not designed with testing in mind.

Advocacy for writing tests first to facilitate easier testing.

The importance of testing in close proximity to writing production code.

The challenge of knowing what tests to write and how to write them.

The critical role of thinking in the test writing process.

How documenting decisions through tests improves code understanding.

The issue of long-running tests disrupting development flow.

Strategies for making tests run faster.

The benefits of running tests on a local environment.

Techniques for efficient test execution, such as parallel testing.

The importance of treating testing as a first-class citizen in development.

Emphasizing the need for prompt testing close to development time.

Concluding remarks on the value of testing and thinking in development.

Transcripts

I don't have time for testing we've all

thought it we've probably all said it

and we certainly know of organizations

that think it in this video I want to

look at what makes testing bad for

productivity and what we can do about

[Music]

it hello I'm Trish G welcome to the

continuous delivery Channel if this is

your first time here please hit the

Subscribe button button and if you like

the video please press like now before

Dave kicks me off the channel before

I've even had a chance to get started I

want to cave out what I said in the

introduction about tests perhaps leading

to poor developer productivity there are

some practices and some patterns which

get in the way of us being productive as

developers linked to testing these kinds

of things can lead to frustration and

making us feel like we're not getting

the job done properly the first pattern

I want to explore is writing tests for

existing code now this existing code

might be something that we wrote a week

ago or it might be something that

someone else wrote 3 years ago certainly

the latter case is more challenging but

even the former case where we wrote it a

week ago there's a good chance we've

forgotten what we did and why we did it

writing tests for existing code is

challenging we don't always know or

remember what the code is supposed to do

we don't always know why it works in a

particular way we don't necessarily know

which decisions were made and which

trade-offs were made and so we don't

always know which tests to write for the

code what's usually more challenging

about writing test for existing code

particularly code that's not already

tested is that this code is usually very

difficult to test it's often difficult

to poke it with the kinds of values that

we want to poke it with and request

their answers from it in order to

ascertain what happened and did it work

correctly code which has not been

written with testing in mind is

difficult to test Dave and I would

probably argue that the thing you need

to do is write your tests first

therefore any code that you write

afterwards will by definition be easier

to test this isn't always a luxury that

we have when we're working on a legacy

code base so writing tests for existing

code is a challenging thing the key is

to write tests as close as possible to

the time that we write the production

code ideally perhaps beforehand but if

not beforehand then certainly close

enough after writing the production code

that we can remember what it does and

that if we need to make some changes to

the production code to make it easier to

test we can do that without jeopardizing

some feature that's been used by users

for 3 years or so the next pattern that

I want to look at is it is hard to write

tests even if we're ready willing and

able and we're about to sit down and

write the test for our code it's

difficult to know what the test should

be how to test the code and what should

the code do under certain circumstances

and which circumstances should we be

looking for in order to test that the

code works correctly under certain

circumstances it's hard it requires a

lot of thinking however thinking is our

job this is what we do thinking about

what the code does when the user does

something weird or what the code does

when the connection goes down that's

something we're going to have to do

whether we're writing a test for it or

whether we're writing the code itself

this is valuable thinking time that is

part of our job and if we do this

thinking during the time of writing the

test then presumably it will take less

time to think about it when we're

writing the production code on top of

that we get the added bonus of we start

writing down the questions that we have

what should it do under these

circumstances how should it behave when

something odd happens so we've written

those down in the form of an automated

test and documented the decisions we've

taken when the user puts a malformed

input in then the code should give this

kind of error message message when the

connection is down then the code should

do this so this is now documented it's

readable by other developers and

moreover this is documentation which is

run by a continuous integration server

regularly so the CI server is telling us

your assumptions are still correct and

the code still works the way you thought

it did the next thing that gets in the

way of us being productive as developers

when it comes to testing is tests that

take a long time to run this might be

integration tests test that require an

emulator tests that need to run in a

particular environment with a particular

setup or even local tests which just

happen to have a lot of dependencies and

have a lot going on a test that takes a

long time to run is not giving us the

feedback that we want from our automated

tests anything that takes long than

probably a minute is going to cause us

to get up go somewhere else do something

else or check our email generally

context switch so that when the result

of the test comes in we can't remember

what we were doing and what we're

supposed to do about this test result

whether it's a pass or will fail long

running tests kill our flow what we

really want to do is run the test check

our assertion make sure it works yes it

does great onto the next thing or it

fails okay I better do something about

that we want it to be part of our

development flow all the time we don't

want to break our flow to wait for the

result of a test what can we do about

this we should be making our tests as

fast as possible of course that means

pushing it down the testing pyramid

where possible turning integration tests

into perhaps a number number of unit

tests tests that don't require uh

databases network connections big

modules that have a lot of startup time

things that can run quickly a unit test

is not necessarily one test class for a

production code uh class but it is

something which should be able to run

very quickly these tests should also

ideally run on our laptop so again the

the focus should be trying to make it

run as quickly as possible maybe you're

going to use something like containers

to make your build more reproducible

ideally we don't even really want

something like that if your test is

validation for example you should just

be testing input and output and not

spinning up lots of different services

so where possible our tests should be

running as quickly as possible as far

down the testing pyramid as possible we

can also use techniques like running

tests in parallel where possible our

build tools usually support this where

we can run our tests in parallel um

there are all sorts of other techniques

things like using predictive test

selection to only run a subset of tests

that kind of thing there are things that

we can do to make our tests run as

quickly as possible and we should be

investigating those Solutions so that

we're not having to wait minutes hours

maybe even until the next day to find

the results of our automated test that

we spent so much time thinking about and

implementing overall the answer to the

problem of testing slowing us down as

developers is to treat testing as a

first class citizen it is part of our

job thinking about what the code should

do is of course what we do but since

we're going to be thinking about what

the code should do why not write that

down somewhere in say an automated test

where we can actually check that those

assertions and those assumptions are

correct all the time and when someone

makes some changes they haven't broken

some behavior that we expected we should

also be treating it like a first class

citizen in terms of performance this is

not something that should just kind of

like run you know maybe when we want to

and it takes as long as it takes that's

okay cuz it's over there no fast-

performing tests are what make us

productive so treat tests as a first

class citizen

write tests as soon as possible to the

time at which you write your production

code treat them as first class citizens

in terms of the performance of those

tests and never ever think that the time

that you spent thinking about what to

test is wasted because the thinking is

the thing that takes the time and the

thinking is the thing that we get paid

for anyway I hope you enjoyed this video

if you did please consider supporting

the continuous delivery channel on

patreon thanks

[Music]