Navigating Common Misconceptions of Transitioning from Manual to Automation

Summary

In this discussion, the speakers reflect on the early challenges and misconceptions of automation in the late 1990s. They emphasize how skepticism about its value, the assumption that automation could solve everything, and the belief that automation was a one-time effort were common. They share insights on proving the financial benefits of automation, learning to write maintainable and reusable scripts, and understanding that ongoing human oversight is crucial for successful automation. The conversation highlights the evolving understanding and strategic approach to automation over time.

Takeaways

• 🔍 Early automation efforts faced skepticism, with doubts about its value compared to manual testing.
• ⏳ One major concern was the time investment required for automation, which many believed wouldn't pay off in the long run.
• 📊 Proving the financial benefits of automation was crucial in gaining buy-in from management, showing significant time and cost savings.
• 🧩 A key lesson learned was that automation isn't a one-size-fits-all solution; not everything should or can be automated effectively.
• 🔍 Early misconceptions included the belief that automation could replace all manual testing, which proved to be overly optimistic.
• 🛠 Developing a strategy for automation was essential, particularly in creating smaller, reusable test scripts to ease maintenance.
• ⚙️ Maintaining automation scripts is an ongoing effort; changes in the application can require frequent updates to the automation code.
• 🚦 It’s important to have a dedicated testing environment for automation to avoid interference from other ongoing testing activities.
• 💡 Waiting to automate areas of an application that are still undergoing frequent changes can prevent unnecessary rework and maintenance.
• 🔄 Automation requires ongoing human oversight, particularly in maintaining and updating scripts as the application evolves.

Q & A

What was one of the main fears expressed by the speakers when they started with automation in the late 90s?

One of the main fears was that automation would take too much time initially, and the upfront work required would not be worth the effort on the back end.

How did the speaker demonstrate the value of automation to their company?

The speaker demonstrated the value by showing that automation reduced the time needed for a quarterly release from a week to three days, which also decreased the cost significantly.

What misconception did people have about automation that the speaker had to overcome?

People believed that automation would not provide any real value and that the initial effort to automate was not worth the time and effort.

How did the speaker prove to their company that automation was worth the investment?

The speaker showed that by automating a significant portion of a large Excel spreadsheet, they were able to reduce the time and manpower required for a quarterly release, proving the financial benefits of automation.

What was the initial approach to writing automated tests according to the speakers?

Initially, the speakers created large tests with many conditions, which made it difficult to identify which part of the application was failing when tests did not pass.

What did the speakers learn about test strategy as they began to automate?

They learned that each automated test should result in one outcome for one functionality, leading to the adoption of atomic tests and a more strategic approach to test automation.

Why did the speakers emphasize the importance of having a separate environment for automation?

Having a separate environment ensured that tests were not broken by changes made by others in the shared testing environment, which was crucial for maintaining the reliability of automated tests.

What was a key lesson the speakers learned about the maintenance aspect of automation?

They learned that automation scripts need to be written in a way that allows for easy maintenance, such as creating reusable test chunks that can be updated in one place rather than in multiple tests.

What misconception did the speakers address regarding the ability to automate everything?

They addressed the misconception that automation can replace all manual testing, learning that it's more about having a strategic approach to what should be automated and what should remain manual.

How did the speakers approach the issue of changes in the application affecting automated tests?

They learned to create smaller, more focused tests and to wait to automate areas of the code that were likely to change, reducing the need for constant script maintenance.

What advice would the speakers give to someone new to automation based on their experiences?

They would advise focusing on strategic test automation, maintaining separate environments for testing, and writing maintainable scripts with reusable components to handle changes in the application.

Outlines

00:00

😨 Overcoming Early Fears and Misconceptions in Automation

The speaker reflects on the early challenges and misconceptions surrounding automation in the late 90s, particularly the skepticism about its value and the belief that it would require too much upfront time and effort. They share how they were able to demonstrate the financial benefits and time savings of automation by reducing the time required for quarterly releases from a week to just three days. This success helped convince the company of the value of automation, despite initial resistance. The speaker emphasizes the importance of proving both the effectiveness and economic sense of automation to gain managerial buy-in.

05:01

🤖 Realizing the Limits of Automation

The speaker discusses the initial misconception that everything could be automated, leading to overly complex and error-prone tests. They learned that effective automation requires a well-thought-out test strategy, where each test should be focused on a single functionality. Collaboration with manual testers and test leads became crucial to determine which areas of an application were worth automating. The speaker also mentions the need for a separate, stable environment for automation to avoid interference from ongoing development, highlighting the importance of smaller, reusable test components to facilitate easier maintenance.

10:02

🛠 The Ongoing Need for Maintenance in Automation

The speaker categorizes three major misconceptions about automation: the initial belief that automation wouldn't work, the assumption that automation could solve everything, and the idea that once automated, no further attention would be needed. They emphasize that automation requires continuous human oversight, particularly in maintaining and updating tests as the application evolves. The speaker advises waiting to automate areas of code that are still undergoing significant changes to avoid excessive rework. This underscores the balance needed between automating efficiently and ensuring that the automated scripts remain reliable over time.

Keywords

💡Automation

Automation refers to the use of technology to perform tasks that were previously done manually. In the context of the video, automation is discussed as a transformative approach in testing processes, particularly in software development. The speakers mention initial skepticism about its value, challenges in implementation, and the realization that automation is not a one-time effort but requires ongoing maintenance and strategic planning.

💡Manual Testing

Manual testing involves human testers executing test cases without the help of automation tools. The video contrasts this traditional approach with automation, highlighting the initial reliance on manual testing and the industry's eventual shift towards automation. The transition from manual to automated testing is a central theme, reflecting broader changes in the software development process.

💡Misconceptions

Misconceptions refer to incorrect beliefs or assumptions. The video discusses several misconceptions about automation, including the idea that it wouldn't be valuable, that it could automate everything, and that it was a 'one and done' process. These misconceptions were barriers to adopting automation and were only overcome through experience and demonstration of automation's long-term benefits.

💡Test Strategy

A test strategy is a plan that outlines the testing approach, resources, schedule, and scope. In the video, the speakers emphasize the importance of developing a solid test strategy when implementing automation. They learned that effective automation requires careful planning to ensure that each test focuses on a specific functionality and that tests are maintainable over time.

💡Maintenance

Maintenance in the context of automation refers to the ongoing effort required to keep automated tests functional as the application evolves. The video highlights that automation is not a set-and-forget solution; instead, it requires continuous updates to accommodate changes in the application. Proper maintenance is crucial for the long-term success of automation efforts.

💡Regression Testing

Regression testing is a type of testing that ensures that new changes or updates to an application do not negatively affect existing functionality. The video mentions the importance of automating tests in areas that are likely to change to prevent regression issues. This type of testing is a key reason for adopting automation, as it helps maintain software quality over time.

💡Reusable Test Scripts

Reusable test scripts are pieces of code or tests that can be used across multiple test cases or scenarios. The video explains that creating reusable scripts helps reduce maintenance effort by allowing a single change to propagate across all relevant tests. This concept is central to efficient and scalable automation practices.

💡Financial Impact

The financial impact refers to the cost savings or economic benefits of implementing automation. In the video, one speaker describes how they demonstrated the value of automation by showing the company how it reduced the time and resources required for testing, leading to significant cost savings. This financial argument was crucial in gaining management's support for automation.

💡Testing Environment

A testing environment is a controlled setting where tests are executed. The video highlights the need for a separate testing environment for automation, as having a clean, stable environment ensures that automated tests are not disrupted by ongoing changes in the main application. This was a lesson learned through experience, emphasizing the importance of an isolated environment for reliable automation.

💡Conditional Statements

Conditional statements are programming constructs that execute different actions based on whether certain conditions are met. The video discusses the use of complex conditional statements in early automation efforts, which led to difficulties in troubleshooting and maintaining tests. The speakers learned to simplify their test scripts to make them more manageable and less prone to errors.

Highlights

Overcoming initial fears and misconceptions about automation in the late 90s.

The challenge of proving the value of automation to skeptics within the company.

The misconception that automation would take too much time and not be worth the effort.

Demonstrating the benefits of automation through a quarterly release scenario.

Reducing the all-hands-on-deck period from a week to three days through automation.

The importance of showing the financial benefits of automation to gain management support.

The common misconception that automation could replace all manual testing.

Learning the necessity of a clear test strategy for effective automation.

The realization that not all tests need to be automated due to their complexity or lack of change.

The need for a separate, clean environment for automated tests to avoid interference.

The evolution from writing large, complex scripts to creating smaller, reusable test components.

The importance of maintenance in automation and the need for scripts that are easy to update.

The learning process involved in determining which parts of an application should be automated.

The strategic approach to automation, including when to hold off on automating certain areas.

The role of human supervision in ensuring the effectiveness and accuracy of automated tests.

The economic impact of automation on reducing costs and increasing efficiency within a company.

The evolution of the speakers' understanding of automation from a tool to a strategic business asset.

The importance of aligning automation efforts with the overall business goals and objectives.

Transcripts

[Music]

talked about the fears you talked about

some of the things that you you try to

overcome and it looks like your boss

told you hey that must have been one

fear like hey go ahead and get this

thing going or you know we're done here

uh but then then what along with those

fears what were the misconceptions of

automation back then and like what what

is it and I'm I'm guessing there is a

lot of correlation to what we're seeing

today with AI in testing but we're not

going to go there yet I'm just curious

like back in n in late 90s when you both

were jumping into into the deep end of

automation what uh are some of the

things that that people misunderstood

that ended up being more like uh than an

actual myth or or something that I

didn't become reality

uh for me personally because it was new

and people had just been doing manual

testing I just don't think anyone

believed that there was going to be any

real value that was going to come from

it

and then there was also

some of the

that automating would take too much time

like the initial upfront work that would

would need to be done would not be wor

on on the back end right and so it was

really just

getting um you know managers and stuff

like that to see that it was it was

worth the time and effort and one way

that I was able to prove that

was we used to do quarterly releases and

when we did this quarterly release it

would be like an all handson deck within

the company for like a week and I was

able to show that by automating we had

this huge Excel spreadsheet that had all

of these things that needed to be done

in order for it to be you know deemed

like good good enough to go for the

quarterly release I had started what

once I had gotten the green light to go

and and stuff like that I had gotten um

a fair amount of that spreadsheet

automated and I was able to show the

company um you know I was able to kind

of break down all right this is roughly

how much you're paying for that week of

all Hands-On deck because it wasn't just

the QA we were bringing in other members

of the company you know analysts and

implementation Specialists and stuff

like that and I was able to show you

know hey we able to that down to three

days there was by me being able to

automate a certain percentage of that

spreadsheet that you had

uh we only need two tester that was

handling some of the newer um stories

that had been brought in that had not

been automated yet and I was able to

show but look how much we were able to

cover with just the tool so that was

probably the biggest thing that was I

was able to do to kind of get them to

see hey this is worth it but that but

getting them to to Really buy in for

that initial part of me kind of being

left alone

because I didn't do any manual testing

my job was just just to do everything I

could to automate you know what we had

and once I was able to prove that from a

financial standpoint and show them um

that we're able to do it faster and for

less money that was really the

big uh impact that allowed us to be able

to like to make it like a huge part of

who we are now at our company with the

automation so it it does look like you

had not only the hey I have to show that

these work but also I have to show that

these is e like the economics of these

make sense yeah right otherwise I mean

the investment wasn't going to be worth

it regardless of how fast you could do

it yeah and I you know with a business

and that's ultimately what they're going

to want to see a lot of times right I

mean it really comes down to how can we

save the company money by by doing this

so yeah that was it for me Alex I don't

know what you have but what what

misconceptions did you face Alex when

you were trying to I mean there is a

technology I don't recall the name right

now that both of you mentioned at at the

very beginning but um as you were

learning and implementing these things

did you did you find anything that uh or

or was there any objections really when

you were trying to to adopt it or

implement it yeah so um I'd say um the

common misconception at the time when I

was starting was that uh we could

automate everything so right it's like

oh great so there this tool can you know

do like can run everything that we're

doing manually and and it can basically

like we'll be able to test everything

super quick and then initially as we

were learning as well like yeah I think

so so we'll we would create tests and

and and Dave you'll probably relate to

it but like we would create tests like

huge tests full of conditions like if

then if then else and then choose this

and choose that and then and it worked

you know it could make it work but then

very quickly when there were failures in

the application that caused the tests

the automated test to fail we would go

like huh okay so why did it fail like

which functionality actually is not

working because our tests automated so

many things that uh we had we literally

had to trouble not troubleshoot but

triage the test itself

to find which part of the application

was throwing the error and so very

quickly we learned okay automation is

not just about automation it's there's

more about test strategy than actually

automating right so then we very quickly

learned that okay each automated test

should result uh in one or should create

one result right for one functionality

uh uh like Atomic test basically right

and uh and then that helped us uh Drive

the test strategy for the overall

automation effort uh and then as we

started learning about how to do that we

found ourselves working a lot closer

with the uh manual testing team and also

with the uh the test lead to make sure

hey does this even make sense to to

automate or is this one of those I don't

know admin areas of the application that

is never going to change in in the

foreseeable future so why would we spend

time automating that or no maybe it will

change the surrounding areas so it's

important for us to have that automated

because of regression concerns right so

it started helping us drive the strategy

around testing as a whole and

specifically about automation that was a

big misconception is just not it's not

because you can that you actually should

so uh it's a that was a valuable lesson

learned that cost us a lot of waste well

not wasted but like a lot of time that

we spent um and then eventually we

learned yeah what I had to do too was

the another misconception was oh well

you can just build the scripts into this

environment and there were people that

were in that environment so it wasn't

just me and so you know what I had to

learn to do was kind of get the company

to see I need my own completely separate

environment that no one else will be

touching

um because you know we we had a very

configurable application and so they

could come along and completely remove

or add new things and that that that was

breaking the test too so initially I had

started creating them within the system

that everybody was testing against and

had to say uh I'm going to need like my

completely own separate environment

that's completely clean that no one's

going to be in and that has been crucial

um and then like what you said Alex was

initially I wrote those scripts that

were gez 200 300 I don't know however

many hundreds of lines of code and then

you learned that you had to uh make

those smaller um and then the other the

other important piece was learning how

to either create like a reusable test

that you could call um things that you

found that were going to be used on a

regular basis or in some cases because

regardless of how how much we'd like for

an application to stay the same there

there's going to be enhancements that

are going to change and so learning how

to create these chunks reusable tests or

you could call you know the keyw the

methods or something like that to where

you could only have to make a change

once and then it would be pushed out to

every other test that called that uh was

also um another like important piece in

learning how to write them in a way

because maintenance is a huge part of

you know automation it's not just about

going out and writing the scripts and

then no that's it it'll you never have

to worry about that one again we're just

going to keep creating new ones it's

like no there's a maintenance aspect so

learning how to write the scripts in a

way that if you f notice an area that's

going to have possible more changes or

something like that it's better to have

those smaller reusable test chunks that

you only have to make that call once or

make that change once and it you don't

have to worry about going and trying to

change it in 15 other places that was

another thing that I learned along the

way that was important for making

maintenance easier anyway because you

would said Alex you know we had these

huge scripts with all these conditional

statements and then that one area would

break and you know and if you had like a

bunch of tests that were like that you'd

have to go and find that chunk of code

in every single one of those and make

that so that was another thing that took

time for me to learn as well so yeah so

so it looks like if we if we categorize

the the um I guess the misconceptions is

that fir first is it looks like David

the first one that you faced was this is

not going to work yeah uh the second one

Alex it looks like is like oh it will

automation will solve for everything and

also what I'm learning from David is the

third one sounds like a misconception

would have been like okay you automated

and it's one and done yeah and you know

now you're gonna let it run and and you

don't have to touch that again but it

looks like um ironically automation

sounds like it also need some human

supervision oh 100% and and and in this

case it would be maintenance right

making sure that that things are I mean

you always what I found is you you

definitely want to make sure that the

area that the code that you're

automating is at at some for lack of the

better the word sound like it's not

going to have a lot more changes coming

down the pipe um I have found it's

better to wait if you know that there's

an area of the code that's being worked

on it's going to have a lot of changes

to just wait on that automation so that

you're not having to you know write that

script or maintain it over and over and

over again because they keep making

changes to it so that was another thing

that I learned as well was sometimes

it's better to just hold off on an area

that you know is going to have drastic

changes to it until that that code gets

a little bit more solid and sound

[Music]

[Applause]

[Music]