Navigating Common Misconceptions of Transitioning from Manual to Automation
Summary
TLDRIn this discussion, the speakers reflect on the early challenges and misconceptions of automation in the late 1990s. They emphasize how skepticism about its value, the assumption that automation could solve everything, and the belief that automation was a one-time effort were common. They share insights on proving the financial benefits of automation, learning to write maintainable and reusable scripts, and understanding that ongoing human oversight is crucial for successful automation. The conversation highlights the evolving understanding and strategic approach to automation over time.
Takeaways
- 🔍 Early automation efforts faced skepticism, with doubts about its value compared to manual testing.
- ⏳ One major concern was the time investment required for automation, which many believed wouldn't pay off in the long run.
- 📊 Proving the financial benefits of automation was crucial in gaining buy-in from management, showing significant time and cost savings.
- 🧩 A key lesson learned was that automation isn't a one-size-fits-all solution; not everything should or can be automated effectively.
- 🔍 Early misconceptions included the belief that automation could replace all manual testing, which proved to be overly optimistic.
- 🛠 Developing a strategy for automation was essential, particularly in creating smaller, reusable test scripts to ease maintenance.
- ⚙️ Maintaining automation scripts is an ongoing effort; changes in the application can require frequent updates to the automation code.
- 🚦 It’s important to have a dedicated testing environment for automation to avoid interference from other ongoing testing activities.
- 💡 Waiting to automate areas of an application that are still undergoing frequent changes can prevent unnecessary rework and maintenance.
- 🔄 Automation requires ongoing human oversight, particularly in maintaining and updating scripts as the application evolves.
Q & A
What was one of the main fears expressed by the speakers when they started with automation in the late 90s?
-One of the main fears was that automation would take too much time initially, and the upfront work required would not be worth the effort on the back end.
How did the speaker demonstrate the value of automation to their company?
-The speaker demonstrated the value by showing that automation reduced the time needed for a quarterly release from a week to three days, which also decreased the cost significantly.
What misconception did people have about automation that the speaker had to overcome?
-People believed that automation would not provide any real value and that the initial effort to automate was not worth the time and effort.
How did the speaker prove to their company that automation was worth the investment?
-The speaker showed that by automating a significant portion of a large Excel spreadsheet, they were able to reduce the time and manpower required for a quarterly release, proving the financial benefits of automation.
What was the initial approach to writing automated tests according to the speakers?
-Initially, the speakers created large tests with many conditions, which made it difficult to identify which part of the application was failing when tests did not pass.
What did the speakers learn about test strategy as they began to automate?
-They learned that each automated test should result in one outcome for one functionality, leading to the adoption of atomic tests and a more strategic approach to test automation.
Why did the speakers emphasize the importance of having a separate environment for automation?
-Having a separate environment ensured that tests were not broken by changes made by others in the shared testing environment, which was crucial for maintaining the reliability of automated tests.
What was a key lesson the speakers learned about the maintenance aspect of automation?
-They learned that automation scripts need to be written in a way that allows for easy maintenance, such as creating reusable test chunks that can be updated in one place rather than in multiple tests.
What misconception did the speakers address regarding the ability to automate everything?
-They addressed the misconception that automation can replace all manual testing, learning that it's more about having a strategic approach to what should be automated and what should remain manual.
How did the speakers approach the issue of changes in the application affecting automated tests?
-They learned to create smaller, more focused tests and to wait to automate areas of the code that were likely to change, reducing the need for constant script maintenance.
What advice would the speakers give to someone new to automation based on their experiences?
-They would advise focusing on strategic test automation, maintaining separate environments for testing, and writing maintainable scripts with reusable components to handle changes in the application.
Outlines
😨 Overcoming Early Fears and Misconceptions in Automation
The speaker reflects on the early challenges and misconceptions surrounding automation in the late 90s, particularly the skepticism about its value and the belief that it would require too much upfront time and effort. They share how they were able to demonstrate the financial benefits and time savings of automation by reducing the time required for quarterly releases from a week to just three days. This success helped convince the company of the value of automation, despite initial resistance. The speaker emphasizes the importance of proving both the effectiveness and economic sense of automation to gain managerial buy-in.
🤖 Realizing the Limits of Automation
The speaker discusses the initial misconception that everything could be automated, leading to overly complex and error-prone tests. They learned that effective automation requires a well-thought-out test strategy, where each test should be focused on a single functionality. Collaboration with manual testers and test leads became crucial to determine which areas of an application were worth automating. The speaker also mentions the need for a separate, stable environment for automation to avoid interference from ongoing development, highlighting the importance of smaller, reusable test components to facilitate easier maintenance.
🛠 The Ongoing Need for Maintenance in Automation
The speaker categorizes three major misconceptions about automation: the initial belief that automation wouldn't work, the assumption that automation could solve everything, and the idea that once automated, no further attention would be needed. They emphasize that automation requires continuous human oversight, particularly in maintaining and updating tests as the application evolves. The speaker advises waiting to automate areas of code that are still undergoing significant changes to avoid excessive rework. This underscores the balance needed between automating efficiently and ensuring that the automated scripts remain reliable over time.
Mindmap
Keywords
💡Automation
💡Manual Testing
💡Misconceptions
💡Test Strategy
💡Maintenance
💡Regression Testing
💡Reusable Test Scripts
💡Financial Impact
💡Testing Environment
💡Conditional Statements
Highlights
Overcoming initial fears and misconceptions about automation in the late 90s.
The challenge of proving the value of automation to skeptics within the company.
The misconception that automation would take too much time and not be worth the effort.
Demonstrating the benefits of automation through a quarterly release scenario.
Reducing the all-hands-on-deck period from a week to three days through automation.
The importance of showing the financial benefits of automation to gain management support.
The common misconception that automation could replace all manual testing.
Learning the necessity of a clear test strategy for effective automation.
The realization that not all tests need to be automated due to their complexity or lack of change.
The need for a separate, clean environment for automated tests to avoid interference.
The evolution from writing large, complex scripts to creating smaller, reusable test components.
The importance of maintenance in automation and the need for scripts that are easy to update.
The learning process involved in determining which parts of an application should be automated.
The strategic approach to automation, including when to hold off on automating certain areas.
The role of human supervision in ensuring the effectiveness and accuracy of automated tests.
The economic impact of automation on reducing costs and increasing efficiency within a company.
The evolution of the speakers' understanding of automation from a tool to a strategic business asset.
The importance of aligning automation efforts with the overall business goals and objectives.
Transcripts
[Music]
talked about the fears you talked about
some of the things that you you try to
overcome and it looks like your boss
told you hey that must have been one
fear like hey go ahead and get this
thing going or you know we're done here
uh but then then what along with those
fears what were the misconceptions of
automation back then and like what what
is it and I'm I'm guessing there is a
lot of correlation to what we're seeing
today with AI in testing but we're not
going to go there yet I'm just curious
like back in n in late 90s when you both
were jumping into into the deep end of
automation what uh are some of the
things that that people misunderstood
that ended up being more like uh than an
actual myth or or something that I
didn't become reality
uh for me personally because it was new
and people had just been doing manual
testing I just don't think anyone
believed that there was going to be any
real value that was going to come from
it
and then there was also
some of the
that automating would take too much time
like the initial upfront work that would
would need to be done would not be wor
on on the back end right and so it was
really just
getting um you know managers and stuff
like that to see that it was it was
worth the time and effort and one way
that I was able to prove that
was we used to do quarterly releases and
when we did this quarterly release it
would be like an all handson deck within
the company for like a week and I was
able to show that by automating we had
this huge Excel spreadsheet that had all
of these things that needed to be done
in order for it to be you know deemed
like good good enough to go for the
quarterly release I had started what
once I had gotten the green light to go
and and stuff like that I had gotten um
a fair amount of that spreadsheet
automated and I was able to show the
company um you know I was able to kind
of break down all right this is roughly
how much you're paying for that week of
all Hands-On deck because it wasn't just
the QA we were bringing in other members
of the company you know analysts and
implementation Specialists and stuff
like that and I was able to show you
know hey we able to that down to three
days there was by me being able to
automate a certain percentage of that
spreadsheet that you had
uh we only need two tester that was
handling some of the newer um stories
that had been brought in that had not
been automated yet and I was able to
show but look how much we were able to
cover with just the tool so that was
probably the biggest thing that was I
was able to do to kind of get them to
see hey this is worth it but that but
getting them to to Really buy in for
that initial part of me kind of being
left alone
because I didn't do any manual testing
my job was just just to do everything I
could to automate you know what we had
and once I was able to prove that from a
financial standpoint and show them um
that we're able to do it faster and for
less money that was really the
big uh impact that allowed us to be able
to like to make it like a huge part of
who we are now at our company with the
automation so it it does look like you
had not only the hey I have to show that
these work but also I have to show that
these is e like the economics of these
make sense yeah right otherwise I mean
the investment wasn't going to be worth
it regardless of how fast you could do
it yeah and I you know with a business
and that's ultimately what they're going
to want to see a lot of times right I
mean it really comes down to how can we
save the company money by by doing this
so yeah that was it for me Alex I don't
know what you have but what what
misconceptions did you face Alex when
you were trying to I mean there is a
technology I don't recall the name right
now that both of you mentioned at at the
very beginning but um as you were
learning and implementing these things
did you did you find anything that uh or
or was there any objections really when
you were trying to to adopt it or
implement it yeah so um I'd say um the
common misconception at the time when I
was starting was that uh we could
automate everything so right it's like
oh great so there this tool can you know
do like can run everything that we're
doing manually and and it can basically
like we'll be able to test everything
super quick and then initially as we
were learning as well like yeah I think
so so we'll we would create tests and
and and Dave you'll probably relate to
it but like we would create tests like
huge tests full of conditions like if
then if then else and then choose this
and choose that and then and it worked
you know it could make it work but then
very quickly when there were failures in
the application that caused the tests
the automated test to fail we would go
like huh okay so why did it fail like
which functionality actually is not
working because our tests automated so
many things that uh we had we literally
had to trouble not troubleshoot but
triage the test itself
to find which part of the application
was throwing the error and so very
quickly we learned okay automation is
not just about automation it's there's
more about test strategy than actually
automating right so then we very quickly
learned that okay each automated test
should result uh in one or should create
one result right for one functionality
uh uh like Atomic test basically right
and uh and then that helped us uh Drive
the test strategy for the overall
automation effort uh and then as we
started learning about how to do that we
found ourselves working a lot closer
with the uh manual testing team and also
with the uh the test lead to make sure
hey does this even make sense to to
automate or is this one of those I don't
know admin areas of the application that
is never going to change in in the
foreseeable future so why would we spend
time automating that or no maybe it will
change the surrounding areas so it's
important for us to have that automated
because of regression concerns right so
it started helping us drive the strategy
around testing as a whole and
specifically about automation that was a
big misconception is just not it's not
because you can that you actually should
so uh it's a that was a valuable lesson
learned that cost us a lot of waste well
not wasted but like a lot of time that
we spent um and then eventually we
learned yeah what I had to do too was
the another misconception was oh well
you can just build the scripts into this
environment and there were people that
were in that environment so it wasn't
just me and so you know what I had to
learn to do was kind of get the company
to see I need my own completely separate
environment that no one else will be
touching
um because you know we we had a very
configurable application and so they
could come along and completely remove
or add new things and that that that was
breaking the test too so initially I had
started creating them within the system
that everybody was testing against and
had to say uh I'm going to need like my
completely own separate environment
that's completely clean that no one's
going to be in and that has been crucial
um and then like what you said Alex was
initially I wrote those scripts that
were gez 200 300 I don't know however
many hundreds of lines of code and then
you learned that you had to uh make
those smaller um and then the other the
other important piece was learning how
to either create like a reusable test
that you could call um things that you
found that were going to be used on a
regular basis or in some cases because
regardless of how how much we'd like for
an application to stay the same there
there's going to be enhancements that
are going to change and so learning how
to create these chunks reusable tests or
you could call you know the keyw the
methods or something like that to where
you could only have to make a change
once and then it would be pushed out to
every other test that called that uh was
also um another like important piece in
learning how to write them in a way
because maintenance is a huge part of
you know automation it's not just about
going out and writing the scripts and
then no that's it it'll you never have
to worry about that one again we're just
going to keep creating new ones it's
like no there's a maintenance aspect so
learning how to write the scripts in a
way that if you f notice an area that's
going to have possible more changes or
something like that it's better to have
those smaller reusable test chunks that
you only have to make that call once or
make that change once and it you don't
have to worry about going and trying to
change it in 15 other places that was
another thing that I learned along the
way that was important for making
maintenance easier anyway because you
would said Alex you know we had these
huge scripts with all these conditional
statements and then that one area would
break and you know and if you had like a
bunch of tests that were like that you'd
have to go and find that chunk of code
in every single one of those and make
that so that was another thing that took
time for me to learn as well so yeah so
so it looks like if we if we categorize
the the um I guess the misconceptions is
that fir first is it looks like David
the first one that you faced was this is
not going to work yeah uh the second one
Alex it looks like is like oh it will
automation will solve for everything and
also what I'm learning from David is the
third one sounds like a misconception
would have been like okay you automated
and it's one and done yeah and you know
now you're gonna let it run and and you
don't have to touch that again but it
looks like um ironically automation
sounds like it also need some human
supervision oh 100% and and and in this
case it would be maintenance right
making sure that that things are I mean
you always what I found is you you
definitely want to make sure that the
area that the code that you're
automating is at at some for lack of the
better the word sound like it's not
going to have a lot more changes coming
down the pipe um I have found it's
better to wait if you know that there's
an area of the code that's being worked
on it's going to have a lot of changes
to just wait on that automation so that
you're not having to you know write that
script or maintain it over and over and
over again because they keep making
changes to it so that was another thing
that I learned as well was sometimes
it's better to just hold off on an area
that you know is going to have drastic
changes to it until that that code gets
a little bit more solid and sound
[Music]
[Applause]
[Music]
Browse More Related Video
SKOOL - What is first principles thinking?
ISTQB CTAL TAE Session 36 - 5.1 - Selection of TAS Metrics
FASTEST Way to Learn Automation and ACTUALLY Get a Job
Katalon Academy: Create Automated Tests with Record & Playback (Intro)
Marketing Automation 101: The Definitive Guide for 2024
5 ESTRATÉGIAS INFALÍVEIS PARA LEVANTAR CAIXA RÁPIDO NO MARKETING DIGITAL
5.0 / 5 (0 votes)