ISTQB FOUNDATION 4.0 | Tutorial 4 | 1.3 Testing Principles | ISTQB Foundation Tutorials | TM SQUARE

Summary

This tutorial delves into the ISTQB Foundation Level certification's fundamental principles of software testing. It explains seven key principles, emphasizing that testing reveals defects but cannot guarantee their absence, exhaustive testing is impractical, and early testing is cost-effective. The script discusses the clustering of defects, the 'pesticide paradox' highlighting the need for evolving test cases, the context-dependency of testing, and the fallacy of assuming no defects mean success. The tutorial aims to guide testers in understanding and applying these principles throughout the testing lifecycle.

Takeaways

• 🔍 Testing is a process that can reveal the presence of defects but not their absence, indicating that a product cannot be guaranteed to be defect-free just because defects have been found during testing.
• 🌐 Exhaustive testing, which would involve testing all possible input combinations, is impossible due to the infinite possibilities and the impracticality of covering every scenario.
• 📅 Early testing, including static testing, can save time and money by identifying defects early in the software development life cycle, reducing the cost and effort needed for fixing them later.
• 👥 Defects tend to cluster together, meaning they are not evenly distributed across all modules of a system, which highlights the importance of thorough testing even in seemingly simple areas.
• 🔄 Tests can wear out, also known as the pesticide paradox, where the same test cases repeatedly executed may not reveal new defects, suggesting the need to periodically revise and update test cases.
• 🛠️ Testing is context-dependent, meaning that different types of applications require different testing approaches and strategies based on their unique characteristics and requirements.
• 🔄 The absence of detected defects can be a fallacy, as it does not necessarily mean the product meets all user requirements or is without defects; testers must also ensure the product's functionality aligns with user expectations.
• 🔑 Testers play a crucial role not only in finding and fixing defects but also in validating that the final product meets the specified requirements and is useful to the end-user.
• 📚 The tutorial emphasizes the importance of understanding and applying the seven standard principles of software testing to improve the testing process and product quality.
• 🛑 The principle of 'early testing saves time and money' underscores the value of static testing in identifying defects before they propagate through the development stages, thus reducing rework.
• 🔄 The tutorial also discusses the need for test case evolution, especially in regression testing, to ensure that the test suite remains effective as the product evolves over time.

Q & A

What are the seven standard principles of software testing mentioned in the tutorial?

The seven standard principles of software testing are: 1) Testing shows the presence of defects but not their absence. 2) Exhaustive testing is impossible. 3) Early testing saves time and money. 4) Defects can cluster together. 5) Tests wear out, also known as the pesticide paradox. 6) Testing is context dependent. 7) Absence of defects is a fallacy.

Why does testing show the presence of defects but not their absence?

Testing is a process that helps in finding defects, but it cannot assure that all possible defects in the system have been found. It is an endless journey, and various factors like missing data, environmental constraints, and different testing approaches can limit the ability to find every defect.

What does it mean by 'exhaustive testing is impossible'?

It means that it is impractical to test a simple application with all possible combinations of inputs and their respective outputs due to the vast number of permutations and combinations that can exist, especially when considering invalid inputs.

How does early testing save time and money?

Early testing, including static testing, allows defects to be found in the earlier stages of the software development life cycle, such as in requirements or design. Fixing defects early on is less costly and time-consuming than fixing them later in the development process.

What is meant by the term 'defects cluster together'?

It refers to the principle that defects are not evenly distributed across all modules of a system. They can accumulate in certain areas, potentially leading to an illusion that simpler modules without defects are error-free.

Can you explain the 'pesticide paradox' or 'tests wear out' principle?

The pesticide paradox suggests that the same set of test cases, if repeatedly executed without changes, may not be effective in finding new defects over time. Test cases may need to be revised and updated as the product evolves to maintain their effectiveness.

What does it mean for testing to be 'context dependent'?

Testing is context dependent means that there is no single approach or strategy that can be universally applied to test all types of applications. The testing approach should be tailored to the specific context and requirements of the product being tested.

Why is the absence of defects considered a fallacy?

The absence of defects being a fallacy implies that even if no defects are found, it does not necessarily mean that the product meets all user requirements and expectations. Testers must ensure that the product is not only defect-free but also useful and satisfactory to the end-user.

What is the significance of the principle 'testing shows presence of defects but not their absence' in the context of software testing?

This principle emphasizes that testing is not a guarantee of a defect-free product. It is a risk reduction process aimed at finding as many defects as possible within the given constraints, but it cannot assure the complete absence of defects.

How can the principle of 'early testing saves time and money' be practically applied in a software development project?

It can be applied by involving testers early in the software development life cycle to review requirements, design documents, and code. Identifying and fixing issues at this stage can prevent the propagation of defects and reduce the overall cost and effort required for defect resolution.

What are some strategies to address the principle that 'defects can cluster together'?

Strategies include thorough testing of all modules, regardless of their perceived complexity, and avoiding biases based on previous test outcomes. It also involves risk-based testing and continuous reassessment of testing approaches to ensure all areas are adequately covered.

How can testers ensure that their test cases do not 'wear out' over time?

Testers can ensure their test cases remain effective by regularly reviewing and updating them to reflect changes in the product, adding new test cases to cover new features, and removing obsolete test cases. This helps maintain the relevance and effectiveness of the test suite.

Can you provide an example of how 'testing is context dependent' might play out in different types of software projects?

For instance, testing a safety-critical system like an automotive control system would involve rigorous testing for reliability and safety, while testing an e-commerce website might focus more on user experience, performance, and security.

What steps can be taken to ensure that the principle of 'absence of defects is a fallacy' is not overlooked?

Testers should validate that the product not only meets the technical requirements but also the user needs and expectations. This involves user acceptance testing, usability testing, and ensuring that the product provides value to the end-user.

Outlines

00:00

🔍 Introduction to ISTQB Foundation Level Certification

This paragraph introduces the ISTQB Foundation Level Certification tutorial, focusing on the fundamentals of testing. It sets the stage for discussing the seven principles of testing, emphasizing their importance in the testing life cycle. The paragraph outlines the principles, including the presence of defects, the impossibility of exhaustive testing, early testing benefits, defect clustering, test wear-out (pesticide paradox), context-dependent testing, and the fallacy of defect absence.

05:00

🛠️ Principle 1: Testing Shows Presence of Defects

This section delves into the first principle, explaining that testing can show the presence of defects but not their absence. It highlights that no testing process can guarantee a defect-free product, as it's impossible to find every defect. The principle underscores the limitations of testing, emphasizing that various constraints and incomplete data prevent exhaustive defect detection. The key message is that testing helps find defects but does not ensure the absence of all possible defects.

10:03

🔬 Principle 2: Exhaustive Testing is Impossible

This paragraph discusses the second principle, which states that exhaustive testing is impossible. It uses the example of testing a login system to illustrate that even simple applications have countless input combinations. The principle highlights the impracticality of testing all possible scenarios and the need for efficient test case reduction techniques. It emphasizes that testers must focus on creating effective test cases without compromising coverage, a concept to be further explored in chapter 4.

⏰ Principle 3: Early Testing Saves Time and Money

Here, the focus is on the third principle, advocating for early testing to save time and money. The principle explains the importance of finding defects early in the life cycle, such as in requirements or design phases, to prevent costly fixes later. It differentiates between static testing (reviewing documentation) and dynamic testing (executing test cases). The paragraph underscores the cost implications of late defect detection and the value of early testing in preventing defects from propagating through the development process.

🔗 Principle 4: Defects Cluster Together

This section elaborates on the fourth principle, which states that defects often cluster together rather than being evenly distributed. It explains that critical and complex modules tend to receive more attention, reducing defects, while simpler modules might be neglected, accumulating more defects. The principle advises testers to remain unbiased and test all modules thoroughly, regardless of their perceived simplicity, to avoid missing clustered defects.

♻️ Principle 5: Tests Wear Out (Pesticide Paradox)

The fifth principle, also known as the pesticide paradox, is discussed here. It emphasizes that test cases can become ineffective over time, especially in regression testing. The principle suggests revising and updating test cases regularly to ensure they remain effective in detecting defects. It compares sticking to an old study plan that yielded poor results to reiterating outdated test cases, advocating for continuous improvement and adaptation of test strategies.

🎯 Principle 6: Testing is Context Dependent

This paragraph covers the sixth principle, asserting that testing approaches vary based on the context of the application. It contrasts the testing strategies for safety-critical systems and e-commerce websites, illustrating that different applications require tailored testing methods. The principle emphasizes that there is no one-size-fits-all approach to testing, and testers must adapt their strategies to the specific context of the product being tested.

🚫 Principle 7: Absence of Defects Fallacy

The final principle, the absence of defects fallacy, is explained here. It argues that even if no defects are found, it doesn't guarantee that the product meets the requirements. Testers are responsible for ensuring that the product fulfills user needs and expectations, not just finding and fixing defects. The principle highlights the importance of validating the product against requirements to ensure it is useful and meets customer expectations.

Keywords

💡ISTQB Foundation Level Certification

The ISTQB Foundation Level Certification is an internationally recognized qualification for software testers. It signifies a fundamental understanding of software testing principles and practices. In the video, the tutorial is focused on this certification, specifically discussing the principles of software testing that are essential for individuals pursuing this qualification.

💡Testing Principles

Testing principles are the fundamental guidelines that govern the software testing process. They provide a framework for testers to ensure that their testing is thorough and effective. The video script delves into seven standard testing principles, emphasizing their importance in the testing lifecycle and how they should be applied in practice.

💡Defects

In the context of software testing, defects refer to faults or errors in a program that cause it to behave incorrectly or not at all. The script mentions that testing is aimed at identifying these defects, but it does not guarantee the absence of all possible defects, highlighting the limitations of testing in ensuring a defect-free product.

💡Exhaustive Testing

Exhaustive testing refers to the process of testing a system with all possible combinations of inputs to ensure complete coverage. The script explains that exhaustive testing is impossible due to the vast number of potential combinations, thus emphasizing the need for efficient testing strategies that can provide confidence in the system's behavior without trying every possible scenario.

💡Early Testing

Early testing is the concept of involving testing activities as early as possible in the software development lifecycle. The video script suggests that testing at early stages, such as during requirement and design phases, can prevent defects from propagating further into the development process, thereby saving time and money.

💡Static Testing

Static testing involves the examination of the software's work products, such as requirements documents and design specifications, without executing the software. The script points out that static testing can identify defects early on, which is more cost-effective than finding them later during dynamic testing.

💡Defect Clustering

Defect clustering is the phenomenon where defects are not evenly distributed across a software system but tend to cluster in certain areas. The video script uses this concept to caution testers against becoming complacent with modules that have previously shown no defects, as they may still harbor undiscovered issues.

💡Pesticide Paradox

The Pesticide Paradox, also known as 'tests wear out,' refers to the idea that the same set of test cases, when run repeatedly, may not be effective in finding new defects over time. The script mentions this to emphasize the need for testers to periodically review and update their test cases to maintain their effectiveness.

💡Testing Context

Testing context refers to the specific conditions, environment, and requirements that influence how testing should be conducted for a particular software product. The video script explains that testing is context-dependent, meaning that different types of applications require different testing strategies and approaches.

💡Requirement Validation

Requirement validation is the process of ensuring that the software product meets the specified requirements of the users and customers. The script highlights the tester's responsibility not only to find and fix defects but also to validate that the product fulfills the intended requirements, which is crucial for the product's usefulness and success.

💡Regression Testing

Regression testing is a type of testing that verifies that changes made in the software have not adversely affected existing functionality. The script discusses the importance of evolving the regression test suite over time to adapt to changes in the product, as running the same test cases repeatedly may not be sufficient to detect new defects.

Highlights

Introduction to ISTQB Foundation Level Certification tutorial.

Exploring the fundamental principles of software testing.

Seven standard principles governing the testing process.

Testing shows the presence of defects but not their absence.

Exhaustive testing is impossible due to infinite possible combinations.

Early testing saves time and money by preventing defects.

Defects can cluster together and are not evenly distributed.

Tests wear out, emphasizing the need for evolving test cases.

Testing is context-dependent and requires different approaches for different applications.

Absence of defects is a fallacy and does not guarantee product usefulness.

Importance of understanding the testing life cycle and its principles.

The role of testers in ensuring requirements are met and not just defects are found.

The illusion of defect-free modules due to clustered defects.

The concept of static testing to find defects in requirements and design.

The cost of fixing defects increases the later they are found.

The necessity to revise and update test cases, especially for regression testing.

Different testing strategies for safety-critical systems versus e-commerce websites.

The tester's responsibility to validate the product against user needs and expectations.

Transcripts

Hello friends and greetings for the day

welcome back to another tutorial on

istqb Foundation level certification we

are in chapter one talking about the

fundamental soft testing and moving on

to the next segment of this chapter that

is 1.3 principal soft testing so let's

quickly understand what are these

principles and how does that get applied

to our testing process Al

together

as a part of this tutorial all we are

trying to understand that what are those

governing principles of testing which

one should follow at most at any point

of time during the testing life cycle so

we have seven standard principles what

we are talking about the principles what

we are referring to is called as of

course the very first one is the testing

shows presence of defects exhaustive

testing is

impossible and uh then we have the third

one which is early test testing saves

time and money the defects can cluster

together tests wear out which was

earlier called as pesticide Paradox

testing is context dependent and absence

of defects is a fallacy so let's go deep

dive into each one of these principles

and try understanding that what these

principles are trying to convey and how

does that even apply to testing the very

first principle we are talking about is

testing shows presence of defect but not

their absence which simply means that

testing is a process which does not

assure you that you can find all

possible defects in the system at any

point of time no matter you have found

any defects or not testing is not an

assurance that you have a defect free

product because testing can help you

find defects the one of the objective of

testing is to find defects but at any

point of time whether you found any

defects or not it does not guarantee

that you have found all possible defects

which you had because testing is an

endless Journey right and people people

should not be overestimating from the OV

expecting from the testing as a process

by interacting with a application by you

know conducting testing on an

application while you're trying to find

as many defects as possible but at any

point of time given that your approaches

could be different you may not have all

the information what you may need

sometime the data is missing sometime

the environment may have a constraint

and not every single thing can be

actually tested as a part of the testing

process thus testing at any point of

time can not assure you that you have

found all possible defects in the system

however at some point of time if you say

that we have found 500 defects and tried

all possible test cases which is

something which is very hypothetical to

talk about you may not come up with all

combination of data you may not just

test everything which you might think

about and just because of that I cannot

make a statement that testing can assure

you 100% defect free product so in that

context the very first principle is

dedic dedicated to the meaning of

testing that testing is only helpful in

finding defects it helps you to detect

the presence of defect but does not

prove the absence of it moving on to

principle number two when we talk about

exhaustive testing is impossible all we

are talking about is testing a simple

application with all possible

combinations of inputs with their

respective outputs of course in order to

test a s simple application let's take

an example here if I'm talking about

testing login then login has two fields

that is username and password now if I

just talk about the four combinations

which I can test that is valid username

valid password invalid username valid

password valid username invalid password

and invalid username invalid password

quite often we think that the four test

cases would be enough to test the system

but if you deep dip you realize that

invalid may have multiple possibilities

thus you may have permutation and

combination of these with the valid set

of data like 1 is to 1 1 is to 2 1 is to

3 2 is to 1 2 is to2 2 is to 3 and

there's no end to it so even 40 test

cases may be not enough so in that

context we say that hey coming up with

all possible combination of these test

data and trying out the basic you know

set of all the inputs combination is

something considered as exhaustive you

may not just come up with everything

what comes to your mind you rather have

to reduce your test cases write only the

efficient number of test cases which are

enough at that point of time to validate

and gain the required confidence but the

question is how would you do that answer

is you do have test techniques to reduce

the number of test cases but at the same

time not compromising on the coverage so

that is what the chapter 4 will be and

we'll be talking more about that in

detail moving on to the principle number

three we are talking about static

testing or in other words we are saying

early testing saves time and money so

early testing saves time and money is

all about preventing defects or at least

talking about static test testing in

this context all a tester is requested

to understand is that a defect can be

found even in the requirement can be

found in the design can be found in the

code and that should be done much

earlier than the tester case executions

so testing is certainly done in two

parts like in previous tutorial we

understood that it is static and dynamic

static helps you to find defects related

to work products and dynamic helps you

to find defs related to the product so

of course a tester is someone who's

responsible to get involved as early as

possible in reviewing the documentation

what is being written for the project

and identify the anomalies written

related to that for example if the

requirements are published much earlier

in the life cycle as a document then a

tester is supposed to review the

requirements and raise any sort of

anomaly related to that by doing so you

are identifying the mistakes done by the

business analyst or missing information

in the requirements at that phe itself

which is preventing the bugs or defects

to to be propagated to design

development and so on so that's where

the objective of this principle is to

identify defects as soon as possible and

one way the reason why we are saying

early testing saves time and money is

because it helps you to reduce the cost

of fixing it if I find a defect in

requirement all I have to do is rectify

the requirement but if this missing

requirement goes to code and then I'm

doing a dynamic level of testing that is

integration then I would need more time

to get to the root cause and and then

fixing the root cause will be again

fixing the requirement itself but it

will invite a lot of rework like

redesign recode retest so as you see the

number of activities increases as I

found it later the cost of fixing a

defect goes higher when the defect is

identified at a later point of time so

the earlier you find it it is cheaper to

fix the later you find it it is

expensive to fix so preventing bugs or

defects is more valued today compared to

that of finding defects so early testing

saves time and money principle number

four is talking about defects clustered

together that means defects can be

clustered together now can be in the

sense like we are not doing it

purposefully it may be possible so what

do what is that we are talking about

here in simple words defects are not

evenly distributed it's not necessary

that your defects will be evenly

distributed between or among

modules so it is even possible that

defects can get accumulated to one of

the module rather than being distributed

in all the modules so assume that you

have 10 modules to be tested first few

modules like 1 2 3 4 are critical and

complex modules indeed everyone

including the business analyst the

designer the developer will put their

100% effort to make sure that these

critical and complex modules do not go

wrong so they did all the effort and

when it came to testing of course when

you tested you did not find any defects

which simply means that you may create

an illusion

that the other modules which are simpler

may not have any defect but through this

principle all we want to tell you that

that maybe a developer or designer would

be overconfident on the simpler modules

and make 50 mistakes there what about

that so tester does not you know get

influenced by the outcome of previous

test executions or previous modules

execution outcomes a tester is always

supposed to be as independent as biased

as possible so that they test every

dependent thing separately okay and

that's where it is very important for a

tester to not to be influenced by the

execution because defects can cluster

together so it's not necessary that

defects will be distributed evenly

between the modules sometime first nine

modules may not have any defects and the

10th module may have 50 defects so

principle number five we are talking

about tests wear out which was earlier

very popularly known as pesticide

Paradox this principle really wanted to

convey one of the most important part of

testing that it's not necessary that

what you have written is going to work

forever especially targeting the

regression area so say for example I got

a module to test and I've written test

10 test cases right when I executed that

I did not find any defect being a very

confident tester and experienced tester

I felt that this is not possible so I

thought of repeating the same execution

once again and when I repeated indeed

the module was same the test case was

same so I will just get the same result

that is everything passed if I just go

and keep running it several times it

does not help me heeld the defect what

I'm looking for because I have a strong

intuition that generally there are

defects here so it may be possible that

your test cases are not strong enough or

good enough to the to find that defect

what you're looking for in simple words

there is no point reiterating a plan

which you created in semester 1 to get

90% if after following the plan very

strictly you got only 60% it's not

necessary that your hard work is not

enough sometime your plan the way you

created it was only for 60% not for 90%

so there is no point following the same

plan once again in semester 2 to Target

90% rather tweak the plan make some

changes in your plan reduce the number

of hours you have Leisure reduce the

number of hours when you play and

increase the number of hours to study or

change the timings of studies and see

that whether the that works and exactly

the same thing what you're talking about

here if you think that test cases are

just passing it does not mean that it's

a proof of correctness so in that

context you may look forward to revise

your test cases especially the

regression because regression test suets

are just blindly iterated every time we

make any changes thus the regression

test Suite Remains the Same which we

created long back so over a period of

time when the product evolves the

regression test Suite should also evolve

moving to the the next principle that is

testing is context dependent all we are

trying to say here is that like a

testing does not have a universally

accepted approach or strategy that means

there's no single approach to test all

types of applications as the products

are different from each other the

strategy or approach to test that should

also be different for example a safety

critical system is not tested the way an

e-commerce website is tested e-commerce

website is just for shopping products

does have their own set of criticality

and uh risk involved but when I talk

about safety critical devices things

like Automotive elevators escalators

right where human lives are involved I

totally understand that my Approach of

testing that would be completely

different so two different products are

not tested with the same approach now

what are the approaches how I can Define

my set of approach for a project answer

is we will discuss this in more detail

in chapter 5 so right now this is just

enough to understand that two different

products are not tested with the same

approach and they have different

approaches to test them finally talking

about the last principle absence of

defects fallacy U looks like very

critical terminology to understand

absence of defect is also a failure that

is what the principle says now how is

that even a failure absence of defect is

a failure that means you just removed

all the defects from the system does

that mean it could be a failure yes

absolutely it is possible why because we

are test engineers and we just don't

find and fix effects we are equally

responsible to meet the requirements

does that mean the developer and

designer may not be following it yeah if

they are following it then why are we

here as a tester I do understand that

designers and developers are also taking

care of the requirements but they're not

testing it as for the requirement

they're testing it more from a technical

driven point of view so it'll be not so

good to say that they're not following

the requirements they are indeed

following the requirements to implement

it but they are not testing it or they

are not even worried about that whether

this works as per the user needs and

expectation or not but we as a tester

are held responsible to understand the

needs of the user and customer

expectations and to validate that that

means we are not here only to find and

fix def at the end the product which is

built is useless to the customer that

does not help us so as a tester we have

full responsibility on meeting the

requirements or testing the product

according to the given requirements okay

so we just don't uh test find defs and

fix them which may not make any sense at

some point of time if the product built

is useless to the customer so one

principle dedicated to let you know that

the requirements are equally important

for tester we are not driven by code we

are not driven by Design we are all we

are driven by is the set of requirements

provided to us so testers should always

be worried about that however we are

finding defects and fixing them but does

that meet the given requirement or does

that help a user to use the system so

that's all from this particular tutorial

team should you have anything else feel

free to comment below I'm always there

to address your queries and answer them

well till then keep learning keep

exploring keep understanding the context

thanks for watching the video team and

happy

learning

[Music]