ISTQB FOUNDATION 4.0 | Tutorial 48 | Test Execution Schedule | Test Prioritization | CTFL Tutorials

Summary

This tutorial delves into the ISTQB Foundation level certification, focusing on test planning and prioritization of test cases. It explains the criticality of prioritizing test cases based on factors like risk, complexity, and dependencies to ensure smooth execution. The video outlines common prioritization strategies, including risk-based, coverage-based, and requirement-based approaches, and illustrates how to integrate these with dependencies to establish a test execution schedule. The tutorial aims to guide viewers in understanding the nuances of test case prioritization and execution planning for effective software testing.

Takeaways

• 📘 Test case prioritization is a critical activity in the test process to ensure efficient execution and adherence to project specifics.
• 🔄 Test cases can be written in any order, but they must be prioritized for execution based on various factors such as complexity, risk, and dependencies.
• 🎯 Prioritizing test cases early can be beneficial, but it's also acceptable to write them in any order and prioritize later when more clarity on project requirements is available.
• 🔑 Common prioritization strategies include risk-based, coverage-based, and requirement-based prioritization, each focusing on different aspects of test case importance.
• 🚨 Risk-based prioritization focuses on executing test cases related to the most significant risks first to ensure critical issues are addressed promptly.
• 🌐 Coverage-based prioritization ensures that test cases covering the most functionality are executed first, maximizing the breadth of testing early in the process.
• 📋 Requirement-based prioritization aligns test case execution with the priority of the requirements as defined by stakeholders, ensuring high-priority features are tested first.
• 🔄 Dependencies between test cases can alter the execution order; a lower-priority test case must be executed first if it is a prerequisite for a higher-priority one.
• 📝 A test execution schedule is a structured plan that defines the order of test case execution, taking into account prioritization strategies and dependencies.
• 🔍 Understanding the balance between priority, technical, and logical dependencies is essential for creating an effective test execution schedule.
• 🛠️ The final execution order may require adjustments based on the interplay of priority and dependencies, emphasizing the importance of a flexible and informed approach to testing.

Q & A

What is the significance of test case prioritization in the test process?

Test case prioritization is crucial because it ensures that the most critical and high-risk test cases are executed first, allowing for early detection of major issues and ensuring smooth test execution without blockers.

What are some factors to consider when prioritizing test cases?

Factors to consider include the complexity of the test case, the associated risks, dependencies between test cases, and the prioritization of requirements.

How can requirement prioritization influence test case writing?

If requirements are prioritized early, test cases can be written to align with these priorities, ensuring that the most important aspects of the project are tested first.

What is risk-based prioritization?

Risk-based prioritization involves executing test cases based on the results of risk analysis, prioritizing test cases that address the highest risks first.

What is coverage-based prioritization?

Coverage-based prioritization focuses on the extent of code or requirement coverage achieved by test cases. Test cases that achieve the highest coverage are executed first.

What is requirement-based prioritization?

Requirement-based prioritization orders test execution based on the priorities of requirements as defined by stakeholders, with test cases for the most important requirements executed first.

How do dependencies affect test case execution order?

If a test case has a higher priority but is dependent on the execution of another test case, the dependent test case must be executed first to remove the dependency.

What is the difference between technical and logical dependencies in test execution?

Technical dependencies are based on functional relationships between elements, such as needing to sign up before logging in. Logical dependencies are based on test execution order, like running valid test cases before invalid ones.

How can highly independent tests be managed in the execution schedule?

Highly independent tests, especially those with high priority, can be executed before removing dependencies of other high-priority test cases to optimize the execution schedule.

What is the final order of execution when considering all factors (priority, technical dependency, and logical dependency)?

The final order of execution prioritizes independent and high-priority tests first, followed by dependent high-priority tests, and then medium and low-priority tests, while ensuring all dependencies are resolved appropriately.

Outlines

00:00

📝 Test Case Prioritization Techniques

This paragraph introduces the importance of test case prioritization in the software testing process. It explains that while test cases can be written in any order, their execution must be prioritized to ensure critical aspects of the project are addressed first. Factors such as complexity, risk, and dependencies play a crucial role in determining the sequence of test case execution. The paragraph also touches on the idea that not all factors may be known at the beginning of a project, which can affect the initial writing of test cases but should be considered when prioritizing them for execution.

05:01

🔍 Exploring Test Case Prioritization Strategies

The second paragraph delves into various strategies for prioritizing test cases, including risk-based prioritization where test cases are ordered based on the outcome of risk analysis, ensuring the most critical risks are tested first. It also discusses coverage-based prioritization, where test cases achieving the highest coverage of the software are executed first, and requirement-based prioritization, which aligns test case execution with the priority of the requirements as determined by stakeholders. The paragraph emphasizes the need to consider dependencies that may override the priority order if a higher-priority test case depends on a lower-priority one.

10:02

📚 Understanding Test Execution Schedules

The final paragraph provides an in-depth explanation of how to create a test execution schedule by considering the priority, technical, and logical dependencies of test cases. It uses a hypothetical table to illustrate how the order of execution is determined, emphasizing the importance of running highly independent tests before those with dependencies. The paragraph clarifies that while priority is important, dependencies can alter the execution order, and a test case with a lower priority but no dependencies may be executed before a higher-priority one that has dependencies. The goal is to ensure a smooth testing process with minimal blockers.

Keywords

💡Test Case Prioritization

Test Case Prioritization is the process of arranging test cases in a specific order for execution based on various criteria. It is a critical activity in the test process, ensuring that the most important test cases are executed first. In the video, it is mentioned as a key factor in managing test activities, especially to address complexities, risks, and dependencies. The script discusses prioritization in the context of risk, coverage, and requirement-based strategies.

💡Test Planning

Test Planning is the initial phase of the testing process where the approach, resources, schedule, and strategies for testing are defined. It is part of the tutorial's focus in chapter 5, emphasizing the importance of planning in managing test activities effectively. The script highlights that test case prioritization is an integral part of test planning.

💡Risk Analysis

Risk Analysis is the process of evaluating the potential risks associated with a project and determining their impact. In the context of the video, risk-based prioritization is a strategy where test cases are ordered based on the results of risk analysis, ensuring that the most critical risks are addressed first. The script uses this concept to explain how test cases can be prioritized to manage project risks effectively.

💡Test Execution Schedule

A Test Execution Schedule is a plan that defines the order in which test cases or test suites are to be executed. It is essential for organizing and managing the test process efficiently. The script explains how different factors, including priority, dependencies, and risk, influence the creation of a test execution schedule.

💡Coverage Based Prioritization

Coverage Based Prioritization is a method of prioritizing test cases based on the level of code coverage they achieve, such as statement coverage or decision coverage. Test cases that provide the highest coverage are executed first. The script mentions this as an alternative strategy when risk analysis is not a factor in prioritization.

💡Requirement Based Prioritization

Requirement Based Prioritization involves ordering test cases based on the priority of the requirements they are linked to. Stakeholders define these priorities, and test cases related to the most important requirements are executed first. The script explains this as a method to align test case execution with business priorities.

💡Dependencies

Dependencies refer to the relationships between test cases where the execution of one test case may be contingent upon the successful completion of another. The script discusses technical and logical dependencies and how they can affect the order of test case execution, even when considering priority.

💡Criticality

Criticality is the degree to which a particular feature or function is essential to the success of a project. In the script, criticality is a factor considered when prioritizing test cases to ensure that the most critical aspects of the system are tested first.

💡Complexity

Complexity, in the context of the video, refers to the level of difficulty or intricacy associated with a test case or a feature being tested. It is one of the factors considered during test case prioritization to determine the sequence of execution.

💡Test Cases

Test Cases are defined sets of conditions or variables under which a set of procedures is executed to determine the correctness of a software unit. The script discusses how test cases are written, prioritized, and executed as part of the test planning and management process.

💡ISTQB Foundation Level Certification

ISTQB Foundation Level Certification is a professional certification for software testers that validates their knowledge of fundamental software testing concepts. The video is part of a tutorial series aimed at preparing individuals for this certification, focusing on managing test activities and test case prioritization.

Highlights

Introduction to ISTQB Foundation Level certification tutorial on managing test activities.

Emphasis on the importance of test case prioritization in the test process.

Explanation of factors influencing test case prioritization such as complexity, risk, and dependencies.

The concept that test cases can be written in any order but must be prioritized for execution.

Discussion on the use of requirement prioritization to inform test case prioritization.

Clarification that test case writing can occur without initial clarity on project specifics.

Highlighting the significance of executing high-priority test cases first for critical item confirmation.

Introduction of different test case prioritization strategies: risk-based, coverage-based, and requirement-based.

Risk-based prioritization explained, focusing on executing the most critical risk-related test cases first.

Coverage-based prioritization strategy, prioritizing test cases based on the level of code coverage achieved.

Requirement-based prioritization, aligning test case execution with the priority of corresponding requirements.

The impact of technical and logical dependencies on the test execution order.

Example of creating a test execution schedule considering priority, technical, and logical dependencies.

Explanation of how to adjust the execution order when high-priority test cases are dependent on lower-priority ones.

The approach to blend priority, dependencies, and risk factors to define the right order of execution.

Final thoughts on the importance of understanding the test execution schedule for effective testing.

Invitation for further queries and the conclusion of the tutorial.

Transcripts

Hello friends and greetings for the day

welcome back to another tutorial on

istqb Foundation level certification we

are in chapter 5 talking about managing

the test activities and continuing ahead

with our 5.1 that is test planning and

as a part of today's tutorial we'll be

talking about how to do test case

[Music]

prioritization

prioritizing a test case is indeed one

of the very critical activities to be

taken as a part of the test process

altoe writing test cases can be done in

any manner in any order in any sequences

but when it comes to executing them the

test case needed needs to be prioritized

now when we talk about prioritizing a

test case it's all about making the

right sequence of execution in order to

adhere to those of the other specific

things related to the project for

example we talk about complexities risk

associated and many such other factors

like dependencies to be removed Etc it

to be taken into account in order to run

the tests smoothly without any kind of

blockers any kind of Stoppers alog

together thus prioritization becomes

very important to be taken into account

however when it comes to writing the

test cases we may not just consider

everything at the very beginning because

sometime we don't have all the clarity

what we may need right at the beginning

of the project thus if you have already

requirements prioritized you can take

that also into account in order to

prioritize your test cases or write test

cases early for that but when certain

things do not exist like a requirement

prioritization may not have been done

then test case writing can happen in any

order and later we may sit down together

to take multiple factors into account

like the criticality the complexity or

the risk factor as well and then

determine the right sequence of

execution executing high priority test

cases first always gives us a

confirmation that the critical items are

dealt with and all remaining test cases

are now something which are non-critical

however sometime we run out of time then

running those critical and risk area

related test cases first would certainly

save your day now let's exactly

understand what are the different ways

by which we can prioritize the test

cases and at the same time we'll also be

looking at what exactly is the way to

understand a test execution schedule so

when we talk about the prioritization

methods of course once the test cases

and the test procedures are specified

and assembled into the test Suites these

test Suites can be arranged in a test

execu ution schedule that defines the

order in which they are supposed to be

run when prioritizing the test cases

different factors can be taken into

account and that's what the factors

we're talking about in the options below

the most commonly used test case

prioritization strategies are as follows

number one we have the risk based

prioritization where the order of the

test execution is based on the results

of the risk analysis test case coverage

the most important risk are executed

first so in simple words risk is taken

as one of the key factors in order to

perform the executions and thus based on

the criticality and the level of risk we

determine which test cases to be

executed first however we always try to

run the highest priority risk related

test first then the medium then the low

so that's a very common procedure indeed

one of the common practice which we

follow in any of the organizations today

that when we conduct risk analysis we

just use it as a part of the

prioritization of the test cases as as

well another factor to consider here is

the coverage based on prioritization the

coverage based prioritization where the

order of the test execution is based on

the coverage including examples like

statement coverage decision coverage Etc

test cases achieving the highest

coverage are executed first in another

variant called as additional coverage

prioritization the test case achieving

the highest coverage is executed first

each subsequent test case is the one

that achieves the highest additional

coverage so point being made here is if

in case your project does not talk about

risk analysis then the alternative is

coverage driven uh prioritization where

we pick up those test cases which has

the number of coverage or amount of

coverage higher on the given requirement

for example sometime a single test like

even that you have covered chapter 4 now

you know that sometime one test is

enough to cover all the statements in a

given code right and that one test

covers 100% of statement so sometime one

test May cover 50% another test May

cover only 20% and another one test will

cover remaining

30% then additional coverage measurement

certainly talks about first I'll run the

test which covers the 50% coverage then

I'll run the second one which covers the

30% and then I'll run the third one

which covers the 20% so put together the

coverage based prioritization certainly

means the test cases which have the

highest coverage on a given requirement

or on a given particular code we run

them first and that's what another

method of prioritization would be the

third option here is certainly the

requirement based prioritization where

the order of the test execution is based

on the priorities of the requirement

traced back to the corresponding test

cases requirement priorities are defined

by the stakeholders test cases related

to most important requirements are

executed first I think we just talking a

moment ago that requirement priority can

also be seen as one of the method to

prioritize your test cases thus the

highest priority requirement related

test cases will be prioritized highest

and will be executed first then the

medium priority requirement then the low

priority requirements so put together

these are three different ways by which

I can go ahead and prioritize my test

cases and put them into a stacked table

called as test execution schedule and

then Define the final order of execution

also to add here uh ideally the test

cases would be ordered to run based on

their priority levels using for example

one of the above mentioned

prioritization strategies however this

practice may not work if the test cases

or features being tested have

dependencies now if a test case with a

higher priority is dependent on a test

case with lower priority then lower

priority test case must be executed

first so of course for this particular

statement we have a separate slide Al

together we will be talking about in a

moment but on a very high level

dependencies are something which are

also seen as blockers so maybe one test

is blocking another test then the

blocking test should be executed first

then the other one and in this case the

priority does not really make sense

because the blocker has to be removed in

order to run the highest priority

however there is a complete approach to

understand how to blend the priority the

dependencies all together to finally

Define the right order of execution and

that's what we'll be looking at next so

to talk about the dependencies and risk

factors Etc and priority together to

actually understand a test execution

schedule let's take some quick example

to understand standard so right here on

my screen we have a small table which is

very very hypothetical I'm not taking

any kind of technical constraints any

kind of features any kind of criticality

into account I'm just creating a dummy

data to help you understand that what

the test execution schedule May look

like and based on that how to define the

order of execution so let's read the

data first the table certainly contains

four columns we have test case ID we

have the defined priority for it and

then we have something called as

technical dependency and logical

dependency depy in simple words

technical dependencies are those where

the elements are technically dependent

on each other for example I cannot log

in without signing up on a particular

application so I have to be registered

user in order to login so I will try to

execute uh sign up uh first then login

and logical dependency for an instance

could be uh related to test execution

order itself for example I run the valid

test cases first compared to that of

invalid test cases so if my valid test

passes then the invalid test will be

executed and that can be referred to as

logical dependency but I'm not using

these examples in my data right here the

data is very very hypothetical so do not

correlate that okay this test belongs to

sign up or login or positive or negative

so please ignore all that so here we

have got four test T1 is high priority

T2 is medium T3 is high T4 is low you

may ask me a question that why didn't I

sort it no test cases can be written in

any order so we have we have just sorted

it by the test case ID so that we don't

lose any particular test however the

priorities may be defined later as you

all understand writing test cases is one

step prioritizing test cases is another

step right so prioritization may be done

at a later point of time then we

identified that T1 and T3 are high

priority and T2 is medium T4 is low also

if you notice uh at this point if I just

consider only the priority the order of

execution would be T1

then T3 then T2 then T4 and I'm sure

pretty much everyone would agree to that

but also to highlight here that it can

also be executed in this manner that is

T3 T1 T2 T4 why because T1 and T3 are

both on the same priority level thus

there is no difference which one gets

executed first point being made here is

that order of execution that is priority

driven is what we have to take care of

the order of writing the test cases does

not make a difference someone has to be

below another right so in that context

it has to be defined in any order but

when it comes to Priority I can run both

are on high priority so I can run them

in any order so T3 T1 then we have T2

and T4 if I just consider only the

priority but if I take the next

constraint that is technical dependency

I see that T1 is dependent on T2 that

means no matter T2 is medium it has to

be executed before T1 in order to

release the dependency so in that

context if I look into account the

considering the priority and Technical

dependency both the order of execution

will be T3 T2 T1 T4 many of you would

have thought like we can run T2 first

then T1 then T3 but for your kind of

information another important aspect of

test execution schedule is that if you

have a highly independent test I repeat

highly independent test then that can be

executed before removing the dependency

of other high test cases

because see it's always important to

understand that what is more independent

and high priority thus T3 is completely

independent and even at high priority

thus it can be executed before T1 so

that's the reason the execution order

reads like T3 so high then T2 then T1

and then T4 okay so always remember we

should run High independent then High

dependent and same way medium

independent medium dependent and then

low independent and low dep dependent so

that's the order of execution now taking

logical dependency also into account we

have T3 also being dependent on T2 that

means both the high level test or high

priority test are dependent on T2 now

now in that context considering the

entire table together that is priority

Technical and logical dependency the

order of final order of execution will

be T2 T1 T3 T4 or T2 T3 T1 T4 see T2 is

a blocker for both the high dependent

test right then if I run T2 both become

independent then again the order of T1

and T3 does not matter I can run them in

any of the order so either D3 first then

D1 or D1 first and then D3 so put

together that's the whole context of

what exactly test execution schedule is

I hope you have got the understanding of

it so that's all from this particular

tutorial team should you have anything

else feel free to comment below I'm

always there to address your queries and

answer them well till then keep learning

keep exploring keep understanding the

context thanks for watching the video

your team and happy

[Music]

learning