CH05. L03. Test planning activities and time estimation
Summary
TLDRThis script delves into the intricacies of test planning, outlining 10 key activities. It emphasizes identifying testing scope, risks, and objectives, choosing between proactive and reactive approaches. The discussion covers integrating test activities within the software lifecycle, deciding what to test, assigning resources, and setting schedules. It also highlights the importance of establishing entry and exit criteria, monitoring, and controlling the testing process. The script further explores test estimation techniques and the influence of product characteristics, development processes, and testing outcomes on the testing effort. Finally, it touches on various test approaches, including analytical, stochastic, methodical, and dynamic heuristic methods, underlining the context-dependent nature of testing.
Takeaways
- 🔍 Identify the scope of testing, the risks involved, objectives, and the approach (proactive or reactive) as the first step in test planning.
- 🔄 Integrate and coordinate test activities within the software life cycle, ensuring each life cycle activity has a corresponding testing activity.
- 📝 Decide on the parts to test, roles needed, and how to evaluate test results during the planning stage.
- 🗓 Set a detailed schedule for each task, including start and end times, participants, and duration.
- 👥 Assign resources to each activity, identifying the level of detail and selecting necessary metrics for monitoring and control.
- 👀 Monitor and control the test process to ensure execution aligns with the plan and take immediate action to fix any deviations.
- 📋 Specify the level of detail in test procedures to guide the preparation and execution of testing activities.
- 🚦 Define entry criteria for when to start testing, including the readiness of the test environment, tools, data, and code.
- 🏁 Establish exit criteria to determine when to stop testing, such as coverage targets, defect density, or risk analysis.
- 📊 Test effort estimation should be based on approaches like matrix-based or expert-based to ensure fairness and accuracy.
- ⚖️ Test effort is influenced by product characteristics, development process, and testing outcomes, including defect trends and rework required.
Q & A
What is the first step in test planning activities?
-The first step is to identify the scope of testing, assess the risks, define the testing objectives, and decide on the test planning approach (either proactive or reactive).
How are test activities integrated into the software life cycle?
-Test activities are integrated and coordinated within the software life cycle by aligning each life cycle activity with its corresponding testing activity in the V-model.
What is meant by 'monitor and control' in the test planning process?
-'Monitor and control' refers to the follow-up process where the execution is monitored to ensure it matches the plan, and immediate actions are taken to correct any deviations.
What are entry criteria in the context of test planning?
-Entry criteria specify the conditions that must be met before testing can start, such as having a ready test environment, necessary tools, and test data.
What are exit criteria in test planning?
-Exit criteria define the conditions under which testing can be concluded, such as achieving a certain level of code coverage, completing a specified number of test cases, or resolving critical defects.
How is the test effort estimated?
-Test effort is estimated using approaches like matrix-based estimation, which relies on data from similar projects, and expert-based estimation, which depends on the experience of task owners.
What factors influence the testing effort?
-Factors influencing the testing effort include the characteristics of the product, the development process, and the outcomes of testing, such as the number of defects found and the required rework.
What is the analytical approach in test strategy?
-The analytical approach, also known as risk-based testing, focuses the testing effort on modules with the greatest risk by analyzing the system to identify high-risk areas.
What is stochastic testing?
-Stochastic testing, or random testing, uses statistical data about failure rates to measure the system's reliability by inputting multiple inputs in a specific sequence and checking the system's response.
What is the methodical approach in testing?
-The methodical approach deals with failures by using techniques like error guessing and fault attacks, relying on the test engineer's experience to determine the testing method.
Outlines
🔍 Overview of Test Planning Activities
This paragraph provides a detailed overview of the test planning activities. It begins by discussing the importance of identifying the scope, risks, and objectives of testing. The paragraph outlines the integration of testing within the software life cycle, the selection of proactive or reactive approaches, and the scheduling of tasks. It also emphasizes the significance of assigning resources, defining entry and exit criteria, and ensuring the availability of necessary tools and environments before testing begins. The discussion concludes with the importance of monitoring and control, highlighting the need for immediate action to correct deviations during testing.
🚦 Entry and Exit Criteria in Testing
This paragraph delves into the concepts of entry and exit criteria in the testing process. Entry criteria refer to the conditions that must be met before testing can begin, such as the availability of a ready test environment, tools, and test data. The paragraph stresses the importance of preparing these elements in advance to avoid delays. Exit criteria determine when testing should be concluded, based on factors like code coverage, defect density, and the completion of specific test levels. It discusses the need to focus on achieving testing objectives and managing risks before concluding the testing process.
🧠 Approaches to Test Estimation and Strategies
This paragraph focuses on the test estimation process and the factors that influence it. It describes how test leaders estimate the effort required for testing tasks using methods like matrix-based and expert-based approaches. The estimation considers the characteristics of the product, development process, and testing outcomes, such as the number of defects found and the amount of rework required. The paragraph also touches on the test strategy and approach, highlighting the importance of context-dependent testing and selecting the appropriate techniques based on the project’s requirements.
⚖️ Different Testing Approaches and Their Applications
This paragraph explores various testing approaches and their applications in specific contexts. It introduces seven common testing approaches, including analytical, stochastic, methodical, process-compliant, dynamic and heuristic, consultative, and regression averse. The paragraph explains each approach, such as the analytical approach focusing on risk-based testing, stochastic testing using statistical models, and methodical approaches relying on error guessing and fault attacks. It also covers the importance of selecting the right approach based on factors like risk, safety, available resources, and the specific needs of the project.
Mindmap
Keywords
💡Test Planning
💡Proactive and Reactive Approaches
💡Test Activities Integration
💡Entry Criteria
💡Exit Criteria
💡Test Estimation
💡Risk-Based Testing
💡Test Design Techniques
💡Defect Density
💡Regression Testing
Highlights
The test planning process includes 10 key activities, starting with identifying the scope, risks, and objectives of testing.
Choosing a test planning approach can be either proactive or reactive.
Integration and coordination of test activities within the software lifecycle are crucial, often following the V-model.
Deciding which parts of the system to test is a critical step in the planning phase.
Scheduling is vital, covering task timelines, participants, and timeframes for implementation, execution, and evaluation.
Resources assignment and detailing the level of detail in tasks and metrics are essential for monitoring and controlling.
Monitor and control involve checking for deviations from the plan and taking corrective actions.
Entry criteria define when to start testing, ensuring readiness in terms of environment, tools, and test data.
Exit criteria determine when to stop testing, based on factors like coverage, risk, defect density, and project timelines.
Test estimation involves predicting effort and resources required, using matrix-based and expert-based approaches.
Product characteristics, development process, and testing outcomes significantly impact the testing effort.
The choice of test strategy and approach, whether proactive or reactive, depends on context and available resources.
There are seven typical test approaches, including analytical, statistical, methodical, and consultative methods.
Analytical approaches focus on risk-based testing, targeting high-risk modules.
Regression-averse approaches emphasize reusing existing test materials and conducting regression testing.
Transcripts
We've talked before about the test management
and had a quick overview of the test planning and what does it mean.
let's know now the test planning activities.
They are 10 activities:
The first step is to identify the scope of testing in which you'll work
and what are the risks you'll face, and what are
the objectives of testing? And to mention
which test planning approach we will use, either
proactive or reactive. The third thing we do
is to integrate and coordinate the test activities inside
the software life cycle. Since, we've mentioned before, each life cycle activity
has in the opposite its testing activity in the V-model.
In this stage, we will put all the testing activities we have
in the cycle of the software development
Then Comes step no. 4
that is to decide which parts we need to test.
All this time , we were still talking about the planning stage to know
what to test, which roles we will need
which will execute all the testing activities
and how to evaluate the test results .
We start by setting a schedule. In planning,
we should always schedule for each task;
when it will happen, who will participate,
the time it takes, when it will start and end.
you should also schedule
the implementation, the execution and the evaluation.
when you are still in planning, don't forget to assign all the resources
of each activity and identify the level of details
you'll work on and select which metrics
you will need during the monitor and control.
The "monitor and control" is the follow-up
we do to monitor and to check if what
you execute in reality is the same as what .
you planned for while the control means taking an immediate action,
to fix any defect and to solve
any problem at once, whenever you find a deviation
between plan and execution. what do we do
in control? It helps us to take an action to improve
ourselves during working on the test process.
You should mention the level of details in the test procedures
in order to tell those who will do the task of preparation
and execution where to start in the test planning.
You must also identify the entry criteria
on which you will start testing, in addition to
identifying the exit criteria. The entry criteria
means "when to start testing?"
for example :I will start testing at the beginning
of each test level.
or when I have a test group which is about execution.
what do the Entry criteria cover?
when you start testing, what will you need?
To start testing, can you start without
an environment which is ready for testing?
or can you start testing while they are not ready?
So I want the test environment to be at least available,
any tools or the test data we need,
or anything we need as a setup to the environment.
Whether what we are going to test ready or not?
If we are using tools, are these tools ready for
the test environment or not?
Testable code: If we work the white box, is our code
and the test data are available or not?
All tools we need before starting testing process,
are called the Entry Criteria. You must ensure
that they are available so as not to be blamed later on when people say
"you wasted time". If you started testing
without test preparation for your data,
you will search for test data.
Also if you start testing while you don't have the tool you will use.
All that will affect the timing of testing
and this is what we call the Entry Criteria.
the stage before testing is like preparing a checklist for
the things I'll need in testing, are they available or not?
If anything is not available, you will stop and ask them to prepare it.
when will we finish testing?
we won't test forever.
you must determine when to start testing
and when to stop testing. I will stop the testing process
at the end of each test level,
or when we finish each group of test on a specific goal.
For example,if we finish registration today
so this will be the end or the point you will stop at
when we reach the ending of registration.
The Exit criteria: where should you focus?
I will focus on "Did we cover our code and functionality
or the risk?" what does it mean? I said here I will stop
testing when I reach 100*100 decision coverage
I am the one who set that. So I'll get back to
the question "did I reach this objective or not?"
I will stop testing when I reach 90%
of the test cases have been executed,
and this is for me measure.
Or I'll stop testing based on the risk analysis
as the most dangerous 2 modules are 1&2
and we've finished testing them. So this is for me
my Exist criteria. This is regarding the 1st point.
Your exist criteria can be defect density.
what is defect density? it's the number of defects we found
per end line of codes.Again,
we test say 1000 lines of code if we found
500 defects per 1000 lines of code
this is for me Exist criteria and I will stop testing here.
The Exist criteria can be
number of the remaining risks, like what?
i.e.what can remain from them?
Do we fix all defects we find at once?
No, I may prioritize the defects I find
and may leave some of them to be done later.
So my Exist criteria may be the defects not fixed
or lack of test coverage in certain areas.
Some areas in the application I didn't give them full coverage.
You may need to finish testing
because of the time to market.
I need to publish the first version so I'll stop testing.
So what factors that determine the Enter and Exist criteria
The availability:
Is the team ready? Are the environment and system ready?
Are the tools available?
and the test items status to start or to end testing
Defects,
measurements, defect trend and severity.
What is it meant by defect trend ?
As long as the project is ongoing, we check if the defects we found are relative to the defect fixed.
if the defect is opened i.e. found, so how much did we finish?
Test execution measurement
what is the measurement of the test execution?
How many passed and failed tests? and how many tests we run?
Coverage measurements
how much did you cover from the requirements and the code?
All that is in terms of coverage measurements.
Quality characteristics measurement
what are the quality attributes which you have checked?
the finding defects cost and our budget
is the exist criteria for me.
what is the risks of delivery of the early or
the late modules or software?
Now, we've finished all about planning
what happen in it and its activities
and Entry and Exist criteria. Now, we will move to a stage
called: Test Estimation. when a test leader
works with say a team of five testers,
he is required to estimate the effort
submitted from each tester in terms of
number of the tasks which assigned to him
and the timing set to each task. The estimation is not
haphazard, i.e. it is not his evaluation. they work according to
some approaches. These approaches can help him
to estimate tasks so as not to be unfair with his team members
by giving one some tasks above
his capabilities, or setting timing less than the task needs.
There are many approaches for the estimation .
we have only 2 of them:
Matrix based and expert based
he shall use the Matrix for estimating
the testing effort on the similar projects.
and check what time did it take and according
to this basis we start estimating. This is one of the ways.
There is another way , which is to estimate the test effort
based on the estimation made by the task owner
or by an expert.
He sets the time for a task to be 2 or 3 days.
So on what it is based here ?!
It is based on the experience of the owner of
this task. Once the test effort was estimated,
we will start make assignments to
the resources it will need.
Then we start to set a schedule for him to follow.
The testing effort depends on a number of factors.
includes: Characteristics of the product
and the characteristics of the development process
and the outcome of testing. So there are some elements
affect the test effort during working.
What are the characteristics of the product?
The quality of specification and information resulted from
the test basis we have,
the size of the product, the complexity
of the problem domain... all those elements
affect the effort. Which principle of testing
does it match, if you remember?
It matches contacts dependence.
According to the contacts, the way should be different.
and as the way is different, so the effort will be different.
Also the resources, we will work on or we may need,
will be different.
One of the factors that may affect the test effort
is the development process and stability
of the organization.
Does the effort I need in working for a professional
environment gives me enough time and resources for testing?
Or they conduct this testing unprofessionally?
Or they ask me to finish quickly under pressure,
while they don't have any processes.
Of course the development process also affects
the testing effort, as we don't work in isolation.
Testing is strongly affected with the development process.
The third element affects the test effort is the outcome of testing
which is the number of defects we found
and the amount of re-work required.
So, Whenever we find a problem that affects the product,
we are obliged to get it back to be fixed. Then it come back to be reviewed again.
All these factors affect the testing effort.
Here, We talk again about the test strategy and the test approach
This is what we call test strategy,
and the implementation of test strategy for specific project.
where is this test approach defined?
It is defined in the test plan and the test design.
in which we mention whether we work proactive or reactive.
In the starting point of a test process,
we should select the test design techniques
and the test types in which we mention and identify
as our entry and exist criteria.
On which basis do you select the approach? As you like?
No, It's based on the context. As we previously
mentioned that test is context dependent.
If there is a risk or safety critical
and based on the available resources,
will we start working early or at the very last end ?
The skills of other participants, the technologies, and the
type of system ; do we work COTS?
what are the testing objectives?
are there any regulations on you or not?
There are typical approaches.
There are 7 approaches, about which you are always asked.
what is required from these seven is to know
the differences between them. The first one is
the analytical approach. This one depends on my analysis
for the risk of the system. This is what we call
the risk based testing.
The test effort is mostly for
the modules that have the greatest risk.
So the analytical one analyzes system and
identifies the high risk modules, then it starts to direct the testing effort
towards them. The second approach is
to work on some statistical models or use
"The stochastic testing"
what is the stochastic testing? Here we use some
statistical data about the failure rate, on which
we will work. the stochastic testing has another
name which is "random testing".
It means that we input multiple inputs in a definite sequence
to the system, then check the system response.
In other words, by using this approach,
It will measure the reliability rate of the system
comparing to the bugs we find and fix in system.
There is another term called "operational profiles"
What is the operational profiles?
It is a model acts like a guide for
the software development in order to
help me to know which parts of
the software are operational and
non-operational.i.e. how much have we finished
of the development and what remains?
The third type is called
the methodical approaches.
This one deals with failures.
It includes an error guessing and fault attacks
This approach depends on the experience of the test engineer
This is an experience based technique, on which we determine
the way of testing or the approach of testing.
We may find another approach
believes that the way of testing is related
to the process or the standard complaince.
Do we follow this process or not?
like the industry specific standards
i.e. if we test medical context, we will follow our testing.
So we will be directed to leading
the available industry standards.
The Dynamic and heuristic
approaches, this is the fifth type.
Here, we work exploratory
which relies on specific hypotheses
based on my experience.
Some people use in testing an approach
called "consultative approaches".
This means that I get some
experts in business domain from outside the team.
Then I get their consultation test
on the coverage which we've finished.
Based on the experience of some people from domain
but out-side the test team.
Here we have a different approach called
regression averse approaches.
It is to reuse or retest all the existing test material
while you do the automation
which was modified.
then I will conduct regression testing and standard test all the time.
浏览更多相关视频
CH01.L04. Fundamental Test Process
ISTQB FOUNDATION 4.0 | Tutorial 5 | 1.4 Test Activities, Testware & Test Roles (Part-1) | CTFL
WhatsApp Video 2024 09 09 at 21 54 26 5e58471e
ISTQB FOUNDATION 4.0 | Tutorial 13 | 2.2.1 Test Levels & Test Types | Component Testing | CTFL
ISTQB FOUNDATION 4.0 | Tutorial 10 | 2.1 Impact of SDLC on Testing | Good Practices of Testing CTFL
CH05. L02. Test planning overview
5.0 / 5 (0 votes)