CH05. L04. Test monitoring and control
Summary
TLDRThis script delves into the crucial aspects of test progress monitoring and control within a project. It outlines the roles and responsibilities of a test team and leader, emphasizing the importance of test planning and its implementation. The video discusses various monitoring methods, from manual to automated tools, and the significance of test metrics to gauge progress. It also covers test control actions, including reactive and preventive measures, and concludes with the creation of a test summary report, adhering to IEEE829 standards, to communicate comprehensive findings and recommendations to stakeholders.
Takeaways
- đ **Test Progress Monitoring**: The test manager observes and compares the actual progress of test activities against the planned results.
- đ **Monitoring Methods**: The choice between manual and automated tools for monitoring depends on the team size and project scale.
- đ **Test Metrics**: Eight key metrics are used for monitoring, including test case preparation, execution, defect density, and test coverage.
- đ **Defect Information**: Monitoring helps in understanding the number of defects found, fixed, and the failure rate, which is crucial for quality assessment.
- đ **Test Environment**: The preparation of the test environment and its impact on the overall testing process is a monitored aspect.
- đ **Defect Density**: A metric that measures the number of bugs found relative to the lines of code, indicating software quality.
- đ **Test Coverage**: The extent to which test cases cover requirements, risks, or code is a critical metric for comprehensive testing.
- đ€ **Subjective Confidence**: The level of confidence a tester has in the testing process, which can be an indicator of test quality.
- đ **Test Milestones**: Monitoring includes tracking the dates of test milestones to ensure the project stays on schedule.
- đ° **Test Costs**: Understanding the costs associated with testing can help in decision-making regarding further defect searching or test execution.
- đ **Test Control**: After monitoring, identifying deviations from the test plan and implementing corrective or preventive measures is essential.
- đ **Test Summary Report**: A comprehensive report submitted to stakeholders, analyzing results, metrics, and providing recommendations for future actions.
- đ **IEEE829 Standard**: The test summary report follows the IEEE829 standard, ensuring a structured and standardized format.
- đ **Report Components**: The report is divided into eight parts, including identification, summary, variations, assessment, evaluation, activities summary, and approvals.
Q & A
What is the primary role of a test manager in the context of test planning?
-The test manager is responsible for setting the test planning to be followed during the project and overseeing the activities related to it.
Why is monitoring test progress important?
-Monitoring test progress is crucial for providing feedback on test activities, observing the progress of test planning activities, and comparing it to the actual plan results.
How does the choice of monitoring method depend on the team size?
-For smaller teams, manual monitoring through documents, spreadsheets, or databases is feasible. However, larger teams working on larger projects require automated tools for efficient data gathering and monitoring.
What are the eight important points that test metrics measure during monitoring?
-Test metrics measure the percentage of work done in test case preparation and execution, test environment preparation, defect information, defects density, defects found and fixed, failure rate and retest results, test coverage of requirements, risks, or code, subjective confidence level of testers, and test costs.
How does monitoring help in understanding the test cases' status?
-Monitoring helps clarify the percentage of test cases that have been run, which ones have passed and failed, and provides an estimate of the work done in test environment preparation.
What is meant by defects density in the context of software testing?
-Defects density refers to the number of found bugs in proportion to the number of lines of code in the software being tested.
What is the purpose of the test control stage after monitoring?
-The test control stage involves identifying deviations from the test plan, deciding on recovery or preventive actions, and determining whether to correct issues or skip them based on their impact.
What are some common test control actions a manager can take?
-Common test control actions include rescheduling testing, increasing the number of testers, re-ordering test priorities, and deciding which parts can be skipped without testing.
What is the significance of the test summary report?
-The test summary report is important for submitting comprehensive reports to stakeholders, analyzing results and metrics, and setting recommendations for future actions.
What standard is commonly followed for the format of the test summary report?
-The IEEE829 standard is commonly followed for the format of the test summary report.
Can you describe the eight parts of the test summary report as per IEEE829 standards?
-The eight parts are: 1) Summary report identification, 2) Summary, 3) Variations, 4) Comprehensive assessment, 5) Overall status of incidents, 6) Evaluation, 7) Summary of activities, and 8) Approvals.
Outlines
đ Test Progress Monitoring and Control
This paragraph discusses the importance of monitoring and controlling test progress within a project. It highlights the roles and responsibilities of a test manager in setting and following test plans, as well as gathering feedback on test activities. The paragraph emphasizes the need for a monitoring method chosen by the team, which could be manual for small teams or automated for larger projects. It also outlines the use of test metrics to measure eight key points, including the percentage of test cases prepared and executed, defect density, and test coverage. The paragraph concludes with the discussion on test control actions, such as rescheduling and resource allocation, and the significance of a test summary report in providing comprehensive insights to stakeholders and setting recommendations for future actions.
đ Test Summary Report Structure
The second paragraph delves into the structure and purpose of a test summary report, as per IEEE829 standards. It underscores the test manager's responsibility in creating this report during the project's execution phase. The summary report serves to submit comprehensive project test process reports to stakeholders and to analyze results and metrics for future recommendations. The paragraph outlines the eight-part structure of the report, starting with identification and including sections on what was tested and its results, deviations from the plan, comprehensive assessment of test objectives, overall incident status, evaluation of quality based on test plan criteria, summary of activities, and finally, the approvals section with signatures and authority confirmations.
Mindmap
Keywords
đĄTest Progress Monitoring
đĄTest Manager
đĄTest Planning
đĄFeedback
đĄTest Metrics
đĄDefect Density
đĄTest Coverage
đĄTest Control
đĄTest Summary Report
đĄIEEE829
đĄStakeholders
Highlights
Test progress monitoring and control are essential for relating all issues together in the testing process.
Understanding the rules and responsibilities of a test team, including the test leader's role in setting and following test plans, is crucial.
Monitoring test progress involves comparing actual results with the test plan to ensure alignment.
The choice of monitoring method depends on the team size and project scale, ranging from manual to automated tools.
Test metrics measure eight key aspects, including test case preparation, execution, and defect information.
Monitoring helps clarify the percentage of test cases run, passed, and failed, and defect density.
Defects found, fixed, and the failure rate are essential metrics for understanding test progress.
Test coverage of requirements, risks, or code is a critical metric for assessing testing thoroughness.
Subjective confidence levels of testers provide an indication of the perceived quality of testing.
Matrix monitoring provides dates of test milestones, aiding in planning and decision-making.
Test costs are an important metric to determine the continuation of defect searching and testing.
The monitor stage is divided into quantity and calculation, covering test case numbers, defect numbers, and subjective confidence levels.
Calculations such as failure rate, defects density, and defect detection percentage are vital for analyzing test effectiveness.
Test control involves observing deviations from the test plan and implementing corrective or preventive actions.
Common test control actions include re-scheduling, increasing tester numbers, and prioritizing test cases.
The test summary report is a comprehensive document detailing the test process, results, and metrics.
The summary report serves to inform stakeholders, analyze results, and provide recommendations for future actions.
The IEEE829 standard outlines the format for the test summary report, consisting of eight parts.
The report includes identification, summary, variations, comprehensive assessment, overall status of incidents, evaluation, summary of activities, and approvals.
Transcripts
We'll talk about Test progress monitoring and control
In order to relate all issues together,
let us mention quickly that
we've learned the rules and responsibilities
of a test team,
the responsibility of a test leader
or a test manager towards the task,
as he set the test planning
to be followed during the project
as well as the activities.
Now, when we start implementing the rest of a test planning
we will need to monitor
the test progress and get a feedback
about the test activities of a tester.
So in the test monitor stage,
the test manager observes the test planning
activities progress,
then he compares it to the actual plan results.
Our choice of a monitoring method
is set by the team.
If a project team has few members,
I will be able to monitor manually,
either by documents, spreadsheets, databases.
However, if we work in a large team for a large project,
in this case we will need a tool
which is on a high a level of efficiency
in order to gather all the monitoring data
so it needs to be done by
automated tools. The test metrics
can be also used during monitoring
to measure 8 important points:
1-The percentage of both
the work done in test cases preparation
and the test cases execution.
Here, Monitoring is useful in
clarifying the percentage of
the test cases we've run
and the test cases which are
passed and failed.
2- We also can estimate the percentage of
work done in test environment preparation,
in addition to the defect information.
3-These information can help us
to know the defects density which means
that the number of found bugs
In proportion to the number of code's lines
of the software we testing
4- we can also know the defects found and which ones are fixed.
In addition, it helps us to know
the failure rate,
and the retest results.
5- It also helps us to know test coverage of requirements, risks
or code. 6- Moreover, it gives us
indication about subjective confidence
level of tester .
7- we'll find that matrix monitor
gives us the dates of test milestones.
8-Through this, We can know the test costs
that can assist me in deciding whether
to continue searching for the next defect
or when I'll be able to run
the next test.
Again, the matrix can divide
the monitor stage into
quantity and calculation.
This means that anything related to
the number of test cases which has been
run or not,
and the number of test cases
either passed or failed,
Also, the Number of defects which we've founded
and fixed, comparing the date of
the test milestones and the scheduled
with the reality, and finally
the subjective confidence level of testers.
these are all for the quality metrics.
what about calculations?
such as: failure rate,
defects density,
and defect detection percentage.
This was the test monitoring part.
So what about the test control?
After finishing the observation
and gathered the needed information
in the test monitoring stage.
So now, I need to find out the deviations that
happened in the test planning
and to know how to recover them
which is called "The Reactive ";
or at least, the test manager,
can prevent this problem from happening in the next stages.
This is called "preventive ",which depends
on the project itself.
Can we skip these bugs
or should we correct them?
Will these deviations affect the testing team?
So what are the common test
control actions that a manager can do?
He can re-schedule the testing.
or increase the number of testers.
and as a last solution, is to re-order the most important
to the least important.
In addition to which parts can be skipped without testing.
The test summary report:
Now we've finished what happens in the test planning
and its activities
and how to monitor and control
the test process.
The next stage is "summary report".
The test manager is the responsible for it
in the execution stage of the project.
So what is the benefit of the summary report?
we always make reports during the test process.
Note that:
the summary report has very important objectives.
- We use it to submit comprehensive reports to the stake holders
about the test process of the project.
- It also helps us to analyze results
and the metrics we gathered through which
we can set recommendations for future actions.
Does the summary report have a definite format to be followed?
yes, of course.
It is a common question in the ISTQB exam.
We always follow the IEEE829 standards
to submit the summary report.
So what is the format of our report?
It is divided into 8 parts:
First part is about the summary report
identification, in which we write the
name and the version of the project
The second part is the summary
in which we write the software tested version.
We also can add a list of tested items and
test environment and all data related to
what was tested and its results
whether interface or modules.
The third part is about variations.
what are the deviations that happened
out of the plan?
and what are their reasons?
The fourth part is comprehensive assessment.
Here we mention what was covered
and know what were my planned test
objectives
in the test planning.
and which objective were covered.
The fifth part:
in which we mention the overall status of
the incidents we found.
It's better in this part
not to use quantities, and use
calculations and ratios.
which should make it easier
to stakeholders.
The following part is about evaluation.
When we were setting the test plan
we had definite specifications
to be followed.
Here, we need to assess
the overall quality which resulted
based on the criteria specified
in the test plan.
Summary of activities: this part is related to
the stuff worked on the testing project.
whether on time or cost basis.
It compares what was written in
the test plan with what was actually done
during the run time.
Of course, if there is a difference between them,
we should mention the reasons of these variances .
The last part is the approvals which is
a list for all the signatures
and approval authority of the IEEE summary report.
Voir Plus de Vidéos Connexes
ISTQB FOUNDATION 4.0 | Tutorial 52 | Test Monitoring & Test Control | Test Metrics | ISTQB Tutorials
ISTQB FOUNDATION 4.0 | Tutorial 53 | Test Progress Report | Test Summary Report | CTFL Tutorials
CH04. L01. Test development process
CH01.L04. Fundamental Test Process
ISTQB FOUNDATION 4.0 | Tutorial 6 | 1.4 Test Activities, Testware & Test Roles (Part-2) | CTFL
CH05. L03. Test planning activities and time estimation
5.0 / 5 (0 votes)