CH01.L04. Fundamental Test Process

Summary

This script outlines the five core activities of software testing: Test planning and control, test analysis and design, test implementation and execution, evaluating exit criteria and reporting, and test closure activities. It emphasizes the iterative nature of testing, detailing how objectives are set, test conditions and cases are designed, and discrepancies are identified and resolved. The importance of monitoring progress, adjusting plans, and confirming fixes through regression testing is highlighted, ensuring a thorough and controlled testing process.

Takeaways

• 📝 **Testing Process Overview**: The script outlines a five-step process for software testing, emphasizing the importance of each step and their interrelated nature.
• 🎯 **Test Planning and Control**: The first step involves setting objectives and monitoring progress, with a focus on adjusting the plan as needed to meet testing goals.
• 🔍 **Test Analysis and Design**: This step transforms test objectives into test conditions and cases, including the creation of a traceability matrix for tracking.
• 🛠️ **Test Implementation and Execution**: The longest phase, where test cases are developed, test data is created, and the environment is set up for testing.
• 🔄 **Discrepancy Management**: Discrepancies found during testing are reported and recorded, with a follow-up to confirm fixes and conduct regression testing.
• 📈 **Evaluating Exit Criteria**: Testing stops when predefined exit criteria are met, which can be based on various factors like test case completion or bug detection rate.
• 📋 **Reporting**: A summary report is prepared for stakeholders, detailing the testing process, outcomes, and any necessary follow-up actions.
• 🔚 **Test Closure Activities**: The final phase includes reviewing deliverables, documenting system acceptance, and archiving test materials for future reference.
• 🔄 **Continuous Monitoring**: Test control is an ongoing activity that involves comparing actual progress with the plan and taking corrective actions when deviations occur.
• 🔑 **Traceability Matrix**: A key tool in the testing process, used to link test conditions to test cases, ensuring comprehensive coverage and tracking of testing stages.
• 🛑 **Regression Testing**: After fixing discrepancies, it's crucial to conduct regression testing to ensure that fixes haven't introduced new issues in other parts of the program.

Q & A

What are the five main activities included in software testing according to the transcript?

The five main activities in software testing are Test planning and control, Test analysis and design, Test implementation and execution, Evaluating exit criteria and reporting, and Test closure activities.

What is the primary objective of the Test planning and control activity?

The primary objective of Test planning and control is to set the objectives of the testing process and to monitor all test activities to ensure they align with the plan, making adjustments as necessary.

How does test control differ from the other activities in the testing process?

Test control is an ongoing activity that involves monitoring the actual progress against the plan, reporting deviations, and taking corrective actions to achieve the project's objectives.

What are the two main tasks involved in the Test analysis and design activity?

The two main tasks in Test analysis and design are collecting and reviewing test bases for writing test cases and evaluating or finding test conditions from the test bases that can be used for testing.

What is a 'traceability matrix' and how is it used in the testing process?

A traceability matrix is an Excel sheet created during the Test analysis and design stage that lists all test conditions and links them to the test cases, allowing for bi-directional traceability to understand which testing stages have been completed and which requirements are yet to be tested.

What is the purpose of creating test cases during the Test implementation and execution activity?

The purpose of creating test cases is to provide a structured approach to testing, ensuring that all aspects of the software are tested systematically and that the testing process is thorough and repeatable.

What is meant by 'test harnesses' in the context of the testing process?

Test harnesses refer to the stubs and drivers prepared during the implementation stage, which are used to support the execution of test cases in a controlled environment.

Why is regression testing important after fixing discrepancies found during test execution?

Regression testing is important to ensure that fixing one bug has not introduced new bugs or affected other parts of the program, maintaining the overall stability and correctness of the software.

What are 'exit criteria' in the context of testing, and why are they significant?

Exit criteria are the targets or conditions that, when met, indicate that testing should stop. They are significant because they help determine when enough testing has been done or when it's time to halt testing due to reaching a predefined limit such as cost or time constraints.

What is the purpose of the 'Evaluating exit criteria and reporting' activity in the testing process?

The purpose of this activity is to assess whether the predetermined exit criteria have been met and to write a test summary report for all stakeholders, informing them of the testing stage reached and any necessary next steps.

What tasks are involved in the Test closure activity of the testing process?

Test closure activities include checking the delivery of planned deliverables, closing any open modifications, documenting system acceptance, finalizing and archiving the test ware, and handing over the test ware to the maintenance organization.

Outlines

00:00

📝 Test Planning and Control Overview

The first paragraph introduces the five main activities of software testing, emphasizing the importance of remembering the major tasks of each activity for certification purposes. The initial activity, Test Planning and Control, is broken down into setting objectives for the testing process, such as defect detection or informing decision-makers about software readiness, and test control, which involves ongoing monitoring of test activities against the plan. Deviations are reported, and corrective actions are taken to ensure the project meets its objectives. Feedback from monitoring is crucial for modifying the plan and implementing necessary changes.

05:00

🔍 Test Analysis, Design, and Implementation

The second paragraph delves into the Test Analysis and Design activity, where test objectives are translated into test conditions and cases. It involves collecting test bases, evaluating existing material for testing, and designing test cases with prioritization. Test data preparation and environment setup are also part of this stage, along with the creation of a traceability matrix for tracking test progress and requirements coverage. The Implementation phase follows, where test cases are developed into procedures and suites, and test harnesses are prepared if needed. The environment setup is verified, and the traceability is confirmed. The Execution phase involves running test procedures, comparing expected results with actual outcomes, and recording discrepancies. It also includes confirmation testing to ensure fixes are effective and regression testing to verify that fixes do not introduce new issues.

Keywords

💡Test Execution

Test Execution refers to the process of running test cases against the software to verify its behavior and identify defects. It is a fundamental part of the testing process, as it directly contributes to the discovery of bugs. In the script, test execution is mentioned as a clear part of testing where testers compare expected results with actual outcomes to identify discrepancies.

💡Test Planning and Control

Test Planning and Control is the activity that sets the objectives and monitors the progress of testing to ensure it meets the project goals. It is crucial for aligning testing efforts with the overall project strategy. The script describes setting objectives such as finding defects or informing decision-makers about software readiness and emphasizes ongoing monitoring and corrective actions based on deviations from the plan.

💡Test Analysis and Design

Test Analysis and Design involves turning test objectives into tangible test conditions and test cases. It is a preparatory phase where testers review requirements and other documents to create a foundation for test cases. The script mentions collecting test bases and evaluating existing conditions for testing, leading to the design of test cases and the creation of a traceability matrix.

💡Test Implementation and Execution

Test Implementation and Execution is the phase where the designed test cases are put into action. It includes developing test procedures, creating test suites, and preparing test data. The script describes this as a two-step process: first, implementing the test cases, and second, executing them to compare expected results with actual results, which may lead to identifying discrepancies.

💡Evaluating Exit Criteria

Evaluating Exit Criteria is the activity of assessing whether the predetermined targets for stopping testing have been met. These criteria can be based on various factors such as the percentage of test cases executed or the number of bugs found. The script explains that testers should check if these criteria have been achieved by reviewing test logs and deciding if further testing is necessary.

💡Reporting

Reporting in the context of software testing refers to documenting the results of testing activities for stakeholders. It includes summarizing test outcomes and providing insights into the software's quality. The script mentions writing a test summary report to communicate the testing stage reached and the results of the testing process.

💡Test Closure Activities

Test Closure Activities mark the final stage of the testing process, where all testing artifacts are reviewed, documented, and handed over. It includes verifying deliverables, finalizing reports, and archiving test materials. The script describes this as an essential step to ensure that all testing efforts are concluded and lessons learned are documented for future projects.

💡Discrepancies

Discrepancies are the differences found between the expected results and the actual results during test execution. They indicate potential defects or issues in the software. The script discusses the importance of reporting and recording discrepancies to ensure they are addressed and verified as fixed through confirmation testing.

💡Regression Testing

Regression Testing is the process of re-testing the software to ensure that changes made to fix defects have not introduced new issues in other parts of the program. It is a safeguard to maintain the overall quality of the software. The script differentiates regression testing from confirmation testing, emphasizing its role in verifying the impact of bug fixes on the entire system.

💡Traceability Matrix

A Traceability Matrix is a tool used in test design to link test conditions to test cases, ensuring comprehensive coverage of requirements. It helps in tracking which test stages have been completed and which requirements are yet to be tested. The script mentions creating a traceability matrix during the Test Analysis and Design phase to facilitate bi-directional traceability.

💡Test Conditions

Test Conditions are the elements or scenarios identified during test analysis that can be tested. They form the basis for creating test cases. The script describes evaluating existing test bases to find what can be used for testing, which are referred to as test conditions, serving as the starting point for test case design.

Highlights

Testing for software includes 5 main activities that may overlap during the test process.

Test planning and control involves setting objectives and monitoring progress against the plan.

Test objectives may include finding the maximum number of defects or informing decision-makers about software readiness.

Test control is an ongoing activity that involves monitoring and taking corrective actions if deviations occur.

Test analysis and design turns test objectives into perceptible test conditions and cases.

Analysis involves collecting and reviewing sources like requirements and risk analysis reports for writing test cases.

Test conditions are identified from test bases and used to design test cases and prioritize them.

Test data preparation and environment setup are part of the test design process.

A traceability matrix is created to link test conditions to test cases for bi-directional traceability.

Test implementation involves developing test procedures and creating test suites.

Test data is created during the implementation stage, and test harnesses are prepared if needed.

Test execution compares expected results with actual results and records discrepancies.

Discrepancies found during execution are reported and recorded in a test log.

Confirmation testing is conducted to ensure that identified discrepancies have been fixed.

Regression testing is performed to ensure that fixing one bug does not introduce new issues.

Evaluating exit criteria involves checking if predetermined targets for stopping testing have been met.

Reporting includes writing a test summary report for stakeholders to understand the testing stage reached.

Test closure activities include checking deliverables, finalizing reports, and documenting system acceptability.

Test ware includes all outputs from the testing process, such as test cases, suites, and procedures.

Handing over test ware to the maintenance organization helps in learning lessons for future projects.

Transcripts

The most clear part of testing is

the test execution process we do for the

test cases. But actually

the testing for any software

includes 5 main activities.

Please note that in this part

you are required to concentrate and remember,

in each activity we will mention,

what are the major tasks which distinguish

this activity from others? Because you will be

be asked about it in details in the certificate.

They are as follows:

Test planning and control,

Test analysis and design,

Test implementation and execution,

Evaluating exit criteria and reporting,

Test closure activities,

Although their order is

logical, but

these activities, during the test process, may

overlap with another activity.

or you may find 2 activities

start simultaneously with each other. Each one of

the previous activities is divided into 2 halves

The first activity in test process

is Test Plan and Control.

so what do we do in a test plan?

and what are the major tasks of it?

Here we set the objectives

of the testing process in general.

like what?

For example, if we need to find the largest

possible number of bugs, which we call

find defects as much as we can.

or if our objective is

to give information to those who

work on decision making

to decide whether this software is ready

to be presented to market now or not

and so on.

So you will find many similar objectives.

and whatever the testing objectives, you'll

always need a description for the test activities

which will achieve your test

objectives. The second part

of the first activity is "test control".

To conduct a correct test control

for your project, you should

keep monitoring all the test activities

as long as you work on the project.

That's why the test control is

the ongoing activity. i.e. as long as

we are testing, the test control

is ongoing and we keep

monitoring all the time. So,

what are we doing in a test control in details?

you observe whether your plan

is the same as what happens in reality

which is called the actual progress

or not. Then if there is

a deviation in the actual progress

from the plan, We should report

the status including the deviation

from the plan. As long as

We have deviations you have to take

an corrective action to be able to

achieve the project's objectives,

and to implement the plan as expected.

For example, if we planning

to finish testing in one week,

then while working we discovered that

one week is not enough,

In this time, we will apply the steps of test

control, which we compare

the actual progress against the plan

then we report the status

and take action. Note that

the feedback we get from monitoring

and controlling the activities is

what the test control depend on

to modify the plan and to

implement the required action for modifications.

The second activity in test process

is Test analysis & design.

in which the test objective

is turned to be perceptible

in the form of test condition & test cases.

This activity is also divided into

2 partitions. In analysis,

we collect and review

the sources we will depend on

to write the test cases

which we call "test bases".

Requirements,

risk analysis reports,

software structure

and interface specifications.

There are a lot of the resources that

can help you in writing test cases.

The second task

is evaluating or finding

what is already exists in the test bases

which can be used for testing

this is what we call "test condition"

which means that anything

can be tested, we call "test condition".

From test conditions,

the partition of test design will start.

The first thing we will design

is the test cases.

Then, we will prioritize the test cases.

which of these cases is more important

which is less important

in testing.

In test cases,

we always have input data,

which we input into the program during the

test case execution, then it results as an output.

This is what we call

the "test data", which

we prepare and identify

in this stage.In addition to designing

the environment setup in this

analysis and design process.

In which we identify the infrastructure

and the tools we will need during testing process.

In this stage we create an excel sheet

and name it "traceability matrix"

where we add all the test conditions

and link them to the test cases.

So through this sheet we can

have a bi-directional tractability

which helps us to know what stages of testing

have finished and what requirements

are not tested yet. In the previous stage

we have designed it by ourselves.

In this stage, the implementation starts.

this can be considered the longest thing

and it is also done

through 2 steps: Implementation

then Execution. we start

the implementation of the test cases,

then we develop the test procedures

which is ordering the test cases

that we will go through during the execution.

then we get out of the test cases,

to create test suites which gathers

the test cases.

we should remember that

the test data was identified

during the previous stage,

while in this stage it is being created.

If our testing needs

stubs and drivers,

so we prepare them during the implementation

stage and call them "test harnesses".

Finally, in the implementation partition

we verify whether the

test environment has been set up or not.

we also verify that

the bi directional traceability sheet

between the test bases and

the test cases is correct.

The second step for this stage

is the execution for the test procedure

whether it is manually or

by using execution tools.

as we've learned,

in execution we compare the expected result

with the actual result we got.

so if we find that

ER doesn't equal AR, we will call it

Discrepancies .

The discrepancies we've resulted during the execution

should be reported and recorded.

it is not a report only for the discrepancies,

but to all the result information

from the test execution we did.

such as: the tester name

the date of execution, the actual result,

the test result (pass / fail)

The tested Build/server, etc.

This file we call the test log.

the verb we will use for

this writing process is logging

and recording. For sure after

finishing the report, we need to

conduct a confirmation testing.

which is making a re-execution or

re-testing again to check whether

the discrepancies found before were

fixed or not. If they were fixed

we mark it with a confirmation

that they are fixed. Last note,

finding a discrepancy in the software

and getting sure that they were fixed

is a good point, but it's not enough.

that's why we should conduct a regression testing.

which means to re-test again

and re-executions for the other modules

in the program, to be sure that when

those defects were fixed, didn't cause

problems in another part in the program,

and the program keeps working correctly.

we should notice the difference between

the purpose of the retesting which is

ensuring that the bug was fixed.

while the purpose of the regression testing

is ensuring that the fixed bug didn't

affect the other parts of the program

and didn't cause new bugs.

For the fourth activity which has to be

applied on each test level, which is

Evaluating exit criteria and reporting.

First of all, from the activity's name

there is a new expression which is

"Exit criteria"

the exit criteria are the targets

that when achieved, we will stop testing.

such as: after 80 % of test

case execution we will stop testing.

or for example, we've reached

a particular percentage of finding bugs

in a software at which we will stop.

or according to a particular cost or

time we will stop testing.

These criteria differ from

one project to another.

Here, we check the exit criteria,

which we determined in the test plan,

have been achieved or no.

By getting back to the test logs

that we wrote in the previous stage and

we recorded all the outcomes of

the execution process,

you can asses whether you need more

testing or not. Does the exit criteria

need to be changed?

As for the reporting section,

it is to write a test summary report

for all the stakeholders to know the stage

we've reached. Finally, the last activity

of testing process.

The Test closure

is the last stage which we apply

at each milestone of the project.

Examples for the milestone

in the project:

a software system is released

a test project is completed or cancelled

a maintenance release has been completed

the test closure tasks:

here you'll check which

planned deliverables were delivered

and which ones were not.

The closing i.e. to close the report

and any open modifications to any

part of the project which weren't finished yet.

Documenting: To document that

the system is acceptable and agreed upon.

Finalizing and archiving the test ware.

What is meant by the test ware is

all we have got from the testing process

like: test hardness ,

the tools we used,

the test cases we have,

test suites and procedures with

their resulted data. The output of

all these test activities is called

the test ware.

handing over:

to hand over the test ware

to the maintenance organization.

the information we've collected will help us

to know the learned lessons, and

to determine what needs to be changed.

and what we can do better in testing

the future projects.

Now we have finished

the test process part

in chapter 1. the completion

of this part with the rest of terms is

in the first video of chapter 4.

you can review the part we've finished

in the file entitled "fundamental test process