L17 Integration Testing
Summary
TLDRThis tutorial by Phil Copeman delves into integration testing, emphasizing its importance post-unit testing to ensure components work harmoniously as per high-level design. It focuses on testing component interfaces and interactions, avoiding common pitfalls like skipping to system testing or lacking traceability. The tutorial illustrates the process with a vending machine example, highlighting the need to check not just the final output but the entire sequence of actions. It also discusses the role of message dictionaries in automotive diagnostics, emphasizing the necessity of testing various message types and values to validate system behavior against design specifications.
Takeaways
- 🔍 Integration testing is conducted after unit testing to see how various pieces of the system work together.
- 🏗️ It traces back to the architectural and high-level design work, ensuring components not only function internally but also interact as intended.
- 🔄 The primary focus of integration testing is on the interfaces between components, assessing if they handle input sequences and interactions correctly.
- 📈 Integration testing aims to confirm that modules align with the high-level design, especially in terms of sequence diagrams.
- 🔄 Avoid the anti-pattern of skipping straight to system testing after unit tests, as this might miss subtle interaction issues.
- 🔗 Ensure traceability from integration tests to the high-level design to maintain the connection between testing and design intent.
- 🚫 Integration tests should not be pass/fail based on system function alone but should focus on the correctness of interfaces and interactions.
- 🛠️ A simple integration test example involves a vending machine sequence diagram, where the test checks if the system correctly processes a coin insertion and subsequent actions.
- 📊 Integration testing should cover all sequence diagrams, testing nominal behaviors, missing inputs, false preconditions, and invalid sequencing.
- 📝 High-level designs often include interface descriptions, such as message dictionaries, which should be tested for message structure, field values, and exception handling.
- 🔑 Best practices in integration testing emphasize the importance of tracing tests to the high-level design and covering all interactions to ensure the system works as intended.
Q & A
What is the primary purpose of integration testing?
-The primary purpose of integration testing is to ensure that the various pieces of a software system work together as intended after individual components have been unit tested. It focuses on the interactions between components and their adherence to the high-level design.
Why is it important to conduct integration testing after unit testing?
-Integration testing is important because unit testing alone cannot identify interaction problems between components. It ensures that the components, which have been individually tested, function correctly when combined and that the overall system matches the high-level design.
What are the potential anti-patterns or pitfalls of integration testing mentioned in the script?
-The script mentions three anti-patterns: skipping straight to system testing after unit testing without conducting integration tests, lacking traceability from integration tests to the high-level design, and basing integration test pass criteria on system functionality rather than on the proper functioning of interfaces.
How does integration testing relate to high-level design?
-Integration testing is intended to verify that the modules of a system match the high-level design, particularly focusing on the sequence diagrams and ensuring that the system behaves as the design specifies.
What is a sequence diagram and how does it relate to integration testing?
-A sequence diagram is a type of diagram in the high-level design that shows the interactions between objects or components in a system over time. Integration testing uses these diagrams to check that all inputs result in correct outputs and that every component interface is exercised as expected.
What should be the focus of an integration test for a vending machine as described in the script?
-The focus of an integration test for a vending machine should be to exercise a specific sequence diagram, ensuring that all inputs lead to correct outputs, every component interface is tested with relevant values, and the sequence of actions matches the expected behavior as per the high-level design.
Why is it incorrect to base the pass/fail criteria of an integration test solely on the final output?
-Basing the pass/fail criteria solely on the final output is incorrect because it overlooks the importance of the intermediate results and the sequence of actions. Integration testing should verify that all arcs appear in the expected sequence and that all side effects and timings happen as expected according to the high-level design.
What is an example of an interface description used in integration testing?
-An example of an interface description is the OBD2 parameter ID message dictionary, which defines automotive operational parameters and their associated data structures or fields in network packets, allowing for the testing of message structures, values, and exception handling.
How should integration testing be approached in terms of best practices?
-Integration testing best practices involve focusing on the interaction of components, tracing tests to the high-level design, exercising all arcs on every sequence diagram, and covering all modules, network interfaces, message types, and data fields to ensure the system works as intended.
What are the two main pitfalls of relying solely on system testing instead of conducting integration testing?
-The two main pitfalls are missing system integration edge cases where the system may appear to work but the internal logic is incorrect, and the difficulty of exercising off-nominal sequence diagrams at a system level, which can lead to overlooking specific situations that the system will not handle as intended.
Why is traceability from integration tests to the high-level design important?
-Traceability is important because it ensures that the integration tests are based on a clear reference point, allowing testers to verify whether the system is behaving as the high-level design specifies. Without traceability, there is no way to know if the system is functioning as intended.
Outlines
🔍 Understanding Integration Testing
This paragraph introduces the concept of integration testing as a process that occurs after unit testing. It emphasizes the importance of ensuring that individual components, which have been verified through unit tests, work together as intended. The focus is on the interfaces between components and whether they can handle all types of data and interactions as specified in the high-level design. The paragraph also highlights common anti-patterns, such as skipping integration testing and basing pass criteria on system functions rather than interfaces. An example of a simple integration test for a vending machine is provided to illustrate the process of verifying sequence diagrams and ensuring that the software implementation matches the high-level design.
🛠️ Integration Testing Best Practices and Pitfalls
The second paragraph delves into best practices for integration testing, such as tracing tests back to the high-level design and ensuring that all sequence diagrams are covered. It discusses the importance of testing message structures, including a range of valid and invalid values, and handling exceptions like bad checksums or sequence numbers. The paragraph also warns about the pitfalls of relying solely on system testing, which can miss critical system integration edge cases and fail to exercise off-nominal sequence diagrams. The goal is to stress the interactions among components and ensure that the system works not only at a functional level but also adheres to the high-level design.
📝 The Importance of Traceability in Integration Testing
The final paragraph underscores the significance of traceability in integration testing. It points out that without a high-level design to reference, there is no clear way to determine if the system is behaving as intended. Skipping integration testing can lead to a lack of understanding of whether the system's internal logic aligns with the high-level design. The paragraph stresses the necessity of integration testing to ensure that the system's components interact correctly and that the overall system functions as designed.
Mindmap
Keywords
💡Integration Testing
💡Unit Testing
💡High-Level Design (HLD)
💡Sequence Diagram
💡Component Interfaces
💡Anti-Patterns
💡Traceability
💡System Function
💡Message Dictionary
💡Validation Test Suite
💡Best Practices
Highlights
Integration testing is a process that occurs after unit testing to ensure that various pieces of a software system work together as intended.
The purpose of integration testing is to exercise all component interfaces and verify that they handle input sequences and interactions correctly according to the high-level design.
Integration testing should not duplicate unit testing efforts but focus on interaction problems that unit testing cannot find.
Skipping integration testing and moving straight to system testing can lead to missing subtle interaction problems.
Traceability from integration tests to the high-level design is crucial for ensuring the implementation matches the intended design.
Integration testing pass criteria should be based on interfaces, not just system functionality.
A simple integration test example involves a vending machine sequence diagram, where the test checks if the machine handles a coin insertion correctly.
Integration tests should monitor the cascade of actions in a sequence diagram to ensure they occur as expected.
The pass or fail of an integration test is determined by whether the system behaves according to the sequence diagrams, not just the final output.
Integration testing coverage includes checking all arcs on sequence diagrams, testing nominal behaviors, and handling missing inputs or preconditions.
Integration tests should also verify the correct implementation of side effects and timing within the system.
High-level designs often include interface descriptions, such as message dictionaries, which should be tested in integration.
Integration testing should exercise message structures, testing a range of values, valid and invalid fields, and message types.
Components should be accompanied by a validation test suite to ensure proper support for different message types.
Best practices in integration testing involve concentrating on component interactions and ensuring they satisfy high-level design aspects.
Pitfalls of system testing alone include missing system integration edge cases and difficulty in exercising off-nominal sequence diagrams.
Skipping high-level design traceability in integration testing leads to guessing the system's intended behavior.
Integration testing is essential to confirm that the system not only works but also operates exactly as the high-level design specifies.
Transcripts
this is phil copeman with a tutorial
on integration testing
integration testing sits part way up the
v
on the right hand side the idea of
integration testing
is after you're done with unit test and
the various pieces
have been checked out you then see how
the various pieces work together
integration test traces back to the left
hand side of the v
where the architectural and high-level
design work takes place
in integration testing several
components
are tested as a set each component has
an internal function
which has already been assessed by unit
test but also has communications
or the ability to control other
components
it's the arrows between components that
are the primary emphasis of
integration testing the idea of
integration testing is to exercise
all the component interfaces and ask
questions such as
did the input sequences lead to correct
responses
and can all types of data and all types
of interactions be handled by the
interfaces
as they're supposed to according to the
high-level design
integration testing is intended to make
sure that the modules match the
high-level design
including especially the sequence
diagrams
when you're doing integration testing
the idea is not to redo
things that have already been found in
unit testing
you assume unit testing has already
happened and instead
concentrate on the types of things unit
testing can't find
which primarily are interaction problems
the anti-patterns for integration
testing include
skipping straight to system test once
you're done with the unit test
you could just try and run the system
but you can easily miss
subtle interaction problems that result
in the system
almost working and you not knowing that
there's some sort of issue
that the implementation does not match
the intended high level design
because things mostly work and you don't
notice the difference
another anti-pattern is no traceability
from integration test to the high level
design
the point of integration test is to
compare against the high level design
a third anti pattern is that the
integration test pass criteria based on
system function
not interfaces that means if you run
an integration test and you ignore all
the interactions and just say yup sure
enough the system seems to work at the
end
that's not an integration test because
you're not paying attention to the
intermediate results
that's simply a system test and you're
not actually doing
integration testing
let's take a look at the types of things
that would happen in a very simple
integration test
the sequence diagram on the right shows
what happens when a coin is put into a
vending machine
an integration test tracing to this
sequence diagram
would in step one initialize all the
modules
in step 2 it would make sure as part of
or
just after the initialization each
precondition is satisfied
so that the sequence diagram can be
activated
in step 3 the initial action that
triggers the cascade of events in the
sequence diagram is fed to the system
in this case a coin is inserted
after that the integration test consists
of monitoring to make sure
that the cascade of actions in the
sequence diagram
actually takes place the coin in signal
is received
in four and five there's a side effect
that happens there may not be a way to
observe it but in number six
the result of that side effect is
observable
in the end the point of the integration
test is
to exercise a specific sequence diagram
you check that all inputs result in
correct outputs
every component interface is exercised
across the whole set of sequence
diagrams
with all the relevant values with all
the relevant timing and sequencing
if any of the sequence diagrams do not
behave as expected
that means the software does not
correctly implement the high level
design
and you have some sort of integration
testing failure
the pass fail criteria is whether or not
the system behaves as the sequence
diagrams
say the system should behave for
integration testing coverage
the questions you tend to ask are are
all arcs on all sequence diagrams
exercised are off nominal behaviors
tested
what happens if one of the inputs to a
sequence diagram is missing
what happens if one of the preconditions
is false does the sequence diagram
incorrectly trigger
do you have invalid sequencing do you
have extraneous outputs
and so on
it's important to realize that the point
of an integration test based on a
sequence
diagram is not simply the end result
in this example there's a sequence
diagram that shows a
vending machine that takes two coins for
a purchase
receiving a third coin and refunding it
automatically
going back to two coins inside the
machine ready for purchase
the integration test for this sets up
the machine
make sure that the machine thinks it
already has two coins and then
pops in another coin that initiates the
test
the remainder of the test is observing
the coin in signal arrives
the coin count increases and it thinks
it has a third coin
then it refunds the coin by exercising
the coin out actuator
and the coin count goes back to two
simply looking that the final coin out
is two
is not how you determine pass fail for
this integration test
this integration test only succeeds if
it notices it has an extra coin and it
refunds the extra coin instead of just
silently eating it
and then goes back to the right number
of coins only observing the final test
output would not tell you whether it
actually refunded the coin which is the
whole point of this sequence diagram
and this integration test what this
example illustrates
is that integration test is not simply a
pass on the final output
but rather did all the arcs appear in
the expected sequence
did all the timings happen as you
expected did all the side effects happen
as you expected
in other words it is not simply that the
pieces manage to work together more or
less
it's that they manage to work together
exactly as they're supposed to according
to the high level design
in addition to sequence diagrams it is
common for high level designs
to also have some sort of interface
description
many interfaces look like messages one
way or another
here's an example of the obd2 parameter
id message dictionary
which makes automotive operational
parameters available
via diagnostic port this is a typical
sort of message dictionary in that
each message which might be an actual
network message or it might be
a data structure in memory that you can
access has a descriptor
with a categorical value saying what
kind of message is it
is this the engine speed is the engine
coolant temperature
is it the accelerator pedal position and
so on
once you know what the enum what the
categorical value is you can then
interpret the associated data in a data
structure or
fields in a network packet based on what
that identifier is
as an example if the enum says it's an
engine speed
then there might be an integer
afterwards that is in
tens of rpms or what have you
integration testing should exercise the
message structure
it should test all types of messages
a range of values inside fields valid
and invalid field values
and invalid message types it should also
test the timing and exception handling
what if there's a bad checksum
indicating a message should be
dropped what if there's a bad sequence
number on a sequence of messages
and so on the hld will have this message
dictionary which should define
all the message types formats and so on
and give you a good basis for writing
the integration tests based on the hld
it's common to see components
accompanied by a validation test suite
so you can know that all the different
types of messages
are supported properly
integration testing best practices
revolve around concentrating on the
interaction of components
integration tests should be traced to
the high level design
including exercising all the arcs on
every sequence diagram
covering every sequence diagram in the
high level design
all the modules all the network
interfaces
all the message types all the data
fields the idea
is assuming unit testing has found
everything unit testing is likely to
find
how can you additionally stress the
interactions among components
to make sure that sure the units each do
what they want to do
but when you put them together they
actually still satisfy
all the aspects of the high level design
the two main integration testing
pitfalls
first system testing alone misses system
integration edge cases
sometimes a misbehaving system appears
to work just fine
but the internal logic isn't quite right
and there's some specific edge case
situation that it will not handle as
intended
because it was just getting lucky in the
common case
also it can be difficult to exercise off
nominal sequence diagrams at a system
level
there are some tests which are very
difficult to reproduce with a physical
system or
downright dangerous integration testing
helps you cover all the fine-grained
interactions to make sure
not only is this system working but it's
working the way you thought it was
supposed to be working
from a traceability point of view if you
skip the high level design
there's nothing to trace your
integration test to so you're just sort
of guessing
what things are supposed to do and if
you don't do integration testing
there's no way to really know whether
the system is behaving as intended
or not
تصفح المزيد من مقاطع الفيديو ذات الصلة
ISTQB FOUNDATION 4.0 | Tutorial 14 | Test Levels | Component Integration Testing | CTFL Tutorials
L16 Unit Testing
Software Testing Tutorial #14 - What is System Testing
ISTQB FOUNDATION 4.0 | Tutorial 16 | Test Levels | System Integration Testing | CTFL | TM Square
ISTQB FOUNDATION 4.0 | Tutorial 36 | Value of White Box Test Techniques | CTFL Tutorials | TM SQUARE
Software Testing Tutorial #11 - Levels in Software Testing
5.0 / 5 (0 votes)