Software Testing
What is Software Testing?
Several definitions:
“Testing is the process of establishing confidence that a program or
system does what it is supposed to.” by Hetzel 1973
“Testing is the process of executing a program or system with the
intent of finding errors.” by Myers 1979
“Testing is any activity aimed at evaluating an attribute or capability
of a program or system and determining that it meets its required
results.” by Hetzel 1983
2
Who does Software Testing?
- Test manager
- manage and control a software test project
- supervise test engineers
- define and specify a test plan
- Software Test Engineers and Testers
- define test cases, write test specifications, run tests
- Independent Test Group
- Development Engineers
- Only perform unit tests and integration tests
- Quality Assurance Group and Engineers
- Perform system testing
- Define software testing standards and quality control process 3
The Test Plan
• The Test Plan
• who
• what
• when
• where
• how
Test Plan Considerations
• What are the critical or most complex modules?
• make sure they get integration tested first
• probably deserve white-box attention
• Where have you had problems in the past?
• Third-Party delivered components?
• What training is required?
• conducting formal reviews
• use of testing tools
• defect report logging
Test Items
• Requirements Specification
• Design
• Modules
• User/Operator Material
• the user interface
• User Guide
• Operations Guide
• Features
• response time, data accuracy, security, etc
• System Validation
• alpha and beta testing
Software Testing Principles
•Principle #1: Complete testing is impossible.
•Principle #2: Software testing is not simple activity.
•Reasons:
•Quality testing requires testers to understand a system/product
completely
•Quality testing needs adequate test set, and efficient testing methods
•A very tight schedule and lack of test tools.
•Principle #3: Testing is risk-based.
•Principle #4: Testing must be planned.
•Principle #5: Testing requires independence (SQA team).
•Principle #6: Quality software testing depends on:
•Good understanding of software products and related domain 7
application
•Cost-effective testing methodology, coverage, test methods, and tools.
•Good engineers with creativity, and solid software testing experience
Software Testing Process
V&V Targets
Unit test Code & Implementation
Integration Software Design
test
Validation Requirements
test
System System engineering
test
Levels of Testing
• Component/Unit testing
• Integration testing
• System testing
• Acceptance testing
• Regression testing
9
Levels of Testing
What users
really need Acceptance testing
Requirements System testing
Design Integration testing
Code Unit testing
Maintenance Regression Testing
SYSTEM TESTING
11
System Testing
• Testing of groups of components integrated to create a system
or sub-system;
• The responsibility of an independent testing team;
• Tests are based on a system specification.
12
System Testing
• Functional testing
– Test end to end functionality
– Requirement focus
• Test cases derived from specification
– Use-case focus
• Test selection based on user profile
13
System Testing
• Non-functional testing
• Quality attributes
– Performance, can the system handle required throughput?
– Reliability, obtain confidence that system is reliable
– Timeliness, testing whether the individual tasks meet their specified
deadlines
– etc.
14
ACCEPTANCE TESTING
15
Acceptance Testing
• User (or customer) involved
• Environment as close to field use as possible
• Focus on:
– Building confidence
– Compliance with defined acceptance criteria in the contract
16
REGRESSION TESTING
17
Re-Test and Regression Testing
• Conducted after a change
• Re-test aims to verify whether a fault is removed
– Re-run the test that revealed the fault
• Regression test aims to verify whether new faults are
introduced
– How can we test modified or newly inserted programs?
• Ignore old test suites and make new ones from the scratch or
• Reuse old test suites and reduce the number of new test suites as many as
possible
– Should preferably be automated 18
TEST STRATEGIES
19
Strategies
• Code coverage strategies, e.g.
– Decision coverage
– Path coverage
– Data-Flow analysis (Defines -> Uses)
• Specification-based testing, e.g.
– Equivalence partitioning
– Boundary-value analysis
– Combination strategies
• State-based testing
• Black-box or behavioral testing
– knowing the specified function a product is to perform and
demonstrating correct operation based solely on its specification 20
without regard for its internal logic
Code Coverage
• Statement coverage
– Each statement should be executed by at least one test case
– Minimum requirement
• Branch/Decision coverage
– All paths should be executed by at least one test case
– All decisions with true and false value
21
Specification-based testing
• Test cases derived from specification
• Equivalence partitioning
– Identify sets of input from specification
• Assumption: if one input from set s leads to a failure, then all inputs from set
s will lead to the same failure
– Chose a representative value from each set
– Form test cases with the chosen values
22
Specification-Based Testing
Expected output
Specification
Apply input
Actual output
Program
Validate the observed output against the expected output
Specification-based testing
• Boundary value analysis
– Identify boundaries in input and output
– For each boundary:
• Select one value from each side of boundary (as close as possible)
– Form test cases with the chosen values
24
Boundary Value Analysis
Black-box technique
focuses on classes and also on the boundaries of the input
domain.
Guidelines:
1. If input condition specifies a range bounded by values a and b,
test cases should include a and b, values just above and just
below a and b
2. If an input condition specifies a number of values, test cases
should exercise the minimum and maximum numbers, as well as
values just above and just below the minimum and maximum
values
25
State-Based Testing
• Model functional behavior in a state machine (communication
– protocol …)
• Select test cases in order to cover the graph
– Each node
– Each transition
– Each pair of transitions
– Each chain of transitions of length n
26
Thank You
mesfin.kifle@aau.edu.et
