Software Testing
Faults/Errors/Failures
Software Faults, Errors and Failures
• Error:
– a human action that produces an incorrect result.
also referred to as Mistake.
– it is the developer mistake that produce a fault.
– Often, it has been caused by human activities such
as the typing errors.
Software Faults, Errors and Failures
• Fault: (commonly named “bug/defect”)
– a defect in a system.
– The result of an error being made is a fault,
– It is something that is wrong in the software
– A failure may be caused by the presence of
one or more faults on a given system.
However, the presence of a fault in a system
may or may not lead to a failure, e.g., a
system may contain a fault in its code but on
a fragment of code that is never exercised so
this kind of fault do not lead to a software
failure.
Software Faults, Errors and Failures
• Failure:
– a deviation of the software from its
expected delivery or service.
– Occurs when fault executes
– an observable incorrect behavior or state of
a given system.
– In this case, the system displays a behavior
that is contrary to its
specifications/requirements. Thus, a failure
is tied (only) to system
executions/behaviors and it occurs at
runtime when some part of the system
enters an unexpected state.
Example … LOC Code
1 program double ();
2 var x,y: integer;
3 begin
4 read(x);
5 y := x * x;
6 write(y)
7 end
Failure: x = 3 means y =9 àFailure!
• This is a failure of the system since the correct output would be 6
Fault: The fault that causes the failure is in line 5. The * operator is
used instead of +.
Error: The error that conduces to this fault may be:
• a typing error (the developer has written * instead of +)
• a conceptual error (e.g., the developer doesn't know how to
double a number)
Another Example
Fault: Should start
searching at 0, not 1
public static int numZero (int [ ] arr)
Test 1
{ // Effects: If arr is null throw NullPointerException [ 2, 7, 0 ]
// else return the number of occurrences of 0 in arr Expected: 1
int count = 0; Actual: 1
for (int i = 1; i < arr.length; i++)
{ Error: i is 1, not 0, on Test 2
if (arr [ i ] == 0) the first iteration [ 0, 2, 7 ]
{ Failure: none Expected: 1
count++; Actual: 0
}
} Error: i is 1, not 0
return count; Error propagates to the variable count
} Failure: count is 0 at the return statement
Specification
“if you can’t say it, you can’t do it”
• You have to know what your product is before
you can say if it has a bug.
• A specification defines the product being
created and includes:
– Functional requirements that describes the
features the product will support. E.g., on a word
processor
• Save, print, check spelling, change font, …
– Non-functional requirements are constraints on
the product. E.g,
• Security, reliability, user friendliness, platform, …
A software bug occurs when at
least one of these rules is true
• The software does not do something that the
specification says it should do.
• The software does something that the
specification says it should not do.
• The software does something that the
specification does not mention.
• The software does not do something that the
product specification does not mention but
should.
• The software is difficult to understand, hard to
use, slow …
Most bugs are not because of
mistakes in the code …
• Specification (~= 55%)
• Design (~= 25%)
• Code (~= 15%)
• Other (~= 5%)
Relative cost of bugs
“bugs found later cost more to fix”
• Cost to fix a bug increases exponentially (10x)
– i.e., it increases tenfold as time increases
• E.g., a bug found during specification costs $1 to
fix.
• … if found in design cost is $10
• … if found in code cost is $100
• … if found in released software cost is $1000
Bug Free Software
• Software is in the news for the wrong reason
– Security breach, Mars Lander lost, hackers getting
credit card information, etc.
• Why can’t software engineers develop software
that just works?
– As software gets more features and supports more
platforms it becomes increasingly difficult to make it
create bug-free.
Discussion …
• Do you think bug free software is
unattainable?
– Are their technical barriers that make this
impossible?
– Is it just a question of time before we can do
this?
