CONCEPTS OF SOFTWARE
ENGINEERING
Testing
Dr. Abdul Karim ABED
Testing
In general, Testing software is operating the
software under controlled conditions, to (1) verify
that it behaves “as specified”; (2) to detect errors,
and (3) to validate that what has been specified is
what the user actually wanted.
Testing
In XP, Testing is done consistently throughout the
process. Programmers design the tests first and
then write the software to fulfill the requirements
of the test. The customer also provides acceptance
tests at each stage to ensure the desired results are
achieved.
XP Process
An XP developer's daily routine
Pair-up
Select Task
Quick Design
Write Test
Code Refactor
Integrate
Go Home
Testing
Continuous Testing: Before programmers add a
feature, they write a test for it. When the suite runs,
the job is done. Tests in XP come in two basic
flavors.
Unit Tests are automated tests written by the
developers to test functionality as they write it. Each
unit test typically tests only a single class, or a small
cluster of classes. Unit tests are typically written using
a unit testing framework, such as JUnit.
Testing
Acceptance Tests (also known as Functional Tests)
are specified by the customer to test that the overall
system is functioning as specified. Acceptance tests
typically test the entire system, or some large chunk of
it. When all the acceptance tests pass for a given user
story, that story is considered complete.
At the very least, an acceptance test could consist of a
script of user interface actions and expected results that
a human can run. Ideally acceptance tests should be
automated, either using the unit testing framework, or a
separate acceptance testing framework.
Iterative Software development
7
Write
and
execute
unit tests Execute
Write
acceptance
acceptance
tests
tests
increment
+ system
Prioritized ”Written before“ ”Executed after the development“
functionalities
XP Testing
Every piece of code has a set of automated unit tests,
which are released into the code repository along with
the code.
The programmers write the unit tests before they write
the code, then add unit tests whenever one is found to
be missing.
No modification or refactoring of code is complete
until 100% of the unit tests have run successfully.
XP Testing
Acceptance tests validate larger blocks of
system functionality, such as user stories.
When all the acceptance tests pass for a given
user story, that story is considered complete.
Test-driven development
Test first: before adding a feature, write a test
for it!
If code has no automated test case, it is assumed it
does not work
When the complete test suite passes 100%, the
feature is accepted
What is Unit Testing
Method of testing that verifies the individual units
of the code is working properly.
Test the smallest unit in source code
Unit Tests
Unit Tests automate testing of functionality as
developers write it
Each unit test typically testing framework, such as JUnit
(xUnit)
Experiments show that test-driven development reduces
debugging time
Increases confidence that new features work, and work
with everything
If a bug is discovered during development, add a test case
to make sure it doesn’t come back!
tests only a single class, or a small cluster of classes
Testing with JUnit
Junit is a unit test environment for Java programs
developed by Erich Gamma and Kent Beck.
Writing test cases
Executing test cases
Pass/fail? (expected result = obtained result?)
?How to write tests
All tests must be pass/fail style tests.
A bad test output:
Now Testing Person.java:
First Name: Tanner
Last Name: Burke
Age: 1
A good test output:
Now Testing Person.java:
Failure: Expected Age 2, but was 1 instead.
JUnit 4.x
JUnit is a simple “testing framework” that provides:
Annotations for marking methods as tests
Annotations for marking methods that setting up
and cleaning up test data (“fixtures”)
methods for making assertions
Simple JUnit
Create test class and test case.
Use an assert method for ensuring method output
assertEquals()
assertTrue()
assertNotNull()
Can be invoked manually by running the test class
or automated by using ant script
How to write JUnit-based
testing code (Minimum)
Create a Java class
package Test;
public class MyClass
{
public int multiply(int x, int y) {
return x / y;
}
}
How to write JUnit-based
testing code (Minimum)
import org.junit.Test;
import static org.junit.Assert.assertEquals;
public class MyClassTest
{
@Test
public void testMultiply() /* must use test*/
{
MyClass tester = new MyClass();
assertEquals("Result", 50, tester.multiply(10, 5)); }
}
How to write JUnit-based
testing code (Minimum)
Run your test
The test should be failing (indicated via a red bar).
With the following statement:
Java.lang.AssertionError: Result expected <50> but
was: <2>
Junit in Netbeans
You don’t need to load the jar into netbeans project.
By default the jar is embedded in test libarary
folder
And also netbeans has test class and test case code
generation menu
Lets Do The Code
Lets start with heating up our Netbeans 6.1 and
create new java project.
Make a simple class having both return valued and
void method.
Let the return valued method do simple process for
example addition or substraction.
Just print something in the void method.
SimpleMath.java
Create Unit Test
Choose this menu in netbeans
Tools > Create Junit Test
Or just simply press Ctrl + Shift + U.
A window dialogue will appear, choose suitable
options.
Or you can leave it as is.
Test case will automatically build inside the test
package folder.
Unit Test Menu
Unit Test Window
SimpleMathTest.java
Unit Testing
Assign the variable value for the test case.
Remove the fail() method in return valued method
test.
Run the test class using Shift + F6.
See the test result
Test Result
Another example (Junit 4.x)
public class Add {
29 public static int sum (int a, int b) {
return a+b;
}
}
import math.Add;
import org.junit.*;
import static org.junit.Assert.*;
public class Testcase1_add_Junit4 {
@Test
public void testAdd() {
Add add=new Add();
int sum=add.sum(3, 2);
assertEquals(5, sum);
}
}
Another example (Junit 4.x)
public class Add {
30 public static int sum (int a, int b) {
return a- b;
}
}
import math.Add;
import org.junit.*;
import static org.junit.Assert.*;
public class Testcase1_add_Junit4 {
@Test
public void testAdd() {
Add add=new Add();
int sum=add.sum(3, 2);
assertEquals(5, sum);
}
}
Another example (Junit 4.x)
Test case 2
31
import junit.framework.TestCase;
import math.Add;
public class TestCase1_add_Junit3
extends TestCase {
public void testAdd() {… }
public void testAdd_2() {
Add add=new Add();
int sum=add.sum(3, -2);
assertEquals(5, sum);
}
}
Example: Old way vs. new way
int max(int a, int b) {
if (a > b) {
return a;
} else {
return b;
}
}
void testMax() { @Test
int x = max(3, 7);
if (x != 7) { void testMax() {
System.out.println("max(3, 7) gives " + x);
} assertEquals(7,
x = max(3, -7);
if (x != 3) { max(3, 7));
System.out.println("max(3, -7) gives " + x);
} assertEquals(3,
}
public static void main(String[] args) {
max(3, -7));
}
new MyClass().testMax();
}
Test fixtures
A test fixture is the context in which a test case
runs.
Typically, test fixtures include:
Objects or resources that are available for use by any
test case.
Activities required to make these objects available
and/or resource allocation and de-allocation.
Test fixtures
Before
You can define one or more methods to be executed before
each test; typically such methods initialize values, so that
each test starts with a fresh set
@Before public void
createOutputFile()
{
output = new File(...);
}
Test fixtures
After
You can define one or more methods to be executed
after each test; typically such methods release
resources, such as files
@After public void
deleteOutputFile()
{
output.delete();
}
Test fixtures
BeforeClass
If you wish, you can declare one method to be executed just
once, when the class is first loaded
This is for expensive setup, such as connecting to a
database
@BeforeClass
public static void setUpClass() throws Exception
{
// one-time initialization code
}
Test fixtures
AfterClass
If you wish, you can declare one method to be
executed just once, to do cleanup after all the tests
have been completed
@AfterClass
public static void tearDownClass() throws
Exception {
// one-time cleanup code
}
Example
import org.junit.*;
import static org.junit.Assert.*;
import java.util.*;
public class JunitTest1
{
private Collection collection;
private Collection collection;
@BeforeClass public static void oneTimeSetUp()
{ // one-time initialization code
System.out.println("@BeforeClass - oneTimeSetUp");
}
Example
@AfterClass
public static void oneTimeTearDown() {
// one-time cleanup code
System.out.println(“("@AfterClass - oneTimeTearDown");
}
@Before public void setUp()
{ collection = new ArrayList();
System.out.println("@Before - setUp");
}
@After public void tearDown()
{ collection.clear();
System.out.println("@After - tearDown"); }
Example
@Test
public void testEmptyCollection()
{ assertTrue(collection.isEmpty());
System.out.println("@Test -testEmptyCollection"); }
@Test
public void testOneItemCollection() { collection.add("itemA");
assertEquals(1, collection.size()); System.out.println("@Test -
testOneItemCollection");
}}
Example
Results:
@BeforeClass - oneTimeSetUp
@Before – setUp
@Test - testEmptyCollection
@After – tearDown
@Before - setUp
@Test – testOneItemCollection
@After - tearDown
@AfterClass - oneTimeTearDown
Assertions
Assertions are defined in the JUnit class Assert
If an assertion is true, the method continues executing.
If any assertion is false, the method stops executing at
that point, and the result for the test case will be fail.
If any other exception is thrown during the method, the
result for the test case will be error.
If no assertions were violated for the entire method, the
test case will pass.
All assertion methods are static methods
Assertions
A test method is annotated with @Test, takes no
parameters, and returns no result
@Test
public void sum() {
assertEquals(15, program.sum(array));
assertTrue(program.min(array) > 0);
}
Assertion
Boolean conditions are true or false
assertTrue(condition)
assertFalse(condition)
Objects are null or non-null
assertNull(object)
assertNotNull(object)
Objects are identical (i.e. two references to the same
object), or not identical.
assertSame(expected, actual)
true if: expected == actual
assertNotSame(expected, actual)
Assertion methods (2)
“Equality” of objects:
assertEquals(expected, actual)
valid if: expected.equals( actual )
“Equality” of arrays:
assertArrayEquals(expected, actual)
arrays must have same length
for each valid value for i, check as appropriate:
assertEquals(expected[i],actual[i])
or
assertArrayEquals(expected[i],actual[i])
Floating point assertions
When comparing floating point types (double or
float), there is an additional required parameter delta.
The assertion evaluates
Math.abs( expected – actual ) <= delta
to avoid problems with round-off errors with floating point
comparisons.
Example:
assertEquals( aDouble, anotherDouble, 0.0001 )
?When is JUnit appropriate
As the name implies…
for unit testing of small amounts of code
On its own, it is not intended for complex testing, system
testing, etc.
In the test-driven development methodology, a JUnit test
should be written first (before any code), and executed.
Then, implementation code should be written that would be the
minimum code required to get the test to pass – and no extra
functionality.
Once the code is written, re-execute the test and it should pass.
Every time new code is added, re-execute all tests again to be
sure nothing gets broken.
Acceptance Test
Acceptance tests are created from user stories.
During an iteration the user stories selected during
the iteration planning meeting will be translated
into acceptance tests.
Acceptance Test
Acceptance Tests in XP
Simple version
Customer writes the acceptance tests with help from the developer and
the user stories
Developers write code to make the acceptance tests pass, reports results
to the customer
Acceptance Test
The customer specifies scenarios to test when a
user story has been correctly implemented.
A story can have one or many acceptance tests,
what ever it takes to ensure the functionality works.
Example
One story that all the groups wrote a test for was
"Allow customer to make a deposit." A sample
acceptance test for this story follows.
Example
Acceptance test
Establish account
Make deposit
Verify balance is correct
Example
An example of a more specific version of the same
acceptance test follows.
Acceptance test
Establish account
Account number = 1231234556
Account owner = John Smith Starting
balance = 921.53
Make deposit Deposit
amount = 51.21
Verify balance is correct
Updated balance = 972.74
Acceptance Test
Acceptance tests are black box system tests. Each
acceptance test represents some expected result
from the system.
Testing Techniques
Result
Actual output
compared
requirements … Black box with
required output
White box
Confirmation
design… of expected
elements behavior
White Box Testing
Software testing approach that uses inner structural
and logical properties of the program for
verification and deriving test data
Black-box Testing
Unlike white-box testing, no knowledge about
the internals of the code
Test cases are designed based on specifications
Example: search for a value in an array
Postcondition: return value is the index of some
occurrence of the value, or -1 if the value does not occur
in the array
Acceptance Test
Customers are responsible for verifying the
correctness of the acceptance tests and reviewing
test scores to decide which failed tests are of
highest priority
Acceptance Test
A user story is not considered complete until it has
passed its acceptance tests. This means that new
acceptance tests must be created each iteration or
the development team will report zero progress.
Benefits to Acceptance Tests
Can serve as a contract for the client/developers
Requires stories are testable
User stories are understandings; acceptance tests are
requirements the developers must meet
Client can track progress by observing the total
number of acceptance tests growing and % of passing
tests increasing
Developers get more confidence that work is being
done and can cross stories off their list when
acceptance tests pass
Sample Acceptance Test
Writing cash register software
Acceptance Test: Shopping cart for generating a receipt
Create a shopping cart with:
1 lb. coffee, 3 bags of cough drops, 1 gallon milk
Prices: Coffee $6/lb, cough drops $2.49/bag, milk $4.95/gallon
Verify total is $18.42
Test might span multiple stories (fill shopping cart,
checkout, view receipt…)
Other tests might verify sales tax is calculated
correctly, coupons properly discounted, etc.
Acceptance Tests Are Important
Gives customer some satisfaction that features are
correctly implemented
Not the same as Unit Test
Unit tests could pass but acceptance tests fail,
especially if acceptance test requires the integration of
components that were unit-tested
