What is User Acceptance Testing?

Introduction:

This article attempts to explain the process of User Acceptance Testing.

Once the application is ready to be released the crucial step is User Acceptance
Testing.

In this step a group representing a cross section of end users tests the application.
The user acceptance testing is done using real world scenarios and perceptions
relevant to the end users.

What is User Acceptance Testing?

User Acceptance Testing is often the final step before rolling out the application.

Usually the end users who will be using the applications test the application before
‘accepting’ the application.

This type of testing gives the end users the confidence that the application being
delivered to them meets their requirements.

This testing also helps nail bugs related to usability of the application.

User Acceptance Testing – Prerequisites:

Before the User Acceptance testing can be done the application is fully developed.
Various levels of testing (Unit, Integration and System) are already completed
before User Acceptance Testing is done. As various levels of testing have been
completed most of the technical bugs have already been fixed before UAT.

User Acceptance Testing – What to Test?

To ensure an effective User Acceptance Testing Test cases are created.

These Test cases can be created using various use cases identified during the
Requirements definition stage.
The Test cases ensure proper coverage of all the scenarios during testing.

During this type of testing the specific focus is the exact real world usage of the
application. The Testing is done in an environment that simulates the production
environment.
The Test cases are written using real world scenarios for the application

User Acceptance Testing – How to Test?

The user acceptance testing is usually a black box type of testing. In other words,
the focus is on the functionality and the usability of the application rather than the
technical aspects. It is generally assumed that the application would have already
undergone Unit, Integration and System Level Testing.
However, it is useful if the User acceptance Testing is carried out in an environment
that closely resembles the real world or production environment.

The steps taken for User Acceptance Testing typically involve one or more of the
following:
.......1) User Acceptance Test (UAT) Planning
.......2) Designing UA Test Cases
.......3) Selecting a Team that would execute the (UAT) Test Cases
.......4) Executing Test Cases
.......5) Documenting the Defects found during UAT
.......6) Resolving the issues/Bug Fixing
.......7) Sign Off

User Acceptance Test (UAT) Planning:

As always the Planning Process is the most important of all the steps. This affects
the effectiveness of the Testing Process. The Planning process outlines the User
Acceptance Testing Strategy. It also describes the key focus areas, entry and exit
criteria.

Designing UA Test Cases:

The User Acceptance Test Cases help the Test Execution Team to test the
application thoroughly. This also helps ensure that the UA Testing provides
sufficient coverage of all the scenarios.
The Use Cases created during the Requirements definition phase may be used as
inputs for creating Test Cases. The inputs from Business Analysts and Subject
Matter Experts are also used for creating.

Each User Acceptance Test Case describes in a simple language the precise steps to
be taken to test something.

The Business Analysts and the Project Team review the User Acceptance Test
Cases.

Selecting a Team that would execute the (UAT) Test Cases:

Selecting a Team that would execute the UAT Test Cases is an important step.
The UAT Team is generally a good representation of the real world end users.
The Team thus comprises of the actual end users who will be using the application.

Executing Test Cases:

The Testing Team executes the Test Cases and may additional perform random
Tests relevant to them

Documenting the Defects found during UAT:

The Team logs their comments and any defects or issues found during testing.

Resolving the issues/Bug Fixing:

The issues/defects found during Testing are discussed with the Project Team,
Subject Matter Experts and Business Analysts. The issues are resolved as per the
mutual consensus and to the satisfaction of the end users.

Sign Off:
Upon successful completion of the User Acceptance Testing and resolution of the
issues the team generally indicates the acceptance of the application. This step is
important in commercial software sales. Once the User “Accept” the Software
delivered they indicate that the software meets their requirements.

The users now confident of the software solution delivered and the vendor can be
paid for the same.

What are the key deliverables of User Acceptance Testing?

In the Traditional Software Development Lifecycle successful completion of User

Acceptance Testing is a significant milestone.

The Key Deliverables typically of User Acceptance Testing Phase are:

1) The Test Plan- This outlines the Testing Strategy

2) The UAT Test cases – The Test cases help the team to effectively test the
application

3) The Test Log – This is a log of all the test cases executed and the actual results.

4) User Sign Off – This indicates that the customer finds the product delivered to
their satisfaction

Summary:

In this article we studied the process of ‘User Acceptance Testing’.

System Testing: Why? What? & How?

Introduction:

‘Unit testing’ focuses on testing each unit of the code.

‘Integration testing’ focuses on testing the integration of “units of code” or

components.
Each level of testing builds on the previous level.

‘System Testing’ is the next level of testing. It focuses on testing the system as a
whole.

This article attempts to take a close look at the System Testing Process and analyze:

Why System Testing is done? What are the necessary steps to perform System
Testing? How to make it successful?

How does System Testing fit into the Software Development

Life Cycle?

In a typical Enterprise, ‘unit testing’ is done by the programmers. This ensures that
the individual components are working OK. The ‘Integration testing’ focuses on
successful integration of all the individual pieces of software (components or units
of code).

Once the components are integrated, the system as a whole needs to be rigorously
tested to ensure that it meets the Quality Standards.

Thus the System testing builds on the previous levels of testing namely unit testing
and Integration Testing.

Usually a dedicated testing team is responsible for doing ‘System Testing’.

Why System Testing is important?

System Testing is a crucial step in Quality Management Process.

........- In the Software Development Life cycle System Testing is the first level where

...........the System is tested as a whole

........- The System is tested to verify if it meets the functional and technical
...........requirements
........- The application/System is tested in an environment that closely resembles the

...........production environment where the application will be finally deployed

........- The System Testing enables us to test, verify and validate both the Business
...........requirements as well as the Application Architecture

Prerequisites for System Testing:

The prerequisites for System Testing are:

........- All the components should have been successfully Unit Tested
........- All the components should have been successfully integrated and Integration
..........Testing should be completed
........- An Environment closely resembling the production environment should be
...........created.

When necessary, several iterations of System Testing are done in multiple

environments.

System Testing: Why? What? & How?

Introduction:

‘Unit testing’ focuses on testing each unit of the code.

‘Integration testing’ focuses on testing the integration of “units of code” or

components.
Each level of testing builds on the previous level.

‘System Testing’ is the next level of testing. It focuses on testing the system as a
whole.
This article attempts to take a close look at the System Testing Process and analyze:

Why System Testing is done? What are the necessary steps to perform System
Testing? How to make it successful?

How does System Testing fit into the Software

Development Life Cycle?

In a typical Enterprise, ‘unit testing’ is done by the programmers. This ensures that
the individual components are working OK. The ‘Integration testing’ focuses on
successful integration of all the individual pieces of software (components or units
of code).

Once the components are integrated, the system as a whole needs to be rigorously
tested to ensure that it meets the Quality Standards.

Thus the System testing builds on the previous levels of testing namely unit testing
and Integration Testing.

Usually a dedicated testing team is responsible for doing ‘System Testing’.

Why System Testing is important?

System Testing is a crucial step in Quality Management Process.

........- In the Software Development Life cycle System Testing is the first level where

...........the System is tested as a whole

........- The System is tested to verify if it meets the functional and technical
...........requirements
........- The application/System is tested in an environment that closely resembles the

...........production environment where the application will be finally deployed

........- The System Testing enables us to test, verify and validate both the Business
...........requirements as well as the Application Architecture

Prerequisites for System Testing:

The prerequisites for System Testing are:

........- All the components should have been successfully Unit Tested
........- All the components should have been successfully integrated and Integration
..........Testing should be completed
........- An Environment closely resembling the production environment should be
...........created.

When necessary, several iterations of System Testing are done in multiple

environments.

Steps needed to do System Testing:

The following steps are important to perform System Testing:
........Step 1: Create a System Test Plan
........Step 2: Create Test Cases
........Step 3: Carefully Build Data used as Input for System Testing
........Step 3: If applicable create scripts to
..................- Build environment and
..................- to automate Execution of test cases
........Step 4: Execute the test cases
........Step 5: Fix the bugs if any and re test the code
........Step 6: Repeat the test cycle as necessary

System Testing: Why? What? & How?

Introduction:

‘Unit testing’ focuses on testing each unit of the code.

‘Integration testing’ focuses on testing the integration of “units of code” or

components.
Each level of testing builds on the previous level.

‘System Testing’ is the next level of testing. It focuses on testing the system as a
whole.

This article attempts to take a close look at the System Testing Process and analyze:

Why System Testing is done? What are the necessary steps to perform System
Testing? How to make it successful?

How does System Testing fit into the Software

Development Life Cycle?

In a typical Enterprise, ‘unit testing’ is done by the programmers. This ensures that
the individual components are working OK. The ‘Integration testing’ focuses on
successful integration of all the individual pieces of software (components or units
of code).

Once the components are integrated, the system as a whole needs to be rigorously
tested to ensure that it meets the Quality Standards.

Thus the System testing builds on the previous levels of testing namely unit testing
and Integration Testing.

Usually a dedicated testing team is responsible for doing ‘System Testing’.

Why System Testing is important?

System Testing is a crucial step in Quality Management Process.

........- In the Software Development Life cycle System Testing is the first level where

...........the System is tested as a whole

........- The System is tested to verify if it meets the functional and technical
...........requirements
........- The application/System is tested in an environment that closely resembles the

...........production environment where the application will be finally deployed

........- The System Testing enables us to test, verify and validate both the Business
...........requirements as well as the Application Architecture

Prerequisites for System Testing:

The prerequisites for System Testing are:

........- All the components should have been successfully Unit Tested
........- All the components should have been successfully integrated and Integration
..........Testing should be completed
........- An Environment closely resembling the production environment should be
...........created.

When necessary, several iterations of System Testing are done in multiple

environments.

Steps needed to do System Testing:

The following steps are important to perform System Testing:

........Step 1: Create a System Test Plan
........Step 2: Create Test Cases
........Step 3: Carefully Build Data used as Input for System Testing
........Step 3: If applicable create scripts to
..................- Build environment and
..................- to automate Execution of test cases
........Step 4: Execute the test cases
........Step 5: Fix the bugs if any and re test the code
........Step 6: Repeat the test cycle as necessary

What is a ‘System Test Plan’?

As you may have read in the other articles in the testing series, this document
typically describes the following:
.........- The Testing Goals
.........- The key areas to be focused on while testing
.........- The Testing Deliverables
.........- How the tests will be carried out
.........- The list of things to be Tested
.........- Roles and Responsibilities
.........- Prerequisites to begin Testing
.........- Test Environment
.........- Assumptions
.........- What to do after a test is successfully carried out
.........- What to do if test fails
.........- Glossary
How to write a System Test Case?

A Test Case describes exactly how the test should be carried out.

The System test cases help us verify and validate the system.
The System Test Cases are written such that:
........- They cover all the use cases and scenarios
........- The Test cases validate the technical Requirements and Specifications
........- The Test cases verify if the application/System meet the Business &
Functional
...........Requirements specified
........- The Test cases may also verify if the System meets the performance standards

Since a dedicated test team may execute the test cases it is necessary that System
Test Cases. The detailed Test cases help the test executioners do the testing as
specified without any ambiguity.

The format of the System Test Cases may be like all other Test cases as illustrated
below:

 Test Case ID
 Test Case Description:
o What to Test?
o How to Test?
 Input Data
 Expected Result
 Actual Result

Sample Test Case Format:

Test
What To How to Expected Actual
Case Input Data Pass/Fail
Test? Test? Result Result
ID
. . . . . . .

Additionally the following information may also be captured:

........a) Test Suite Name
........b) Tested By
........c) Date
........d) Test Iteration (The Test Cases may be executed one or more times)

Working towards Effective Systems Testing:

There are various factors that affect success of System Testing:

1) Test Coverage: System Testing will be effective only to the extent of the
coverage of Test Cases. What is Test coverage? Adequate Test coverage implies the
scenarios covered by the test cases are sufficient. The Test cases should “cover” all
scenarios, use cases, Business Requirements, Technical Requirements, and
Performance Requirements. The test cases should enable us to verify and validate
that the system/application meets the project goals and specifications.
2) Defect Tracking: The defects found during the process of testing should be
tracked. Subsequent iterations of test cases verify if the defects have been fixed.

3) Test Execution: The Test cases should be executed in the manner specified.
Failure to do so results in improper Test Results.

4) Build Process Automation: A Lot of errors occur due to an improper build.

‘Build’ is a compilation of the various components that make the application
deployed in the appropriate environment. The Test results will not be accurate if the
application is not ‘built’ correctly or if the environment is not set up as specified.
Automating this process may help reduce manual errors.

Test Automation: Automating the Test process could help us in many ways:

a. The test can be repeated with fewer errors of omission or oversight

b. Some scenarios can be simulated if the tests are automated for instance
simulating a large number of users or simulating increasing large amounts
of input/output data

6) Documentation: Proper Documentation helps keep track of Tests executed. It

also helps create a knowledge base for current and future projects. Appropriate
metrics/Statistics can be captured to validate or verify the efficiency of the technical
design /architecture.

Summary:

In this article we studied the necessity of ‘System Testing’ and how it is done.

What is Regression Testing?

Introduction:

This article attempts to take a close look at the process and techniques in
Regression Testing.

What is Regression Testing?

If a piece of Software is modified for any reason testing needs to be done to ensure
that it works as specified and that it has not negatively impacted any functionality
that it offered previously. This is known as Regression Testing.

Regression Testing attempts to verify:

- That the application works as specified even after the

changes/additions/modification were made to it

- The original functionality continues to work as specified even after

changes/additions/modification to the software application
- The changes/additions/modification to the software application have not introduced
any new bugs

When is Regression Testing necessary?

Regression Testing plays an important role in any Scenario where a change has
been made to a previously tested software code. Regression Testing is hence an
important aspect in various Software Methodologies where software changes
enhancements occur frequently.

Any Software Development Project is invariably faced with requests for changing
Design, code, features or all of them.

Some Development Methodologies embrace change.

For example ‘Extreme Programming’ Methodology advocates applying small

incremental changes to the system based on the end user feedback.

Each change implies more Regression Testing needs to be done to ensure that the
System meets the Project Goals.

Why is Regression Testing important?

Any Software change can cause existing functionality to break.

Changes to a Software component could impact dependent Components.

It is commonly observed that a Software fix could cause other bugs.

All this affects the quality and reliability of the system. Hence Regression Testing,
since it aims to verify all this, is very important.

Making Regression Testing Cost Effective:

Every time a change occurs one or more of the following scenarios may occur:
- More Functionality may be added to the system
- More complexity may be added to the system
- New bugs may be introduced
- New vulnerabilities may be introduced in the system
- System may tend to become more and more fragile with each change

After the change the new functionality may have to be tested along with all the
original functionality.

With each change Regression Testing could become more and more costly.

To make the Regression Testing Cost Effective and yet ensure good coverage one or
more of the following techniques may be applied:

Test Automation: If the Test cases are automated the test cases may be executed
using scripts after each change is introduced in the system. The execution of test
cases in this way helps eliminate oversight, human errors,. It may also result in
faster and cheaper execution of Test cases. However there is cost involved in
building the scripts.

- Selective Testing: Some Teams choose execute the test cases selectively. They do
not execute all the Test Cases during the Regression Testing. They test only what
they decide is relevant. This helps reduce the Testing Time and Effort.

Regression Testing – What to Test?

Since Regression Testing tends to verify the software application after a change has
been made everything that may be impacted by the change should be tested during
Regression Testing. Generally the following areas are covered during Regression
Testing:

- Any functionality that was addressed by the change

- Original Functionality of the system

- Performance of the System after the change was introduced

Regression Testing – How to Test?

Like any other Testing Regression Testing Needs proper planning.

For an Effective Regression Testing to be done the following ingredients are
necessary:

- Create a Regression Test Plan: Test Plan identified Focus Areas, Strategy, Test
Entry and Exit Criteria. It can also outline Testing Prerequisites, Responsibilities,
etc.

- Create Test Cases: Test Cases that cover all the necessary areas are important.
They describe what to Test, Steps needed to test, Inputs and Expected Outputs. Test
Cases used for Regression Testing should specifically cover the functionality
addressed by the change and all components affected by the change. The
Regression Test case may also include the testing of the performance of the
components and the application after the change(s) were done.

- Defect Tracking: As in all other Testing Levels and Types It is important Defects
are tracked systematically, otherwise it undermines the Testing Effort.

Summary:

In this article we studied the importance of ‘Regression Testing’, its role and how it
is done.

Integration Testing: Why? What? & How?

Introduction:

As we covered in various articles in the Testing series there are various levels of
testing:
Unit Testing, Integration Testing, System Testing

Each level of testing builds on the previous level.

“Unit testing” focuses on testing a unit of the code.

“Integration testing” is the next level of testing. This ‘level of testing’ focuses on
testing the integration of “units of code” or components.

How does Integration Testing fit into the Software Development

Life Cycle?

Even if a software component is successfully unit tested, in an enterprise n-tier

distributed application it is of little or no value if the component cannot be
successfully integrated with the rest of the application.

Once unit tested components are delivered we then integrate them together.
These “integrated” components are tested to weed out errors and bugs caused due
to the integration. This is a very important step in the Software Development Life
Cycle.

It is possible that different programmers developed different components.

A lot of bugs emerge during the integration step.

In most cases a dedicated testing team focuses on Integration Testing.

Prerequisites for Integration Testing:

Before we begin Integration Testing it is important that all the components have
been successfully unit tested.

Integration Testing Steps:

Integration Testing typically involves the following Steps:

Step 1: Create a Test Plan
Step 2: Create Test Cases and Test Data
Step 3: If applicable create scripts to run test cases
Step 4: Once the components have been integrated execute the test cases
Step 5: Fix the bugs if any and re test the code
Step 6: Repeat the test cycle until the components have been successfully
integrated

What is an ‘Integration Test Plan’?

As you may have read in the other articles in the series, this document typically
describes one or more of the following:
- How the tests will be carried out
- The list of things to be Tested
- Roles and Responsibilities
- Prerequisites to begin Testing
- Test Environment
- Assumptions
- What to do after a test is successfully carried out
- What to do if test fails
- Glossary

How to write an Integration Test Case?

Simply put, a Test Case describes exactly how the test should be carried out.
The Integration test cases specifically focus on the flow of data/information/control
from one component to the other.

So the Integration Test cases should typically focus on scenarios where one
component is being called from another. Also the overall application functionality
should be tested to make sure the app works when the different components are
brought together.

The various Integration Test Cases clubbed together form an Integration Test Suite
Each suite may have a particular focus. In other words different Test Suites may be
created to focus on different areas of the application.

As mentioned before a dedicated Testing Team may be created to execute the

Integration test cases. Therefore the Integration Test Cases should be as detailed as
possible.

Sample Test Case Table:

Test
Test Case Input Expected Actual
Case Pass/Fail Remarks
Description Data Result Result
ID

Additionally the following information may also be captured:

a) Test Suite Name
b) Tested By
c) Date
d) Test Iteration (One or more iterations of Integration testing may be performed)

Working towards Effective Integration Testing:

There are various factors that affect Software Integration and hence Integration
Testing:

1) Software Configuration Management: Since Integration Testing focuses on

Integration of components and components can be built by different developers and
even different development teams, it is important the right version of components
are tested. This may sound very basic, but the biggest problem faced in n-tier
development is integrating the right version of components. Integration testing may
run through several iterations and to fix bugs components may undergo changes.
Hence it is important that a good Software Configuration Management (SCM) policy
is in place. We should be able to track the components and their versions. So each
time we integrate the application components we know exactly what versions go
into the build process.

2) Automate Build Process where Necessary: A Lot of errors occur because the
wrong version of components were sent for the build or there are missing
components. If possible write a script to integrate and deploy the components this
helps reduce manual errors.

3) Document: Document the Integration process/build process to help eliminate

the errors of omission or oversight. It is possible that the person responsible for
integrating the components forgets to run a required script and the Integration
Testing will not yield correct results.

4) Defect Tracking: Integration Testing will lose its edge if the defects are not tracked
correctly. Each defect should be documented and tracked. Information should be captured
as to how the defect was fixed. This is valuable information. It can help in future
integration and deployment processes.

Summary:

Integration testing is the most crucial steps in Software Development Life Cycle.
Different components are integrated together and tested. This can be a daunting
task in enterprise applications where diverse teams build different modules and
components. In this article you learned the steps needed to perform Integration
Testing.

Unit Testing: Why? What? & How?

In this tutorial you will learn about unit testing, various levels of testing, various
types of testing based upon the intent of testing, How does Unit Testing fit into the
Software Development Life Cycle? Unit Testing Tasks and Steps, What is a Unit Test
Plan? What is a Test Case? and Test Case Sample, Steps to Effective Unit Testing.

There are various levels of testing:

 Unit Testing
 Integration Testing
 System Testing

There are various types of testing based upon the intent of testing such as:

 Acceptance Testing
 Performance Testing
 Load Testing
 Regression Testing

Based on the testing Techniques testing can be classified as:

 Black box Testing

 White box Testing
How does Unit Testing fit into the Software Development Life
Cycle?

This is the first and the most important level of testing. As soon as the programmer
develops a unit of code the unit is tested for various scenarios. As the application is
built it is much more economical to find and eliminate the bugs early on. Hence Unit
Testing is the most important of all the testing levels. As the software project
progresses ahead it becomes more and more costly to find and fix the bugs.

In most cases it is the developer’s responsibility to deliver Unit Tested Code.

Unit Testing Tasks and Steps:

Step 1: Create a Test Plan
Step 2: Create Test Cases and Test Data
Step 3: If applicable create scripts to run test cases
Step 4: Once the code is ready execute the test cases
Step 5: Fix the bugs if any and re test the code
Step 6: Repeat the test cycle until the “unit” is free of all bugs

What is a Unit Test Plan?

This document describes the Test Plan in other words how the tests will be carried
out.
This will typically include the list of things to be Tested, Roles and Responsibilities,
prerequisites to begin Testing, Test Environment, Assumptions, what to do after a
test is successfully carried out, what to do if test fails, Glossary and so on

What is a Test Case?

Simply put, a Test Case describes exactly how the test should be carried out.
For example the test case may describe a test as follows:
Step 1: Type 10 characters in the Name Field
Step 2: Click on Submit

Test Cases clubbed together form a Test Suite

Test Case Sample

Test Test Case Input Expected Actual

Pass/Fail Remarks
Case ID Description Data Result Result

Additionally the following information may also be captured:

a) Unit Name and Version Being tested
b) Tested By
c) Date
d) Test Iteration (One or more iterations of unit testing may be performed)
Steps to Effective Unit Testing:

1) Documentation: Early on document all the Test Cases needed to test your code.
A lot of times this task is not given due importance. Document the Test Cases, actual
Results when executing the Test Cases, Response Time of the code for each test
case. There are several important advantages if the test cases and the actual
execution of test cases are well documented.

a. Documenting Test Cases prevents oversight.

b. Documentation clearly indicates the quality of test cases
c. If the code needs to be retested we can be sure that we did not miss anything
d. It provides a level of transparency of what was really tested during unit testing.
This is one of the most important aspects.

e. It helps in knowledge transfer in case of employee attrition

f. Sometimes Unit Test Cases can be used to develop test cases for other levels of
testing

2) What should be tested when Unit Testing: A lot depends on the type of
program or unit that is being created. It could be a screen or a component or a web
service. Broadly the following aspects should be considered:

a. For a UI screen include test cases to verify all the screen elements that need to
appear on the screens
b. For a UI screen include Test cases to verify the spelling/font/size of all the
“labels” or text that appears on the screen
c. Create Test Cases such that every line of code in the unit is tested at least once in
a test cycle
d. Create Test Cases such that every condition in case of “conditional statements” is
tested once
e. Create Test Cases to test the minimum/maximum range of data that can be
entered. For example what is the maximum “amount” that can be entered or the
max length of string that can be entered or passed in as a parameter
f. Create Test Cases to verify how various errors are handled
g. Create Test Cases to verify if all the validations are being performed

3) Automate where Necessary: Time pressures/Pressure to get the job done may
result in developers cutting corners in unit testing. Sometimes it helps to write
scripts, which automate a part of unit testing. This may help ensure that the
necessary tests were done and may result in saving time required to perform the
tests.

Summary:

“Unit Testing” is the first level of testing and the most important one. Detecting and
fixing bugs early on in the Software Lifecycle helps reduce costly fixes later on. An
Effective Unit Testing Process can and should be developed to increase the Software
Reliability and credibility of the developer. The Above article explains how Unit
Testing should be done and the important points that should be considered when
doing Unit Testing.
Many new developers take the unit testing tasks lightly and realize the importance
of Unit Testing further down the road if they are still part of the project. This article
serves as a starting point for laying out an effective (Unit) Testing Strategy.

Software Quality Management

This article gives an overview of Software Quality Management and various
processes that are a part of Software Quality Management. Software Quality is a
highly overused term and it may mean different things to different people. You will
learn What is Software Quality Management?, What does it take to Manage
Software Quality?, Quality Planning, Quality Assurance, Quality Control, Importance
of Documentation and What is Defect Tracking?

The definition of the ISO 8204 for quality:

“Totality of characteristics of an entity that bears on its ability to satisfy stated and
implied needs.”

This means that the Software product delivered should be as per the requirements
defined. We now examine a few more terms used in association with Software
Quality.

Quality Planning:
In the Planning Process we determine the standards that are relevant for the
Software Product, the Organization and the means to achieve them.

Quality Assurance:
Once the standards are defined and we start building the product. It is very
important to have processes that evaluate the project performance and aim to
assure that the Quality standards are being followed and the final product will be in
compliance.

Quality Control:
Once the software components are built the results are monitored to determine if
they comply with the standards. The data collected helps in measuring the
performance trends and as needed help in identifying defective pieces of code.

What is Software Quality Management?

Software Quality Management simply stated comprises of processes that ensure

that the Software Project would reach its goals. In other words the Software Project
would meet the clients expectations.

The key processes of Software Quality Management fall into the following three
categories:

1) Quality Planning
2) Quality Assurance
3) Quality Control

What does it take to Manage Software Quality?

The Software Quality Management comprises of Quality Planning, Quality
Assurance and Quality Control Processes. We shall now take a closer look at each of
them.

1) Quality Planning

Quality Planning is the most important step in Software Quality Management.

Proper planning ensures that the remaining Quality processes make sense and
achieve the desired results. The starting point for the Planning process is the
standards followed by the Organization. This is expressed in the Quality Policy and
Documentation defining the Organization-wide standards. Sometimes additional
industry standards relevant to the Software Project may be referred to as needed.
Using these as inputs the Standards for the specific project are decided. The Scope
of the effort is also clearly defined. The inputs for the Planning are as summarized
as follows:

a. Company’s Quality Policy

b. Organization Standards
c. Relevant Industry Standards
d. Regulations
e. Scope of Work
f. Project Requirements

Using these as Inputs the Quality Planning process creates a plan to ensure that
standards agreed upon are met. Hence the outputs of the Quality Planning process
are:

a. Standards defined for the Project

b. Quality Plan

To create these outputs namely the Quality Plan various tools and techniques are
used. These tools and techniques are huge topics and Quality Experts dedicate years
of research on these topics. We would briefly introduce these tools and techniques
in this article.

a. Benchmarking: The proposed product standards can be decided using the

existing performance benchmarks of similar products that already exist in the
market.

b. Design of Experiments: Using statistics we determine what factors influence

the Quality or features of the end product

c. Cost of Quality: This includes all the costs needed to achieve the required
Quality levels. It includes prevention costs, appraisal costs and failure costs.

d. Other tools: There are various other tools used in the Planning process such as
Cause and Effect Diagrams, System Flow Charts, Cost Benefit Analysis, etc.

All these help us to create a Quality Management Plan for the project.

2) Quality Assurance
The Input to the Quality Assurance Processes is the Quality Plan created during
Planning.
Quality Audits and various other techniques are used to evaluate the performance of
the project. This helps us to ensure that the Project is following the Quality
Management Plan.

The tools and techniques used in the Planning Process such as Design of
Experiments, Cause and Effect Diagrams may also be used here, as required.

3) Quality Control

Following are the inputs to the Quality Control Process:

- Quality Management Plan.

- Quality Standards defined for the Project
- Actual Observations and Measurements of the Work done or in Progress

The Quality Control Processes use various tools to study the Work done. If the Work
done is found unsatisfactory it may be sent back to the development team for fixes.
Changes to the Development process may be done if necessary.

If the work done meets the standards defined then the work done is accepted and
released to the clients.

Importance of Documentation:

In all the Quality Management Processes special emphasis is put on documentation.

Many software shops fail to document the project at various levels. Consider a
scenario where the Requirements of the Software Project are not sufficiently
documented. In this case it is quiet possible that the client has a set of expectations
and the tester may not know about them. Hence the testing team would not be able
test the software developed for these expectations or requirements. This may lead
to poor “Software Quality” as the product does not meet the expectations.

Similarly consider a scenario where the development team does not document the
installation instructions. If a different person or a team is responsible for future
installations they may end up making mistakes during installation, thereby failing to
deliver as promised.

Once again consider a scenario where a tester fails to document the test results
after executing the test cases. This may lead to confusion later. If there were an
error, we would not be sure at what stage the error was introduced in the software
at a component level or when integrating it with another component or due to
environment on a particular server etc. Hence documentation is the key for future
analysis and all Quality Management efforts.

Steps:
In a typical Software Development Life Cycle the following steps are necessary for
Quality Management:

1) Document the Requirements

2) Define and Document Quality Standards
3) Define and Document the Scope of Work
4) Document the Software Created and dependencies
5) Define and Document the Quality Management Plan
6) Define and Document the Test Strategy
7) Create and Document the Test Cases
8) Execute Test Cases and (log) Document the Results
9) Fix Defects and document the fixes
10) Quality Assurance audits the Documents and Test Logs

Various Software Tools have been development for Quality Management. These
Tools can help us track Requirements and map Test Cases to the Requirements.
They also help in Defect Tracking.

What is Defect Tracking?

This is very important to ensure the Quality of the end Product. As test cases are
executed at various levels defects if any are found in the Software being tested. The
Defects are logged and data is collected. The Software Development fixes these
defects and documents how they were fixed The testing team verifies whether the
defect was really fixed and closes the defects. This information is very useful. Proper
tracking ensures that all Defects were fixed. The information also helps us for future
projects.

The Capability Maturity Model defines various levels of Organization based on the
processes that they follow.

Level 0
The following is true for “Level 0” Organizations -
There are no Processes, tracking mechanisms, no plans. It is left to the developer or
any person responsible for Quality to ensure that the product meets expectations.

Level 1 – Performed Informally

The following is true for “Level 1” Organizations -
In Such Organizations, Typically the teams work extra hard to achieve the results.
There are no tracking mechanisms, standards defined. The work is done but is
informal and not well documented.

Level 2 – Planned and Tracked

The following is true for “Level 2” Organizations -
There are processes within a team and the team can repeat them or follow the
processes for all projects that it handles.

However the process is not standardized throughout the Organization. All the teams
within the organization do not follow the same standard.

Level 3 – Well-Defined
In “Level 3” Organizations the processes are well defined and followed throughout
the organization.

Level 4 – Quantitatively Controlled

In “Level 4” Organizations -
- The processes are well defined and followed throughout the organization
- The Goals are defined and the actual output is measured
- Metrics are collected and future performance can predicted

Level 5 – Continuously Improving

“Level 5” Organizations have well defined processes, which are measured and the
organization has a good understanding of IT projects affect the Organizational
goals.
The Organization is able to continuously improve its processes based on this
understanding.

Summary:

In this article we studied the Software Quality Management process.