Chapter 8:
Project Quality Management
Information Technology
Project Management,
Fifth Edition
The Importance of Project Quality
Management
Many people joke about the poor quality of IT products
People seem to accept systems being down occasionally
or needing to reboot their PCs
But quality is very important in many IT projects
What Is Project Quality?
The International Organization for Standardization (ISO) defines
quality as “the degree to which a set of inherent characteristics
fulfills requirements” (ISO9000:2000)
Other experts define quality based on:
Conformance to requirements: the project’s processes
and products meet written specifications
Fitness for use: a product can be used as it was
intended
What Is Project Quality Management?
Project quality management ensures that the project will satisfy
the needs for which it was undertaken
Processes include:
Quality planning: identifying which quality standards are
relevant to the project and how to satisfy them
Quality assurance: periodically evaluating overall project
performance to ensure the project will satisfy the relevant quality
standards
Quality control: monitoring specific project results to ensure that they
comply with the relevant quality standards
Project Quality Management Summary
Quality Planning
Implies the ability to anticipate situations and prepare
actions to bring about the desired outcome
Important to prevent defects by:
Selecting proper materials
Training and indoctrinating people in quality
Planning a process that ensures the appropriate
outcome
Design of Experiments
Design of experiments is a quality planning technique that helps
identify which variables have the most influence on the overall outcome
of a process
Computer chip designer would determine what combination of
materials and equipment will produce the most reliable chips at a
reasonable cost
Also applies to project management issues, such as cost and schedule
trade-offs
Junior programmers cost less than senior programmers but will not
produce the same level of work in the same amount of time
Design of Experiments
An appropriately designed experiment to compute` project costs
and durations for various combinations of staff can help determine
an optimal mix of personnel
Involves documenting important factors that directly contribute to
meeting customer requirements
It is often difficult for customers to explain exactly what they want in an
IT project.
Important scope aspects of IT projects that affect quality include:
Functionality is the degree to which a system performs its intended
function
Features are the system’s special characteristics that appeal to users.
It is important to specify which are required and which are optional
System outputs are the screens and reports the system generates.
Need to define clearly what they look like
Scope Aspects of IT Projects
Performance addresses how well a product or service performs the
customer’s intended use.
Need to know volumes of data and transactions, number of
simultaneous users, required response time, etc.
Reliability is the ability of a product or service to perform as
expected under normal conditions (customers must define expected
level of service)
Maintainability addresses the ease of performing maintenance on a
product
Quality Assurance
Quality assurance includes all the activities related to satisfying the
relevant quality standards for a project
Another goal of quality assurance is continuous quality
improvement
Benchmarking generates ideas for quality improvements by comparing
specific project practices or product characteristics to those of other
projects or products within or outside the performing organization
A quality audit is a structured review of specific quality management
activities that help identify lessons learned that could improve
performance on current or future projects
Performed by in-house auditors or third parties
Quality Control
Although one of the main goals of QC is to improve quality, its
main outcomes are:
Acceptance decisions- are the products/services
acceptable or should they be rejected and rework is then
necessary
Rework – action taken to bring rejected items into
compliance with products specs. Can be very expensive
Process adjustments – correct or prevent further quality
problems based on quality control measurements (purchase
faster server if response time is too slow)
7 Tools & Techniques for Quality Control
1. Cause-and-effect diagrams trace complaints about quality
problems back to the responsible production operations
They help you find the root cause of a problem
Also known as fishbone or Ishikawa diagrams
Can also use the 5 whys technique where you repeat the question
“Why” (five is a good rule of thumb) to peel away the layers of
symptoms that can lead to the root cause
1. Why the users can not get into the system
2. Why they keep forgetting passwords
3. Why didn’t they reset their passwords
4. Why didn’t they check the box to save their password, etc.
Sample Cause-and-Effect Diagram
Sample Cause-and-Effect Diagram
Possible causes of staff leaving before the end of a project
They may include environment, ambition, career prospects,
satisfaction (variety, challenges, recognition), remuneration (basic
pay, benefits - car, health, pension).
2. Quality Control Charts
Graphic display of data that illustrates the results of a process over time
Main use is to prevent defects, rather than to detect or reject them
Quality control charts allow you to determine whether a process is
in control or out of control
When a process is in control, any variations in the results of the
process are created by random events; processes that are in control
do not need to be adjusted
When a process is out of control, variations in the results of the process
are caused by nonrandom events; you need to identify the causes of
those nonrandom events and adjust the process to correct or eliminate
them
The Seven Run Rule
You can use quality control charts and the seven run rule to look for
patterns in data
The seven run rule states that if seven data points in a row are all
below the mean, above the mean, or are all increasing or decreasing,
then the process needs to be examined for nonrandom problems
Example: The following slide is a control chart for the manufacture
of 12” rulers
Upper and lower specifications are 12.10” and 11.9” – this is the
range specified as acceptable by the customer for purchase
The controls limits of 11.91” and 12.09” mean that the
manufacturing process is designed to produce rulers within that
range
Sample Quality Control Chart
The rule violations indicate that a calibration device may need adjustment
3. Run Chart
A run chart displays the history and pattern of variation of a process
over time
It is a line chart that shows data points plotted in the order in which
they occur
Can be used to perform trend analysis to forecast future outcomes
based on historical patterns e.g., of defects
4. Scatter Diagram
A scatter diagram helps to show if there is a relationship between
two variables
The closer data points are to a diagonal line, the more closely the two
variables are related
5. Histograms
A histogram is a bar graph of a distribution of variables
Each bar represents an attribute or characteristic of a problem or
situation, and the height of the bar represents its frequency
6. Pareto Charts
A histogram that can help you identify and prioritize problem areas
The variables are ordered by frequency of occurrence to help identify
the key contributors that account for most quality problems
(hopefully following the 80-20 rule)
Pareto analysis is also called the 80-20 rule, meaning that 80 percent
of problems are often due to 20 percent of the causes
In the following chart, Log-in Problems account for about 55% of
the complaints and together with System lock-ups accounts for about
80%
Fixing these two problems can greatly reduce the volume of compalints
Small problems should be investigated before addressing them in case
the user is in error
Sample Pareto Diagram
7. Flowcharts
Flowcharts are graphic displays
of the logic and flow of
processes that help you analyze
how problems occur and how
processes can be improved
They show activities, decision
points, and the order of how
information is processed
Statistical Sampling
Statistical sampling involves choosing part of a
population of interest for inspection
This is needed when the population is too large be to be
completely sampled
The size of a sample depends on how representative you want the
sample to be
Sample size formula:
Sample size = .25 X (certainty factor/acceptable error)2
Desired certainty Certainty factor Sample size
95% 1.960 384
90% 1.645 68
80% 1.281 10
Six Sigma
Six Sigma is “a comprehensive and flexible system for achieving,
sustaining, and maximizing business success.
Six Sigma is uniquely driven by close understanding of customer
needs, disciplined use of facts, data, and statistical analysis, and
diligent attention to managing, improving, and reinventing business
processes.”
Basic Information on Six Sigma
The target for perfection is the achievement of no more than 3.4
defects per million opportunities
The principles can apply to a wide variety of processes – design
and production of a product, a Help Desk or other customer-service
process
Six Sigma projects normally follow a five-phase improvement
process called DMAIC
DMAIC
DMAIC is a systematic, closed-loop process for continued improvement that is
scientific and fact based
Define: Define the problem/opportunity, process, and customer requirements.
Tool used include project charter, requirements, Voice of the Customer data.
Measure: Define measures (in terms of defects per million), then
collect, compile, and display data
Analyze: Scrutinize process details to find improvement opportunities;
seeks root cause of problems
Improve: Generate solutions and ideas for improving the problem; pilot test
the solution
Control: Track and verify the stability of the improvements and the
predictability of the solution
How Is Six Sigma Quality Control Unique?
It requires an organization-wide commitment at all levels. Often huge
training investements but pay off in higher quality goods and services at
lower costs
Training follows the “Belt” system as in a karate class
Six Sigma organizations have the ability and willingness to adopt
contrary objectives: reducing errors and getting things done faster;
creative and rational; focus on the big picture and minute details; make
customers happy and make a lot of money
It is an operating philosophy that is customer-focused and strives to
drive out waste, raise levels of quality, and improve financial
performance at breakthrough levels
Six Sigma and Project Management
Joseph M. Juran stated, “All improvement takes place project by project, and
in no other way”
It’s important to select projects carefully, apply higher quality where it makes
sense; companies that use Six Sigma do not always boost their stock values
Minimizing defects does not matter if an organization is making a product
that no one wants to buy. As Mikel Harry puts it, “I could genetically
engineer a Six Sigma goat, but if a rodeo is the marketplace, people are still
going to buy a Four Sigma horse.”
Six Sigma projects must focus on a quality problem or gap between the
current and desired performance, not have a clearly understood problem,
the solution should not be predetermined and an optimal solution should
not be apparent
Six Sigma Projects Use Project Management
The training for Six Sigma includes many project management
concepts, tools, and techniques
For example, Six Sigma projects often use business cases, project
charters, schedules, budgets, and so on
Six Sigma projects are done in teams;
the project manager is often called the team leader, and the sponsor is
called the champion
Six Sigma projects are projects that focus on supporting the Six
Sigma philosophy by being customer-focused and striving to
drive out waste, raise levels of quality and improve financial
performance at breakthrough levels
Six Sigma and Statistics
The term sigma means standard deviation
Standard deviation measures how much variation exists in a
distribution of data
Standard deviation is a key factor in determining the acceptable
number of defective units found in a population.
A small s.d. means the data clusters closely around the middle of a
distribution and there is little variability in the data.
Six Sigma projects strive for no more than 3.4
defects per million opportunities, yet this number is confusing to many
statisticians
Six Sigma Uses a Conversion Table
Using a normal curve, if a process is at six sigma, there would be no
more than two defective units per billion produced
Six Sigma uses a scoring system that accounts for time, an important
factor in determining process variations
Yield represents the number of units handled correctly through
the process steps
3
Six Sigma Uses a Conversion Table
A defect is any instance where the product or service fails to meet
customer requirements
Because most products or services have multiple customer
requirements, there can be several opportunities to have a defect
Ex: a company is trying to reduce errors on their bills. There
could be several errors – misspelled name, wrong address,
calculation error, etc.
Instead of measuring the number of defects per billing
statement, Six Sigma measures the number of defects based
on the number of opportunities
3
Normal Distribution and Standard Deviation
Normal Distribution and Standard Deviation
Specification Range % of Defective units
(in +/- Sigmas) population per billion
within range
1 68.27 317,300,000
2 95.45 45,400,000
3 99.73 2,700,000
4 99.9937 63,000
5 99.999943 57
6 99.9999998 2
Sigma Conversion Table
Sigma Yield Defects per
Million
Opportunities
1 31.0% 690,000
2 69.2% 308,000
3 93.3% 66,800
4 99.4% 6,210
5 99.97% 230
6 99.99966% 3.4
Six 9s of Quality
Six 9s of quality is a measure of quality control equal to 1 fault in 1
million opportunities
In the telecommunications industry, it means 99.9999 percent
service availability or 30 seconds of down time a year
This level of quality has also been stated as the target goal for the
number of errors in a communications circuit, system failures, or errors
in lines of code
To achieve six 9s of quality requires continual testing to find and
eliminate errors or enough redundancy and back-up equipment to
reduce the overall system failure to that low a level
Testing
Many IT professionals think of testing as a stage that comes near the end
of IT product development
Testing should be done during almost every phase of the IT product
development life cycle
Types of Tests:
Unit testing tests each individual component (often a program) to
ensure it is as defect-free as possible
Integration testing occurs between unit and system testing to test
functionally grouped components
System testing tests the entire system as one entity
User acceptance testing is an independent test performed by end
users prior to accepting the delivered system
3
Testing Tasks in the Software Development Life Cycle
One way of portraying
the systems life cycle
Shows 17 main tasks
involved in a s/w
development project
and shows their
realtionship to each
other
Testing Alone Is Not Enough
Watts S. Humphrey, a renowned expert on software quality, defines a software
defect as anything that must be changed before delivery of the program
Testing does not sufficiently prevent software defects because:
As code gets more complex, the number of defects missed by testing increases
and becomes the problem of not just the testers but also of the paying
customers
He estimates that finished code, after all testing, contains 5-6
defects per thousand lines of code
The number of ways to test a complex system is huge
Users will continue to invent new ways to use a system that its developers
never considered
Humphrey suggests that people rethink the software development process to
provide no potential defects when you enter system testing; developers must be
responsible for providing error-free code at each stage of testing
Modern Quality Management
Modern quality management:
Requires customer satisfaction
Prefers prevention to inspection
Recognizes management responsibility for quality
Noteworthy quality experts include Deming, Juran, Crosby, Ishikawa,
Taguchi, and Feigenbaum
Quality Experts
Deming was famous for his work in rebuilding Japan after WWII and his 14
Points for Management
His ideas were not accepted by US industry until Japan started producing
products that seriously challenged American products, particularly in the auto
industry
Juran wrote the Quality Control Handbook and ten steps to quality improvement
Stressed the difference between manufacturer’s view of quality focus on
conformance to quality) and the customer’s view (fitness for use).
Crosby wrote Quality is Free and suggested that organizations strive
for zero defects
He suggested that the cost of poor quality is so understated that companies
can profitably spend unlimited amounts of money on improving quality
Quality Experts
Ishikawa developed the concepts of quality circles and fishbone
diagrams
Quality circles are groups of non-supervisors and work leaders in
a single company department who volunteer to conduct group
studies on how to improve the effectiveness of work in their
department
In Japan quality is a company wide commitment while in the
US it is delegated to a few staff members
Quality Experts
Taguchi developed methods for optimizing the process of
engineering experimentation
Quality should be designed into the product and not inspected into it
Quality is best achieved by minimizing deviation from the target value
Robust design methods – focus on eliminating defects by substituting
scientific inquiry for trial-and-error methods
Feigenbaum developed the concept of total quality control
Responsibility for quality should rest with the people who do the work
Product quality is more important that production rates and workers
are allowed to stop production whenever a quality problem occurs
Malcolm Baldrige Award
The Malcolm Baldrige National Quality Award originated in
1987 to recognize companies that have achieved a level of world-
class competition through quality management
Given by the President of the United States to U.S. businesses
Three awards each year in different categories
Manufacturing
Service
Small business
Education and health care
ISO Standards
ISO 9000 is a quality system standard that:
Is a three-part, continuous cycle of planning, controlling,
and documenting quality in an organization
Provides minimum requirements needed for an organization to
meet its quality certification standards
Helps organizations around the world reduce costs and improve
customer satisfaction
Improving Information Technology Project Quality
Several suggestions for improving quality for IT projects include:
Establish leadership that promotes quality
Understand the cost of quality
Focus on organizational influences and workplace factors that affect
quality
Improving the organization’s overall maturity level in
software development and project management
Leadership
As Joseph M. Juran said in 1945, “It is most important that top management
be quality-minded. In the absence of sincere manifestation of interest at the
top, little will happen below.”
A large percentage of quality problems are associated with management,
not technical issues
Leadership
As globalization increases and customers become more demanding,
creating quality products quickly at a reasonable price is essential for
staying in business
In 1988, Motorola Corp. became one of the first companies to receive
the Malcolm Baldrige National Quality Award.
One of Motorola's innovations that attracted a great deal of attention
was its Six Sigma program.
Top management stressed the need to develop and use quality
standards and provided resources (training, staff, customer input) to
help improve quality
The Cost of Quality
The cost of quality is the cost of conformance plus the cost of
nonconformance
Conformance means delivering products that meet
requirements and fitness for use
Cost of nonconformance means taking responsibility for failures or
not meeting quality expectations
A 2002 study reported that software bugs cost the
U.S. economy $59.6 billion (6% of GDP) each year and that one-third of
the bugs could be eliminated by an improved testing infrastructure
Gartner Research estimated that the cost of downtime for
computer networks is about
$42,000/hour.
A worse than average system with a downtime of 30 minutes per day
can cost more than $7 million per year.
Five Cost Categories Related to Quality
Prevention cost: cost of planning and executing a project so it is error-free
or within an acceptable error range
Appraisal cost: cost of evaluating processes and their outputs to ensure
that a project is either error-free or within an acceptable error range
Internal failure cost: cost incurred to correct an identified defect before
the customer receives the product (rework, inventory costs due to
defects, premature failure of products)
External failure cost: cost that relates to all errors not detected and
corrected before delivery to the customer (warranty costs, product liability
suits, future business losses)
Measurement and test equipment costs: capital cost of equipment used
to perform prevention and appraisal activities
Five Cost Categories Related to Quality
Demarco found that the average large company devoted more than
60% of its s/w development efforts to maintenance
Around 50% of development costs are typically spent on testing
and debugging software
Top management is primarily responsible for the high cost of
nonconformance in IT
Top managers often rush their organizations to develop new systems
and do not give project teams enough time ot resources to do a project
right the first time
Top management must create a culture that embraces quality
Organizational Influences, Workplace Factors, and Quality
Study by DeMarco and Lister showed that organizational issues had a
much greater influence on programmer productivity than the technical
environment or programming languages
Programmer productivity varied by a factor of one to ten across all
participants across all organizations, but only by 21% within the same
organization
Study found no correlation between productivity and programming
language, years of experience, or salary
A dedicated workspace and a quiet work environment were key factors
to improving programmer productivity
Organizational Influences, Workplace Factors, and Quality
Major problems in with work performance and project failures are
sociological, not technological, in nature
They suggest minimizing office politics and giving smart people
physical space, intellectual responsibility and strategic direction
and then just letting them work
Manager should not make people work, but make it possible for
people to work by removing political roadblocks
Expectations and Cultural Differences in
Quality
Project managers must understand and manage stakeholder
expectations
Expectations also vary by:
Organization’s culture – even within the organization
Geographic regions
Maturity Models
Maturity models are frameworks for helping organizations improve
their processes and systems
An evolutionary path of increasingly organized and systematically
more mature processes
The Software Quality Function Deployment Model focuses on
defining user requirements and planning software projects resulting in
a set of measurable technical product specifications and their priorities
Clearer requirements can lead to fewer design changes, increased
productivity and ultimately s/w products that are more likely to
satisfy stakeholder requirements
Maturity Models
The Software Engineering Institute’s Capability Maturity Model
Integration is a process improvement approach that provides
organizations with the essential elements of effective processes
Companies may not get to bid on government projects unless they
have a CMMI Level 3
CMMI Staged Representation
Optimizing
5 Focus on process
improvement
Quantitatively
4 Process measured Managed
and controlled
Defined
3 Process characterized
for the organization
and is proactive
Managed
2 Process characterized for
projects and is often Improvement
reactive
Performed planning, execution
1 Process unpredictable, and measurement is
poorly controlled and
reactive sequential through
the CMMI levels
PMI’s Maturity Model
PMI released the Organizational Project Management Maturity
Model (OPM3) in December 2003
Model is based on market research surveys sent to more than 30,000
project management professionals, and incorporates 180 best
practices and more than 2,400 capabilities, outcomes, and key
performance indicators
Addresses standards for excellence in project, program, and
portfolio management best practices and explains the capabilities
necessary to achieve those best practices
