SOFTWARE QUALITY ENGINEERING
LEC # 2: WHAT IS SOFTWARE QUALITY?
WHAT IS QUALITY?
Software Quality
– may include many different attributes and may be defined and
perceived differently based on people’s different roles and
responsibilities
We adopt the correctness-centered view of quality, that is,
– high quality means none or few problems
– of limited damage to customers
WHAT IS QUALITY?
Software was particularly problematical for the following reasons:
Software has no physical existence
The lack of knowledge of client needs at the start
The change of client needs over time
The rapid rate of change in both hardware and software
The high expectations of customers, particularly with respect to
adaptability
Within the software quality area, the need to provide a solution that
matches user needs is often considered as “design quality”, whilst
ensuring a match to the specification is considered“manufacturing
quality”
VIEWS OF QUALITY
Quality is a multidimensional construct. It may therefore be considered
using a polyhedron metaphor. Within this metaphor, a three-dimensional
solid represents quality. Each face represents a different quality aspect,
such as correctness, reliability, and efficiency
It has been classified according to various ‘views’ or perspectives. These
views are often diverse and may conflict with each other. Each view comes
from a particular context
DIFFERENT VIEWS OF QUALITY
Examine the different views of quality based on the different roles
Researchers have divided it into five major views
Transcendental view: some intangible properties that can’t be seen but
delight the users (quality is hard to define or describe in abstract terms)
(Transcendental view e.g. efficient algorithm can do things with speed, can’t
be seen by the user but can delight him)
User view: fitness for purpose, meeting users’ needs
Manufacturing view: conformance to process standards
Product view: the focus is on the inherent characteristics of the product itself
in the hope that controlling these internal metrics will improve the external
behavior of the product
Value-based view: quality is the customers’ willingness to pay
QUALITY CHARACTERISTICS
External:
– What a systems user is interested in; typically properties of any single
particular system
Internal:
– What programmers/management are interested in; properties of the
development of a collection of systems
EXTERNAL CHARACTERISTICS
Correctness- The degree to which the system is free from faults in specification,
design, and implementation
Usability- The Ease with which users can learn and use the system
Efficiency- Minimal use of system resources including memory and execution
time
Reliability- The ability of a system to perform whenever required without/with
few failures
Integrity- Prevention of unauthorized or improper use
Adaptability- Usability in other applications than the original one
Accuracy- Degree of “quantitative” correctness
Robustness- Functioning of the system in presence of invalid inputs, stress
environment
INTERNAL CHARACTERISTICS
Maintainability: Ease of modifying software for changing/adding capabilities,
and improving performance
Flexibility: Extend of modifying system for other uses/environments
Portability: Ease of modifying the system for operating in a different
environment
Reusability: Extend of using parts in other systems
TECHNIQUES FOR IMPROVING SQ
Explicit software quality objectives
Explicit quality assurance activities
Testing strategy
Software Engineering guidelines
Informal technical reviews
Formal technical reviews
External audits
Development process
TECHNIQUES FOR IMPROVING SQ
Change control procedures
Measurement of results
Prototyping
Mathematical proof
Modular programming techniques
THE SOFTWARE PROJECT ROLES
Project manager
Business analyst
Implementation programmer
Quality auditor
End user
Line manager
Project sponsor
PEOPLE’S ROLES & RESPONSIBILITIES
Different people would have different expectations based on their roles
and responsibilities
We can divide the people into two broad groups:
Consumers of software products or services
– customers, users
– can be extended to non-human or invisible users
Producers of software products
– anyone involved in managing, developing, marketing, and service of
software products
– can be extended to third-party participants for add-on or testing
EXTERNAL/INTERNAL VIEW
External view → Consumers’ perspective
People who are more concerned with the external behavior
Internal view → Producers’ perspective
Because they are typically familiar with or at least aware of various
internal characteristics of the product
In other words:
External view mostly sees a software system as a black box
– Because one can observe its behavior but not see through inside
Internal view mostly sees it as a white box
– Because one can see what is inside and how it works
EXTERNAL/CONSUMER QUALITY EXPECTATIONS
Good enough for the price
Fit-for-use, doing the right thing
Conformance, doing the thing right
Reliability- functions correctly over repeated use or over a long period of
time
Related to Validation and Verification (V&V)
QUALITY EXPECTATIONS ON THE CONSUMER
SIDE 2/2
Difference in priority on expectations:
For many users especially novice users: Usability, ease of installation
For non-human users: inter-operability and adaptability, so that the
software can work well with others and within its surrounding
environment
QUALITY EXPECTATIONS ON THE PRODUCER SIDE
Contractual Obligations: conform to product specifications or provide
services that confirm to the service agreement
Software methodologies, languages, and tools: For product and service
managers, adherence to pre-selected software process and relevant
standards, proper choice of software methodologies, languages, and tools,
as well as other factors, related to quality
They are also interested in managing and satisfying user’s quality
expectations, by translating such quality expectations into realistic quality
goals that can be defined and managed internally, selecting appropriate
and effective QA strategies
QUALITY EXPECTATIONS ON THE PRODUCER SIDE
Producer quality views: people on the producer side, their different
concerns may also produce quality views and expectations different
– For example, usability and modifiability may be paramount for people involved
with software service, maintainability for maintenance personnel, portability for
third-party or software packaging service providers, and profitability and
customer value for product marketing
QUALITY STANDARDS AND FRAMEWORKS
Two approaches to software that can be followed to ensure software
quality:
Process based: assurance of the process by which a product is
developed (ISO 9001, ISO 9000-3 provides guidelines for the application
of the ISO 9001)
Product based: the evaluation of the quality of the end product (ISO
9126)
Both approaches are important and both require the presence of a system
for managing quality
QUALITY FRAMEWORKS
Two main approaches:
Standard Models:
– ISO 9126
– McCall
Application or company specific quality models (adaptations of standard
models for specific application environments)
– FURPS: Functionality, Usability, Reliability, Performance and Supportability
– GQM Approach
ISO-9126 QUALITY FRAMEWORK
ISO 9126 is the software product evaluation standard from International
Organization for Standardization (ISO)
ISO-9126 is the most influential one in the Software Engineering
community
ISO-9126 provides a hierarchical framework for quality definition,
organized into quality characteristics and sub-characteristic
ISO-9126 QUALITY FRAMEWORK
Functionality: A set of attributes that bear on the existence of a set of functions
and their specified properties. The functions are those that satisfy stated or
implied needs. The sub-characteristics include:
Suitability- to the user’s needs
Accuracy- of results
Interoperability- with other systems
Security- against unintended access
Reliability: A set of attributes that bear on the capability of software to maintain
its level of performance under stated conditions for a stated period of time. The
sub-characteristics include:
Maturity- frequency of failures
Fault Tolerance- performance in case of faults
Recoverability- of functionality and data loss
GQM (GOAL, QUESTION, METRIC) BY SOFTWARE
ENGINEERING LAB AT THE NASA
A measurement program can be more successful if designed with the
goals in mind
GQM approach provides a framework with 3 steps:
List the major goals of the development/maintenance project
Derive from each goal the questions that must be answered to determine if
the goals are being met
Decide what must be measured to answer the questions adequately
GQM BY SOFTWARE ENGINEERING LAB AT
THE NASA
GOAL Evaluate effectiveness of coding standard
Who is What is What is
QUESTIONS using coder code
Standard? productivity? quality?
Proportion of Experience of Code Effort Errors
Coders Coders size
METRICS -using standard -with standard
-using language -with language
-with environment
etc 23
GQM BY SOFTWARE ENGINEERING LAB AT THE NASA
Benefits :
Generates only those measures relevant to the goal
Several measurements may be needed to answer a single question
A single measurement may apply to more than one question
The goal provides the purpose for collecting the data
Disadvantages:
Additional efforts to derive the goals and metrics
Error prone compared to standard models
CORRECTNESS & DEFECTS: DEFINITIONS
Error: refers to a missing or incorrect human action such as human
misconceptions, misunderstandings, etc. resulting in certain fault(s) being
injected into a software during design, coding and data entry
– Data entry error example: writing someone’s DOB a century back. How to
prevent it?
Fault: internal characteristics - cause for failures
– An incorrect step, process, or data definition in a computer program
Failure: external behavior
– Deviation from expected/specified behavior
– The inability of a system or component to perform its required functions within specified
performance requirements
CORRECTNESS & DEFECTS: DEFINITIONS
Defect:
– Generally refers to some problem, either with external behavior or with
internal characteristics
– error/fault/failure are collectively referred to as defects
EXAMPLE: ERROR, FAULT, FAILURE
Example: Software for finding factorial
– Error: factorial is the sum 1 to N (instead of product!)
– Fault: use + operator instead of *
– Failure: wrong value produced
ERROR, FAULT, FAILURE: RELATION
ERROR, FAULT, FAILURE: RELATION
A causal relation exists among the three aspects of defects:
– errors → faults → failures
However, this relationship is not necessarily1-to-1:
– a single error may cause many faults
– a single fault may cause many failures in repeated executions
– conversely, the same failure may be caused by several faults
– the same fault may be there due to different errors
CORRECTNESS-CENTERED PROPERTIES AND
MEASUREMENTS - EXTERNAL
With Correctness-centered approach, the consumer’s concern is that the
‘no failure, or few failures with min impact’
This concern can be captured by:
– Failure Properties and Direct Failure Measurement
Failure Properties include:
– Information about specific failures, what they are, how they occur etc.
– Failure count, distribution, density, etc.
• Failure Likelihood and Reliability Measurement: How often or how likely
• Failure Severity Measurement and Safety Assurance: Failure impact
CORRECTNESS-CENTERED PROPERTIES AND
MEASUREMENTS - INTERNAL
Fault Properties and Related Measurement
– Individual faults: types, relation to specific failures, their causes, time, and
injection circumstances
– Collective faults: distribution and density over development phases and
different software components
