Development Methods

 commercial quality systems require a

systematic development approach
◦ ad hoc approaches may be suitable for
research prototypes or personal use, but not
for widely used or critical systems
 some software engineering methods are
suitable for the development of expert
systems
 Problem Selection
 the development of an expert system should be
based on a specific problem to be addressed by
the system
 it should be verified that expert systems are the
right paradigm to solve that type of problem
◦ not all problems are amenable to ES-based solutions
 availability of resources for the development
◦ experts/expertise
◦ hardware/software
◦ users
◦ sponsors/funds
Project Management
 activity planning
◦ planning, scheduling, chronicling, analysis
 product configuration management
◦ product management
◦ change management
 resource management
◦ need determination
◦ acquisition resources
◦ assignment of responsibilities
◦ identification of critical resources
 ES Development Stages
 feasibility study
 paper-based explanation of the main idea(s)
 no implementation
 rapid prototype
 quick and dirty implementation of the main idea(s)
 refined system
 in-house verification by knowledge engineers, experts
 field test
 system tested by selected end users
 commercial quality system
 deployed to a large set of end users
 maintenance and evolution
 elimination of bugs
 additional functionalities
Error Sources in ES Development
 knowledge errors
 semantic errors
 syntax errors
 inference engine errors
 inference chain errors
 limits of ignorance errors
Knowledge Errors
 problem: knowledge provided by the expert
is incorrect or incomplete
reflection of expert’s genuine belief
omission of important aspects
inadequate formulation of the knowledge by the
expert
 consequences
existing solution not found
wrong conclusions
 remedy
validation and verification of the knowledge
may be expensive
 Semantic Errors
 problem: the meaning of knowledge is not
properly communicated
◦ knowledge engineer encodes rules that do not reflect
what the domain expert stated
◦ expert misinterprets questions from the knowledge
engineer
 consequences
◦ incorrect knowledge, inappropriate solutions,
solutions not found
 remedy
◦ formalized protocol for knowledge elicitation
◦ validation of the knowledge base by domain experts
Syntax Errors
 problem: rules or facts do not follow the
syntax required by the tool used
◦ knowledge engineer is not familiar with the
method/tool
◦ syntax not clearly specified
 consequences
◦ knowledge can’t be used
 solutions
◦ syntax checking and debugging tools in the ES
development environment
Inference Engine Errors
 problem: malfunctions in the inference
component of the expert system
◦ bugs
◦ resource limitations
 e.g. memory
 consequences
◦ system crash
◦ incorrect solutions
◦ existing solutions not found
 remedy
◦ validation and verification of the tools used
 Inference Chain Errors
 problem: although each individual inference step
may be correct, the overall conclusion is
incorrect or inappropriate
◦ causes: errors listed above; inappropriate priorities of
rules, interactions between rules, uncertainty, non-
monotonicity
 consequences
◦ inappropriate conclusions
 remedy
◦ formal validation and verification
◦ use of a different inference method
Limits of Ignorance Errors
 problem: the expert system doesn’t
know what it doesn’t know
◦ human experts usually are aware of the
limits of their expertise
 consequences
◦ inappropriate confidence in conclusions
◦ incorrect conclusions
 remedy
◦ meta-reasoning methods that explore the
limits of the knowledge available to the ES
Expert Systems and
Software Engineering
 software process models
◦ waterfall
◦ spiral
 use of SE models for ES development
 ES development models
◦ evolutionary model
◦ incremental model
◦ spiral model
Generic Software Process Models
 waterfall model
◦ separate and distinct phases of specification and
development
 evolutionary development
◦ specification and development are interleaved
 formal systems development
◦ a mathematical system model is formally
transformed to an implementation
 reuse-based development
◦ the system is assembled from existing
components
 Waterfall Model
Requirements
definition

System and
software design

Implementation
and unit testing

Integr ation and

system testing

Operation and
maintenance

[Sommerville 2001]
 Suitability of Software Models for ES
Development
 the following worksheets help with the evaluation of
software models for use in the development of expert
systems
 identify the key differences between conventional software
development and ES development with respect to a specific model
 what are the positive and negative aspects of the model for ES
development
 evaluate the above issues, and give the model a score
 10 for perfectly suited, 0 for completely unsuitable
 determine the overall suitability
 high, medium low
 explanation
 Waterfall Worksheet
Aspect Evaluation Score
key
differences

positive

negative

 overall suitability: • high • medium • low

 explanation
Evolutionary Development
 exploratory development
◦ objective is to work with customers and to
evolve a final system from an initial outline
specification. should start with well-
understood requirements
 throw-away prototyping
◦ objective is to understand the system
requirements. should start with poorly
understood requirements

[Sommerville 2001]
 Evolutionary Development
Concurr ent
activities

Initial
Specification
version

Outline Intermediate
Development
description versions

Final
Validation
version

[Sommerville 2001]
 Evolutionary Dev. Worksheet
Aspect Evaluation Score
key
differences

positive

negative

 overall suitability: • high • medium • low

 explanation
Incremental Development
 development and delivery is broken down
into increments
◦ each increment delivers part of the required
functionality
 user requirements are prioritised
◦ the highest priority requirements are included in
early increments
 once the development of an increment is
started, the requirements are frozen
◦ requirements for later increments can continue
to evolve

[Sommerville 2001]
 Incremental Development

Define outline Assign requirements Design system

requirements to increments architecture

Develop system Valida te Integrate Valida te

increment increment increment system
Final
system
System incomplete

[Sommerville 2001]
Spiral Development
 process is represented as a spiral rather than
as a sequence of activities with backtracking
◦ each loop in the spiral represents a phase in the
process.
◦ no fixed phases such as specification or design
 loops in the spiral are chosen depending on what is
required
◦ risks are explicitly assessed and resolved
throughout the process
 similar to incremental development

[Sommerville 2001]
Spiral Model Sectors
 forquadrants in the coordinate system
represent specific aspects
objective setting
specific objectives for the phase are identified
risk assessment and reduction
risks are assessed and activities put in place to reduce
the key risks
development and validation
a development model for the system is chosen which
can be any of the generic models
planning
the project is reviewed and the next phase of the
spiral is planned

[Sommerville 2001]
Spiral Model
De term ine ob jectiv es
Ev aluate a lt ern atives
alternatives and id en tify, resol ve risk s
cons traint s R isk
analys is
R isk
analys is
R isk
analys is Opera-
P rot otyp e 3 ti onal
P rot otyp e 2 prot oyp e
Risk
R EVIEW analysis P rot o-
ty pe 1
Requi rement s pl an S im ul ati ons, m odels, b en ch marks
Li fe-cycle pl an C oncept o f
Operati on S /W
requi re ment s P rod uct
desi gn Detail ed
Requi rement desi gn
Develop ment
pl an va lid ati on C ode
Desi gn Uni t t es t
Integrati on
and t e st p lan V& V Integr ati on
P lan next p has e test
Accep tance
S erv ice test Develop, v erify
next -l evel p rod uct

[Sommerville 2001]
 Spiral Model Worksheet
Aspect Evaluation Score
key
differences

positive

negative

 overall suitability: • high • medium • low

 explanation
Formal systems development
 based on the transformation of a
mathematical specification through
different representations to an executable
program
 transformations are ‘correctness-
preserving’
◦ it is straightforward to show that the program
conforms to its specification
 embodied in the ‘cleanroom’ approach to
software development

[Sommerville 2001]
 Formal Transformation Model

Formal transformations
T1 T2 T3 T4

Formal R1 Executable
R2 R3
specification program

P1 P2 P3 P4

Proofs of transformation correctness

[Sommerville 2001]
 Formal Transformations Worksheet
Aspect Evaluation Score
key
differences

positive

negative

 overall suitability: • high • medium • low

 explanation
Reuse-Oriented Development
 based on systematic reuse
◦ systems are integrated from existing
components or COTS (commercial-off-the-
shelf) systems
 process stages
◦ component analysis
◦ requirements modification
◦ system design with reuse
◦ development and integration
 this approach is becoming more important
but still limited experience with it
[Sommerville 2001]
 Reuse-oriented development

Requirements Component Requirements System design

specification analysis modification with reuse

Development System
and integration validation

[Sommerville 2001]
Reuse-Oriented Model Worksheet
Aspect Evaluation Score
key
differences

positive

negative

 overall suitability: • high • medium • low

 explanation
 Generic System Design Process

Requirements
specification

Design ac tivities

Architectur
al Inte rface Component Data Algorithm
Abstract
design design design str uctur
e design
specification
design

Software Data
System Inte rface Component Algorithm
specification str uctur
e
architectur
e specification specification specifica
tion
specification

Design pr
oducts

[Sommerville 2001]
 System Evolution

Define system Assess existing Propose system Modify

requirements systems changes systems

Existing New
systems system

[Sommerville 2001]
 Linear Model of ES Development
 the life cycle repeats a sequence of stages
◦ variation of the incremental model
◦ once iteration of the sequence roughly
corresponds to one circuit in the spiral model
 stages
◦ planning
◦ knowledge definition
◦ knowledge design
◦ code & checkout
◦ knowledge verification
◦ system evaluation
 Linear Model Diagram
Knowledge Design Product
Baseline Baseline Baseline

Code & Checkout

Knowledge Knowledge Knowledge
Definition Design Verification

Evaluation
Planning

System
Identification

& Extraction
& Selection
Acquisition

Definition
Analysis

Analysis
Detailed
Source

Design

Formal
Test

Test
Work Knowledge Prelim. Knowl. Test Test Final
Plan Review Data System Readiness Audit Interme
Review Design Review Review Review
Review
Planning
 feasibility assessment
 resource management
 task phasing
 schedules
 high-level requirements
 preliminary functional layout
Knowledge Definition
 knowledge source  knowledge acquisition,
identification and analysis and extraction
selection  acquisition strategy
 source identification  knowledge element
 source importance identification
 source availability  knowledge classification
system
 source selection
 detailed functional layout
 preliminary control flow
 preliminary user’s manual
 requirements
specifications
 knowledge baseline
Knowledge Design
 knowledge definition  detailed design
 knowledge representation  design structure
 detailed control structure  implementation strategy
 internal fact structure  detailed user interface
 preliminary user interface  design specifications and
 initial test plan report
 detailed test plan
Code & Checkout
 coding
 tests
 source listings
 user manuals
 installation and operations guide
 system description document
Knowledge Verification
 formal tests  test analysis
◦ test procedures ◦ results evaluation
◦ test reports ◦ recommendations
System Evaluation
 results evaluation
◦ summarized version of the activity from the
previous stage
 recommendations
◦ as above
 validation
◦ system conforms to user requirements and
user needs
 interim or final report
Important Concepts and Terms
 evolutionary development  knowledge definition
 expert system (ES)  knowledge design
 expert system shell  knowledge representation
 explanation  knowledge verification
 limits of ignorance
 feasibility study
 linear model ES life cycle
 inference
 maintenance
 inference mechanism  rapid prototyping
 If-Then rules  reasoning
 incremental development  rule
 knowledge  semantic error
 knowledge acquisition  software development life
 knowledge base cycle
 knowledge-based system  spiral development
 syntactic error
 waterfall model
Summary Expert System Design
 the design and development of
knowledge-based systems uses similar
methods and techniques as software
engineering
◦ some modifications are necessary
◦ the linear model of ES development is an
adaptation of the incremental SE model
 possible sources of errors are
◦ knowledge and limits of knowledge errors
◦ syntactical and semantical errors
◦ inference engine and inference chain errors