SOFT COMPUTING

UNIT-01
Basic tools of soft Computing:

Fuzzy logic, Neural Networks and

Evolutionary Computing, Approximations
of Multivariate functions, Non – linear Error
surface and optimization.

2
 INTRODUCTION TO
ARTIFICIAL NEURAL NETWORKS
(ANN)
Outline

Definition, why and how are neural

networks being used in solving problems
Human biological neuron
Artificial Neuron

Applications of ANN

Comparison of ANN vs conventional AI

methods
The idea of ANNs..?
NNs learn relationship between cause and effect or
organize large volumes of data into orderly and
informative patterns.
It’s a frog
frog

lion

bird

What is that?
 Neural networks to the rescue…

• Neural network: information processing

paradigm inspired by biological nervous
systems, such as our brain
• Structure: large number of highly interconnected
processing elements (neurons) working together
• Like people, they learn from experience (by
example)

6
Definition of ANN
“Data processing system consisting of a
large number of simple, highly
interconnected processing elements
(artificial neurons) in an architecture inspired
by the structure of the cerebral cortex of the
brain”

(Tsoukalas & Uhrig, 1997).

7
 Inspiration from Neurobiology
Human Biological Neuron

8
Biological Neural Networks

Biological neuron
 Biological Neural Networks

A biological neuron has

three types of main
components; dendrites,
soma (or cell body) and
axon.
Dendrites receives
signals from other
neurons.
The soma, sums the incoming signals. When
sufficient input is received, the cell fires; that is it
transmit a signal over its axon to other cells.
 Artificial Neurons

ANN is an information processing system that has

certain performance characteristics in common
with biological nets.
Several key features of the processing elements of
ANN are suggested by the properties of biological
neurons:

1. The processing element receives many signals.

2. Signals may be modified by a weight at the receiving
synapse.
3. The processing element sums the weighted inputs.
4. Under appropriate circumstances (sufficient input), the
neuron transmits a single output.
5. The output from a particular neuron may go to many other
neurons.
 Artificial Neurons
• From experience: A physical neuron
examples / training
data
• Strength of connection
between the neurons
is stored as a weight-
value for the specific
connection.
• Learning the solution
to a problem =
changing the
connection weights

An artificial neuron
12
 Artificial Neurons

ANNs have been developed as generalizations of

mathematical models of neural biology, based on
the assumptions that:

1. Information processing occurs at many simple elements

called neurons.
2. Signals are passed between neurons over connection links.
3. Each connection link has an associated weight, which, in
typical neural net, multiplies the signal transmitted.
4. Each neuron applies an activation function to its net input
to determine its output signal.
 Model Of A Neuron
Wa
X1

Wb Y
X2  f()

Wc
X3

Input units Connection Summing

computation
weights function

(dendrite) (synapse) (axon)

(soma)
14
• A neural net consists of a large number of
simple processing elements called neurons,
units, cells or nodes.

• Each neuron is connected to other neurons by

means of directed communication links, each
with associated weight.

• The weight represent information being used by

the net to solve a problem.

15
• Each neuron has an internal state, called
its activation or activity level, which is a
function of the inputs it has received.
Typically, a neuron sends its activation as
a signal to several other neurons.

• It is important to note that a neuron can

send only one signal at a time, although
that signal is broadcast to several other
neurons.
16
• Neural networks are configured for a specific
application, such as pattern recognition or
data classification, through a learning
process
• In a biological system, learning involves
adjustments to the synaptic connections
between neurons
 same for artificial neural networks (ANNs)

17
 Artificial Neural Network
Synapse Nukleus

x1 w1
 
y
Axon
x2 w2 Activation Function:
yin = x1w 1 + x2w 2 (y-in) = 1 if y-in >= 
and (y-in) = 0

Dendrite

-A neuron receives input, determines the strength or the weight of the input, calculates the total
weighted input, and compares the total weighted with a value (threshold)

-The value is in the range of 0 and 1

- If the total weighted input greater than or equal the threshold value, the neuron will produce the
output, and if the total weighted input less than the threshold value, no output will be produced

18
 History
• 1943 McCulloch-Pitts neurons
• 1949 Hebb’s law
• 1958 Perceptron (Rosenblatt)
• 1960 Adaline, better learning rule (Widrow,
Huff)
• 1969 Limitations (Minsky, Papert)
• 1972 Kohonen nets, associative memory

19
• 1977 Brain State in a Box (Anderson)
• 1982 Hopfield net, constraint satisfaction
• 1985 ART (Carpenter, Grossfield)
• 1986 Backpropagation (Rumelhart, Hinton,
McClelland)
• 1988 Neocognitron, character recognition
(Fukushima)

20
 Characterization
• Architecture
– a pattern of connections between neurons
• Single Layer Feedforward
• Multilayer Feedforward
• Recurrent
• Strategy / Learning Algorithm
– a method of determining the connection weights
• Supervised
• Unsupervised
• Reinforcement
• Activation Function
– Function to compute output signal from input signal

21
Where can neural network systems help…
• when we can't formulate an algorithmic
solution.
• when we can get lots of examples of the
behavior we require.
‘learning from experience’
• when we need to pick out the structure
from existing data.

22
 Who is interested?...
• Electrical Engineers – signal processing,
control theory
• Computer Engineers – robotics
• Computer Scientists – artificial
intelligence, pattern recognition
• Mathematicians – modelling tool when
explicit relationships are unknown

23
 Problem Domains
• Storing and recalling patterns
• Classifying patterns
• Mapping inputs onto outputs
• Grouping similar patterns
• Finding solutions to constrained
optimization problems

24
Coronary
Disease

Classification

STOP

01 10
Neural
11
Net 11 10 00 00 Input patterns
00 11

Input layer

Output layer

00 01 10 11
Sorted
00 10 11 patterns
.

00 11 25
 Clustering

00 11
10
11 10

00 11

00
01

26
 ANN Applications

Medical Applications

Information
Searching & retrieval
Chemistry

Education
Business & Management
 Applications of ANNs

• Signal processing
• Pattern recognition, e.g. handwritten
characters or face identification.
• Diagnosis or mapping symptoms to a
medical case.
• Speech recognition
• Human Emotion Detection
• Educational Loan Forecasting
28
 Abdominal Pain Prediction

Intensity Duration
Male Age Temp WBC Pain Pain
adjustable
1 20 37 10 1 1
weights

AppendicitisDiverticulitis Ulcer Pain Cholecystitis Obstruction Pancreatitis

Duodenal Non-specific
Small Bowel
0 0 Perforated
0 0 0 0
1

29
Voice Recognition

30
FUZZY LOGIC
Outlook Temp Humidit Windy Decision
(0) (T) y (W) to play
(H) (D)
Sunny High High F N
Sunny High High T N
Cloudy High High F Y
Rain Med High F Y
Rain Cold Low N Y
Rain Cold Low T N
Cloudy Cold Low T Y
To-play-or-not-to-play-tennis data vs. Climatic-Condition from
Ross Quinlan’s paper on ID3 (1986), C4.5 (1993)
Weather Temp Humidit Windy Decision
(0) (T) y (W) (D)
(H)

Sunny Med High F N

Sunny Cold Low F Y
Rain Med Low F Y
Sunny Med Low T Y
Cloudy Med High T Y
Cloudy High Low F Y
Rain High High T N
 Outlook

Sunny Cloudy Rain

Humidity Yes Windy

High Low T F

No Yes No Yes
 Rule Base
R1: If outlook is sunny and if humidity is high then
Decision is No.

R2: If outlook is sunny and if humidity is low then

Decision is Yes.

R3: If outlook is cloudy then Decision is Yes.

WHAT IS FUZZY LOGIC?

 Definition of fuzzy
 Fuzzy – “not clear, distinct, or precise;
blurred”

 Definition of fuzzy logic

 A form of knowledge representation
suitable for notions that cannot be
defined precisely, but which depend
upon their contexts.
TRADITIONAL REPRESENTATION OF
LOGIC

Slow Fast
Speed = 0 Speed = 1
bool speed;
get the speed
if ( speed == 0) {
// speed is slow
}
else {
// speed is fast
}
FUZZY LOGIC REPRESENTATION
Slowest
 For every problem
[ 0.0 – 0.25 ]
must represent in
terms of fuzzy sets.
Slow
 What are fuzzy sets? [ 0.25 – 0.50 ]

Fast
[ 0.50 – 0.75 ]

Fastest
[ 0.75 – 1.00 ]
FUZZY LOGIC REPRESENTATION
CONT.

Slowest Slow Fast Fastest

float speed;
get the speed
if ((speed >= 0.0)&&(speed < 0.25)) {
// speed is slowest
}
else if ((speed >= 0.25)&&(speed < 0.5))
{
// speed is slow
}
else if ((speed >= 0.5)&&(speed < 0.75))
{
// speed is fast
}
else // speed >= 0.75 && speed < 1.0
{
// speed is fastest
}
ORIGINS OF FUZZY LOGIC
 Traces back to Ancient Greece

 Lotfi Asker Zadeh ( 1965 )

 First to publish ideas of fuzzy logic.

 Professor Toshire Terano ( 1972 )

 Organized the world's first working group on fuzzy
systems.

 F.L. Smidth & Co. ( 1980 )

 First to market fuzzy expert systems.
FUZZY LOGIC IN CONTROL SYSTEMS

 Fuzzy Logic provides a more efficient and

resourceful way to solve Control Systems.

 Some Examples
 Temperature Controller

 Anti – Lock Break System ( ABS )

TEMPERATURE CONTROLLER
 The problem
 Change the speed of a heater fan, based off the room
temperature and humidity.
 A temperature control system has four settings
 Cold, Cool, Warm, and Hot
 Humidity can be defined by:
 Low, Medium, and High
 Using this we can define
the fuzzy set.
BENEFITS OF USING FUZZY LOGIC
ANTI LOCK BREAK SYSTEM ( ABS )
 Nonlinear and dynamic in nature
 Inputs for Intel Fuzzy ABS are derived from
 Brake
 4 WD
 Feedback
 Wheel speed
 Ignition
 Outputs
 Pulsewidth
 Error lamp
FUZZY LOGIC IN OTHER FIELDS

 Business

 Hybrid Modeling

 Expert Systems
SUMMARY

 Fuzzy logic provides an alternative way to

represent linguistic and subjective attributes of the
real world in computing.

 It is able to be applied to control systems and other

applications in order to improve the efficiency and
simplicity of the design process.
 Introduction to
Evolutionary Computation
 Q What is the most powerful
problem solver in the Universe?

The (human) brain

that created “the wheel, New York, wars and so on” (after
Douglas Adams)

The evolution mechanism

that created the human brain (after Darwin et al.)
 Building problem solvers by looking at and
mimicking:

• brains  neurocomputing

• evolution  evolutionary computing

 Table of Contents
• Taxonomy and History
• The Metaphor
• The Evolutionary Mechanism
• Domains of Application
• Performance
• Sources of Information
 Taxonomy

COMPUTATIONAL
INTELLIGENCE
or
SOFT COMPUTING

Neural Evolutionary Fuzzy

Networks Algorithms Systems

Evolutionary Evolution Genetic Genetic

Programming Strategies Algorithms Programming
 History
• L. Fogel 1962 (San Diego, CA):
Evolutionary Programming
• J. Holland 1962 (Ann Arbor, MI):
Genetic Algorithms
• I. Rechenberg & H.-P. Schwefel 1965
(Berlin, Germany): Evolution Strategies
• J. Koza 1989 (Palo Alto, CA):
Genetic Programming
 Evolution
• Evolution is the change in the inherited traits of a
population from one generation to the next.

• Natural selection leading to better and better species

 Evolution – Fundamental Laws
• Survival of the fittest.
• Change in species is due to change in genes over
reproduction or/and due to mutation.

An Example showing the concept of survival of the fittest and reproduction over
generations.
 Evolutionary Computation

• Evolutionary Computation (EC) refers to computer-based problem solving

systems that use computational models of evolutionary process.

• Terminology:
– Chromosome – It is an individual representing a candidate solution of the
optimization problem.

– Population – A set of chromosomes.

– gene – It is the fundamental building block of the chromosome, each gene in a

chromosome represents each variable to be optimized. It is the smallest unit of
information.

• Objective: To find a best possible chromosome to a given optimization

problem.
The Metaphor
 The Ingredients
t reproduction t+1

selection

mutation
recombination
 The Evolution Mechanism
• Increasing diversity by • Decreasing diversity by
genetic operators selection
– mutation – of parents
– recombination – of survivors
The Evolutionary Cycle
Selection
Parents

Recombination

Population
Mutation

Replacement
Offspring
 What is Evolutionary Computation?
 Simulation of Darwinian Evolution on a
Computer
A set of
possible Computer How good is
solutions Model each solution?

Next
generation
Survival
of the
Mutation
fittest
Duplicate, Keep a set of
Mutate & the best
Crossover solutions
 Domains of Application
• Numerical, Combinatorial Optimisation
• System Modeling and Identification
• Planning and Control
• Engineering Design
• Data Mining
• Machine Learning
• Artificial Life
 Performance
• Acceptable performance at acceptable
costs on a wide range of problems
• Intrinsic parallelism (robustness, fault
tolerance)
• Superior to other techniques on
complex problems with
– lots of data, many free parameters
– complex relationships between
parameters
– many (local) optima
 Advantages
• No presumptions w.r.t. problem space
• Widely applicable
• Low development & application costs
• Easy to incorporate other methods
• Solutions are interpretable (unlike NN)
• Can be run interactively, accommodate
user proposed solutions
• Provide many alternative solutions
 Disadvantages
• No guarantee for optimal solution within
finite time
• Weak theoretical basis
• May need parameter tuning
• Often computationally expensive, i.e. slow
 Summary
EVOLUTIONARY COMPUTATION:
• is based on biological metaphors
• has great practical potentials
• is getting popular in many fields
• yields powerful, diverse applications
• gives high performance against low
costs
• AND IT’S FUN !