Lovely Professional University, Punjab

Course Code ECE427 Course Category Course Title NEURAL NETWORK AND FUZZY LOGIC Courses with conceptual focus Course Planner 15698::Manpreet Kaur Lectures 3.0 Tutorials Practicals Credits 1.0 0.0 4.0

TextBooks Sr No T-1 T-2 Title Fuzzy Logic With Engineering Applications Neural Network Design Reference Books Sr No R-1 R-2 R-3 Title Neural Networks and Learning Machines Author Simon Haykin Edition 3rd 2nd 2nd Year 2009 2006 2009 Publisher Name Pearson Education Tata McGraw Hill Pearson Education Author Timothy J. Ross Hagan,Demuth Edition 3rd 3rd Year 2010 2010 Publisher Name Mc graw Hill Cengage Learning

Introduction to Neural Networks using S.N.Sivanandam, MATLAB 6.0 S.Sumathi,S. N. Deepa Neural Networks James A Freeman and Davis Skapura

Relevant Websites Sr No RW-1 RW-2 RW-3 RW-4 RW-5 RW-6 RW-7 RW-8 RW-9 RW-10 RW-11 (Web address) (only if relevant to the course) Salient Features http://ocw.mit.edu/courses/brain-and-cognitive-sciences/9- 641j-introduction-to-neural- Consist of complete lecture notes, assignments from MIT, USA networks-spring-2005/index.htm http://www.cs.umbc.edu/~ypeng/F04NN/NN-lecture-notes.htm http://staff.aist.go.jp/utsugi-a/Lab/Links.html http://staff.aist.go.jp/utsugi-a/research.html http://neuron.eng.wayne.edu/software.html http://aiworkshop.tamucc.edu/index_files/FuzzyLogic.pdf http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol3/cs11/test. html#question10 http://tralvex.com/pub/nap/ http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11/rep ort.html http://www.dementia.org/~julied/logic/applications.html http://www.dementia.org/~julied/logic/sets.html Lecture notes of complete course Applets for Neural Networks and Artificial Life Statistical Study of Artificial Neural Networks Java Demonstrations of Neural Net Concepts Demonstration of Matlab Fuzzy Logic Packages Quiz Neural Network Applications NEURAL NETWORKS by Christos Stergiou and Dimitrios Siganos applications for fuzzy logic Introduction to Fuzzy Sets

LTP week distribution: (LTP Weeks) Weeks before MTE Weeks After MTE Spill Over 7 6 2

Detailed Plan For Lectures

Week 1

Lecture 1

Introduction to Neural Networks (Introduction)

T-2:1-1 1-2

Introduction, Objectives, Introduce the concepts Slide Show by teacher History of Neural of neural network and Networks its objective,History of some of the main neural network contributors Introduction, Objectives, Introduce the concepts Slide Show by teacher History of Neural of neural network and Networks its objective,History of some of the main neural network contributors Introduction, Objectives, Introduce the concepts Slide Show by teacher History of Neural of neural network and Networks its objective,History of some of the main neural network contributors Appliations and Schematic drawing of biological neurons How neural networks Slide Show by teacher are applied in different fields

Introduction to Neural Networks (Objective)

T-2:1-1 1-2

Introduction to Neural Networks (History of Neural Networks)

T-2:1-1 1-2

Lecture 2

Introduction to Neural Networks (Applications and Biological Inspiration) Introduction to Neural Networks (Neuron Model)

T-2:1-5 1-8

Lecture 3

T-2:2-2 2-3 2-7

SingleInput Neuron, To introduce our Slide Show by teacher Transfer simplified mathematical Functions,MultipleInput model of neuron Neuron A Layer of Neurons, Multiple Layers of Neurons,Recurrent Networks A Layer of Neurons, Multiple Layers of Neurons,Recurrent Networks To learn how artifical neurons can be interconnect to form variety of network architectures To learn how artifical neurons can be interconnect to form variety of network architectures Slide Show by teacher

Week 2

Lecture 4

Introduction to Neural Networks (Network Architectures)

T-2:2-7 2-9 2-10 213

Lecture 5

Introduction to Neural Networks (Network Architectures)

T-2:2-7 2-9 2-10 213

Slide Show by teacher

Week 2

Lecture 6

Single & Multilayer Perceptrons (Preceptron)

T-2:3-3 3-4 3-5

TwoInput Case Pattern Recognition

To study how simple pattern recognition problem can be solved using neural network architecture

Slide Show by teacher

Week 3

Lecture 7

Single & Multilayer Perceptrons (Hopfield Network)

T-2:3-12

Feedforward layer recurrent layer

To study how simple Slide Show by teacher pattern recognition problem can be solved using hopfield network To study how simple Slide Show by teacher pattern recognition problem can be solved using hamming network To study training Slide Show by teacher preceptron networks to solve classification problems To study training Slide Show by teacher preceptron networks to solve classification problems To study training Slide Show by teacher preceptron networks to solve classification problems To study training Slide Show by teacher preceptron networks to solve classification problems To study How hebb rule Slide Show by teacher can be used to train neural networks for pattern recognition To study How hebb rule Slide Show by teacher can be used to train neural networks for pattern recognition

Single & Multilayer Perceptrons (Hamming Network)

T-2:3-8 3-9

Feedforward layer recurrent layer

Lecture 8

Single & Multilayer Perceptrons (Perceptron Architecture)

T-2:4-2 4-3 4-5 4-8

Learning Rules,SingleNeuron Perceptron, MultipleNeuron Perceptron Learning Rules,SingleNeuron Perceptron, MultipleNeuron Perceptron Constructing Learning Rules, Unified Learning Rule,Training MultipleNeuron Perceptrons Constructing Learning Rules, Unified Learning Rule,Training MultipleNeuron Perceptrons Linear associator,Associative memory Linear associator,Associative memory

Lecture 9

Single & Multilayer Perceptrons (Perceptron Architecture)

T-2:4-2 4-3 4-5 4-8

Week 4

Lecture 10

Single & Multilayer Perceptrons (Preceptron learning rule)

T-2:4-8 4-9 4-10 412 4-13

Lecture 11

Single & Multilayer Perceptrons (Preceptron learning rule)

T-2:4-8 4-9 4-10 412 4-13

Lecture 12

Learning of Neural Networks (Linear Associator)

T-2:7-3 7-4 7-5

Week 5

Lecture 13

Learning of Neural Networks (Linear Associator)

T-2:7-3 7-4 7-5

Learning of Neural Networks(The Hebb Rule)

T-2:7-3 7-4 7-5

The Hebb To study How hebb rule Slide Show by teacher rule,Performance analtsis can be used to train neural networks for pattern recognition pseudoinverse rule equations To study several procedures to reduce errors due to hebb rule Slide Show by teacher

Lecture 14

Learning of Neural Networks (Pseudoinverse Rule)

T-2:7-7 7-10 7-12

Week 5

Lecture 15

Backpropagation(Multilayer Perceptrons)

T-2:11-2 11-3 11-4

pattern classification,function approximation

To study the Slide Show by teacher capabilities of the multilayer preceptron for pattern classification Slide Show by teacher

Backpropagation(The Backpropagation Algorithm) Backpropagation(Drawbacks of Backpropagation)

T-2:11-7 11-8 11-9 11-11 T-2:12-3 12-7

performance index,chain to demonstrate how to rule,Backpropagating the use the chain rule Sensitivities Performance Surface Example, Convergence To study several variations of backpropagation to make the algorithm more practical To study several variations of backpropagation to make the algorithm more practical To study several variations of backpropagation to make the algorithm more practical

Slide Show by teacher

Week 6

Lecture 16

Backpropagation(Drawbacks of Backpropagation)

T-2:12-3 12-7

Performance Surface Example, Convergence

Slide Show by teacher

Backpropagation(Heuristic Modifications of Backpropagation)

T-2:12-9 12-12

Momentum,Variable Learning Rate

Slide Show by teacher

Lecture 17 Lecture 18 Backpropagation(LevenbergMarquardt Algorithm) T-2:12-19

Homework,Term Paper,Test 1 algorithm To study several variations of backpropagation to make the algorithm more practical To study associative learning rules to classify patterns To study associative learning rules to classify patterns To study associative learning rules to classify patterns To study associative learning rules to classify patterns To study associative learning rules to classify patterns Slide Show by teacher

Week 7

Lecture 19

Competitive Networks(Hamming Network) Competitive Networks(Hamming Network, Competitive Layer)

T-2:14-3 14-4

Layer 1, Layer 2

Slide Show by teacher

T-2:14-3 14-4

Layer 1, Layer 2

Slide Show by teacher

Lecture 20

Competitive Networks (Competitive Layer)

T-2:14-4 14-7 14-9

Competitive Learning, Problems with Competitive Layers Improving feature maps

Slide Show by teacher

Competitive Networks(SelfOrganizing Feature Maps) Lecture 21 Competitive Networks(Learning Vector Quantization)

T-2:14-12 14-15

Slide Show by teacher

T-2:14-16 14-18 1421

LVQ learning, Improving LVQ networks

Slide Show by teacher

MID-TERM

Week 8

Lecture 22

Grossberg Networks(Biological Motivation) Grossberg Networks(Basic Nonlinear Model)

T-2:15-3 15 4 15-8

Biological Motivation, Illusions, Vision Normalization to discuss basic nonlinear model

to study the grossberg network in biological context To study self organizing continuous time competitive networks To study self organizing continuous time competitive networks To demonstrate grossberg network network in its simplest form

Slide Show by teacher

Lecture 23

T-2:15-9

Slide Show by teacher

Grossberg Networks(Two-Layer Competitive Network)

T-2:15-12 15-20 1522

Choice of Transfer Function,Learning Law

Slide Show by teacher

Lecture 24

Grossberg Networks(Relation to Kohonen Law.)

T-2:15-24

Kohonen law

Slide Show by teacher

Week 9

Lecture 25

Hopfield Networks(Lyapunov Function)

T-2:18-5 18-7 18-11

Invariant Sets,Hopfield Attractors

to study how lyapunov Slide Show by teacher stability theory can be used to analyze network operation to study how lyapunov Slide Show by teacher stability theory can be used to analyze network operation Slide Show by teacher

Hopfield Networks(Hopfield Model)

T-2:18-3

Hopfield network

Lecture 26

Hopfield Networks(Effect of gain) Hopfield Networks(Hopfield Design)

T-2:18-12 T-2:18-16 18-18 1822 18-24

Inverse tangent amplifier To study the effect of characteristics gain ContentAddressable Memory,Hebb Rule, Lyapunov Surface Invariant Sets,Hopfield Attractors

to study how a hopfield Slide Show by teacher network can be designed to work as an associative memory to study how lyapunov Slide Show by teacher stability theory can be used to analyze network operation to study how a hopfield Slide Show by teacher network can be designed to work as an associative memory to study the basics of classical sets to study basic of fuzzy relations Slide Show by teacher Slide Show by teacher

Hopfield Networks(Lyapunov Function)

T-2:18-5 18-7 18-11

Lecture 27

Hopfield Networks(Hopfield Design)

T-2:18-16 18-18 1822 18-24

ContentAddressable Memory,Hebb Rule, Lyapunov Surface properties, Operations Crisp and fuzzy fuzzy relations,membership fuzzy relations,membership features of membership functions

Week 10

Lecture 28 Lecture 29 Lecture 30

Introduction to Fuzzy Theory (Classical & Fuzzy Sets) Introduction to Fuzzy Theory (Relations) Introduction to Fuzzy Theory (Fuzzy sets) Introduction to Fuzzy Theory (Fuzzy sets) Introduction to Fuzzy Theory (Membership Functions)

T-1:Pg 24- 35 T-1:Pg No. 53-59 T-1:Pg. no. 90-94 T-1:Pg. no. 90-94 T-1:90-94

to study different fuzzy Slide Show by teacher sets in fuzzy theory to study different fuzzy Slide Show by teacher sets in fuzzy theory basics of membership function Slide Show by teacher

Week 11

Lecture 31 Lecture 32

Week 11

Lecture 32 Lecture 33

Introduction to Fuzzy Theory (Cardinalities) Fuzzy Logic System Components (Membership value assignment) Fuzzy Logic System Components (Fuzzification)

T-1:Pg. no.90-94 T-1:Pg. No. 94-179 T-1:94-179 T-1:94-179

membership functions intuition,inference fuzzification method rule base systems

basics of membership function fuzzification theory basics fuzzification theory basics

Slide Show by teacher Slide Show by teacher Slide Show by teacher

Week 12

Lecture 34

Fuzzy Logic System Components (Development of rule base and decision making system) Fuzzy Logic System Components (Defuzzification to crisp sets) Fuzzy Logic System Components (Defuzzification methods)

to study algorithms for Slide Show by teacher development of membership functions to study basics of crisp Slide Show by teacher functions to study basics of crisp Slide Show by teacher functions to study basics of crisp Slide Show by teacher functions practical applications of Slide Show by teacher neural network practical applications of Slide Show by teacher neural network practical applications of Slide Show by teacher fuzzy logic practical applications of Slide Show by teacher fuzzy logic

T-1:Pg no. 96 T-1:94-179 T-1:Pg no. 96 T-1:Pg. No. 517 RW-11

crisp sets different methods crisp sets Process identification, control, fault diagnosis Process identification, control, fault diagnosis Fuzzy logic control and Fuzzy classification Fuzzy logic control and Fuzzy classification

Lecture 35 Lecture 36

Fuzzy Logic System Components (Defuzzification to crisp sets) Applications of Neural Networks & Fuzzy Logic(Applications of neural network) Applications of Neural Networks & Fuzzy Logic(Applications of neural network) Applications of Neural Networks & Fuzzy Logic(Applications of Fuzzy logic) Applications of Neural Networks & Fuzzy Logic(Applications of Fuzzy logic)

Week 13

Lecture 37

T-1:Pg. No. 517

RW-11

Lecture 38

T-1:Pg.517

RW-8

Lecture 39

T-1:Pg.517

RW-8

SPILL OVER
Week 14 Lecture 40 Lecture 41 T-2:1 T-1:2 rivision rivision rivision rivision

Scheme for CA:

Details of Academic Task(s)

AT No.

Objective

Topic of the Academic Task

Nature of Academic Task (group/individuals/field work Individual

Evaluation Mode

Allottment / submission Week 3/4

Homework 1

To improve the academic standard of the student To improve the academic standard of the student

Neural network introduction, Models, single and multi layer perceptron, learning rules

checking

Test 1

Supervised Hebbian Learning: Linear Associator,The Hebb Individual Rule,Performance Analysis, Pseudoinverse Rule, Application, Variations of Hebbian Learning,Multilayer Perceptrons, Pattern Classification, Function Approximation, The Backpropagation Algorithm, Performance Index, Chain Rule, Backpropagating the Sensitivities, Drawbacks of Backpropagation, Performance Surface Example, Convergence, Heuristic Modifications of Backpropagation, Momentum,Variable Learning Rate, Levenberg-Marquardt Algorithm, Competitive networks: Hamming Network, Competitive Layer, Competitive Learning, Problems with Competitive Layers, Self-Organizing Feature Maps Improving Feature Maps term paper should be given in group Group

checking

5/6

Term Paper 1

To improve the academic standard of the student

presentation

3 / 12

List of suggested topics for term paper[at least 15] (Student to spend about 15 hrs on any one specified term paper) Sr. No. Topic 1 Development and limitations of expert systems and neural networks. 2 Application of neural network in Medicine. 3 Neural Network in Business 4 Neural networks versus conventional computers - a comparison 5 Human and Artificial Neurones - investigating the similarities 6 Architecture of neural networks 7 The Learning Process: The Back-Propagation Algorithm 8 Advances in Artificial Intelligence 9 Fuzzy Systems in Computer Science 10 Foundations of Neuro-Fuzzy Systems. 11 Information processing in the nervous system. 12 Applications for Fuzzy Logic 13 Traffic support systems using Fuzzy Logic 14 Development of fuzzy logic

15 Logic and Perception

Plan for Tutorial: (Please do not use these time slots for syllabus coverage)
Tutorial No. Lecture Topic Type of pedagogical tool(s) planned (case analysis,problem solving test,role play,business game etc)

Tutorial 1 Tutorial 2 Tutorial 3 Tutorial 4 Tutorial 5 Tutorial 6 Tutorial 7

Neural Networks and its applications Neuron Model Network Architectures, Recurrent Networks Perceptron Perceptron Learning Rule Supervised Hebbian Learning The Backpropagation Algorithm

Case analysis Problem solving Problem solving Problem solving Problem solving Problem solving Problem solving

After Mid-Term
Tutorial 8 Tutorial 9 Tutorial 10 Tutorial 11 Tutorial 12 Tutorial 13 convergence examples Drawbacks of Backpropagation cardinalities membership functions Defuzzification to crisp sets Defuzzification methods Problem solving Problem solving Problem solving Problem solving Problem solving Problem solving