Engineering Systems Design and Analysis

(ENG 504)
504)
Lecture 7
/


Email: amhm@zu.edu.eg / amhm162@gmail.com
Webpage: http://www.staff.zu.edu.eg/amhm/

In the previous lectures

Expert systems
Propositional logic
Predicate logic
Fuzzy logic

Dr. A. Helmi Eng. Sys. Design and Analysis (ENG 504)

Zagazig Univ-Eng. Faculty

2013/2014

Agenda

Expert systems
Artificial Neural Networks

Dr. A. Helmi Eng. Sys. Design and Analysis (ENG 504)

Zagazig Univ-Eng. Faculty

Real & Artificial

Neurons

2013/2014

Real Neurons

Human brain contains a bout 1011 neurons.

Neurons can receive, process and transmit electrochemical signals over neural
pathways that make up the brains communication system.
Structure of a neuron:
Cellular body: called soma
Dendrites: branches conduct information into a cell
Axon: branches conduct information (reaction) out of the cell
Action potential: is an activation of a neuron.
A spike: an emitted signal from a neuron and characterized by frequency,
duration and amplitude.

Dr. A. Helmi Eng. Sys. Design and Analysis (ENG 504)

Zagazig Univ-Eng. Faculty

2013/2014

Real Neurons

A Neuron has many inputs but only one output.

Synapse is the connection between neurons.
The brain consists of different types of neurons with different functionality.
Dr. A. Helmi Eng. Sys. Design and Analysis (ENG 504)

Zagazig Univ-Eng. Faculty

2013/2014

Artificial Neurons
w1

x1
x2

Output

w2

Inputs

x3

z = wi xi ; y = H ( z )

w3
.

xn-1

i =1

wn-1

xn

wn

The McCullogh-Pitts model

Binary inputs.
Binary output.
Fixed activation threshold.
Dr. A. Helmi Eng. Sys. Design and Analysis (ENG 504)

Zagazig Univ-Eng. Faculty

Artificial Neurons

2013/2014

Parameters:
1.

Input connections with weights.

2. Input function: the aggregate net input signal: z=f(x,w)

3. Activation signal: calculates the activation level: a = s(z)
4. Output function: calculates the output signal: y = H(a)

Dr. A. Helmi Eng. Sys. Design and Analysis (ENG 504)

Zagazig Univ-Eng. Faculty

2013/2014

Artificial Neurons

Input and output values can be:

Binary: {0,1}
Bivalent: {-1,1}
Continuous: [0,1]
Discrete numbers in a defined interval.

Dr. A. Helmi Eng. Sys. Design and Analysis (ENG 504)

Zagazig Univ-Eng. Faculty

Artificial Neurons

2013/2014

Most used activation functions, a=s(z):

a=

1
1 + e u

Dr. A. Helmi Eng. Sys. Design and Analysis (ENG 504)

Zagazig Univ-Eng. Faculty

2013/2014

Structure of Neural Networks

Input layer
Hidden layer

Inputs

Output layer
Output

This is the feed-forward style.

There is also the feed-back style.
Dr. A. Helmi Eng. Sys. Design and Analysis (ENG 504)

Zagazig Univ-Eng. Faculty

2013/2014

Neural Networks

10

An artificial neural network is a computational model defined by four parameters:

1. Type of neurons.
2. Connectionist architecture.
3. Learning algorithm.
4. Recall algorithm.
Notice that:
The weight matrix represents the "knowledge", the long-term memory of
the system.
The activation of the neurons represents the current state,
the short-term memory.

Dr. A. Helmi Eng. Sys. Design and Analysis (ENG 504)

Zagazig Univ-Eng. Faculty

2013/2014

Learning of Neural
Networks

Real Neurons

11

The brain's ability to analyze complex problems and to react adequately to

unfamiliar situations is due to its heuristic faculty of taking decisions on the
basis of previously stored knowledge and its ability to adapt to new situations.
The human brain has the ability to learn and to generalize (recall).
The information we accumulate as a result of our learning is stored in the
synapses in the form of concentrated chemical substances.

Dr. A. Helmi Eng. Sys. Design and Analysis (ENG 504)

Zagazig Univ-Eng. Faculty

2013/2014

Real Neurons

12

Learning
is achieved in the brain through the process of chemical change
in synaptic connections.

Recall
activates a collection of neurons in time.

Dr. A. Helmi Eng. Sys. Design and Analysis (ENG 504)

Zagazig Univ-Eng. Faculty

2013/2014

Artificial Neural Networks

13

A neural network has to be trained: we apply a set of input vectors,

then we change the weights until the desired output obtained.
Three categories of learning algorithms:
1. Supervised: Training is performed until the neural network "learns" to
associate each input vector x to its corresponding and desired output
vector y. (with a teacher)
2. Unsupervised: Only input vectors are supplied, then the neural network
learns some internal features of the whole set of all the input vectors.
3. Reinforcement: is a combination of the above two techniques.
is called also reward-penalty learning or learning with a critic (without
teacher).

Dr. A. Helmi Eng. Sys. Design and Analysis (ENG 504)

Zagazig Univ-Eng. Faculty

2013/2014

Artificial Neural Networks

as a problemproblem-solving
paradigm

Solving problems paradigm

14

Tasks to be solved by artificial neural networks:

Controlling the movements of a robot based on self-perception and other
information (e.g., visual information).
Predicting where a moving object goes, when a robot wants to catch it.
Deciding the category of potential food items
(e.g., edible or non-edible) in an artificial world;
Image/speech recognition.
Weather forecasting.

Dr. A. Helmi Eng. Sys. Design and Analysis (ENG 504)

Zagazig Univ-Eng. Faculty

2013/2014

Solving problems paradigm

15

The problem-solving process, when using neural networks, comprises two phases:

Dr. A. Helmi Eng. Sys. Design and Analysis (ENG 504)

Zagazig Univ-Eng. Faculty

Connectionist Expert Systems

Dr. A. Helmi Eng. Sys. Design and Analysis (ENG 504)

2013/2014

16

Zagazig Univ-Eng. Faculty

2013/2014

System Example

Optical Character Recognition (OCR)

17

A sample set of handwritten characters:

Dr. A. Helmi Eng. Sys. Design and Analysis (ENG 504)

Zagazig Univ-Eng. Faculty

2013/2014

OCR

18

Two ANNs schemes for OCR.

Results (around 1996) for isolated
characters:
98% for digits.
96% for uppercase characters.
87% for lowercase characters.

Dr. A. Helmi Eng. Sys. Design and Analysis (ENG 504)

Zagazig Univ-Eng. Faculty

Thanks

2013/2014