ARTIFICIAL NEURAL

NETWORKS
Grupo Lambda SAC

Mg. William Snchez Tapia

Applied Econometrics
December 2013
1

What is Intelligence?
Intelligence is the faculty of understanding

Intelligence is not to make no mistakes but

quickly to understand how to make them good

(German Poet)

What Is Artificial Intelligence?

Artificial Intelligence (AI) is usually

defined as the science of making
computers do things that require
intelligence when done by humans.

A.I is the study of ideas that enable

computers to be intelligent

How Does AI Works?

Artificial intelligence works with the help of

Artificial Neurons (Artificial Neural

Network)
And

Scientific theorems(If-Then Statements,

Logics)

What is Neural Networking?

Artificial neural networks are composed of interconnecting
artificial neurons (programming constructs that mimic the
properties of biological neurons).

Artificial Neural Network

Biological Motivation
Human brain is a densely interconnected network of
approximately 1011 neurons, each connected to, on
average, 104 others.
Neuron activity is excited or inhibited through
connections to other neurons.
.

Biological Basis of ANNs

Biological Basis of ANNs

Animals exhibit intelligence.

Biological neural networks..

Human beings can benefit from simulation of biological

neural networks on computers. These are Artificial Neural
Networks (ANN)

Artificial Neural Networks (ANN) are.

ANNs represent an attempt to simulate

8

ANNs have many names: connectionist

systems, neural nets, neurocomputers, parallel
distributed processing systems, machine
learning algorithms, etc
Each neuron is linked to its neighbors with
varying coefficients of connectivity that represent
the strengths of these connections
Learning..

Based Learning Examples

Based Learning
Examples

test

Reinforcement Learning

Reinforcement Learning

Dendrites: Accepts Inputs

Soma: Processes the Inputs
Axon: Turns the processed inputs into outputs

Synapses: The electrochemical contact between

neurons

13

From Biological Neuron to

Artificial Neuron

Dendrite

Cell Body

Axon

14

From Biology to
Artificial Neural Networks

15

Properties of Artificial Neural

Networks

A network of artificial
neurons

Characteristics
t
t
t
t
t

Nonlinear I/O mapping

Adaptivity
Generalization ability
Fault-tolerance
(graceful degradation)
Biological analogy

16

Biological neural network

Synapse
Axon

Soma

Synapse

Dendrites
Axon

Soma

Dendrites
Synapse

17

Input Signals

Output Signals

Architecture of a typical artificial neural network

Middle Layer
Input Layer

Output Layer

18

Analogy between biological and artificial

neural networks
Biological Neural Network
Soma
Dendrite
Axon
Synapse

Artificial Neural Network

Neuron
Input
Output
Weight

19

The neuron as a simple computing element

Diagram of a neuron
Input Signals

Weights

Output Signals

x1

w1
x2

w2
Neuron
wn

xn

Y
Y

20

 The neuron computes the weighted sum of the input

signals and compares the result with a threshold value,
q. If the net input is less than the threshold, the neuron
output is 1. But if the net input is greater than or equal to
the threshold, the neuron becomes activated and its
output attains a value +1.

The neuron uses the following transfer or activation

function:

X xi wi
i 1

1, if X q
Y
1, if X q

This type of activation function is called a sign function.

21

Activation functions of a neuron

Step function

Sign function Sigmoid function Linear function

+1

+1

-1

-1

0
-1

1, if X 0 sign 1, if X 0 sigmoid
step
Y

Y
Y

0, if X 0

1, if X 0

-1

1
1 e X

Y linear X

22

Can a single neuron learn a task?

In 1958, Frank Rosenblatt introduced a training algorithm
that provided the first procedure for training a simple
ANN: a perceptron.
The perceptron is the simplest form of a neural network.
It consists of a single neuron with adjustable synaptic
weights and a hard limiter.

23

Single-layer two-input perceptron

Inputs
x1
w1

Linear
Combiner

Hard
Limiter

Output
Y

w2

x2

q
Threshold
24

Multilayer neural networks

A multilayer perceptron is a feedforward neural network
with one or more hidden layers.
The network consists of an input layer of source neurons,
at least one middle or hidden layer of computational
neurons, and an output layer of computational neurons.
The input signals are propagated in a forward direction on
a layer-by-layer basis.

25

Input
layer

First
hidden
layer

Second
hidden
layer

Output Sign
als

Input Signal
s

Multilayer perceptron with two hidden layers

Output
layer

26

What does the middle layer hide?

A hidden layer hides its desired output. Neurons in the
hidden layer cannot be observed through the input/output
behaviour of the network. There is no obvious way to
know what the desired output of the hidden layer should
be.
Commercial ANNs incorporate three and sometimes four
layers, including one or two hidden layers. Each layer can
contain from 10 to 1000 neurons. Experimental neural
networks may have five or even six layers, including three
or four hidden layers, and utilise millions of neurons.

27

Back-propagation neural network

Learning in a multilayer network proceeds the same way
as for a perceptron.
A training set of input patterns is presented to the
network.

The network computes its output pattern, and if there is

an error - or in other words a difference between actual
and desired output patterns - the weights are adjusted to
reduce this error.

28

 In a back-propagation neural network, the learning

algorithm has two phases.

First, a training input pattern is presented to the network

input layer. The network propagates the input pattern
from layer to layer until the output pattern is generated by
the output layer.
If this pattern is different from the desired output, an error
is calculated and then propagated backwards through the
network from the output layer to the input layer. The
weights are modified as the error is propagated.

29

Three-layer back-propagation neural network

Input signals
1

x1
x2

xi

y1

y2

yk

yl

wij

wjk

m
n

xn
Input
layer

Hidden
layer

Output
layer

Error signals
30

Benefits of Neural Networks (1/2)

1.
2.
3.
4.

Nonlinearity
Adaptivity
Evidential Response
Contextual Information

31

Benefits of Neural Networks (2/2)

6.
7.
8.
9.

Fault Tolerance
Implementability
Uniformity of Analysis and Design
Neurobiological Analogy

32

Possible Applications(6/6)

7. Military
--Classifying radar signals
--Creating smart weapons
--Doing reconnaissance
--Optimizing use of scarce resources
--Recognizing and tracking targets

Applications
Medicina

1.
2.
3.
4.
5.
6.
7.

Diagnstico de cardiopatas
Deteccin de tumores cancergenos.
Caracterizacin de la dinmica en la variabilidad
cardaca.
Prediccin de enfermedades degenerativas
Prediccin del riesgo de intoxicacin por digoxina.
Prediccin de la respuesta emtica.
Prediccin del nivel de ciclosporina.

Applications
Procesamiento de seales
1. Ecualizacin de canales de
comunicacin
2. Reconocimiento de patrones en
imgenes
3. Reconocimiento de voz
4. Sonar y Radar.
5. Eliminacin activa de ruido.
6. Control

Applications
Reconocimiento facial

Applications
Economa
1. Concesin de crditos.
2. Deteccin de posibles fraudes en tarjetas de crdito.
3. Determinacin de la posibilidad de quiebra de un banco
Prediccin del gasto elctrico de empresas y centrales.
5. Cambio de moneda.
6. Tendencias a corto y medio plazo en bolsas de valores.
7. Prediccin de stocks

Prediccin de Series de Tiempo

Applications
Biologa
1. Aprender ms acerca
del cerebro y otros
sistemas
2. Obtencin de modelos
de la retina
3. Modelamiento de
procesos biolgicos

Applications
Empresas
1. Reconocimiento de
caracteres escritos
2. Identificacin de candidatos
para posiciones especficas
3. Optimizacin de plazas y
horarios en lneas de vuelo
4. Explotacin de bases de
datos

Applications
Finanzas
1. Previsin de la
evolucin de los
precios.
2. Valoracin del
riesgo de los
crditos
3. Interpretacin de
firmas

Applications
Manufactura
1. Robots automatizados y
sistemas de control (visin
artificial y sensores de
presin, temperatura, gas,
etc.)
2. Inspeccin de calidad.

Applications
Robtica

Navegacin Autnoma

ARTIFICIAL NEURAL
NETWORKS
Grupo Lambda SAC

Mg. William Snchez Tapia

Applied Econometrics
December 2013
45