ANN Chapter3 4

Gii thiu
Gi thiu v cc phng php hun luyn Tm hiu mt s gii thut thng dng hun luyn ANN. Phn ny tp trung vo gii thut Gradient descent v cc gii thut ci tin ca n Hm mc tiu Mt li v cc im cc tiu cc b Mt s v d v phng php hun luyn mng bng MATLAB Qui trnh thit k mt ANN Cc k thut ph tr Hin tng qu khp ca ANN
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Cc phng php hun luyn

Hun luyn mng l qu trnh thay i cc trng s kt ni v cc ngng ca n-ron, da trn cc mu d liu hc, sao cho tha mn mt s iu kin nht nh. C 3 phng php hc:
Hc gim c st (supervised learning) Hc khng gim st (unsupervised learning) Hc tng cng (reinforcement learning). Sinh vin tham kho ti liu [1].
Gio trnh ny ch tp trung vo phng php hc c gim st. Hai phng php cn li, sinh vin s c hc trong chng trnh Cao hc.
Gii thut hun luyn ANN (1)

Trong phn ny chng ta tm hiu v gii thut truyn ngc (backpropagation) v cc gii thut ci tin ca n, p dng cho phng php hc c gim st. Gii thut truyn ngc cp nht cc trng s theo nguyn tc: wij(k+1) = wij(k) + g(k) trong :
wij(k) l trng s ca kt ni t n-ron j n n-ron i, thi im hin ti l tc hc (learning rate, 0< 1) g(k) l gradient hin ti
C nhiu phng php xc nh gradient g(k), dn ti c nhiu gii thut truyn ngc ci tin.
Gii thut hun luyn ANN (2)

cp nht cc trng s cho mi chu k hun luyn, gii thut truyn ngc cn 2 thao tc:
Thao tc truyn thun (forward pass phase): p vect d liu vo trong tp d liu hc cho ANN v tnh ton cc ng ra ca n. Thao tc truyn ngc (backward pass phase): Xc nh sai bit (li) gia ng ra thc t ca ANN v gi tr ng ra mong mun trong tp d liu hc. Sau , truyn ngc li ny t ng ra v ng vo ca ANN v tnh ton cc gi tr mi ca cc trng s, da trn gi tr li ny.
p1(k) p2(k)
wi1(k) wi2(k) wij(k)
ai(k)
+ -
ti(k)
ei(k)
pj(k)
wRj(k)
pR(k)
Minh ha phng php iu chnh trng s nron th j ti thi im k

8
Gii thut gradient descent (1)

Xt mt MLP 2 lp:
p1 p2 a21 a2j a2m pn w1ij l trng s lp n, t j n i
Lp n Ng ra n-ron n l a1i
pi
w2ij l trng s lp ra, t j n i

Thao tc truyn thun
Tnh ng ra lp n (hidden layer): n1i (k) = j w1ij (k) pj (k) ti thi im k a1i(k) = f1( n1i(k) ) vi f1 l hm kch truyn ca cc n-ron trn lp n. Ng ra ca lp n l ng vo ca cc n-ron trn lp ra. Tnh ng ra ANN (output layer): n2i (k) = j w2ij (k) a1j (k) ti thi im k a2i(k) = f2( n2i(k) ) vi f2 l hm truyn ca cc n-ron trn lp ra
10

Thao tc truyn ngc
2 1 Tnh tng bnh phng ca li: E (k ) = ti (k ) a 2 i (k ) 2 i vi t(k) l ng ra mong mun ti k Tnh sai s cc n-ron ng ra: E (k ) i (k ) = 2 = [ti (k ) ai (k )] f 2 ' n 2 i (k ) n i (k ) Tnh sai s cc n-ron n: E (k ) i (k ) = 1 = f 1 ' n1i (k ) j (k ) w ji n i (k ) j Cp nht trng s lp n: 1 w ij (k + 1) = wij (k ) + i (k ) p j (k )
lp ra: w 2 ij ( k + 1) = w 2 ij ( k ) + i ( k ) a1 j ( k )
Sinh vin tham kho ti liu [3], trang 161-175.

11
Minh ha gii thut hun luyn (1)

Xt mt ANN nh hnh v, vi cc n-ron tuyn tnh.
p1
w111= -1 w121= 0
w211= 1 w221= -1 w212= 0
a21
p2
w112= 0 w1 b11= 1 b12= 1

22=
1 b21= 1
w222= 1
a22
b22= 1
Minh ha gii thut truyn ngc nh sau

12

n gin, ta cho c cc ngng bng 1, v khng v ra y
p1=0
w111= -1 w121= 0 w112= 0 w122= 1
w211= 1 w221= -1 w212= 0 w222= 1
a21
p2=1
a22
Gi s ta c ng vo p=[0 1] v ng ra mong mun t=[1 0] Ta s xem xt tng bc qu trnh cp nht trng s ca mng vi tc hc =0.1
13

Thao tc truyn thun. Tnh ng ra lp n: a11 = 1
w111= -1 w121= 0 w112= 0 w122= 1 w211= 1 w221= -1 w212= 0 w222= 1
p1=0
a21
p2=1
a22
a12 = 2 a11 = f1(n11) = n11 =(-1*0 + 0*1) +1 = 1 a12 = f1(n12)=n12 = (0*0 + 1*1) +1 = 2
14

Tnh ng ra ca mng (lp ra): a11 = 1
w111= -1 w121= 0 w112= 0 w122= 1 w211= 1 w221= -1 w212= 0 w222= 1
p1=0
a21=2
p2=1
a22=2
a12 = 2 a21 = f2(n21) = n21 =(1*1 + 0*2) +1 = 2 a12 = f2(n22)=n22 = (-1*1 + 1*2) +1 = 2 Ng ra a2 khc bit nhiu vi ng ra mong mun t=[1 0]
15

Thao tc truyn ngc w 2 ij (k + 1) = w 2 ij ( k ) + (k )a1 j ( k ) i p1=0
w1 w112= 0 w122= 1
a11 = 1
w111= -1
21=
w211= 1 w221= -1 w212= 0 w222= 1
1= -1
p2=1
2= -2
a12 = 2 Vi ng ra mong mun t =[1, 0], Ta c cc error ng ra: 1 = (t1 - a21 )= 1 2 = -1 2 = (t2 - a22 )= 0 2 = -2
16

Tnh cc gradient lp ra a11 = 1
w111= -1 w121= 0 w112= 0 w122= 1 w211= 1 w221= -1 w212= 0 w222= 1
p1=0
1a11= -1 1a12= -2 2a11= -2 2a12= -4
p2=1
a1
=2
17

Cp nht trng s lp ra w 2 ij (k + 1) = w 2 ij ( k ) + (k )a1 j ( k ) i p1=0 a11 = 1
w111= -1 w121= 0 w112= 0 w122= 1
w211= 0.9 w221= -1.2 w212= -0.2 w222= 0.6
p2=1
a12 = 2 w211= 1 + 0.1*(-1) = 0.9 w221= -1 + 0.1*(-2) = -1.2 w212= 0 + 0.1*(-2) = -0.2 w222= 1 + 0.1*(-4) = 0.6
18

Tip tc truyn ngc
p1=0
w111= -1 w121= 0 w112= 0 w122= 1
1 w11= -1 2 w21= 2 1 w12= 0 2 w22= -2
1= -1
p2=1
2= -2
S dng li cc trng s trc khi cp nht cho lp ra, tnh gradient lp n i (k ) = f 1 ' n1i (k ) j (k ) w ji
j
19

Tnh cc error trn lp n 1= 1
p1=0
w1 w112= 0
w111= -1
21=
1= -1
p2=1
w122= 1
2= -2 2 = -2
1 = 1 w11 + 2 w21 = -1 + 2 = 1 2 = 1 w12 + 2 w22 = 0 - 2 = -2
20

Tnh gradient lp n 1 p1 = 0 1 p2 = 1 2 p1 = 0 2 p2 = -2
p1=0
w1
11=
-1
w121= 0 w112= 0 w122= 1
1= -1
p2=1
2= -2
21

Cp nht trng s lp n w1ij (k + 1) = wij (t ) + i (k ) p j (k ) 1 p1 = 0 1 p2 = 1 2 p1 = 0 2 p2 = -2 w111= -1 + 0.1*0 = -1 w121= 0 + 0.1*0 = 0 w112= 0 + 0.1*1 = 0.1 w122= 1 + 0.1*(-2) = 0.8
p1=0
w111= -1 w121= 0
p2=1
w112= 0.1 w122= 0.8
22

Gi tr trng s mi:
w111= -1 w121= 0 w112= 0.1 w122= 0.8
w211= 0.9 w221= -1.2 w212= -0.2 w222= 0.6
Qu trnh cp nht cc gi tr ngng hon ton tng t.
23

Truyn thun 1 ln na xc nh ng ra ca mng vi gi tr trng s mi p1=0
w111= -1 w121= 0
a11 = 1.2
w211= 0.9 w221= -1.2 w212= -0.2 w222= 0.6
p2=1
w112= 0.1 w122= 0.8
a12 = 1.6
24

a11 = 1.2
w2
11=
p1=0
w111= -1 w121= 0
0.9
a21 = 1.66
w221= -1.2 w212= -0.2 w222= 0.6
p2=1
w112= 0.1 w122= 0.8
= 1.6 Gi tr ng ra by gi l a2 = [1.66 0.32]gn vi gi tr mong mun t=[1 0] hn. Bi tp: T kt qu ny, sinh vin hy thc hin thao tc truyn ngc v cp nht trng s ANN 1 ln na. a1
2
a22 = 0.32
25
Hun luyn n khi no?

s thi k (epochs) n nh trc Hm mc tiu t gi tr mong mun Hm mc tiu phn k
26
Hm mc tiu MSE (1)

Mean Square Error MSE l li bnh phng trung bnh, c xc nh trong qu trnh hun luyn mng. MSE c xem nh l mt trong nhng tiu chun nh gi s thnh cng ca qu trnh hun luyn. MSE cng nh, chnh xc ca ANN cng cao. nh ngha MSE:
Gi s ta c tp mu hc: {p1,t1}, {p2,t2}, , {pN,tN}, vi p=[p1, p2, pN] l vect d liu ng vo, t= [t1, t2, , tN] l vect d liu ng ra mong mun. Gi a=[a1, a2, , aN] l vect d liu ra thc t thu c khi a vect d liu vo p qua mng. MSE:
1 MSE = N
N i =1
ti ai
c gi l hm mc tiu
27
Hm mc tiu MSE (2)

V d 1: Cho ANN 2 lp tuyn tnh nh hnh v. 1 p=[1 2 3; 0 1 1] l cc vect ng vo. p1 t=[2 1 2] l vect ng ra mong mun. 0 Tnh MSE. 1 p2 Gii: 2 Ng ra lp n: a11=f(n11)=n11=[1.5 3.5 4.5] a12=f(n12)=n12=[0.2 2.2 2.2] Lp ra: a2=f(n2)=n2=[3.2 9.2 11.2]
a1 1 .5 2
1
a2 0 1
1 a1
2
.2
1
1 3 1 2 2 2 2 MSE = (ti ai ) = (2 3.2 ) + (1 9.2 ) + (2 11.2) 3 i =1 3

MSE = 51.1067
28
Hm mc tiu MSE (3)

M phng:
net=newff([-5 5; -5 5], [2 1], {'purelin', 'purelin'}); net.IW{1,1}=[1 1; 0 2]; % gn input weights % gn layer weights % gn ngng n-ron lp n % gn ngng n-ron lp ra % vect d liu vo % ng ra mong mun % ng ra thc t ca ANN % tnh mse
net.LW{2,1}=[2 1]; net.b{1}=[.5; .2]; net.b{2}=0; p=[1 2 3; 0 1 1]; t=[2 1 2]; a=sim(net,p) mse(t-a) ans = 51.1067
29
Hm mc tiu MSE (4)

MSE c xc nh sau mi chu k hun luyn mng (epoch) v c xem nh 1 mc tiu cn t n. Qu trnh hun luyn kt thc (t kt qu tt) khi MSE nh.
30
V d v GT Gradient descent (1)

Bi ton: Xy dng mt ANN nhn dng m hnh vo ra ca h thng iu khin tc motor DC sau:
31

Nguyn tc:
ng vo V Motor DC ng ra
M hnh ANN
gradient descent
e=-
(k)=fANN[V(k), V(K-1), V(K-2), (k-1), (k-2)]
Cc bc cn thit:
Thu thp v x l d liu vo ra ca i tng Chn la cu trc v xy dng ANN Hun luyn ANN bng gii thut gradient descent Kim tra chnh xc ca m hnh bng cc tn hiu khc
32

Thc hin:
M hnh Simulink thu thp d liu: data_Dcmotor.mdl
V(k), V(k-1), V(k-2)
(k), (k-1), (k-2)
Chun b d liu hun luyn: ANN_Dcmotor.m load data_DCmotor; % nap tap du lieu hoc P=[datain'; dataout(:, 2:3)']; % [V(k), V(k-1), V(k-2), (k-1), (k-2)] T=dataout(:,1)'; % theta(k)
33

Thc hin:
To ANN v hun luyn: ANN_Dcmotor.m
>> net=train(net, Ptrain,Ttrain, [], [], VV,TV);
Nhn xt:
Tc hi t ca gii thut Gradient descent qu chm. Sau 5000 Epochs, MSE ch t 6.10-4. Kt qu kim tra cho thy li ln. Cn gii thut ci tin.
34

Thc hin:
M hnh kim tra chnh xc ANN: Test_Dcmotor.mdl
(k)=fANN[V(k), V(K-1), V(K-2), (k-1), (k-2)]

35

Kt qu:
0.4 0.3 0.2 0.1 DC_motor output Model output Testing result of DC_motor model
(rad)
0 -0.1 -0.2 -0.3 -0.4
10
15
20
25 Time(s)
30
35
40
45
50
36
Cc tiu cc b
nh hng ca tc hc
Qu trnh hun luyn mng, gii thut cn vt qua cc im cc tiu cc b (v d: c th thay i h s momentum), t c im global minimum.
Local minimum
global minimum
37
Mt li
Cc tiu mong mun

38
Gradient des. with momentum (1)

Nhm ci tin tc hi t ca gii thut gradient descent, ngi ta a ra 1 nguyn tc cp nht trng s ca ANN:
wij (k + 1) = wij ( k ) + wij (k ) wij (k ) = g (k ) + wij (k 1)

vi g(k): gradient; : tc hc ij(k-1) l gi tr trc ca ij(k) : momentum
t hiu qu hun luyn cao, nhiu tc gi nghi gi tng gi tr ca moment v tc hc nn gn bng 1: [0.8 1]; [0 0.2]
39
Gradient des. with momentum (2)

p dng cho bi ton nhn dng m hnh ca motor DC:
Nhn xt: Sau 5000 Epochs, MSE t 4.10-4, nhanh hn gii thut gradient descent.
40
GD vi tc hc thch nghi (1)

Tc hi t ca gii thut Gradient descent ph thuc vo tc hc . Nu ln gii thut hi t nhanh nhng bt n. Nu nh thi gian hi t s ln. Vic gi tc hc l mt hng s sut qu trnh hun luyn, t ra km hiu qu. Mt gii thut ci tin nhm thay i thch nghi tc hc theo nguyn tc:
41
GD vi tc hc thch nghi (1)

Nhn xt: Sau 5000 Epochs, MSE t 10-4, nhanh hn gii thut gradient descent with momentum
42
(1)
L gii thut gradient descent ci tin, m c tc hc v h s momentum c thay i thch nghi trong qu trnh hun luyn. Vic thay i thch nghi c thc hin tng th nh vic thch nghi tc hc . Th tc cp nht trng s ging nh gii thut GD with momentum:
wij (k + 1) = wij (k ) + wij (k ) wij (k ) = g (k ) + wij (k 1)

vi g(k): gradient; : tc hc thch nghi ij(k-1) l gi tr trc ca ij(k) : momentum thch nghi
43
(2)
Nhn xt: Sau 5000 Epochs, MSE t 3.10-5, nhanh hn gii thut gradient descent with thch nghi
44
GT. truyn ngc Resilient (1)

Cc hm truyn Sigmoid nn cc ng vo v hn thnh cc ng ra hu hn lm pht sinh mt im bt li l cc gradient s c gi tr nh, lm cho cc trng s ch c iu chnh mt gi tr nh, mc d n cn xa gi tr ti u. Gii thut Resilient c pht trin nhm loi b im bt li ny bng Gradient direction cch s dng o hm ca hm li quyt nh hng tng/gim ca gradient.
E E Nu k & k +1 cng du: w w ij ij
wij(k+1) c tng thm 1 lng inc Nu Ek & Ek +1 khc du: wij(k+1) c gim i 1 lng dec w w ij ij
45
46

Nhn xt: Sau 5000 Epochs, MSE t 8.10-6, nhanh hn gii thut gradient descent with & thch nghi
47
Gii thut BFGS Quasi-Newton

Sinh vin t c ti liu [1] v [3] p dng cho bi ton nhn dng m hnh motor DC:
Nhn xt: Sau 1100 Epochs, MSE t 2.10-7, nhanh hn gii thut Resilient
48
Gii thut Levenberg-Marquardt (1)

Gii thut Levenberg-Marquardt c xy dng t tc hi t bc 2 m khng cn tnh n ma trn Hessian nh gii thut BFGS Quasi-Newton. Ma trn Hessian c tnh xp x: H=JTJ v gi tr gradient c xc nh: g=JTe trong , J l ma trn Jacobian, cha o hm bc nht ca hm li (e/wij), vi e l vect li ca mng. Nguyn tc cp nht trng s: wij(k+1)=wij(k) [JTJ + mI]-1 JTe
Nu m=0, th y l gii thut BFGS Quasi-Newton. Nu m c gi tr ln n l gii thut gradient descent. Gii thut Levenberg-Marquardt lun s dng gi tr m nh, do gii thut BFGS Quasi-Newton tt hn gii thut gradient descent.
49
Gii thut Levenberg-Marquardt (1)

Nhn xt: Sau 20 Epochs, MSE t 1,7.10-7, nhanh hn gii thut Newton
50
So snh cc gii thut

So snh trn bi ton nhn dng m hnh mt i tng phi tuyn, c trnh by trong ti liu [1]:
51
Cc gii thut ca NN. toolbox

Trainb Trainbfg Trainbr Trainc Traincgb Traincgf Traincgp Traingd Traingdm Traingda Traingdx Trainlm Trainoss Trainr Trainrp Trains Trainscg Batch training with weight & bias learning rules. BFGS quasi-Newton backpropagation. Bayesian regularization. Cyclical order incremental training w/learning functions. Powell-Beale conjugate gradient backpropagation. Fletcher-Powell conjugate gradient backpropagation. Polak-Ribiere conjugate gradient backpropagation. Gradient descent backpropagation. Gradient descent with momentum backpropagation. Gradient descent with adaptive lr backpropagation. Gradient descent w/momentum & adaptive lr backpropagation. Levenberg-Marquardt backpropagation. One step secant backpropagation. Random order incremental training w/learning functions. Resilient backpropagation (Rprop). Sequential order incremental training w/learning functions. Scaled conjugate gradient backpropagation.
52
Qui trnh thit k mt ANN

B A
A
53
54
Tin x l d liu (1)

Phng php chun ha d liu:
Chun ha tp d liu nm trong khong [-1 1].
D liu vo p[pmin, pmax] Chun ha ps[-1, 1] ANN a Hu x l at
Gi p[pmin, pmax] l vect d liu vo
min ps l vect d liu sau khi chun ha, th: ps = 2 p p 1 max min
p p
Nu ta a tp d liu c x l vo hun luyn mng, th cc trng s c iu chnh theo d liu ny. Nn gi tr ng ra ca mng cn c thao tc hu x l. Gi a l d liu ra ca mng, at gi tr hu x l, th: 1 at = (a + 1)( pmax pmin ) + pmin 2
55
Tin x l d liu (2)

Phng php chun ha d liu (v d)
56
Tin x l d liu (3)

Phng php tr trung bnh v lch chun:
Tin x l tp d liu c tr trung bnh bng 0 (mean=0) v lch chun bng 1 (standard deviation=1). Gi p l vect d liu vo, c tr trung bnh l meanp v lch chun l stdp, th vect d liu c x l l:
ps =
p mean p std p
Gi a l vect d liu ra ca ANN, th vect d liu ng ra sau khi thc hin thao tc hu x l:
at = a * std p + mean p
57
Tin x l d liu (4)

Phng php tr trung bnh v lch chun (v d)
Mean 0 Std 1
Mean = 0 Std = 1
58
Nng cao kh nng tng qut ha (1)

Mt vn xut hin trong qu trnh hun luyn mng, l hin tng qu khp (overfitting). Khi kim tra mng bng tp d liu hun luyn, n cho kt qu tt (li thp). Nhng khi kim tra bng d liu mi, kt qu rt ti (li ln). Do mng khng c kh nng tng qut ha cc tnh hung mi (hc vt). C 2 phng php khc phc: Phng php nh ngha li hm mc tiu v phng php ngng sm.
59

Phng php nh ngha li hm mc tiu:
Thng thng hm mc tiu c nh ngha l:
Hm mc tiu c nh ngha li bng cch thm vo i lng tng bnh phng trung bnh ca cc trng s v ngng, MSW, khi :
Vi
l mt hng s t l v n tng s trng s v ngng ca mng

60

Phng php ngng sm:
Phng php ny i hi chia tp d liu hc thnh 3 phn, gm d liu hun luyn, d liu kim tra v d liu gim st. P = [Ptrain, Ptest, Pvalidation] Sau li thi k hun luyn, tp d liu dm st Pvalidation c a vo mng kim tra li. Nu li thu c gim, qu trnh hun luyn c tip tc. Nu li thu c bt u tng (hin tng qu khp bt u xy ra), qu trnh hun luyn c dng li gi l ngng sm. Sinh vin c thm ti liu [1].
61
Minh ha bng MATLAB

Bi ton: Nhn dng m hnh h thng m t, c phng trnh vi phn m t h:
Vi i(t) [0, 4A] dng in ng vo y(t) l khong cch t nam chm vnh cu n nam chm in. cc tham s: =12, =15, g=9.8 v M=3.
62
Bi tp
1. Sinh vin thc hin li bi ton nhn dng m hnh motor DC v hun luyn mng bng tt c cc gii thut ca NN toolbox ca MATLAB. So snh tc hi t ca cc gii thut.
63
Chng 4
Mt s ng dng ca ANN
Gii thiu Nhn dng k t (OCR) Nhn dng ting ni Thit k cc b iu khin Kt lun Bi tp
64
Gii thiu
Gii thiu mt s hng ng dng ANN Pht trin thnh Lun vn tt nghip hay ti NCKH sinh vin
65
Nhn dng k t
Ma trn ha bitmap ca k t Gi lp cc hnh thc nhiu Tp hp d liu hun luyn, mi k t l 1 vect d liu ng vo Qui c ng ra, gi s l m ASCII tng ng ca k t. Xy dng cu v hun luyn ANN
66
Nhn dng ting ni
Trch c trng tn hiu ting ni Tp hp d liu vo, qui c d liu ra Hun luyn v th nghim Phng php LPC & AMDF xc nh c trng ting ni
67

ANN Chapter3 4

Cargado por

Información del documento

Derechos de autor

Formatos disponibles

Compartir este documento

Compartir o incrustar documentos

Opciones para compartir

¿Le pareció útil este documento?

¿Este contenido es inapropiado?

Copyright:

Formatos disponibles

ANN Chapter3 4

Cargado por

Copyright:

Formatos disponibles

Gii thiu

Cc phng php hun luyn

Gii thut hun luyn ANN (1)

Gii thut hun luyn ANN (2)

Minh ha phng php iu chnh trng s nron th j ti thi im k

Gii thut gradient descent (1)

w2ij l trng s lp ra, t j n i

Gii thut gradient descent (2)

Gii thut gradient descent (3)

Sinh vin tham kho ti liu [3], trang 161-175.

Minh ha gii thut hun luyn (1)

w211= 1 w221= -1 w212= 0

w112= 0 w1 b11= 1 b12= 1

Minh ha gii thut truyn ngc nh sau

Minh ha gii thut hun luyn (2)

w111= -1 w121= 0 w112= 0 w122= 1

w211= 1 w221= -1 w212= 0 w222= 1

Minh ha gii thut hun luyn (3)

Minh ha gii thut hun luyn (4)

Minh ha gii thut hun luyn (5)

w211= 1 w221= -1 w212= 0 w222= 1

Minh ha gii thut hun luyn (6)

1a11= -1 1a12= -2 2a11= -2 2a12= -4

Minh ha gii thut hun luyn (7)

w211= 0.9 w221= -1.2 w212= -0.2 w222= 0.6

Minh ha gii thut hun luyn (8)

w111= -1 w121= 0 w112= 0 w122= 1

1 w11= -1 2 w21= 2 1 w12= 0 2 w22= -2

Minh ha gii thut hun luyn (9)

1 = 1 w11 + 2 w21 = -1 + 2 = 1 2 = 1 w12 + 2 w22 = 0 - 2 = -2

Minh ha gii thut hun luyn (10)

w121= 0 w112= 0 w122= 1

Minh ha gii thut hun luyn (11)

w112= 0.1 w122= 0.8

Minh ha gii thut hun luyn (12)

w111= -1 w121= 0 w112= 0.1 w122= 0.8

w211= 0.9 w221= -1.2 w212= -0.2 w222= 0.6

Qu trnh cp nht cc gi tr ngng hon ton tng t.

Minh ha gii thut hun luyn (13)

w112= 0.1 w122= 0.8

Minh ha gii thut hun luyn (14)

w221= -1.2 w212= -0.2 w222= 0.6

w112= 0.1 w122= 0.8

Hun luyn n khi no?

Hm mc tiu MSE (1)

Hm mc tiu MSE (2)

1 3 1 2 2 2 2 MSE = (ti ai ) = (2 3.2 ) + (1 9.2 ) + (2 11.2) 3 i =1 3

Hm mc tiu MSE (3)

Hm mc tiu MSE (4)

V d v GT Gradient descent (1)

V d v GT Gradient descent (2)

(k)=fANN[V(k), V(K-1), V(K-2), (k-1), (k-2)]

V d v GT Gradient descent (3)

V(k), V(k-1), V(k-2)

(k), (k-1), (k-2)

V d v GT Gradient descent (4)

>> net=train(net, Ptrain,Ttrain, [], [], VV,TV);

V d v GT Gradient descent (5)

(k)=fANN[V(k), V(K-1), V(K-2), (k-1), (k-2)]

V d v GT Gradient descent (6)

0 -0.1 -0.2 -0.3 -0.4

Cc tiu mong mun

Gradient des. with momentum (1)

wij (k + 1) = wij ( k ) + wij (k ) wij (k ) = g (k ) + wij (k 1)