Está en la página 1de 61

Trng i hc Hoa Sen

Mc Lc
Ch thch hnh nh ........................................................................................................................................ 3
Ch thch thut ng ....................................................................................................................................... 5
1 t Vn ........................................................................................................................................... 6
2 Tng quan v Splunk .......................................................................................................................... 7
2.1 Splunk l g? ................................................................................................................................. 7
2.2 S Splunk ph bin .................................................................................................................. 7
2.3 Splunk thu thp nhng g? ............................................................................................................ 8
2.4 Splunk c th lm g? .................................................................................................................... 8
2.5 Splunk cung cp cho chng ta nhng g?...................................................................................... 8
2.6 Splunk, Gii php ti u cho Big Data? ....................................................................................... 8
2.7 Ti sao chn Splunk? .................................................................................................................... 9
3 Gii php vi Splunk ........................................................................................................................ 10
3.1 Qun l cc ng dng: ................................................................................................................ 10
3.1.1 Gii quyt vn nhanh hn, gim thi gian b downtime: ............................................... 10
3.1.2 Ti sao Splunk l gii php tt cho vic qun l ng dng ................................................ 11
3.2 Qun l hot ng IT .................................................................................................................. 11
3.2.1 Phn tch hot ng IT: ....................................................................................................... 12
3.2.2 Gim st c s h tng: ....................................................................................................... 12
3.2.3 Splunk cho h iu hnh ..................................................................................................... 13
3.2.4 Qun l o ha .................................................................................................................... 13
3.3 An ninh trong lnh vc IT ........................................................................................................... 15
3.3.1 Mi e da an ninh ngy mt tng:..................................................................................... 15
3.3.2 Qun l log:......................................................................................................................... 15
3.3.3 ng dng Splunk dnh cho an ninh: ................................................................................... 15
4 Cc tnh nng chnh trong hot ng Gim st mng ca Splunk ............................................... 16
4.1 Map Reduce ................................................................................................................................ 16
4.1.1 Map reduce l g? ................................................................................................................ 16
4.1.2 u im ca mapreduce ..................................................................................................... 16
4.1.3 Nguyn tc hot ng ca Mapreduce ................................................................................ 16
4.1.4 4.Chi tit 2 hm Map v Reduce ......................................................................................... 17
4.1.5 Thc thi Mapreduce trong h thng .................................................................................... 19
4.2 Hng dn tm kim v s dng Splunk hiu qu ..................................................................... 21

1
Trng i hc Hoa Sen

4.2.1 Mt s iu cn lu khi tm kim d liu trong Splunk: ................................................. 21


4.2.2 Tm hiu v Boolean v nhm cc iu kin ...................................................................... 22
4.2.3 S dng * tm kim 1 cch hiu qu .............................................................................. 22
4.2.4 Tm kim cc s kin bng thi gian .................................................................................. 22
4.2.5 Chia s kt qu tm kim vi ngi khc ............................................................................ 22
4.2.6 Lu kt qu tm kim s dng li................................................................................... 23
4.2.7 To alerts t kt qu tm kim............................................................................................. 24
4.3 Table , chart trong Splunk ........................................................................................................... 26
4.3.1 Gii thiu mt s hm c bn trong vic to table ............................................................. 26
4.3.2 V d v mt table c th: ................................................................................................... 26
4.4 Dashboard ................................................................................................................................... 29
4.4.1 SEARCH LANGUAGE trong Splunk ................................................................................ 30
4.4.2 nh ngha chc nng mt s hm tm kim ...................................................................... 30
4.4.3 Mt s c php search language trong splunk: ................................................................... 33
4.5 Splunk Forwarder........................................................................................................................ 35
4.5.1 Cc loi Forwarder: ............................................................................................................. 35
4.5.2 So snh universal forwarder vi Splunk full:...................................................................... 35
4.5.3 So snh universal forwarder vi light forwarder: ............................................................... 36
4.6 Mt s khi nim v cc file Splunk.conf ................................................................................... 36
4.7 Hng dn cu hnh input log t syslog server vo splunk server ............................................. 37
4.8 Hng dn cu hnh input log Window vo splunk server ......................................................... 39
4.9 Cch to Dashboards................................................................................................................... 42
5 Demo Lab ly log t h thng mng nh ........................................................................................ 45
5.1 M hnh: ...................................................................................................................................... 45
5.1.1 Bc 1: Ly log t Pfsense vo Splunk .............................................................................. 46
5.1.2 Bc 2: Ly log t Window Server 2k8 DC vo Splunk.................................................... 51
6 Kt lun v hng pht trin ti ................................................................................................ 61
6.1 Kt lun ....................................................................................................................................... 61
6.2 Hng pht trin ......................................................................................................................... 61

2
Trng i hc Hoa Sen

Ch thch hnh nh
Hnh 1 : S trin khai Splunk ph bin trong doanh nghip.
Hnh 2 : Cc loi data , log m Splunk index c.
Hnh 3 : H thng index v tm kim c phn phi gia nhiu core Splunk.
Hnh 4 : S hot ng ca Mapreduce.
Hnh 5 : V d minh ha cch m Mapreduce hot ng.
Hnh 6 : Phn nh d liu u vo.
Hnh 7 : Sao chp chng trnh.
Hnh 8 : Thc hin hm Map cho ra kt qu <key,value>.
Hnh 9 : Thc hin hm Reduce v thng bo kt qu cho Master.
Hnh 10 : Thng bo chng trnh mapreduce hon tt v kt qu c lu tr trn R tp tin.
Hnh 11 : V d tm cc s kin xy ra trong 60 pht trc.
Hnh 12 : Lu v chia s kt qu tm.
Hnh 13 : Kt qu c th c chia s di dng link.
Hnh 14 : Lu kt qu tm kim.
Hnh 15 : Nhng kt qu phi tha nhng iu kin c thit lp mi c lu.
Hnh 16 : Kt qu tm kim s xut hin trong menu Search & Report.
Hnh 17 : To mt alert.
Hnh 18 : t tn alert v iu kin kch hot alert.
Hnh 19 : Chy kt qu tm kim event mi gi, khi ng arlet khi kt qu tm kim ln hn 0.
Hnh 20 : Nu s lng event tm c trong 5 pht b hn 5 th kch hot alert.
Hnh 21 : Cc option trong Alert.
Hnh 22 : Mt table dng s.
Hnh 23 : Mt table dng chart.
Hnh 24 : Biu chart d liu nhn c trong mt khong thi gian.
Hnh 24 : Cc ty chn formating ca chart.
Hnh 25 : V d v mt dashboard c bn.
Hnh 26 : Tt selinux.
Hnh 27 : Cu hnh mc nh trong file rsyslog.conf.
Hnh 28 : Cu hnh m port 514 cho syslog.

3
Trng i hc Hoa Sen

Hnh 29 : Giao din web ca Splunk.


Hnh 30 : Giao din splunk c thm add-on Windows.
Hnh 31 : Cu hnh Forwarding and Receiving.
Hnh 32 : Ty chn cc loi log m universalforwarder s gi.
Hnh 33 : Splunk nhn c log ca Windows.
Hnh 34 : Menu chnh ca Splunk.
Hnh 35 : To mt Dashboard mi.
Hnh 36 : Ty chnh kiu Dashboard s xut ra.
Hnh 37 : Biu biu din log h thng Window dng pie.
Hnh 37 : Biu biu din log h thng Window dng ct.
Hnh 38 : Thm ghp nhiu biu s tr thnh mt dashboard.

4
Trng i hc Hoa Sen

Ch thch thut ng

Big Data: L tp hp cc d liu ln t nhiu ngun nh h thng my tnh, mysql, cc ng dng.v.v.v


Map Reduce : l mt thut ton gip cc ng dng x l nhanh mt lng d liu ln.
UniversalForwarder : l mt phin bn ca splunk nhng ch c tnh nng thu thp v gi d liu.
Light Forwarder : l mt phin bn ca Splunk , khng c tnh nng phn tch m ch forward d liu. t
c s dng cc phin bn splunk 6.0 .
Heavy Forwarder : L mt phin bn ca Splunk, c th phn tch v gi nhng khng c kh nng tm
kim phn phi d liu.
Dashboard : L mt bng bao gm nhiu biu vi nhiu kiu nh dng khc nhau.
Pfsense : Phn mm firewall m ngun m.

5
Trng i hc Hoa Sen

1 t Vn
Trong mi doanh nghip, h thng cng ngh thng tin l h thng v cng quan trng. Ngy nay vi
mc pht trin cng ngh nhanh chng, th ngoi vic m bo kh nng vn hnh, hot ng lin tc
v chnh xc th vic m bo an ninh thng tin l mt thch thc ln.

Nguy c bn ngoi: Tin tc bn ngoi li dng l hng h thng t nhp


Nguy c bn trong: do hnh vi ngi dng , thc v mc an ton d liu cn cha c cao.
Tnh thng nht trong qun tr : Khi h thng cng ln th mc phc tp trong qun l cng s
tang cao.

SIEM l mt gii php hon chnh, y cho php cc t chc thc hin vic gim st cc s kin cho
m h thng. Cc thnh phn chnh ca SIEM bao gm: thnh phn thu thp nht k, thnh phn phn
tch, thnh phn lu tr, thnh phn qun tr tp trung. Ngoi ra cn c cc thnh phn khc nh: thnh
phn gim st Network mc lp 7 trong m hnh OSI, cc module to bo co (Complaince Report,
Dashboard)

Gii php SIEM c nhng u im sau:

H tr thu thp, phn tch cc s kin theo thi gian thc c thu thp t cc h thng gi v,
c kt hp cng vi cc thng tin lin quan n ngi dng, cc thnh phn trong h thng v
d liu.
Cung cp kh nng lu tr log di, ton din (log management) v kh nng phn tch theo ng
cnh (Correlation).
Cung cp cc chc nng c xy dng sn v cho php thay i (Customized) theo cc yu cu
ca cc t chc.
D dng trin khai v s dng.

Splunk l mt cu ni gia vic qun l log mt cch n gin v bo mt thng tin, thu thp s kin. Ci
m phn bit Splunk so vi cc server Syslog hay cc cng c SIEM khc l Splunk Apps. Mt th vin
qun l hn 200 add-on khc nhau. Chnh v iu lm cho Splunk tr nn khc bit, tng kh nng
thu thp thng tin cc loi log khc nhau, c giao din gn gi v thn thin, cung cp nhng tnh nng
tm kim v phn tch d liu thu c.

6
Trng i hc Hoa Sen

2 Tng quan v Splunk


2.1 Splunk l g?
Splunk l h thng c th captures, trch ra cc d liu thi gian thc c lin quan ti nhau t n c th
to ra cc th, cc bo co, cc cnh bo v cc biu .
Mc ch ca Splunk l gip cho vic xc nh m hnh d liu v thu thp d liu my trn ton h
thng d dng hn.N cung cp s liu, chn on cc vn xy ra , phc v tt cho hot ng kinh
doanh
Splunk c th tm kim cc s kin v ang xy ra, ng thi cng c th bo co v phn tch thng
k cc kt qu tm c. N c th nhp cc d liu ca my di dng c cu trc hoc khng cu trc.
Hot ng tm kim v phn tch s dng SPL(Search Processing Language), c to qun l Big
Data. Do c pht trin t Unix Piping v SQL nn Splunk c kh nng tm kim d liu, lc, sa i,
chn v xa d liu.

2.2 S Splunk ph bin

Hnh 1 : S trin khai Splunk ph bin trong doanh nghip

7
Trng i hc Hoa Sen

M hnh trn bao gm cc thnh phn nh:


+Nhiu thit b Forwarders trung gian phc v cho qu trnh load, tnh sn sang cao, v ci thin tc
x l cc event sp ti.
+Mt Indexer lin kt vi nhiu h thng. Vi nhiu search-peer(indexer) ci thin hiu nng ca qu
trnh nhp d liu v tm kim. N gip gim thi gian tm kim v cung cp tnh d phng cao.
+C nhiu u tm kim. Nhng h thng ring bit ny s phn phi bt k yu cu tm kim trn tt c
cc search-peer cu hnh trc ci thin hiu nng tm kim.
+u tm kim ring bit c th hin y h tr ng dng Splunks Enterprise Security(ES).
+Server trin khai. H thng nay c th c tch hp vi cc dch v Splunk khc, hoc trin khai c
lp. Nu mun trin khai h thng ln, mt h thng c lp l rt quan trng.

2.3 Splunk thu thp nhng g?


*Splunk thu thp d liu h thng do my mc to ra
D liu h thng bao gm nhiu hng mc record ca tt c cc hot ng v hnh vi- hnh vi ca khch
hng, giao dch ca user, hnh vi ca h thng.

2.4 Splunk c th lm g?
-Server Metrics -Vulnerability Data
-Custom Applications -Physical Security
-Windows registries -Scripts
-Card key -Patch Mgmt
-Server Logs -Host Config
-DNS Logs -Virtual Logs
-Host ID -Database Logs
-Router -Email Logs
-RAS VPN -Application Logs

2.5 Splunk cung cp cho chng ta nhng g?


Splunk cung cp 1 giao din chung cho tt c d liu IT nh tm kim d liu, nhng cnh bo, nhng
bo co(report), hay chng ta c th chia s d liu cho mt ai . Splunk cung cp gii php tm kim
ti u.

2.6 Splunk, Gii php ti u cho Big Data?


-Splunk tm kim nhng d liu c lin quan vi nhau, gip thu hp phm vi tm kim , tit kim thi
gian, v lm cho cng tc qun tr mng tt hn.

8
Trng i hc Hoa Sen

2.7 Ti sao chn Splunk?


Splunk cn c goi l Google ca log, c cng c search mng m n chp nhn d liu bt k nh
dng no.

Hnh 2 : Cc loi data , log m Splunk index c.


Splunk t ng list ra thi gian c th ca tng s kin xy trong h thng m n ang gim st.
Cnh bo trong thi gian thc. Ta c th chnh ty chn, nh ngha cc loi cnh bo v c th ch nh
ai nhn c cnh bo .
Splunk cung cp thng tin tm kim thng minh: Kt qu tm kim c sp xp hp l, c lin quan vi
nhau, kh nng hin th thi gian thc, phn tch lch s cc s kin xy ra.
Splunk c th lu tr khi lng d liu ln ca h thng IT v d liu ny c th c cu trc bt k,
song tc truy vn d liu nhanh.
Tm kim phn tn s dng Map Reduce( 1 phn mm ca Google, phc v cho vic tnh ton phn tn
cc tp d liu ln trn cc cm my tnh)

9
Trng i hc Hoa Sen

Hnh 3 : H thng index v tm kim c phn phi gia nhiu core Splunk.
D liu cn tm kim c phn phi gia nhiu cores
Mi indexter x l tp hp con ca ton b d liu v to ra mt phn ca kt qu tm kim tng th ri
a n vo vo u ca qu trnh tm kim gim ti.
Tham kho:
http://docs.splunk.com/Documentation/Splunk/6.0.2/installation/RunSplunkasadifferentornon-rootuser

3 Gii php vi Splunk


3.1 Qun l cc ng dng:
3.1.1 Gii quyt vn nhanh hn, gim thi gian b downtime:
-Troublesshoot vn 1 cch nhanh chng, gim chi ph v gim thi gian iu tra v khc phc s
c ti 70%.
-Gim s phc tp bng cch cung cp cho cc nh pht trin c truy cp vo log ca ng dng
thng qua 1 v tr trung tm m khng cn quyn truy cp vo h thng .
-Gim st ton b mi trng ng dng ca chng ta trong thi gian thc ngn chn cc vn nh
hng ti ngi dung, gi li log t cc s kin nh k ngn nga mt mt.
-Nm c hot ng ca ton b ng dng:
-Truy vt v gim st cc giao dch ca ng dng thng qua cc tng ca kin trc phn tn v t nhiu
ngun d liu.
-Pht hin cc bt thng hoc cc vn trong hot ng, thi gian p ng v ch ng gii quyt
chng trc khi n nh hng ti ngi dung ng dng.

10
Trng i hc Hoa Sen

-Theo di s liu hot ng quan trng nh thi gian p ng end-to-end, di thng ip hng i
v m s ln giao dch tht bi m bo ng dng p ng c nhu cu cn thit.
-Nm c ton b hot ng ca ng dng trong thi gian thc trn ton b c s h tng ng dng
ca chng ta.
-t c ci nhn ton din v cch m ngi dung s dng dch v ca chng ta, t c th cung
cp dch v tt hn.
-Lm phong ph h thng ca chng ta bng cch thm cc ngun phi CNTT nh gi c c s d liu,
thng tin khch hng v thng tin v tr.

3.1.2 Ti sao Splunk l gii php tt cho vic qun l ng dng


Khng ging cc cng c qun l truyn thng, splunk c th index, phn tch, khai thc d liu t bt k
tng ng dng no. N cung cp 1 gc nhn trung tm v ton b h thng c s h tng ca chng ta.
Ngn ng tm kim trong splunk gip ngi s dng so snh cc s kin, cc giao dch v ch s hot
ng quan trng khc.
Quyn iu khin c trao cho nhiu nhm trong mt t chc. Nhng hiu bit v d liu ng dng c
th kt hp vi thng tin c cu trc nh thng tin user hoc gi c thng tin doanh nghip quyt nh
tt hn.
Nh sn xut qun l hot ng ng dng AppDynamics v Extrahop pht trin ng dng Splunk
gip khch hng qun l tt hn cc d liu ng dng nh log, cc s kin, hot ng ca c s h tng
v nhiu hn th na.

3.2 Qun l hot ng IT


Trung tm IT d liu trn ton th gii ang tr nn cc k phc tp, vi hng trm cng ngh khc nhau
v thit b nhiu layer. o ha v in ton m my cng ang tr nn phc tp, c bit l cc vn
lin quan n hiu sut hot ng. i ng qun tr v qun l CNTT lng ph nhiu thi gian trong vic
di chuyn t mt giao din iu khin ti giao din iu khin khc , c gng theo di cc d liu cn
thit m bo hiu sut v tnh sn sng cao.
Splunk cung cp 1 cch tip cn tt hn m khng cn phi phn tch c php hay ty chnh n. Splunk
thu thp v lp indexes cha tt c d liu c to ra bi h thng IT ca chng ta (h thng mng,
server, OS, o ha, v.v.) . N hot ng vi bt k d liu m my to ra, bao gm log, file cu hnh, s
liu hiu sut, SNMP trap v cc ng dng log ty chnh.
+ Gii quyt vn nhanh hn , gim thi gian Downtime:
Gip nm bt c hot ng o ha, h thng cloud private v public t 1 giao din trung tm.
Gip tm c ngun gc ca vn nhanh hn 70% m khng cn phi tm kim trong h thng ,server
hay my o.
Qun l h thng ca chng ta trong thi gian thc, ngn nga vn xy ra trc khi n nh hng ti
ngi dng v c thm kinh nghim x l cc s kin xy ra nh k trnh mt mt.
Ch cn 1 ngi qun l c quyn truy cp trc tip, m bo an ton cho d liu, gip trnh leo thang
c quyn.
+Tng quan cc s kin tt c cc tng layer ca h thng:

11
Trng i hc Hoa Sen

Tm cc lin kt gia ngi s dng, hiu sut cc s kin lien quan ti c s h tng c cung cp bi
splunk
Kt hp phn tch d liu thi gian thc tng quan , so snh vi hng triu terabytes d liu lch s.
Phn tch pht hin thnh phn kh nghi c th gip d on v ngn nga mt mt hoc vn v hiu
nng.
Tn ti d liu t khp ni trn mi tng ca trung tm d liu. Qun l mi trng ca chng ta nhn
bit c s thay i, so snh ngay lp tc bit thiu ht hiu nng ca h thng, nhng vn c
sn hoc vn bo mt, an ninh.
+ Gim chi ph cung cp dch v CNTT:
S dng sc mnh v kh nng m rng ca splunk khng ch cho hot ng qun l CNTT m cn
dng h tr kim ton, an ninh.
Gim s lng cc cng c v k nng cn thit duy tr qun l c s h tng phc tp ca chng ta.

3.2.1 Phn tch hot ng IT:


Splunk dng trong hot ng phn tch IT cung cp nhng hiu bit ton din theo nhiu tng gip cho
nh hng ca doanh nghip tt hn ty theo tng trng hp c th.
Ch ng trong vic nhn din v khc phc li dch v m bo s hi long ca khch hng v gip
tng s lng khch hng s dng.
t hiu qu trong qu trnh hot ng do nm bt c nhng nguy him tim tng trong qu trnh hot
ng kinh doanh.
Gip t c cc mc tiu kinh doanh bng cch cung cp tm nhn ton din trn ton h thng cng
ngh khng ng nht, cc dch v, cch qun l, ln k hoch v dung lng, phn tch mc s dng ca
ngi dng v nhiu hn na.

3.2.2 Gim st c s h tng:


+My ch: Vi Splunk, chng ta c th
Ch ng gim st cc my ch v hiu bit su hn v hiu sut, cu hnh, truy cp v cc li pht sinh.
Tng quan hiu sut my ch, cc li v d liu s kin vi ngi dng, o ha v ng dng thnh phn
ngn nga v khc phc li.
Phn tch v ti u ha chi ph cho vic theo di dung lng my ch, bo co an ninh trong thi gian
thc.
+H thng lu tr: Vi Splunk, chng ta c th
Tng quan log, s liu hiu sut v cc s kin t h thng lu tr ca chng ta vi my ch, mng v
d liu t cc ng dng gii quyt cc vn v lm tng s hi long ca khch hng.
S dng cng c phn tch mnh m khc phc s c trong thi gian thc v phn tch hiu sut h
thng lu tr ca chng ta.
Gim thi gian pht trin v ct gim chi ph bng vic d dng tch hp vi cc nh cung cp dch v lu
tr, nh NetApp v EMC.

12
Trng i hc Hoa Sen

+H thng mng: Vi Splunk, chng ta c th:


Gim st v theo di d liu mng t cc thit b khng dy, switch, router, firewall v trn nhng thit bi
khc bng cch s dng SNMP, Netflow, syslog, PCAP,v.v.
Ch ng nhn din cc vn an ninh mng v thc hin phn tch vn . Tng quan d liu mng
vi cc ng dng, h thng lu tr v phn tch my ch gi cho mng ca chng ta an ton v hot
ng mi lc.
t c ch s ROI ti a bng cch ti u ha dung lng mng li ca chng ta, xc nh tr,
qun l bang thng, xc nh top 10 ti nguyn mng thng c s dng v m hnh s dng.

3.2.3 Splunk cho h iu hnh


Splunk v ng dng ca splunk c th gip chng ta:
Tng quan s liu h thng v d liu s kin vi c d liu cc tng cng ngh khc mt cch d
dng.
Tm lin kt gia vn hiu sut ng dng v h iu hnh, o ha, h thng lu tr, mng, v c s h
tng my ch.
Nm c ton b hot ng h thng bng cch cung cp bng iu khin trung tm sc khe h thng
xuyn sut mi trng khng ng b.
Nm c nng lc hn ch ca h thng hoc tnh trng nhn ri.
Theo di nhng thay i v m bo an ninh cho mi trng ca chng ta bng cch gim st mi trng
pht hin nhng hot ng bt ng, thay i vai tr ca ngi s dng, truy cp tri php,v.v..

3.2.4 Qun l o ha
C s h tng o ha to ra mi trng nng ng, ni m ti nguyn my tnh nh my ch, storage,
phn cng mang c o ha t cc ng dng, h iu hnh v ngi s dng. Mi trng o phc tp
i hi cch tip cn mi vi cc dch v IT truyn thng nh x l s c hiu sut, qun l v phn tch
ri ro.
ng dng o ha ca Splunk kt hp sc manh v tnh nng ca Splunk Enterprise c thit k dnh
ring cho cng ngh o ha. N gip tng tc d liu thu thp c c s h tng o. Kt hp d liu h
tng o ha vi d liu tng cng ngh khc s cho 1 gc nhn bao qut hn v h thng trung tm d
liu.
Splunk App cho o ha c th tng thch v thu thp d liu o ha t cc cng ngh o ha nh
WMware vSphere, Citrix XenServer v Microsoft Hyper-V, v cng ngh o ha my tnh bn nh
Citrix XenApp v Citrix XenDesktop.
N to cc bo co a dng , ng nht v cc cng ngh o ha t tt c cc lp ng dng v c s h
tng ca chng ta.
Gip ch ng ngn chn , qun l vn hiu sut, tc nghn c chai, nhng s kin bt ng, nhng
thay i v li an ninh bo mt nguy him. N phn tch v bo co chnh xc gip cho ngi dng c
tri nghim ti u.
Tng quan d liu o ha, gip vic tm ra cc s kin c lin quan mt cch d dng hn, tng quan
cc vn v hiu nng, mng v kin trc h thng my ch.

13
Trng i hc Hoa Sen

Gi li s liu v hiu sut hot ng ca my theo di v phn tch. Thu thp d liu c chiu su t
my ch, my o, h thng my tnh. Cung cp kh nng hin th hot ng v phn tch hon chnh bng
cch xc nh kh nng ca my ch, cc my o nhn ri, cc my ch s dng ng mc, sc cha d
liu, theo di thng k hiu sut tm m hnh s dng v trnh kh nng tt nghn c th.
Theo di nhng thay i v bo co v ti sn. theo di chi tit s thay i m ngi dng thc hin, t
ng ha cc tc v ca vSphere cng nh bo co tnh trng cc thnh phn o.
Ci thin an ninh bng cch gim st mi trng tm cc hot ng ng ng, vai tr ca ngi s
dng b thay i, truy cp tri php v nhiu hn na.
Vi VMware vSphere
-Splunk App cho VMware cung cp kh nng hin th cc hot ng 1 cch chi tit, hiu sut, log, cc tc
v, s kin v lu t my ch, cc my o v cc trung tm o ha. Cung cp hnh nh bao qut v
chnh xc v tnh trng sc khe ca mi trng o ha, ch ng xc nh cc vn v hiu sut, bo
mt, kh nng hot ng v nhng thay i ca my o.
- Nm c thng tin sc khe my o trong thi gian thc. C th xc nh lp tc khu vc my o, my
ch c vn . Phn tch d liu theo thi gian xc nh xem n c nh hng n cu hnh ti nguyn.
Nhn bo co chi tit da trn mi 20s. Khm ph li v cc trng hp ngoi l bng vic ch ra cc s
kin c lin quan ti nhau bng d liu log VC v ESXi trong mt giao din iu khin duy nht.
-C th bit c tnh trng sc khe ca tng my o. Tng tc troubleshoot nh vo vic so snh
gia cc my o vi nhau.
-Ch ng trong vic qun l hnh vi m m ca user, cc cuc tn cng tim nng bng nhng bo co
an ninh
-Nm bt c thng tin CPU, b nh , a v dung lng disk s dng trong thi gian thc. Ch ng
cnh bo khi thiu ht dung lng xy ra. Ly li khng gian lu tr khng s dng cho ngi dng
c tri nghim ti u. S dng xu hng theo thi gian v ti u ha da trn tiu th. D bo thng tin
CPU, b nh, nhu cu cng cn thit v hiu nng ca tng my ch, my o VMs thng qua lch s s
dng ti nguyn ca my o.
Vi Citrix XenServer v Microsoft Hyper-V:
-Cung cp gc nhn theo thi gian thc v cc yu t nh hiu nng, ch s tiu th ti nguyn , cu trc
lin kt trn nn tng my ch o ha bng cch s dng mt khun kh bo co chung. N bao gm mt
chui cc biu lin quan n hot ng IT, gim st hot ng, ln k hoch kh nng chu ti, v thay
i theo di.
-Dashboard Out-of-the-box cho 1 ci nhn c th trong thi gian thc v tnh trng sc khe ca my o
v my ch.
-o su vo lu cho ci nhn chuyn su v hiu nng, log, thay i v cu hnh, cc cnh bo v
hn na.
-Truy cp vo lch s d liu cho vic phn tch v x l s c.
-Cu hnh cnh bo da trn kch bn c sn cho cc vn thng gp nh b nh, CPU, dung lng
a thp.

14
Trng i hc Hoa Sen

-Gim st v theo di ti nguyn m my o tiu th h tr cho vic ln k hoch v kh nng hot


ng ca my o.
-Cho gc nhn 360 v kh nng hin th my o vi d liu t tng cng ngh khc gip gii quyt v
x l x c nhanh hn.

3.3 An ninh trong lnh vc IT


3.3.1 Mi e da an ninh ngy mt tng:
Hin ti cc phn mm malware tr nn tng hnh, v thng trng ging nh mt dch hay 1 ng
dng bnh thng no . N c xy dng ly lan trn ton b h thng. K tn cng c th ty
nghin cu chnh sa h thng ca chng ta, nu b pht hin, k tn cng c th kch hot malware khc
tip tc thu thp d liu. Splunk c th thu thp v index bt k d liu no m khng quan tm n
nh dng hoc kch c v thc hin tm kim t ng trn hng petabyte d liu. Splunk c mt ngn
ng lnh phn tch mnh m, thng minh, gip cc nh phn tch t ra nhng cu hi v bo mt da
trn d liu ca chng ta. Cch tip cn c bit ny gip chng ta ch ng trong vic tm c mi e da
bng cch kim tra hot ng ca d liu trong mi trng hot ng bnh thng.

3.3.2 Qun l log:


Phn mm Splunk gip khch hng ci thin vn phn tch d liu log qun l vic kinh doanh ca
h tt hn. Splunk t ng index d liu, bt k c cu trc hay khng cu trc. , cho php chng ta
nhanh chng tm kim, bo co, v chn on cc hot ng v cc vn an ninh mt cch t tn km
hn. Vi Spunk-vic qun l log ca chng ta s d hn bao gi ht.

3.3.3 ng dng Splunk dnh cho an ninh:


Vi ng dng an ninh ca Splunk chng ta c th s dng s liu thng k trn bt k d liu no tm
kim cc mi e da tim n, trong khi vn c th gim st lin tc cc mi e da a4 b pht hin bi
nhng sn phm an ninh truyn thng.
ng dng an ninh Splunk chy pha trn Splunk Enterprise v cung cp cng c gim st, cnh bo
v phn tch cn thit xc nh v gii quyt cc mi e da bit v cha bit. N ph hp vi i
ng an ninh nh hoc mt trung tm hot ng bo mt.
Bng iu khin an ninh cung cp mt cch xem hon ton ty bin vi cc t kha bo mt quan trng
trong lnh vc an ninh domain. ng dng an ninh Splunk cha 1 th vin dng sn cc s liu an ninh
h tr ngi dng nhn din c cc tnh hung v gim st lin tc cc nguy c bo mt trn domain.
V tt c thng tin u c th hin r trn bng iu khin Dash board.
Tnh nng xem xt li cc s kin xy ra: Cung cp chi tit quy trnh cng vic phn tch cn thit
cc u tin ca v vic, bi cnh ca s c, loi ca n v cc my ch c lin quan. Ch mt click chut
v chng ta c th thy c cc d liu th m ng dng an ninh splunk lu tr.
Tnh nng bo v ti sn v iu tra nhn dng mi nguy him cung cp cho nh phn tch an ninh kh
nng xem xt cc mi e da da trn mt lot cc s kin an ninh. n gin ch cn chn mt khung
thi gian s kin hoc nhiu s kin i din cho nhng hot ng ng ng v Splunk s t ng hin
th mt bn tm tt m hnh an ninh. Vi 1 c click chut, chng ta c th xem tt c cc d liu th c
t ra theo th t thi gian, a ra 1 ci nhn trc tip cho ng nghip hoc to ra mt tm kim mi
xem cc s kin xut hin ny c ti tc xut hin hay khng.
Phn tch v d on: Bng iu khin phn tch cung cp mt im. Nhp vo im s hin cc gii
php bit c hng i tng lai ca im v d bo gi tr da trn m hnh d liu. Ch cn

15
Trng i hc Hoa Sen

chn kiu d liu, bt k i tng cha kiu d liu , kiu hm trnh din, thuc tnh v chu k phn
tch m chng ta mun to.
Danh sch cc mi e da: Splunk cung cp dch v out-of-the-box h tr cho 18 m ngun m e da
ti d liu nhm tng thm tnh bo mt cho h thng ca chng ta. Splunk cho php chng ta thm m
ngun m ca ring chng ta v ngun cung cp d liu thanh ton ch vi vi click chut m khng cn
mt cam kt dch v. Splunk cn cng tc vi trung tm bo mt Norse Security, 1 trung tm bo mt uy
tn ton cu. Splunk cn cho khch hng cm gic tri nghim dch v an ninh Splunk cho h thng doanh
nghip trong vng 30 ngy.

4 Cc tnh nng chnh trong hot ng Gim st mng ca Splunk


4.1 Map Reduce
4.1.1 Map reduce l g?
Mapreduce l 1 phng thc thc thi gip cc ng dng c th x l nhanh 1 lng d liu ln(big
data). Cc d liu ny c t ti cc my tnh phn tn. Cc my tnh ny s hot ng song song c
lp vi nhau. iu ny lm rt ngn thi gian x l ton b d liu. D liu u vo c th l d liu c
cu trc ( d liu lu tr dng bng quan h 2 chiu ) hoc d liu khng cu trc ( d liu dng tp tin h
thng )

4.1.2 u im ca mapreduce
X l tt bi ton v lng d liu ln c cc tc v phn tch v tnh ton phc tp khng lng trc
c
C th tin hnh chy song song trn cc my phn tn 1 cch chnh xc v hiu qu. D liu hot
ng mt cch c lp, khng cn phi theo di x l cc tc v, x l li.
C th thc hin m hnh Mapreduce trn nhiu ngn ng (Java,C++,Python,Perl,Ruby,C) vi cc th
vin tng ng.

4.1.3 Nguyn tc hot ng ca Mapreduce


Mapreduce hot ng gm 2 qu trnh thc hin 2 hm "Map" v "Reduce"
tng chnh ca Mapreduce chnh l thc hin vic "Chia tr"
-Chia vn cn x l (d liu ) thnh cc phn nh x l
-X l cc vn nh 1 cch song song trn cc my tnh phn tn hot ng c lp
-Tng hp cc kt qu thu c a ra kt qu cui cng
Nh vy ton b qu trnh mapreduce c th hiu nh sau
-c d liu u vo
-Thc hin x l cc phn d liu vo (x l tng phn mt ) (Thc hin hm Map)
-Trn v sp xp cc kt qu thu c t cc my tnh lm sao c kt qu tin li nht so vi
mc ch ca qu trnh
-Tng hp cc kt qu trung gian thu c t cc my tnh phn tn (Thc hin hm reduce)
-a ra kt qu cui cng

16
Trng i hc Hoa Sen

Hnh 4 : S hot ng ca Mapreduce

4.1.4 4.Chi tit 2 hm Map v Reduce


Thay v nh ngha d liu di dng bng gi tr c quan h , Mapreduce thc hin nh ngha d liu
di dng cc cp gm <key,value>
i vi 1 tp tin "key" c th l tn ca tp tin cn "value" c th l ni dung ca tp. Mt v d
khc "key" l a ch 1 trang web cn value l s ln ngi dng truy cp trang web . Hai hm Map v
Reduce tp trung x l d liu di dng cc cp <key,value> nh trn
Hm Map: D liu c a vo hm map l cc d liu c phn nh thnh cc phn. u vo
ca hm Map l cc cp <k1,v1>. Sau khi x l ton b d liu u vo (gm nhiu phn sau khi c
phn nh) kt qu thu c l tp hp gm cc cp <k2,v2>. Cc d liu ny c gi l cc d liu
trung gian
Cc d liu trung gian ny c th c ghp li vi nhau theo danh sch cc kha thun tin cho qu
trnh reduce sau ny
Hm Reduce:T d liu u ra ca hm map (gm danh sch cc cp <k2,v2>) ca cc my tnh phn
tn, hm reduce thc hin vic tng hp cc gi tr ny li. Kt qu u ra l cc cp <k3,v3> c x
l
Qu trnh thc hin mapreduce vi bi ton "WordCount"

17
Trng i hc Hoa Sen

Hnh 5 : V d minh ha cch Mapreduce hot ng.


Hm Map:
Input: 1 dng vn bn
Output:Danh sch cc cp <key,value> ng vi tng ch trong dng vn bn . Trong "key" l ch
,value=1.
Hm Reduce:
Input :danh sch cc cp key, gi tr m c ca mi t.
Output: key=t trong c on, value=s lng t tng ng trong on.

18
Trng i hc Hoa Sen

4.1.5 Thc thi Mapreduce trong h thng


-Phn nh d liu u vo
Thng qua th vin Mapreduce ng vi tng ngn ng , chng trnh c nhim v phn mnh tp d liu
u vo. D liu vo c chia thnh cc phn nh.

Hnh 6 : Phn nh d liu u vo


-Sao chp chng trnh
Chng trnh mapreduce lm nhim v sao chp chng trnh chy thnh cc tin trnh song song ln cc
my tnh phn tn. Cc my gm c Master v Worker. Trong my Master lm nhim v iu phi s
hot dng ca qu trnh thc hin Mapreduce trn cc my Worker. Cc my Woker lm nhim v thc
hin qu trnh Map v Reduce vi d liu m n nhn c

Hnh 7 : Sao chp chng trnh


-Thc hin hm Map

19
Trng i hc Hoa Sen

My master s phn phi cc tc v Map v Reduce vo cc worker ang rnh ri. Cc tc v ny c


Master phn phi cho cc my da trn v tr ca d liu lin quan trong h thng. My Woker khi nhn
c tc v Map s c d liu m n c nhn t phn vng d liu gn cho n v thc hin hm
Map. Kt qu u ra la cc cp <key,value> trung gian. Cc cp ny c lu tm trn b nh m ca
cc my.

Hnh 8 : Thc hin hm Map cho ra kt qu <key,value>.


-Sau khi thc hin xong cng vic Map . Cc my Worker lm nhim v chia cc gi tr trung gian thnh
R vng (tng ng vi R tc v Reduce) lu xung a v thng bo kt qu ,v tr lu cho my Master

Hnh 9 : Thc hin hm Reduce v thng bo kt qu cho Master.


-Thc thi tc v Reduce
Master s gn cc gi tr trung gian v v tr ca cc d liu cho cc my thc hin cng vic Reduce.
Cc my reducer lm nhim v x l sp xp cc key, thc hin hm reduce v a ra kt qu cui.

20
Trng i hc Hoa Sen

Hnh 10 : Thng bo chng trnh mapreduce hon tt v kt qu c lu tr trn R tp tin.


-Thng bo kt qu.
Master s kch hot thng bo cho chng trnh ngi dng qu trnh mapreduce hon tt. Kt qu u
ra c lu tr trn R tp tin.

4.2 Hng dn tm kim v s dng Splunk hiu qu


Cha kha to mt cu lnh tm kim hiu qu chnh l tn dng li th ca index. Index ca
Splunk l mt kho t ln v nhn t nh hng ti kt qu tm kim, l c bao nhiu event c ly ra
t disk.

4.2.1 Mt s iu cn lu khi tm kim d liu trong Splunk:


- Splunk khng phn bit ch hoa, thng. Cc t ng tm kim nh error, ErRoR, ERROR u tr v kt
qu tm kim nh nhau.
- Splunk truy vn d liu ti mt thi gian c th.
- C th kt hp cc t kha tm kim vi Bolean (AND , OR ,NOT..) hoc nhm cc iu kin vi nhau
tm kim hiu qu hn. Bolean khi s dng phi vit hoa.
- T kha tm kim phi nguyn 1 t, khng phi 1 phn ca t. Tm kim t kha foo s khng khp
vi kt qu foobar.
- T kha l nhng t c bao quanh bi khong cch hoc du chm cu. V d 1 on log 2014-02-14
Hello world th t kha c index l 2014,02,14,Hello,world.
- Con s cha phi l nh dng s cho ti khi n c phn tch ti thi im tm kim.
- Tn ca cc field phi vit thng. V d: host=hoasen s hot ng, Host=hoasen s khng hot ng.

21
Trng i hc Hoa Sen

4.2.2 Tm hiu v Boolean v nhm cc iu kin


- AND kt qu tm kim phi tha c hai gi tri. V d : error AND mary.
- OR kt qu tm kim ch cn tha 1 hay c hai gi tr. V d : error OR mary
- NOT p dng cho iu kin tm kim tip theo . V d: error NOT mary. Kt qu s tm kim cc
event c t error v khng cha t mary.
- dng tm mt cu theo ng th t. V d: Out of order . Kt qu tm kim s tr v cu theo
ng th t. Nu khng dng vi cu khi tm kim, kt qu tm kim c th s khng ng theo th t
cc t trong cu.
- () dng nhm cc iu kin. v d ( bob AND (error OR mary)) AND NOT debug
- = c dnh ring xc nh cc fields
- [ ] dng thc hin subsearch(tm kim con)

4.2.3 S dng * tm kim 1 cch hiu qu


-Mc d index da vo t tm kim nhng ta c th dng * khi ta khng bit chnh xc t .
-Nn s dng * sau cng, sau khi p dng cc t kha tm kim trc m vn khng tm c kt qu
nh .
V d: Bob* s cho kt qu tm kim Bobby

4.2.4 Tm kim cc s kin bng thi gian


-C th ty chnh thi gian xem trong khong thi gian bao nhiu pht c bao nhiu s kin xy ra

Hnh 11 : V d tm cc s kin xy ra trong 60 pht trc.


-C th dng lnh tm kim bng thi gian trn thanh search
+ tm kim error nh hng user bob trong 60 pht va qua, s dng earliest = -60m bob error
+ tm kim error nh hng user bob trong 3 gi trc, s dng earliest = -3h@h bob error
+ tm kim error nh hng user bob ngy hm qua, s dng earliest = -1d@d latest = -0d@d bob
error
+ tm kim errors nh hng user bob t th hai lc na m, s dng earliest = -0@w1 bob error

4.2.5 Chia s kt qu tm kim vi ngi khc


-Sau khi tm c cc kt qu mong mun ta c th nhn chn Save& share result t menu Save

22
Trng i hc Hoa Sen

Hnh 12 : Lu v chia s kt qu tm kim.


-N s m ra panel Save and Share Results

Hnh 13 : Kt qu c th c chia s di dng link.


-Pha di dng Link tho the results l link URL n kt qu tm kim m ta mun chia s . Ch cn copy
link URL v gi cho ngi ta cn chia s.

4.2.6 Lu kt qu tm kim s dng li


-Chn Save search t menu Save

Hnh 14 : Lu kt qu tm kim
-Ca s Save Search xut hin:

23
Trng i hc Hoa Sen

Hnh 15 : Nhng kt qu phi tha nhng iu kin c thit lp mi c lu.


-Nhp vo gi tr cho Search name, trong hnh l ,errors affecting mary. Thi gian l t 24h trc. C th
ty chn private hoc chia s cho user khc.
-Kt qu search s xut hin trong menu Searches & Report di Error

Hnh 16 : Kt qu tm kim s xut hin trong menu Search & Report.

4.2.7 To alerts t kt qu tm kim


-T menu Create chn alert

Hnh 17 : To mt alert.
-Menu Create Alert xut hin

Hnh 18 : t tn alert v iu kin kch hot alert.


Hnh 19 : Chy kt qu tm kim event mi gi, khi ng arlet khi kt qu tm kim ln hn 0.
+Option Trigger in real-time whenever result matches c ngha l kt qu tm kim s chy theo thi gian
thc v s t ng cnh bo khi tm thy event.

24
Trng i hc Hoa Sen

Hnh 19 : Chy kt qu tm kim event mi gi, khi ng arlet khi kt qu tm kim ln hn 0.


+Option Run on a schedule once every : lm xut hin nhiu option khc

Hnh 20 : Nu s lng event tm c trong 5 pht b hn 5 th kch hot alert.


+Monitor in real-time over a rolling window of: rt hu ch trong vic to cnh bo. V d nu s lng
event din ra trong 1 pht di 100 th gi cnh bo.
-Sau khi ty chnh xog, nhn Next qua phn Action

25
Trng i hc Hoa Sen

Hnh 21 : Cc option trong Alert.


-Bng action gip chng ta quyt nh s lm g i vi kt qu ca alert. Mt s option:
+Send mail:gi mail da trn danh sch e-mail nhp.
+Run a script: chy script vi kt qu ca qu trnh tm kim.
+Show triggered alerts in Alert manager: Lit k cc alerts ph bin trong Saved search
-Sau khi xong cc ty chn, c th nhn Next v thc hin chc nng Sharing nu c nhu cu.

4.3 Table , chart trong Splunk


4.3.1 Gii thiu mt s hm c bn trong vic to table
-Hm pipe (|) trong splunk dng a kt qu output ca 1 tin trnh thnh input cho 1 tin trnh khc.
-Mt s hm to fields :eval, rex
-Hm lc event: head, where
-Hm thay th event vi report : top, stats

4.3.2 V d v mt table c th:


-S dng cu lnh search: source=impl_splunk_gen error | top logger. Kt qu tm kim tr v l mt
table

26
Trng i hc Hoa Sen

Hnh 22 : Mt table dng s.


-Sau c th nhn vo icon chart pha trn table chuyn i table thnh chart

Hnh 23 : Mt table dng chart.


-Mt dng khc ca chart l timechart, dng biu din d liu s theo thi gian
+G cu lnh sourcetype= impl_splunk_gen error |timechart count

Hnh 24 : Biu chart d liu nhn c trong mt khong thi gian.


-Formating options pha trn chart cho ta nhiu la chn ty bin

27
Trng i hc Hoa Sen

Hnh 25 : Cc ty chn formating ca chart.

28
Trng i hc Hoa Sen

4.4 Dashboard
Dashboard l cng c gip chng ta nm bt, nhm v ty chnh cc bng , biu mt cch hiu qu.
N cha nhiu bng thng tin, mi bng chy mt truy vn khc nhau. Mi dashboard c 1 link URL
ring bit, d dng trong vic chia s. Dashboard c th ty bin, ty chnh hin th cc gi tr cn thit,
thanh tm kim trong Dashboard c loi b. Nhiu cng ty s dng dashboard trn my chiu ca h
a 1 ci nhn lt qua v mi trng ca cng ty ti khch hng. Dashboard cn c th lp lch send
file pdf bng email
V d v mt dashboard c bn:

Hnh 26 : V d v mt dashboard c bn.

29
Trng i hc Hoa Sen

4.4.1 SEARCH LANGUAGE trong Splunk


4.4.2 nh ngha chc nng mt s hm tm kim
Cc lnh tm kim

Lnh M t Xem thm


abstract a ra cc bn tm tt cho mi Highlight
kt qu tm kim.
accum Gi hot ng ca 1 s trng s Delta,streamstats, trendline
c th.
Addcoltotals Tnh ton s kin cha cc Stats
trng s cho s kin trc .
Addinfo Thm 1 trng cha cc thng Search
tin v cc lnh tm kim thong
thng ca lnh tm kim hin
ti.
Addtotals Tnh tng cc trng s cho mi Stats
kt qu.
Append Thm cc kt qu ca subsearch appendcols, appendcsv,
cho kt qu hin ti appendlookup, join, set
appendcols Thm vo trng ca kt qu Append, join, set, appendcsv
subsearch vo kt qu hin ti.
Audit Tr li nhng thng tin c
cha trong audix index.
chart Tr li kt qu trong 1 bn, d bucket, sichart, timechart
liu u ra l dng biu .
Cluster Gom, tng hp nhng s kin anomalies, anomalousvalue,
tng t. cluster, kmeans, outlier
Collect, stash em nhng kt qu tm kim vo overlap
index tm tt.
concurrency Dng nhng trng tn ti timechart
kim s s kin ng thi ca
tng s kin.
convert Chuyn i trng gi tr sang eval
gi tr s.
crawl Thu thp file h thng lm ti
nguyn cho index mi.
Dbinspect Tr li thng tin cho 1 index c
th no .
dedup Xa cc chui kt qu ng vi uniq
cc tiu ch c th.
Delete Xa cc s kin c th hoc tm
kim kt qu
Diff Tr v s khc nhau gia 2 kt
qu tm kim.
erex Cho php ch nh v d hoc extract, kvform,
m gi tr v d t ng xut multikv, regex, rex,
ra nhng trng c gi tr tng xmlkv
ng.
Lnh M t Xem thm

30
Trng i hc Hoa Sen

Eval Tnh ton cc hm v y gi tr where


vo 1 trng.
Eventcount Tr v s s kin trong index. Dbinspect
Extract, kv Xut ra trng gi tr t kt qu kvform, multikv,
tm kim. xmlkv, rex
Eventstats Chn tm tt vo tt c cc gi stats
tr tm kim.
filldown Thay th gi tr rng vi gi tr Fillnull
cui cng khng phi l rng.
fillnull Thay th gi tr rng vi 1 gi tr
c th.
findtypes To ra 1 danh sch ngh cc typer
loi s kin.
format Ly kt qu ca Subsearch v
nh dng ca n vo 1 kt qu
ring.
Genttimes Khi to thi gian tm kim kt
qu.
Head Tr v kt qu u tin ca 1 kt Reverse, tail
tm kim.
history Tr v lch s tm kim, nh search
dng nh l 1 danh sch s kin
hoc nh l 1 bng.
Input Thm d liu vo splunk hoc
lm v hiu ha cc ngun t
splunk.
Multisearch THc hin 1 lc nhiu qu trnh Append,join
tm kim.
overlap Tm s kin trong index tm tt collect
m b trng thi gian hoc b
mt.
rangemap Thit lp trng khong cc tn
Rare Hin th cc gi tr t nht trong sirare, stats, top
1 trng.
Replace Thay th gi tr 1 trng c th
vi 1 gi tr mi c th.
return Ch ra gia tr tr v t 1 format, search
subsearch
run Hin th script
sort Sp xp kt qu tm kim bi 1 reverse
trng c th.
table To ra 1 bng s dng cc trng fields
c th
tail Tr v gi tr cui cng . Head, reverse
uniq Xa cc tm kim c trng vi dedup
kt qu trc .

31
Trng i hc Hoa Sen

Cc nh dng bin ngy thng, gi.


Bin thi gian

Bin M t
%Ez Splunk ch ra vng thi gian trong pht.
%H Gi ( nh dng 24h ) l s decimal gm t 00 ti 23
%I Gi ( nh dng 12 ) bao gm s t 01-12
%k Ging %H nhng s 0 u b thay th bng khong trng ( 0 ti 23)
%M Pht, l s decimal ( 00 ti 59)
%p AM hoc PM
%S Giy , l s decimal ( 00 ti 60 )
%T Thi gian trong 24 gi , nh dng ( %H:%M:%S)

Bin d liu

Bin M t
%F nh dng %Y-%m-%d ( theo chun ISO 8601 nh dng ngy thng)
%A C tun ( ch nht ti th 2)
%d Ngy trong thng, l s decimal gm cc s t 01 ti 31
%e Nh %d nhng s 0 u tin thay bng khong trng ( t 1 ti 31 )
%j S ngy trong nm , l s decimal gm cc s t 001 ti 366
%w Th trong tun bng s decimal ( Sunday=0,.. Satuday =6)
Bin thng

Bin M t
%b Tn vit tt tn thng (Jan, Feb, etc.)
%B Tn y ca thng . (January, February, etc.)
%m Thng t theo s decimal ( 01 12 )

Bin nm

Bin M t
%y S nm theo dng decimal ( 00-99)
%Y S nm theo dng y (2014)

32
Trng i hc Hoa Sen

4.4.3 Mt s c php search language trong splunk:


Ch thch:
*(.):t u cu lnh search nhm bo rng c tc v tm kim no trc khi a vo
pipe
* | : t u cu lnh search nhm ngn khng cho thm vo trc cu lnh tm kim.
+administrative

Xem thng tin ca index audit Index=_audit | audit


Thu thp thng tin root v th mc gc sau add |crawl root=/;/Users/ | input add
kt qu tm c vo file inputs.conf
Hin th biu trong khong thi gian mt ngy | dbinspect index=_internal span=1d
Tr v gi tr host cho cc s kin trong index | metadata type=hosts index=_internal
_internal
Tr v thng tin typehead cho sources trong index | typehead prefix=source count=10
_internal index=_internal
+alerting

Gi kt qu tm kim ti mt a ch mail c th | sendmail to=tuan@splunk.com


+add

Lu li s ln xut hin ca total_count | accum count AS total_count


Thm thng tin v tm kim cho mi event | addinfo
Tm kim cc event 404 v thm cc fields | appendcols [search 404]
trong mi s kin vo cc kt qu tm kim trc.
So snh bin count vi gi tr trc ca n v | delta count AS countdiff
lu kt qu vo coundiff
Trch xut gi tr 7/01 v a vo thuc tnh | erex monthday examples=7/01
ngy thng
Thit lp tc v dng distance/time | eval velocity=distance/time
Trch xut gi tr v thit lp li qu trnh tr1ich | extract reload=true
xut field t a
Trch xut gi tr gii hn bi |; v =:. | extract pairdelim= |;, kvdelim==:, auto=f
Thm thng tin v a ch ip | iplocation
Trch xut gi tr t eventtype nu file tn ti | kvform field=eventtype
t range l green nu gi tr date_second t 1- | rangemap field=date_second green=1-30
30; blue nu t 31-39, red nu t 40-59 v blue=31-39 red=40-59 default=gray
gray l cc gi tr cn li.
Tnh ton s lin quan ca php tnh tm kim v Disk error | relevancy | sort relevancy
sp xp kt qu theo th t gim dn
Trch field d liu author t nh dng XML |spath output=author path=book{@author}
hoc JSON (p dng cho sch)
Thm field comboIP. Gi tr ca n = | strcat sourceIP / destIP comboIP
sourceIP + / + destIP
+convert
Chuyn i gi tr ca tt c field thnh gi tr s | convert auto(*) none(foo)
ngoi tr gi tr ca field foo

33
Trng i hc Hoa Sen

Thay i gi tr memory trong field virt thnh | convert memk(virt)


Kilobytes.
Thay i nh dng n v ca | convert dur2sec(delay)
syslog(D+HH:MM:SS) thnh giy
Chia gi tr foo thnh nhiu gi tr | makemv delim=: allowempty=t foo
Kt hp gi tr ca field gi thnh mt gi tr v Eventtype=sendmail | nomv senders | top
hin th 10 gi tr u tin(Dng trong hot ng senders
sendmail)
+filter

Gi field host v ip v hin th theo th t | fields + host, ip


host, ip
Xa field host v ip | fields host, ip
+modify

Xy dng biu thi gian cc s kin web ca Sourcetype=web | timechart count by host |
host v in cc fields trng = NULL fillnull value=NULL
Thay i field _ip thnh IPAddress. | rename _ip as IPAddress
Thay i cc host c gi tr kt thc l localhost | replace *localhost with localhost in host
thnh localhost
+formatting

Hin th bng tm tt 5 dng cho mi kt qu tm | abstract maxlines=5


kim
So snh gi tr ip ca kt qu tm kim th nht | diff pos1=1 ps2=3 attribute=ip
v th ba
Lm ni bt cc t login v logout | highlight login,logout
+delete

Xa events c t invalid trong index imap Index=imap invalid | delete


+summary

t events download trong index tn l Eventtypetag=download | collect


downloadcount index=downloadcount
Tm events trng lp trong summary Index=summary | overlap
+reporting

Tnh tng cc fields s ca mi kt qu v vo | addtotals fieldname=sum


fields sum
Phn tch fields s d on gi tr | af classfield=is_activated
is_activated
Tr v s lng events trong index _internal | eventcount index=_internal
Loi b cc gi tr trng lp cng gi tr host v | stats dc(host)
tr v tng s ln trng lp
Tm log truy cp v tr v 100 gi tr du tin ca Sourcetype=access_combined | top limit=100
referrer domain referer_domain | stats sum(count)
Tnh ton gi tr trung bnh ca CPU mi pht | timechart span=1m avg(CPU) by host
ca tng host
Tnh ton trung bnh CPU v MEM mi pht | timechart span=1m eval(avg(CPU) *
trn mi host avg(MEM)) by host
nh dng li kt qu tm kim | timechart avg(delay) by host | untable _time

34
Trng i hc Hoa Sen

host avg_delay
+results
Tr v nhng events bt thng | anomalies
Xa kt qu trng cng gi tr host | dedup host
Join kt qu ca n vi field id | selfjoin id
Tm t ngy 25/10 n nay | gentimes start=10/25/14
Tm events c to ra bi job vi id=123.2 | loadjob 123.2 events=t
Tr v 20 kt qu u tin | head 20
Tr v 20 kt qu cui cng | tail 20
Hin th events t file messages.1 nu events | inputcsv all.csv | search error | outputcsv
c indexed vo splunk errors.csv
Xut kt qu tm kim ra file csv mysearch.csv | outputcsv mysearch
+search

Gi kt qu tm kim c gi tr src v dst nh Src=10.9.165.* OR dst=10.9.165.8


trc
Tm gi tr URL cha chui 404 hoc 303 |set diff [search 404 | fields url] [search 303 |
nhng khng phi c hai fields url]

Tham kho: https://sites.google.com/site/chapterhut/hoc-tap/mon-hoc/map-reduce

4.5 Splunk Forwarder


Nhim v ca Splunk Forwarder l forward d liu v Splunk server index.

4.5.1 Cc loi Forwarder:


Universal forwarder l mt lightweight forwarder mi ca splunk. N c chc nng thu thp d liu t
nhiu input v forward d liu ti Splunk server index(cha) v tm kim
Light forwarder l mt phin bn nh ca forwarder, c lc b hu ht cc tnh nng ca Splunk full
nhm phc v cho mc ch ti u, n khng phn tch m ch forward d liu ti h thng Splunk
Enterprise hoc h thng ca bn th ba(third-party). Light forwarder nh v cu hnh n gin. N t
c s dng phin bn splunk 6.0.
Heavy forwarder l phin bn Splunk full, vi mt vi tnh nng c lc b ti u ha. N l mt
loi forwarder, c th phn tch d liu v forward d liu ti H thng Splunk Enterprise khc hoc h
thng third-party khc. N khng c kh nng thc hin tm kim phn phi. Nhiu chc nng mc nh
ca n nh splunk web c th b disable ti u hn. N cng c th index(cha) d liu ni b trong
khi forward d liu ti mt Splunk index khc. N chim gp i dung lng b nh, CPU so vi Light
Forwarder v cu hnh phc tp hn.

4.5.2 So snh universal forwarder vi Splunk full:


Mc ch duy nht ca universal forwarder l forward d liu. N khng th index d liu hoc tm kim
d liu. Universal forwarder c mt s hn ch:
+Khng c tnh nng tm kim , index(cha d liu), hay tnh nng cnh bo.

35
Trng i hc Hoa Sen

+Khng phn tch d liu


+Khng y d liu ra ngoi di dng syslog
+Khng ging nh Splunk full, n khng c h tr Python.
Universal forwarder c ti u ch bao gm cc thnh phn cn thit forward d liu ti Splunk
indexers. Universal forwarder c th ni l 1 cng c tt nht forward d liu ti indexer.

4.5.3 So snh universal forwarder vi light forwarder:


+ Universal forwader s dng t CPU, chim t b nh v khng gian a.
+ Universal forwarder c tc truyn d liu mc nh l 256Kbps
+ Universal forwarder khng h tr Python
+ Universal forwarder ch lm nhim v forward, khng th chuyn i thnh Splunk full.

4.6 Mt s khi nim v cc file Splunk.conf


Props.conf: nh ngha cc s kin no theo tn host, source v sourcetype
Input.conf: iu khin d liu vo Splunk , c chc nng blacklist v whilelist, ngn chn hoc cho php
loi d liu no vo splunk, ty chn b qua khng index cc d liu c, input d liu bng cch lng
nghe trn port, c th input d liu bng scripts
Transforms.conf: ni chuyn i v tra cu cc events, c th c tham chiu theo tn trong file
props.conf, to ra cc field
Fields.conf: ni add d liu
Outputs.conf: l file cu hnh splunk forward event ra ngoi.
Indexes.conf: l file quyt nh ni lu tr d liu trn a, lu gi bao nhiu, v trong bao lu. Index
thc cht l tn ca th mc c cu trc c bit. Bn trong cha th mc con gi l bucket v d liu
index.
Authorize.conf: Lu thng tin nh ngha vai tr v cc roles.N nh hng n qu trnh tm kim v
giao din web.
Savedsearches.conf: Ni lu tr kt qu ca cc qu trnh tm kim
Time.conf: nh ngha thi gian xut hin trong bng chn thi gian.
Commands.conf: gm nhng lnh c bit cung cp bi app.
Web.conf: thay i port cho web server, chng ch SSL.

36
Trng i hc Hoa Sen

4.7 Hng dn cu hnh input log t syslog server vo splunk server


+1 my centos hostname splunk.local , ip address 192.168.0.114 ng vai tr l splunk server
+1 my centos client hostname l logserver, ip address 192.168.0.115 ng vai tr l syslog server gi log
v cho splunk server.
Tt firewall, selinux trn c 2 my:
+Tt firewall : # service iptables stop
+Tt Selinux: # vi /etc/sysconfig/selinux
Sa dng lnh t

Hnh 27 : Cu hnh mc nh ca Selinux.


Thnh

Hnh 28 : Tt selinux.
Trn my Splunk server:
Ti gi splunk rpm 64 bit t trang web www.splunk.com v ci t di quyn admin
Gi ci t trong th mc opt
Trn terminal, cd vo th mc opt, g rpm ivh splunk-6.0.3-204106-linux-2.6-x86_64.rpm tin hnh
ci t
G ng dn chy splunk:
# /opt/splunk/bin/splunk start
Bng license agreement hin ra, chn y khi ng.
Splunk c ci t thnh cng
splunk khi ng mi khi restart my g lnh:
# /opt/splunk/bin/splunk enable boot-start

37
Trng i hc Hoa Sen

Vo giao din web ca splunk, chn m trnh duyt(firefox) ri g ng dn 192.168.0.114:8000 vo


giao din splunk web. Mt khu truy cp nh mc nh s l admin/changeme, chng ta phi thay i mt
khu mc nh.
Sau khi truy cp vo giao din web splunk, nhn log t syslog server , ta vo input data, chn tcp, add
port listen l 514 source type l syslog v c th ty chn lng nghe t nhiu ngun hoc t ngun ch
nh(192.168.0.115).Sau save.
Vo input data nhp chn udp v lm tng t nh trn tcp
Hon tt qu trnh ci t v cu hnh splunk.
Trn my SyslogServer :
Ci t syslog server
# Yum install rsyslog
Sau khi ci t xong, vo /etc/rsyslog.conf chnh sa ni dung file

Hnh 29 : Cu hnh mc nh trong file rsyslog.conf.


Thnh

Hnh 30 : Cu hnh m port 514 cho syslog.


ng thi thm vo 2 dng di section #####RULES#####
*.* @192.168.0.114
Mail.* @192.168.0.114
Sau thot ra save file li v restart rsyslog bng lnh :
# service rsyslog restart
Sau th switch user trn syslog server sn sinh log.Sau qua kim tra bn splunk server xem log
c send qua hay cha.

38
Trng i hc Hoa Sen

4.8 Hng dn cu hnh input log Window vo splunk server


Ci t splunk
Trn Splunk Server, tin hnh ci t app for windows trn giao din web
Kt ni vo giao din web serv ca splunk, chn apps gc tri mn hnh -> manage apps

Hnh 31 : Giao din web ca Splunk.


y ta tin hnh ci t app vo splunk t source chun b.
Chn Browse tm ng dn th mc cha file ci t sau chn install app from file.
Sau khi install app restart li splunk server hon tt, apps nhn trn server. Ta c th thy
apps ci homepage.

39
Trng i hc Hoa Sen

Hnh 32 : Giao din splunk c thm add-on Windows.


Gn port cho splunk nhn d liu t forwarder
gc phi chn Setting chn tip mc Data ( Forwarding Receiving)

Hnh 33 : Cu hnh Forwarding and Receiving.

40
Trng i hc Hoa Sen

Chn tip Configure Receiving > Add new tin hnh gn port vo. Chn save hon tt
qu trnh.

Ci t Splunk forwarder trn 1 my khc gi log vo splunk server.


Chun b gi ci t:
Gi Universal Forwarder trn trang ch ca Splunk, y ta s dng gi cho windows
splunkforwarder-6.0.2-196940-x86-release.msi.
Sau tin hnh ci t, chn ni ci. mc Receiving Indexer : g IP ca Splunk Server,
port ging nh port to trn.
Chn Local Data only ly log trn my , bm Next, sau chn loi log m ta cn: y ta
ly Windows Event Logs v Performance Logs.

Hnh 34 : Ty chn cc loi log m universalforwarder s gi.


Chn next , kt thc ci t.
Sau ci t Splunk Technology Add-on trn my forwarder. Vo trang ch splunk down gi
Splunk_TA_windows.
Bung gi ra ta s c 1 th mc cng tn. Vo th mc Splunk_TA_windows\default
chp file input.conf vo th mc Splunk_TA_windows\local
Sau chp th mc Splunk_TA_windows vo ng dn ci forwarder\etc\apps
41
Trng i hc Hoa Sen

Khi ng li my hon tt qu trnh.


Vo my ch Splunk chn Apps for Win kim tra

Hnh 35 : Splunk nhn c log ca Window.

4.9 Cch to Dashboards

Hnh 36 : Menu chnh ca Splunk.


giao din chnh mn hnh Search & Reporting chn Dashboards to

42
Trng i hc Hoa Sen

t tn: No1 , v ch thch li nu mun, chn quyn cho Dashboard ( Private / Share ). Bm Create
Dashboard khi to.

Hnh 37 : To mt Dashboard mi.


Sau khi to xong Dashboard, tip theo s to panel a ln Dashboard. Ta th search lnh ly log.
Chn Save as > Dashboard Panel , chn Dashboard to khi ny ( No1), bm Save . Ta c th to mi
1 Dashboard y hoc s dng Dashboard to.
mc Panel Content: cho php chn kiu biu xut ra.

Hnh 38 : Ty chnh kiu Dashboard s xut ra.

43
Trng i hc Hoa Sen

To tng t vy vi 2 biu mu.

Hnh 39 : Biu biu din log h thng Window dng pie.

Hnh 40 : Biu biu din log h thng Window dng ct.


Sau khi add xong panel vo, ta chn Done hon tt vic to Dashboard v chn panel. y ta c th
ty chnh biu theo mun mc Edit > Edit panel , trn gc phi mi panel ta c th ty chnh loi
biu . Bm Done hon tt.

Hnh 41 : Thm ghp nhiu biu s tr thnh mt dashboard.

44
Trng i hc Hoa Sen

5 Demo Lab ly log t h thng mng nh


5.1 M hnh:

Chun b:
+ 1 my Window Server 2008, ln Domain Controler, Ci t DNS v DHCP server vi IP 193.1.1.30
+ 1 my Cetos ci sn Splunk thu log vi IP 193.1.1.50
+ 1 my Pfsense firewall vi 2 card mng LAN v WAN , card LAN c IP l 193.1.1.1
+ 1 my XP Client kt ni vo Pfsense cu hnh trn giao din Web

45
Trng i hc Hoa Sen

5.1.1 Bc 1: Ly log t Pfsense vo Splunk


+Trn my Centos ci sn Splunk:

Chn Setting =>Data inputs

Chn Add new UDP

46
Trng i hc Hoa Sen

Chn port nhn cc gi tin UDP t client l 514, t sourcetype l Manual, chn Save

Splunk s nhn cc gi tin UDP t port 514


+ Trn my Pfsense:
Vo Status=>System Logs

Chn tab Setting

47
Trng i hc Hoa Sen

Tch vo Send log Messages to remote syslog server, IP remote server nhn log l 193.1.1.50(my
Splunk), tch ty chn cc log mun gi qua Splunk, Chn Save.
Lu l Pfsense ch gi log bng giao thc UDP.

48
Trng i hc Hoa Sen

Kt qu:
Splunk nhn c log t Pfsense

Nhn thanh search tm a ch IP 193.1.1.1 ca Pfsense

49
Trng i hc Hoa Sen

50
Trng i hc Hoa Sen

5.1.2 Bc 2: Ly log t Window Server 2k8 DC vo Splunk


+Trn my Window Server 2008:

Ci t Splunk Forwarder

Chn chp nhn cc iu khon ca splunk sau bm Next

51
Trng i hc Hoa Sen

Chn ni ci t cho Splunk

Nhp vo a ch ca my ch splunk v port. Lu : ta chn port trng vi port s cu hnh trn


Splunk ( Setting > Forwarding and Receive data ) d liu c th gi qua Splunk.

52
Trng i hc Hoa Sen

Chn mc Remote Windows Data gi thng tin cc log, event, v performance ca


DomainController

Nhp tn ti khon administrator ca DomainController.

53
Trng i hc Hoa Sen

Chn loi log m ta cn gim st, ta c th ty chnh li file sau khi ci t.

Ta chn ci t lun Splunk Add-on for Windows.

Finish kt thc qu trnh ci t.

54
Trng i hc Hoa Sen

Copy 2 th mc TA-DNSServer-NT6 v TA-DomainController-NT6 vo th mc cu hnh ca


Splunk c th gi cc thng tin ca DC. Sau ta restart server splunk c th hot ng.

Sau khi restart li DC, kim tra thy tin trnh Splunk hot ng.

55
Trng i hc Hoa Sen

+Cu hnh trn Splunk

Vo Forwarding and Receiving add thm port 10000 trng vi port lc ci t DC.

56
Trng i hc Hoa Sen

Sau khi gn port ta kim tra d liu c gi qua cho Splunk, mc Splunk App for
Windows Infrastructure

57
Trng i hc Hoa Sen

58
Trng i hc Hoa Sen

59
Trng i hc Hoa Sen

60
Trng i hc Hoa Sen

6 Kt lun v hng pht trin ti


6.1 Kt lun
Vi mc tiu ra, nhm chng ti hon thnh cng vic tm hiu, nghin cu cng nh trin khai
p dng c Splunk vo m hnh mng thc t. Qua , nhm chng ti kim chng nhng yu t
sau ca splunk:

Nguyn l hot ng trong mi trng bigdata


Mt s tnh nng c bn v nng cao trong vic x dng splunk.
Sc mnh trong vic truy vt x c pht sinh trong h thng.
iu quan ct li l chng ti thu c lng kin thc xoay quanh vn syslog cng nh
hiu thm v tm quan trng ca cng tc an ninh, bo mt, phc hi mng.
Kt qu t c gip chng ti hiu su hn v cc tnh nng m splunk cung cp. Tuy nhin, do hn
ch v ti liu, ph s dng bn quyn cng nh kinh nghim ca nhm nn bi bo co cn nhiu thiu
st. Nhng nhm chng ti s c gng tip tc tm hiu su hn na, k c sau khi kt thc bi bo co
nghin cu khoa hc ny.

6.2 Hng pht trin


Th gii cng ngh thng tin ni chung v mi trng mng ni ring ang ngy cng pht trin vt
bc. Song song vi ngy cng c nhiu l hng mng c khai thc to iu kin thun li cho
hacker xm nhp gy nh hng tiu cc n h thng. Ngoi ra, i vi v tr IT system administrator
chng ta phi lun lng nghe tt c thng ip c pht i t h thng.
Qua cho thy Splunk hon ton ph hp, cn thit v y kh nng p ng nhng yu cu t
ra i vi mt chng trnh qun l, gim st, cnh bo tt c s kin ang m thm din ra trong h
thng. Thc t, Splunk l mt trnh dch vin cao cp gip ngi qun tr giao tip mt cch trc quan
nht i vi h thng. Qua ci nhn trc quan y m ngi qun tr xc nh chnh xc u l nguyn
nhn dn n s c c th khc phc hiu qu nht cng nh xy dng phng hng pht trin h
thng. Tt c nhng iu trn chng minh rng, Splunk c y kh nng hot ng trong mi trng
datacenter, mng doanh nghip, mng dch v, h tng mng,.. cng nh h tr cc thit b cisco, IBM,
Tng hp cc iu trn ch ra, bt k ni no c h thng mng tn ti, ni no c h thng log tn ti l
ni c th ng dng Splunk.

61