Netbackup operational procedures document

ISL/OPS/A869
ISSUE 2
Netbackup Operational
Procedures
Veritas Netbackup Procedures
Netbackup Operational Procedures
Contents
About this document...
Author
The author of this document may be contacted at:
John Wesson
Eldon House
Sheffield S1 3PL
Content approval
This is Issue 2 of this document
The information contained in this document !as a""ro#ed for use
Filing
The filin$ reference for this document is ISL%&PS%'()*
History
Issue Date Author Reason
Issue 1 23%+2%+, -ic. S!eetin$ /0'1T 1
Issue 2 2)%+,%+2 John Wesson Incor"orate &PS "roduced docs
Page 2 of 61
ISL/OPS/A6!" Issue 2 #Error! No text of specified style in document.$
Contents
Contents
1 Introduction................................................................................................6
2 Contacts.......................................................................................................6
21 3ustomer E4"erience -ana$ement 3entre &"erations)
22 3ustomer E4"erience -ana$ement 3entre Second Line Team)
23 3ustomer E4"erience -ana$ement 3entre Second Line /5'2
26 Stora$e mana$ement2
2, &PE0'TI&7S % -E/I' 3&7T'3T LIST2
2) '/I3 80'9 3ontact%Info2
22 Escalation(
2( Hard!are Information(
2* /ri#e Information(
21+ I5- Hard!are 3ontact*
3 Tape Library Naming Conentions...........................................................!
" I#$ Library In%ormation........................................................................1&
61 9sin$ robtest to interro$ate the robot1+
62 mtlib 3ommands1+
63 /ealin$ !ith "roblems !hen a robot $oes into Pause mode11
66 -edia mo#ements : Insertin$ media12
6, E;ectin$ media from the I5- library1,
6) /ri#e "roblems1,
62 Library "roblems1)
6( mtlib commands : /escri"tion and useful commands12
' ADIC Library In%ormation......................................................................1(
,1 9sin$ robtest to interro$ate the robot12
,2 dasadmin <only to be used on '/I3 libraries=12
,3 -ount a >olume in a ta"e dri#e2+
,6 /ismount a >olume from a ta"e dri#e2+
,, -edia -o#ements : Insertin$ -edia2+
,) E;ectin$ -edia from the '/I3 Library21
,2 /ri#e Problems22
,( '-Ls22
,* 3ommon Problems22
,1+ Points To 7ote22
,11 Library Problem26
6 Accessing Net#ac)up ...............................................................................2"
)1 759 6,26
( Net#ac)up Daemon *rob+ems.................................................................2'
Page % of 61
Contents
22 Shutdo!n 7etbac.u"2)
, Netbac)up Actiity Logs and process in%ormation................................2(
(1 Introduction22
(2 1ile system full "roblems22
(3 1ull List of 7ET5'3?9P Processes2(
,." -enera+ Drie Testing and .au+t Reso+ution.......................................3&
(, /ealin$ With Ta"e /ri#e Problems3+
() /eterminin$ !hether "roblem is /ri#e or 0obot3+
(2 /e#ice 3onfi$uration 9tility : T"confi$32
! Decon%igure/Recon%igure 0e1uent Dries...............................................32
1& Resetting the 0C0I e2tenders ................................................................3"
1+1 &#er#ie!36
11 3se%u+ $edia Commands.......................................................................3'
111 5P-E/I': 1ree@eA unfree@eA sus"end or unsus"end media3,
112 Ta"e -ana$ement3)
12 Restore -uidance ...................................................................................3(
121 5ac.$round32
122 0estore information3(
123 -edia related items3*
126 0estore "rocesses6+
12, 1ailo#er 0estores6)
12) 0eduction of 7et5ac.u" dri#e usa$e to allo! a restore to run,+
122 'dditional information,1
13 #ac)ups ..................................................................................................'1
1" Net#ac)up Logmon 4rror $essages.....................................................'1
161 7et5ac.u" Processes and Procmon Error -essa$es,2
1' Net#ac)up 0TAT30 42its 5 the big hitters..........................................'3
1,1 E4it Status 61 B7et!or. connection timed outC,3
1,2 E4it Status 1 B5ac.u" !as "artially successfulC,,
1,3 E4it Status ,2 BTimed out !aitin$ for -edia -ana$er to mount #olumeC
,,
1,6 E4it Status 21 B7one of the files in the file list e4istC,,
1,, E4it Status 21* BThe reDuired stora$e unit is not a#ailableC,,
1,) E4it Status ,6 BTimed out connectin$ to clientC ,)
1,2 E4it Status (6 B-edia !rite errorC,)
1,( E4it Status ,2 B3lient connection refusedC,)
1,* E4it Status *) B9nable to allocate ne! media for bac.u"A stora$e unit
has none a#ailableC,2
1,1+ E4it Status 131 3lient is not #alidated to use the ser#er,(
1,11 '""lication 0esource 'lert,*
Page & of 61
Contents
1,12 Slo! throu$h"ut of bac.u"s)+
16 Netbac)up C+ients...................................................................................6&
1)1 7etbac.u" /ocumentation )+
1)2 Troubleshootin$ 8uide)1
1)3 7et5ac.u" re"ortin$)1
1)6 Su""ortal)1
1( A**4NDIC40 ........................................................................................61
Page ' of 61
Contents
1 Introduction
This document is intended for use in identifyin$ and resol#in$ >eritas
7et5ac.u" "roblems It is not intended to re"lace any soft!are manuals and
should be used in con;unction !ith the current >eritas manuals It also
assumes a certain le#el of 9ni4 command e4"erience and use of dasadmin
commands
2 Contacts
2.1 Customer Experience Management Centre
perations
The Lines of 5usiness &"erations team are based at the Sheffield 3ustomer
E4"erience -ana$ement 3entre and "ro#ides a first line and monitorin$ role
for 7etbac.u" alarms
This $rou" is a#ailable 26 hours a dayA 2 days a !ee.A and can be contacted
as follo!s:
Tel: +(++ 21)))2 &"tion 3
1a4: +116 222 6226
&"erations 0erice $anager6 +(++ 21)))2 &"tion )A2
Email: cclobo"sEbtcom
The 3larify class for the &"erations 8rou" is: CCL7#7*0
2.2 Customer Experience Management Centre Secon!
"ine #eam
The 5T%HP Second Line team are based at the Sheffield 3ustomer
E4"erience -ana$ement 3entreA and "ro#ide second line technical su""ort
for 9ni4A Wintel and 7etbac.u"
The team is a#ailable 26 hours a dayA 2 days a !ee.A and can be contacted as
follo!s:
Tel: +(++ 21)))2 &"tion ,A2
1a4: +116 222 6226
Second Line team leader: +(++ 21)))2 &"tion ,A6
Email: soc2lineEbtcom
The 3larify classes for the Second Line team are:
Page 6 of 61
Contents
9ni4: C89*7*03:
Wintel: C89*7*0NT
7etbac.u": C89*7*00TN#
2.$ Customer Experience Management Centre Secon!
"ine %&A
' Second line /5' is a#ailable 26 hours a dayA 2 days a !ee.A and can be
contacted as follo!s:
Tel: +116 222 6+32
1a4: +116 222 6226
3larify class: D#A7RA2L
2.' Storage management
3ontact information for the stora$e mana$ement su""ort $rou" !ith
res"onsibility for 7etbac.u"
5rid$e 3larify class 3W5'3?9P
&utside office hours 3W5'3?9P F 5rid$e
3allout 7ET5'3?9P
2.( )E*A#I+S , ME%IA C+#AC# "IS#
' full list of site contacts can be found in /e"artmental 3ontacts under
3ontacts at
G3ollHI G5oo.HI GSTa$HI G90LHhtt":%%datainte$rityintrabtcom%I
GHTa$HIG1ile7ameHIGPathH -'30&59TT&7 E4ternal0eference
htt":%%datainte$rityintrabtcom%
2.- A%IC .*AU Contact,In/o
2.-.1 Contact +um0er , /ault logging , hotline
ADIC 5 -RA3 &13"" ",,(,6
-aurice 0utherford
&ffice +11( *22 *1++
-obile +221+ ,2)62,
-i.e Halliday
Page ( of 61
Contents
&ffice +11( *22 *1++
-obile +2(,+ +++1,2
2.-.2 A%IC Callout Agreement
3allout !ill be insti$ated by the Second Line team%7etbac.u" Su""ort out of
hours !here !e consider there to be a serious de$radation of the systemA ie
o#er ,+ J of ta"e dri#es are una#ailable &ther!ise a call !ill be "laced
durin$ normal !or.in$ hours &n site access !ill be arran$ed by the Second
Line team #ia the 3han$e % Problem -ana$ement System
2.-.$ Hours o/ Cover
1ull co#er !ill be "ro#ided on a 2& )ours per da*" %6' da*s per *ear basis
!ith a 2:hour res"onse time
2.1 Escalation
2.1.1
9* ADIC
John &rman
<Tel +1*+( ),)2)2=
8ary Pa$e
<Tel: +11( *22 *1++=
<-obile: +2*1* 33+*6,=
Pete -eade
<Tel +1,1 2+) ((+,=
<-obile: +2(+2 621232=
-aurice 0utherford
<Tel: +11( *22 *1++=
<-obile: +221+ ,2)62,=
2.2 Har!3are In/ormation
Hard!are information for each site can be found in Hard!are Information
under /ocumentation at
2.4 %rive In/ormation
This can be found under Ta"e Library Info under -edia K Library under
/ri#e K -edia at
htt":%%byadsm+3natbtcom%
Page of 61
Contents
2.15 I&M Har!3are Contact
In the e#ent of a failure of any I5- .it a call should be made to I5- #ia
Stora$ete. <+16(3 22(1+1= The follo!in$ information !ill be reDuired in
order for Stora$ete. to "lace a call !ith I5-
The site id number
The 6:di$it machine ty"e number as listed beside hostname
The serial number of the machine
The "ro;ect nameA !hich in most instances is 5runi4
2.15.1 I&M Callout Agreement
3allout !ill be insti$ated by the Second Line team%7etbac.u" Su""ort out of
hours !here !e consider there to be a serious de$radation of the systemA ie
o#er ,+ J of ta"e dri#es are una#ailable &ther!ise a call !ill be "laced
durin$ normal !or.in$ hours &n site access !ill be arran$ed by the Second
Line team%7etbac.u" Su""ort $rou" #ia the 3han$e % Problem -ana$ement
System
2.15.2 Hours o/ Cover
1ull co#er !ill be "ro#ided on a 2& )ours per da*" %6' da*s per *ear basis
!ith a 2:hour res"onse time
$ Tape ibrary Namin! Con"entions
The 7et5ac.u" ta"e libraries <also .no!n as ta"e robots= come from t!o
different manufacturersA I5- and '/I3%80'9
Lou can tell a $ood deal ;ust from the library name
: the first t!o letters $i#e the locationA e$A IP is I"s!ich
: the ne4t three letters are either I5- or 80'A indicatin$ the manufacturer is
either I5- or '/I3%-RA9
: the ne4t t!o letters are L5A to indicate this is a library rather than a normal
ser#er
: finally there is a robot number This is needed as one site may ha#e se#eral
I5- or '/I3%80'9 libraries
9nderstandin$ the namin$ con#ention means you can tell a $ood deal ;ust
from a name li.e IP80'L51A TPI5-L52A etc
'n o#er#ie! of our ta"e libraries at the ma;or sites can be found under
the 90L htt":%%datainte$rityintrabtcom% Just clic. the tab in the left:hand
Page ! of 61
Contents
menu for MTa"e /ri#e Layout /ia$ramsMA then clic. on the site you are
loo.in$ for 'lternati#elyA issuin$ t"confi$ :d on any ser#er !ill indicate the
library ty"e it uses <t+h indicates an I#$ 3"!"; and t+m indicates an
ADIC/-RA3 A$L +ibrary= and sho! the dri#es that ser#er has
' I#$ ibrary Information
N7ote: -ost of our I5- libraries are I5- 36*6MsA and these ha#e a
library mana$er !hich understands the mtlib commands e4"lained
belo! 7e!er I5- 3,(6Ms are comin$ into ser#ice no! !hich do not
ha#e a library mana$er Lou cannot use mtlib commands on theseA so
!ill ha#e to rely on robtest and 7et5ac.u" for library informationN
'.1 Using ro0test to interrogate the ro0ot
The BrobtestC utility in %usr%o"en#%#olm$r%binA can be used to interro$ate and
control the I5- silo from the system runnin$ the daemon in the same !ay
that it can be used to control the '/I3 silo from machines that dri#e an
'/I3 silo
Issue the follo!in$ command
<robtest
9se BOC to $et hel" in robtest
The main commands a#ailable are BdrstatC <to dis"lay dri#e status= and B#ie!C
to e4amine ta"es 7ote that BunloadC cannot be usedA as the ta"e dri#es are
not connected to the S97 systems
1or further information re$ardin$ the robtest utilityA refer to ISL%&PS%5226 :
5097IP media mana$ement "rocedures at
G3ollHI G5oo.HI GSTa$HI G90LHhtt":%%documentsintrabtcom%I
htt":%%documentsintrabtcom%
Note+ Do not +eae your robtest session actie any +onger than you hae to.
8hen you are in the robtest uti+ity; communication bet=een the tlhcd
daemon and the media serers is b+oc)ed. I% a media serer ma)es a
re1uest to tlhcd >either a mount or a dismount re1uest? =hi+e the robtest
uti+ity is actie; the re1uest =i++ be b+oc)ed. This =i++ resu+t in the tape
dries on that media serer being s=itched into '>0 mode unti+ the
robtest session is terminated
'.2 mtli0 Comman!s
The follo!in$ commands can be used to interro$ate the robot usin$ the mtlib
utility <%usr%bin%mtlib=
Page 1, of 61
Contents
Command Resu+t
mtlib :l 36*6c FD I

Query In#entory This command "roduces an
in#entory of the silo and the cate$ory of the ta"e
mtlib :l 36*6c FD L Query Library This "roduces #arious o"tions of
library data includin$: :
State : This should dis"lay N'utomated &"erational
StateN If the library has been P'9SE/ then a "ause
state !ill dis"lay
'#ailable cells : /is"lays the number of em"ty cells
mtlib :l 36*6c FD S Statistical data
mtlib :l 36*6c FD ? 3ount of ta"es
mtlib :l 36*6c :3 :t
Rcate$oryS :>
RbarcodeS
This chan$es the cate$ory of a ta"e
mtlib :l 36*6c :3 :t
111+ :> RbarcodeS
It can also be used to e;ect ta"es from the silo
mtlib :l 36*6c :/ This dis"lays the silo de#ice numbers
mtlib :l 36*6c FD - /is"lays !hich ta"es are loaded in abo#e de#ice
dri#es
1or further information re$ardin$ the robtest utilityA refer to ISL%&PS%5226 :
5097IP media mana$ement "rocedures at
'.$ %ealing 3ith pro0lems 3hen a ro0ot goes into
)ause mo!e
&ccasionally the I5- library may $o into a "aused state This results in no
cartrid$e mo#ement and "robably a number of bac.u" failures The cause is
the I%& station "roblem slotsA of !hich there are t!oA ha#e been filled !ith
cartrid$es that the robot has had "roblems handlin$
If the robot $oes into a "aused state contact the -edia &"s and as.
them to em"ty the I%& station "roblem slots and note the cartrid$e numbers
When this has been done the library should automatically restart and !or. as
normal 3onfirm that bac.u"s are runnin$ and that cartrid$es are bein$
loaded and unloaded &ccasionally the "roblem may be due to a failure in the
robotic arm and all cartrid$e mani"ulation !ill fail The cartrid$es !ould
Page 11 of 61
Contents
then be "laced in the I%& station "roblem slots causin$ the library to $o into a
"aused state a$ain This !ill reDuire callout to the en$ineers to resol#e the
"roblem
If the library does reco#er then the "roblem cartrid$es can be re:
inserted into the library If this fails then they need to be chec.ed to see if
they contain any #alid data and if so need to be "ut into a fro@en state If no
#alid data e4ists then they can be deleted from the -edia -ana$er database
'.' Me!ia movements 6 Inserting me!ia
To insert media to the I5- library is a strai$htfor!ard "rocess There is no
insert command to run as the library !ill automatically ta.e cartrid$es from
the I%& stationA or ho""erA and "lace them in a#ailable slots The only
"roblems that are li.ely to occur are that there are no slots a#ailable or the
library is not functionin$ correctly
&nce cartrid$es ha#e been inserted and acce"ted by the library they can be
added to 7et5ac.u" media mana$ement database as normal #ia the 89I or
command line This should be done from the 7et5ac.u" master ser#er
'.'.1 Chec7ing availa0le slots
There are t!o !ays to find out ho! many a#ailable slots there are
The least disru"ti#e !ay is to use the follo!in$ !eb"a$e G3ollHI G5oo.HI
GSTa$HI G90LHhtt":%%byadsm+3natbtcom%$rauGinfoI
htt":%%byadsm+3natbtcom%$rauGinfoA !hich includes '/I3 and I5- library
information
'n alternati#e is to interro$ate the ta"e library 's the library is bein$
chec.ed this command must be issued from the mount ser#er Lo$ onto the
mount ser#er and run the robtest utility
Caution6 7ote that !hile robtest is runnin$ no further library action can ta.e "laceA ie mount
and dismounts
Select the a""ro"riate libraryA !hich is normally o"tion 1A and to $et a list of
the "ossible o"tions use the % command
d*ads-,1 . robtest
Configured robots /it) local control supporting test utilities+
0L1#,$ L2CP de3ice pat) 4 /de3/l-cp,
5obot Selection
666666666666666
1$ 0L1 ,
2$ none/7uit
8nter c)oice+ 1
5obot selected+ 0L1#,$ L2CP de3ice pat) 4 /de3/l-cp,
In3oking robotic test utilit*+
Page 12 of 61
Contents
/usr/open3/3ol-gr/bin/tl)test 6r /de3/l-cp, 6d /de3/r-t291 ,,%'!,81A,,
6d /de3/r-t191 ,,%'!,81A,1 6d /de3/r-t%91 ,,%'!,81A,2
Opening /de3/l-cp,
8nter tl) co--ands #: returns )elp infor-ation$
:
0o e;it t)e utilit*" t*pe 7 or <9
audit =3olser> 6 Audit librar* for 3olser
catin3 =categor*> ?=count>@ 6 Print librar* in3entor* b* categor*
d- ?=dri3e>A=IB2 Ce3ice Na-e>@ 6 Cis-ount 3olser fro- dri3e
dr-apclear 6 Clear dri3e address -apping
dr-apfreeDe 6 EreeDe dri3e address -apping
dr-aps)o/ 6 S)o/ dri3e address -apping
drstat ?=dri3e>A=IB2 Ce3ice Na-e>@ 6 Print dri3e status
eFect =3ol> ?bulk@ 6 8Fect 3olser to standard #or bulk$ output area
in3 ?=count>@ 6 Print librar* in3entor*
libstat 6 Print librar* status
- =3olser> ?=dri3e>A=IB2 Ce3ice Na-e>@ 6 2ount 3olser
setcat =3olser> =old> =ne/> 6 Set 3olser categor*
t*pes 6 Print list of -edia t*pes
3erbose 6 0oggle 3erbose -ode
3ie/ =3olser> 6 Print 3olser data
SCSI co--ands+
unload ?=dri3e>A=IB2 Ce3ice Na-e>@ 6 Issue SCSI unload
=dri3e> 4 d1 if dri3e 1" d2 if dri3e 2" 999" d2'6 if dri3e 2'6
9se the libstat o"tion to dis"lay the status of the library
libstat
Librar* infor-ation+
state+ Auto-ated Operational State
input stations+ 1
output stations+ 1
input/output status+ All con3enience input stations e-pt*
All con3enience output stations e-pt*
-ac)ine t*pe+ %&!&
se7uence nu-ber+ ,;16''2
nu-ber of cells+ '&%1
a3ailable cells+ %2'
nu-ber of subs*ste-s+ 1(
con3enience capacit*+ 1,
accessor config+ ,1
accessor , status+ Accessor a3ailable
Gripper 1 a3ailable
Gripper 2 not installed
Page 1% of 61
Contents
Vision s*ste- operational
co-p a3ail status+ Pri-ar* librar* -anager installed9
Pri-ar* librar* -anager a3ailable9
Pri-ar* )ard dri3e installed9
Pri-ar* )ard dri3e a3ailable9
Secondar* )ard dri3e installed9
Secondar* )ard dri3e a3ailable9
Con3enience input station installed9
Con3enience input station a3ailable9
Con3enience output station installed9
Con3enience output station a3ailable9
a3ail %&!, cln c*cles+ ,
a3ail %'!, cln c*cles+ !
<H85I LIB5A5I CA0A co-plete
Page 1& of 61
Contents
Note+ To find out the dri#e details on the ser#erA issue the tpconfi! &d command on
the master%media ser#erA !hich uses the library The out"ut !ill loo. similar
to that sho!n belo!
d*ads-,1 . tpconfig 6d
Inde; Cri3eNa-e Cri3ePat) 0*pe S)ared Status
JJJJJ JJJJJJJJJ JJJJJJJJJJ JJJJ JJJJJJ JJJJJJ
, Cri3e1 /de3/r-t291 )cart2 No HP
0L1#,$ IB2 Ce3ice Na-e4,,%'!,81A,,
1 Cri3e2 /de3/r-t191 )cart2 No HP
0L1#,$ IB2 Ce3ice Na-e4,,%'!,81A,1
2 Cri3e% /de3/r-t%91 )cart2 No HP
0L1#,$ IB2 Ce3ice Na-e4,,%'!,81A,2
Currentl* defined robotics are+
0L1#,$ L2CP de3ice pat) 4 /de3/l-cp,"
3olu-e database )ost 4 d*ads-,1
1rom this it can be seen that there are 3 dri#es confi$ured to be used by
this ser#er F dyadsm+1
'.( E8ecting me!ia /rom the I&M li0rary
To e;ect media from a libraryA issue the follo!in$ command for each ta"e:
mtlib &l 'library de"ice name( &C &s ))** &t ))+* &, aaannn
'lirary de"ice name can be found by runnin! -tpconfi! &d. on the master
ser"er(
Where &l is the libraryCs filenameA /C indicates that you !ant to chan$e the
cate$ory of a #olumeA /s is the startin$ #alueA /t is the #alue you !ant to
chan$e it to and &, is the #olume serial number to be chan$ed
This !ill chan$e the state of cartrid$e aaannn from bein$ in the library to
e;ected and mo#e it accordin$ly
1or further details of the mtlib command you can issue mtlib /%A !hich !ill
"roduce a list of all the "ossible "arameters and their meanin$
'.- %rive pro0lems
If it is a dri#e "roblem then the S3SI commands to mani"ulate the dri#e must
be issued from the master%media ser#er 1or e4am"le to re!ind a cartrid$e
and "re"are it to be unloaded from a dri#e you !ould ha#e to issue the mt &
f 0de"0rmt+ re1off command
The most common "roblem is !hen a cartrid$e does not e;ect from a dri#e
Page 1' of 61
Contents
'.-.1 I!enti/ying stuc7 cartri!ges
3om"are the information from the robtest drstat <see belo! for details=
command to that from the "moprcmd command If drstat lists a cartrid$e as
bein$ in the dri#e and "moprcmd does notA and there are no acti#e ;obs usin$
the cartrid$eA then it is safe to assume that the cartrid$e is stuc. To confirm
this it is also !orth chec.in$ the lo$ files for S3SI errors related to the dri#e
These can indicate !hen the failure occurred and hel" identify the cause to
the en$ineers
If you sus"ect a cartrid$e to be stuc.A lo$ onto the master%media ser#er !hich
uses the dri#e and issue the mt &f 0de"0xxxxx re1off command from the root
accountA !here 0de"0xxxxx is the file name for the dri#e This should re!ind
the cartrid$e and e;ect it ready for the robot If this command fails it could be
because the cartrid$e has already been re!ound or there are "roblems
communicatin$ !ith the dri#e To determine !hich you need to lo$ onto the
mount ser#er and use the robtest dismount o"tion If this fails then the
o"erators !ill ha#e to "o!er:cycle the dri#e to force it to e;ect the cartrid$eI
this should be follo!ed by the robtest dismount If no "ro$ress is made after
all this then it is time to $et the en$ineers in#ol#ed
'.1 "i0rary pro0lems
If there are "roblems !ith the libraryA ie dri#es in '>0 and cartrid$es not
bein$ mountedA then attem"t a manual mount of a cartrid$e
1or e4am"le to mount a cartrid$e you !ould run robtest <see abo#e for
details= and then run the a""ro"riate command
' useful command is the libstat command This sho!s the library status on
the first lineA !hich should be:
state: Automated Operational
State
1urther do!n the out"ut the status of the I%& station%ho""er is dis"layed and
!hether it is full and reDuires em"tyin$ The status should be:
input/output status: All convenience input
stations empty
All convenience output stations empty
'nother useful command is drstatA !hich lists information for all the dri#es
1rom this it is "ossible to tell if a cartrid$e is in the dri#e and its identity
Cri3e % infor-ation+
dri3e nu-ber+ %
de3ice na-e+ ,,%'!,B1A,2
de3ice nu-ber+ ,;2,%1&,
de3ice class+ ,;11 6 %'!, 2odel B1A/ot)er
de3ice categor*+ ,;,,,,
-ounted 3olser+ =none>
-ounted categor*+ ,;,,,,
Page 16 of 61
Contents
de3ice states+ Ce3ice installed in A0L9
Ce3 is a3ailable to A0L9
ACL is installed9
In this e4am"le you can see that there is no cartrid$e in dri#e 3
'.2 mtli0 comman!s 6 %escription an! use/ul
comman!s
The mtlib command allo!s the I5- library to be interro$ated and
mani"ulated by a user
To dis"lay all "ossible o"tions use the mtlib /% command
To find out !hat the lo$ical de#ice number is for a library dis"lay the
0etc0ibmatl.conf file and the library !ill be at the end of the file
mtlib &l 2343c &5, &, xxxnnn
This !ill dis"lay the status of a cartrid$eI ie is in the library
mtlib &l 2343c &5$
This !ill dis"lay all mounted cartrid$es and the library de#ice number it
is in
mtlib &l 2343c &6
This !ill dis"lay all the de#ices and their numbers
mtlib &l 2343c &5
This !ill dis"lay the status of the library
( A6IC ibrary Information
Note6 $ost o% our ADIC/-RA3 +ibraries are A$L@s; and these hae a
+ibrary manager =hich understands the dasadmin commands e2p+ained
be+o=. Ne=er ADIC/-RA3 +ibraries do not hae a +ibrary manager. Aou
cannot use dasadmin commands on these; so =i++ hae to re+y on robtest
and Net#ac)up %or +ibrary in%ormation.B
(.1 Using ro0test to interrogate the ro0ot
Please refer to the instructions for the I5- robot abo#e in section 31 as the
"rocess is the same
(.2 !asa!min 9only to 0e use! on A%IC li0raries:
LI0TD6 List Drie 0tatus
This command dis"lays the dri#e status for all clients or a s"ecific client
dasadmin +d; if the robot has more than 1, dri#es use dasadmin +d2
Page 1( of 61
Contents
E$
dyb.u"+1 T dasadmin ld
listd for client: successful
dri#e: /0I>E1 amu dri#e: +1 st: 9P ty"e: 7 sysid: client: dycase+1
#olser: cleanin$ + cleanGcount: 1+
#olser: cleanin$ + cleanGcount: 2*
#olser: cleanin$ + cleanGcount: 2(
#olser: cleanin$ + cleanGcount: 2(
dri#e: /0I>E, amu dri#e: +, st: 9P ty"e: 7 sysid: client: dyes"!"1
#olser: cleanin$ + cleanGcount: 16
dri#e: /0I>E) amu dri#e: +) st: 9P ty"e: 7 sysid: client: dyes"!"1
dri#e: /0I>E2 amu dri#e: +2 st: 9P ty"e: 7 sysid: client: dyes"!"1
#olser: cleanin$ + cleanGcount: 1,
dri#e: /0I>E( amu dri#e: +( st: 9P ty"e: 7 sysid: client: dyes"!"1
dri#e: /0I>E* amu dri#e: +* st: 9P ty"e: 7 sysid: client: dy#sisb1
#olser: cleanin$ + cleanGcount: )
dri#e: /0I>E1+ amu dri#e: 1+ st: 9P ty"e: 7 sysid: client: dy#sisb1
dri#e: /0I>E11 amu dri#e: 11 st: 9P ty"e: 7 sysid: client: dyb.u"+1
#olser: /E1,*, cleanin$ + cleanGcount: 12
dri#e: /0I>E13 amu dri#e: 13 st: 9P ty"e: 7 sysid: client: dyneb.+1
#olser: cleanin$ + cleanGcount: ,
dri#e: /0I>E16 amu dri#e: 16 st: 9P ty"e: 7 sysid: client: dyneb.+1
dri#e: /0I>E1, amu dri#e: 1, st: 9P ty"e: 7 sysid: client: dyneb.+1
#olser: /E36+) cleanin$ + cleanGcount: 2
To disp+ay the drie +ist %or a speci%ic c+ient
dasadmin +d dyb)up&1
dyb.u"+1 T dasadmin ld dyb.u"+1
listd for client: dyb.u"+1 successful
#olser: /E1,*, cleanin$ + cleanGcount: 12
Page 1 of 61
Contents
#olser: /E83!3 c+eaning & c+eanCcount6 16
das options6
/is"lay /escri"tion of Parameter
dri3e /ri#e number
st /ri#e status 9P or /&W7
t*pe /ri#e ty"e
s*sid 0eser#ed
client 3lient name allocated to the dri#e
3olser -ounted #olume on the dri#e
cleaning 'ctual cleanin$ acti#ity
+: no clean acti#ity on the dri#e
1: cleanin$ media mounted on the dri#e
clean count 7umber of mounts until the ne4t cleanin$ inter#al
'LL&3/: 'llocate dri#e to different client
dasadmin a++ocd DRID42 3* c+ient
D78N
To see the ran$e of ta"es assi$ned use the Bdasadmin 1o+srangeE command
Note+ This command returns a list of #olsersA !hich are accessible to the s"ecified
client !ithin the reDuested #olser ran$e
dasadmin 1o+srange begin3olser end3olser count #client na-e$
e$ dasadmin 1o+srange FDT*216G FDT*22&G ,
Parameter /escri"tion of Parameter
begin3olser The be$in#olser s"ecifies the first #olser in the
ran$e
end3olser The end#olser s"ecifies the last #olser in the
ran$e
count S"ecifies the number of #olsers to re"ort !ithin
the ran$e <This number can be lar$er than the
actual number reDuired=
client na-e If the client name is s"ecified the #olser ran$e is
chec.ed for that client onlyA if none is s"ecified
Page 1! of 61
Contents
then all are chec.ed
(.$ Mount a Volume in a tape drive
This command mounts a #olume on a dri#e from the library
dasadmin mount Ht mediaHtype o+ser drie
e$ dasadmin mount Ht 3'!& DT*216 DRID4"
-edia6t*pe S"ecifies the ty"e of media you are usin$ e$
3,*+
3olser S"ecifies the s"ecific media you !ish to be
mounted e$ /TP21)
dri3e S"ecifies the dri#e number on !hich the media
is to be mounted e$ /0I>E6
(.' Dismount a Volume from a tape drive
This command dismounts a #olume and re"laces it bac. into the library
dasadmin dismount Ht mediaHtype o+ser
e$. dasadmin dismount Ht 3'!& DT*216
3,*+
3olser S"ecifies the s"ecific media you !ish to be
dismounted e$ /TP21)
(.( Media Movements - Inserting Media
9nli.e the I5- libraryA it is necessary to issue a command to insert media
from the in"ut ho""er into the library This command !ill insert #olumes
from a s"ecific insert area into the library area
dasadmin insert Ht mediaHtype area
e$ dasadmin insert Ht 3'!& I&1
3,*+
area S"ecifies the area !here the ta"e<s= !ill be
Page 2, of 61
Contents
inserted e$ I+1
(.- !e"ting Media from t#e ADI$ Li%rar&
This command !ill mo#e #olume<s= to the e;ect area to be remo#ed from the
library
dasadmin e;ect <:c= :t -edia6t*pe 3olser6range area
e$ dasadmin eIect 5c Ht 3'!& DT*216;DT*22& 4&1
6c Tells /'S to remo#e the #olser from the catalo$
3,*+
3olser6range S"ecifies one or more #olsers to be e;ected
area S"ecifies the area !here the ta"e<s= !ill be
e;ected e$ E+1
Page 21 of 61
Contents
(.1 %rive )ro0lems
0obotic "roblems are more strai$htfor!ard to identify than dri#e "roblems
/etermine !hat ty"e of robot you are usin$::
Predominantly !e use '/I3 robots at the moment <'-Ls=
(.2 AM"s
The amu is a "cA !hich sits on the front of the robot and handles all reDuests
for action from all the bac.u" ser#ers The ta"e dri#es sit in the robotA but
there is no direct electrical connection bet!een the robot and the dri#es The
robot arm actions reDuests $i#en to it by the amu The rac.in$ is used to store
the ta"es The amu is contacted usin$ standard i" addressin$ '/I3 "ro#ide
a binary that can be used to communicate !ith the libraryA and that command
is dasadmin ' full list of dasadmin commands and their use can be found by
ty"in$ dasadmin :O
(.4 Common )ro0lems
The dries are in ADR mode
3hec. the messa$es%syslo$ for errors Then either:
Pin$ amu from the ser#erA the i" address !ill be found in %etc%hosts
If there is no res"onse from either ser#er callout '/I3 This could also be a
net!or. "roblem such as the net!or. connecti#ity to the bo4 has been lost In
!hich case there is nothin$ to be done until the net!or. has been restored
If you can "in$ the amu Then issue the command:
Udasadmin +d
!hich !ill tal. to the amu and Duery the status of the dri#es If this han$s or
returns Vune4"ected res"onse code recei#ed from the amuW : callout '/I3
If the dasadmin +d returns a list of dri#esA then select a ta"e from media
mana$er !hich does not ha#e a time assi$ned #alue
Try and mount it in a dri#e 9se the command: :
<dasadmin mount 5t 3'!& 3olser DRID4n
If this returns une4"ected res"onse code recei#ed from the amuA then you
callout '/I3 If it !or.s then either use reset !ith dri#e controlA or
<mt H% /drie o%% A
and
Udasadmin dismount 5t 3'!& "olser
(.15 )oints #o +ote
Some more obser#ations on "roblems encountered !ith '/I3 robots
Page 22 of 61
Contents
The t+md daemon netbac.u" uses to tal. !ith the robotA !ill "eriodically test
the state of the ta"e dri#es If dri#es are in '>0 mode <not /&W7:TL-=
then !hen the daemon $ets a "ositi#e res"onse the dri#es come bac. u"
automatically
&n the syslo$ you may see the messa$e Nrobot encountered an error handlin$
a #olumeNA this could cause intermittent "roblems Lou may be able to $et
a!ay !ith free@in$ the ta"e or if it is scratchA to mo#e some from another
"ool <see e4it status *) for detailed instructions on mo#in$ media=
Then call u" '/I3 The command to free@e a ta"e is
<bpmedia H%reeJe He 3olser
If the robot door is o"ened it !ill sto" the robotA and !ill need to call '/I3
out to restart the machine
dasadmin +d2 may ha#e to be used to #ie! ta"e dri#e numbers abo#e 1,
5y usin$ moprcmd it is "ossible to determine the 97IP de#ice files
associated !ith the 7et5ac.u" /ri#e inde4 When the out"ut a""ears it
contains the dri#e inde4 numberA and the de#ice file associated !ith it
3hec. !hether a ta"e is stuc. on a dri#e: :
Is there an RD0N on the drie inde2K
If so it means that 7et5ac.u" can read the ta"e labelA and the dri#e is
functionin$ to some de$ree
9se dri#e control to reset the dri#e This effecti#ely does Vmt Ff X offW and
tells the robot to "ut the ta"e a!ay If this does not !or.A ma.e sure bac.u"s
are not usin$ that ta"e dri#eA and issue mt 5% %de#ice:file o%% If this ta.es you
bac. to the "rom"t <ie It has !or.ed= then instruct the robot to "ut the ta"e
a!ay by issuin$ the dasadmin dismount commandA then try the dri#e out
a$ain If the dri#e fails a$ainA then an en$ineer !ill ha#e to be calledA because
it may be that the dri#e can read the header label but not "osition on the ta"e
The seDuence of e#ents !ould be::
-t :f %de#icefile off
/asadmin dismount :t 3,*+ #olser
0un a bac.u"A and chec. on ;ob monitor to see if the ta"e has "ositioned F
If the mt 5% %de#ice:file o%% re"orts an error then call an en$ineer Ta.e the
dri#e do!n in 7et5ac.u" until the en$ineer has dealt !ith it
I% there is no RD0N on the drie
1irstly interro$ate the robot to determine !hether the robot has been "uttin$
ta"es in the dri#e 9se dasadmin +d to list the dri#es
If there is a ta"e on the dri#eA chec. that the ta"e isnCt ;ust sittin$ on the Vli"W
of the dri#e by instructin$ the robot to "ut the ta"e a!ay This can be done by
issuin$ the command
<dasadmin dismount Ht 3'!& "olser
if the command !as successful then the ta"e !ould be on the li"
Page 2% of 61
Contents
If the robot re"orts that the dri#e cannot be unloadedA then issue
Umt 5% %de#ice:file o%%
If this ta.es you bac. to the "rom"t <ie It has !or.ed= then instruct the
robot to "ut the ta"e a!ayA then try the dri#e out a$ain If it doesnCt !or.A try
<mt 5% /deiceH%i+e status
then call out an en$ineer It is !orth chec.in$ the syslo$ a$ainA because the
"re#ious actions may ha#e forced 97IP to re"ort an I& error
(.11 Li%rar& Pro%lem
If there are "roblems !ith the libraryA ie dri#es in '>0 and cartrid$es not
bein$ mounted 3hec. the messa$es lo$ on the master ser#er for the
follo!in$:
Eeb 1, ,!+%!+%, d*bkup,1 tl-d?2'('%@+ ?IC !(,6, dae-on9error@ 0L2#,$
dis-ount failure for 3olser SCI1' on dri3e C5IV812" dKerrno 4 1," The
A$7 1as unable to communicate 1ith the robot.
Eeb 1, ,!+%!+%, d*bkup,1 tl-d?1661@+ ?IC 16,1%6 dae-on9error@ 0L2#,$
going to COLN state" status8 9obot hard1are or communication error
Eeb 1, ,!+%!+'' d*bkup,1 tl-d?262((@+ ?IC !6!66' dae-on9error@ 0L2#,$
dis-ount failure for 3olser SCI&(, on dri3e C5IV811" dKerrno 4 1," The
A$7 1as unable to communicate 1ith the robot.
Eeb 1, 1,+%%+1% d*bkup,1 tl-d?1661@+ ?IC 61(1! dae-on9error@ 0L2#,$
dri3e C5IV811 #de3ice ,$ is being COLN8C" status+ 9obotic dismount
failure
&n seein$ these messa$esA initiate a call !ith the #endor F '/I3
- Accessin! Net#ackup
-.1 +&U '.(
When you lo$on to a ser#er : it is only "ossible to use this on a 7etbac.u"
-aster ser#erA not a client : "lease access netbac.u" usin$ one of the
follo!in$ methods: :
>ia the #t1++ "anels
<bpadm
from the toolbar::
Select the start button F "ro$rams F >eritas 7et5ac.u" F 7et5ac.u"
administration F from there access the reDuired "anel
clic.in$ your shortcut icon <if one has been set u"=
Aou may a+so =ish to amend your pro%i+e to inc+ude the netbac)up
directories in your *AT9.
Page 2& of 61
Contents
Another method o% accessing the pane+s =ithout the need to amend
pro%i+es; as root; su 5 storage =hich =i++ gie you the pro%i+e necessary to
per%orm the aboe.
1 Net#ackup 6aemon Problems
There are si4 main 7et5ac.u" daemonsA !hich run constantly
7et5ac.u" :: b"rd
5"dbm
-edia -ana$er:: #md
Ltid
'#rd
Tlmd%tlhd
These daemons run on the ser#er onlyA and not on any of the clients &n sla#e
ser#ers only the media mana$er daemons run
&f these the most si$nificant are b"rd and ltid
5"rd starts b"dbmA and ltid starts #mdA a#rdA and tlmd or tlhd
T+md and t+hd are robotic daemons; =hich are di%%erent depending upon
the type o% robot used. T+md re%ers to ADIC robots; =hi+st t+hd re%ers to
I#$ robots.
1.1.1 %aemon %escriptions
#prd : &n master ser#ers this daemon handles reDuests for bac.u"s and
restores and scheduled bac.u"s
#pdbm F &n master ser#erCs b"dbm handles all the confi$urationA error and
file databases
Ltid F &n master and sla#es this daemon controls the reser#ation and
mana$ement of #olumes
Ard F &n sla#es and masters "erforms automatic #olume reco$nitionA ie
bein$ able to reco$nise a #olume that has a label on the ta"e
Dmd F >olume mana$er daemon mana$es the #olume database containin$
details about ta"e usa$e
T+md/t+hd F robotic control daemons "erform robot handlin$
1.1.2 Starting +et&ac7up
The scri"t bp.)i++Ca++ in /usr/open/netbac)up/bin/goodies !ill .ill off all
netbac.u" daemons under normal conditions If after runnin$ bp.)i++Ca++
Page 2' of 61
Contents
there are daemons hun$ <usually b"sched "rocesses= these should be .illedA
usin$ .ill or .ill F* if necessary
'lternati#elyA if you e4"erience "roblems shuttin$ do!n 7etbac.u" usin$ the
abo#e scri"t use nbuC)i++ scri"tA !hich can be found in /usr/open/btscripts.
1.1.$ Starting up in!ivi!ual +et&Ac7up !aemons
To start u" 7etbac.u" daemons you must al!ays be in root
To chec. for 7etbac.u" daemonsA issue bpps 5a; !hich can be found in
directory /usr/open/netbac)up/bin
The 7etbac.u" startu" scri"t netbac)up is in
/usr/open/netbac)up/bin/goodies.
If bprd has crashedA issuin$ /usr/open/netbac)up/bin/initbprd !ill start it
a$ain
If bpdbm has crashed b"rd !ill "eriodically start it u" a$ainA
If md has crashed then it can be started by issuin$::
<madm
sS s"ecial actions
iS initiate -edia -ana$er >olume /aemon
If ;ust t+md has not startedA or has crashedA this can be started usin$ ./t+md
This is the recommend command !hen startin$ 7etbac.u"
If ardA has crashedA issue /usr/open/o+mgr/bin/stop+tid to sto" ltidA
and then /usr/open/o+mgr/bin/+tid to restart ltid
1.2 Shut!o3n +et0ac7up
The scri"t bp.)i++Ca++ in /usr/open/netbac)up/bin/goodies !ill .ill off all
netbac.u" daemons under normal conditions If after runnin$ bp.)i++Ca++
there are daemons hun$ <usually b"sched "rocesses= these should be .illedA
usin$ .ill or .ill F* if necessary
'lternati#elyA if you e4"erience "roblems shuttin$ do!n 7etbac.u" usin$ the
abo#e scri"t use nbuC)i++ scri"tA !hich can be found in /usr/open/btscripts.
1.2.1 )ro0lem resolution
It is rare that there are "roblems !ith the 7et5ac.u" daemons
3ommon "roblems are: :
The daemons !erenCt started after a rebootA in this case lo$on to the
ser#erA and su: rootA then issue ./netbac)up from
/usr/open/netbac)up/bin/goodies.
Page 26 of 61
Contents
The hostname on the bo4 has chan$ed This should be re#ie!edA !ith
7et5ac.u" Su""ort and Ser#ice /eli#ery
Tlmd can crash if there has been a se#ere robot "roblemA to start this
daemon lo$on to root and issue: ./t+md from /usr/open/o+mgr/bin
Tlmd can crash if there is a net!or. "roblem
If after tryin$ the abo#e 7etbac.u" does not startA then it !ill be necessary to
callout 7et5ac.u" su""ort
2 Netbackup Acti"ity o!s and process
information
2.1 Intro!uction
This directory <%usr%o"en#%netbac.u"%lo$s= is !here detailed acti#ity lo$s !ill
be "laced on the 7et5ac.u" client bo4 if certain sub:directories e4ist These
sub:directories should only be created if une4"lained "roblems are occurrin$
!ith the 7et5ac.u" "roduct and more information is reDuired to isolate the
"roblem 1or further information on >eritas 7et5ac.u" Lo$$in$ "rocedures
see ISL%&PS%51*6 at
Warning: 0ome o% these +ogs can potentia++y gro= ery +arge; and shou+d on+y be
enab+ed i% une2p+ained prob+ems e2ist.
2.2 File system /ull pro0lems
In the e#ent of filesystems listed belo! becomin$ full or bein$ re"orted as
o#er a s"ecified "ercenta$eA this should be addressed !ith the immediate
remo#al of lo$ directories under /usr/open/netbac)up/+ogs; lea#in$ only
admin and userGo"s
/usr/open/ mountpoints L 1&&M
If the situation arises !here the mount"oint for 7et5ac.u" reaches or
e4ceeds 1++JA then the follo!in$ action can be ta.en to try and reduce the
utilisation
Lo$on to the ser#er and su to root and issue the follo!in$ command::
b"ima$e :cleanu" :allclients
'n alternati#e is to bounce the daemonsA !hich !ill force the cleanu" "rocess
to start This has been added to the crontab on all 5097IP master ser#ers to
run it re$ularly
Page 2( of 61
Contents
2.$ Full "ist o/ +E#&AC;U) )rocesses
Here are descri"tions of 7et5ac.u" "rocesses:
bprd
:reDuest daemon
:can be terminated and initiated from the admin interfaces
:res"onds to client and administrati#e reDuests
:restores
:bac.u"s
:archi#es
:Nlist files bac.ed:u" or archi#edN
:manual%immediate bac.u"s
:reread confi$uration database
bpsched
:bac.u" scheduler
:started by b"rd on user directed bac.u"s and archi#es
:started by b"rd on immediate%manual bac.u"s
:started by b"rd e#ery NWa.eu" Inter#alN for re$ularly scheduled incremental
and full bac.u"s
:uses information from the class K stora$e unit databases to determine !hat
clients to startA !hen to start themA and
!hat stora$e unit to !rite bac.u"s%archi#es to
bpdm
:dis. mana$er
:used on stora$e units of ty"e /is.
:started by b"brm on bac.u"s and restores
:durin$ bac.u"s and restoresA one of these is started <on the
ser#er !ith the stora$e unit= for each client bac.u" or restore b"tm
:remo#able media <ta"e= mana$er
:used on stora$e units of ty"e Lo$ical Ta"e
:started by b"brm on bac.u"s and restores
:durin$ bac.u"s and restoresA one of these is started <on the ser#er !ith the
stora$e unit= for each client bac.u" or restore
Page 2 of 61
Contents
:also res"onsible for mana$in$ the media database
:used to dis"lay info in the -edia 0e"orts screen !hen you select -edia List
bpbrm
:bac.u"%restore mana$er
:started by b"sched on bac.u"s%archi#es
:started by b"rd on restores
:durin$ bac.u"s and restoresA one of these is started <on the ser#er !ith the
stora$e unit= for each client bac.u" or restore
:res"onsible for mana$in$ both the client and the media mana$er "rocesses
uses error status from both to determine ultimate
status of bac.u" or restore
bpdbm
:database mana$er
:mana$es classA confi$%beha#iorA stora$e unitA and error /5Ms
:started by the inetd<1-= "rocess
bpcd
:Nclient daemonN
:used on clients <and remote ser#ers= to initiate other "roduct "ro$ramsA
!ithout reDuirin$ %rhosts entries for the ser#er on each client
:started by the inetd<1-= "rocess
bparchie
:command:line "ro$ram on clients to initiate archi#es
:communicates !ith b"rd on ser#er
bpbac)up
:command:line "ro$ram on clients to initiate bac.u"s
bpb)ar
:"ro$ram used on standard clients to $enerate bac.u" ima$es
:not used directly by client users
Page 2! of 61
Contents
bp+ist
:command:line "ro$ram on clients to initiate file lists
bprestore
:command:line "ro$ram on clients to initiate restores
tar
:"ro$ram used on standard clients to restore bac.u" ima$es
bp
:menu user interface for bac.u"sA archi#esA and restores
2.' :eneral 6ri"e Testin! and )ault
9esolution
When a media hard!are "roblem is identified by &"s 'nalysts they !ill
attem"t resolution themsel#esA !ithout in#ol#in$ 7et5ac.u" Su""ort This
resolution !ill include callout of onsite su""ort <e$ 7'33 &"sA 3E=A if
reDuiredA and subseDuent liaison to resol#e the fault Ho!e#erA 7et5ac.u"
Su""ort may be contacted for more detailed "roblem analysis if the "roblem
is not immediately identified as a ta"e%dri#e%library fault
2.( %ealing <ith #ape %rive )ro0lems
The ma;ority of bac.u" ser#ice effectin$ "roblems that !ill occur on
7et5ac.u" !ill be because of Ta"e /ri#e failures of one sort or another It is
im"ortant then to understand the com"onents in a dri#e "athA and !hich tools
!e can use to dia$nose the "roblem
When dealin$ !ith dri#e "roblems al!ays chec. first to determine !hether
the "roblem is dri#e or robot relatedA often the error codes are the same
2.- %etermining 3hether pro0lem is %rive or *o0ot.
It is not al!ays $oin$ to be easy to determine !hether the "roblem is that the
dri#e is bro.enA or the robotA but for the most "art you can tell #ery Duic.ly
by loo.in$ at the follo!in$
Page %, of 61
Contents
Is the robot a shared +ibraryK
If it isA are ta"es bein$ mounted on another ser#erO If so it is unli.ely that the
"roblem is robotic
Can you obtain drie statusK
If you cannot you definitely ha#e a robot "roblem
Are the tape dries in ADR modeK
If so it made be a local "roblem !ith either the connection to the robot
Are the tape dries do=n and remain do=n =hen you bring them upK
If so this usually means you ha#e a dri#e related "roblem in the first instance
8hat does the messages/sys+og say
7etbac.u" media mana$ement lo$s useful information to the syslo$A and this
should al!ays be e4amined
't the end of your initial dia$nosis you may not be com"letely sure !hether
the "roblem is robot or dri#e relatedA but if in doubt start loo.in$ at the dri#e
firstA as this is much more common a "roblem
2.-.1 )oints #o +ote
The follo!in$ "oints to bear in mind !hen dealin$ !ith dri#e "roblems are
the result of e4"erienceA and not from methodical dia$nosisA but they ha#e
ha""ened on more than 1 occasionA are not easy to s"ot
4erything seems o) unti+ you ta)e one drie do=n
The H%W en$ineers <usually Sun or SeDuent= ha#e s!a""ed the S3SI cables
around 7et5ac.u" !ill s"ot the ta"e it !ants is in a dri#e and use itA
ho!e#er if the ta"e dri#es are in the !ron$ !ay around then !hen you shut
one do!n the ta"e !ill be "ut in the do!n dri#e This is #ery confusin$ The
resolution is #ery sim"le F chan$e the robot dri#e numbers around and run
sto"ltid and ltid The 7et5ac.u" Su""ort team should do this
LtidA a#rdA #mdA tlmd or tldd !ill not start after a reboot A or after you ha#e
started netbac.u" Loo. in the syslo$ firstA it al!ays tells you !hy they did
not start
: If the de#ice file is not in 97IP then ltid !ill not startA but this is re"orted
in the syslo$
'fter a rebootA it has been .no!n <"articularly on 3om"aD= for the de#ice
files to be renamed This has the same effect as remo#in$ the de#ice files It
is also a ni$htmare tryin$ to ma" the ne! files bac. on There is no easy !ay
to identify !hether this has ha""enedA if an +s 5a+ command is issued on the
directory it should be "ossible to identify if there are ne! de#ice files S3SI
e4tenders com"licate the situationY
2an* of t)e dri3e proble-s are caused b* SCSI e;tenders" eit)er failing or
)a3ing a glitc)9 5esetting t)e e;tenders -a* be enoug) to fi; a proble-"
)o/e3er fro- our point of 3ie/ /e Fust call out an engineerM it /ill be t)e
engineer /)o /ill deter-ine /)at action is re7uired
Page %1 of 61
Contents
0eboot or 7ot 0eboot
It is by no means clear as to !hen a reboot is reDuired after or durin$ !or. on
a dri#e Some manufacturers su""ort "eri"herals better than others do 5y
deconfi$urin$ the S3SI bus on SeDuent bo4es !e can do most o"erations
!ithout reDuirin$ a reboot 3om"aD also are fairly robust HP and Sun are
DuestionableA althou$h Sun should not reDuire a reboot under normal
circumstances If the S3SI cable is un"lu$$ed from the bac. of the host
machineA then most times you !ill ha#e to reboot the machine If you cannot
use a de#ice after it has been fi4edA al!ays see. $uidance from TS8 before
arran$in$ to reboot a bo4
2.1 %evice Con/iguration Utility 6 #pcon/ig
T"confi$ is the netbac.u" utility to confi$ure de#ices used by 7et5ac.u"
1ull details about its use are co#ered in the >eritas 7et5ac.u" manuals
This utility !ill mostly be used by Stora$e 'dministratorsA but there is one
command that the &'s !ill find useful:
tpcon%ig Hd A :this lists all the de#ices confi$ured in 7et5ac.u"
4 6econfi!ure09econfi!ure ;e5uent 6ri"es
4.1.1 %econ/igure
Prior to deconfi$urin$A
Ensure that the dri#e in Duestion is /&W7 in 7etbac.u"A as other!ise
the dri#e !ill be acti#ely "olledA "re#entin$ deconfi$
To "erform this tas. root access is reDuired
3hec. on /e#ice -ana$er for the de#ice name by !hich the dri#e in
Duestion is set or use tpcon%ig 5+ !hich dis"lays the same information
In brief the ste"s are:
1 3hec.
dum"conf Z $re" tc
2 /econfi$ure
dect+ Hd tcn
dect+ Hd scsibusnn
3 0econfi$ure
dect+ Hc 1cicn
6 3hec.
dumpcon% N grep tc
Page %2 of 61
Contents
4.1.2 #o per/orm the %EC+FI.
List current confi$ usin$: dumpcon% N grep tc
Enter Ndect+ Hd tcnN !here n H dri#e number e$ tc2
<this can be found by usin$ t"confi$A list dri#e confi$uration=
If all is !ellA you !ill see a brief statement confirmin$ that the dri#e in
Duestion has been deconfi$ured
N de3ctl 6d tc,
de3ctl+ deconfiguring tc, fro- scsibus1
The out"ut from the abo#e command !ill $i#e the scsibus number
Enter Ndect+ Hd scsibusnnN !here n H scsibus number e$ scsibus1(
If all is !ellA you !ill see a brief statement confirmin$ that the scsibus
in Duestion has been deconfi$ured <see belo!=
N de3ctl 6d scsibus1
de3ctl+ deconfiguring scsibus1 fro- 7cic'
?ee" a note of the dri#enameA scsibus and Dcic <or fcbr= for reconfi$urin$
The en$ineer should no! be able to carry out any !or. necessary
4.1.$ *econ/iguration
Prior to reconfi$urin$A
If you ha#e mis"laced the deconfi$ detailsA you can find them in the rele#ant
.tlo$ #ia %usr%adm%.tlo$%yyyy%mm%dd : e$ %usr%adm%.tlo$%1***%1+%+6 Lou
should find out"ut similar to the follo!in$:
N%(fc'21 16+1(+'% tolog/note p'! de3ctl 6dC tc1
N%(fc'21 16+1(+'% tolog/note p'! NA28 CEG0IP8
C8VNH2 HNI0 ELAGS OnBHS OnC8VIC8
N%(fc'21 16+1(+'% tolog/note p'! deconfig+ tc1 tc
1 ,;,,,,,,,, S scsi scsibus22
N%(fc'22 16+1(+'& tolog/note p'! de3ctl+ deleted tc1+ t*pe+ tc+
de3nu- ,;1
N%(fc'%b 16+1+1! tolog/note p6%' de3ctl 6dC scsibus22
N%(fc'%b 16+1+1! tolog/note p6%' NA28 CEG0IP8
C8VNH2 HNI0 ELAGS OnBHS OnC8VIC8
N%(fc'%b 16+1+1! tolog/note p6%' deconfig+ scsibus22
scsibus 22 ,;,,,,,,6, S2 -scsi 7cic6
Page %% of 61
Contents
N%(fc'%c 16+1+2, tolog/note p6%' de3ctl+ deleted scsibus22+
t*pe+ scsibus+ de3nu- ,;16
This information can also be listed usin$ the ktmes! co--and
4.1.' #o per/orm the *EC+FI.
Enter Ndect+ Hc 1cicnN !here n H Dcic number e$ Dcic13
If all is !ellA you !ill see a brief statement confirmin$ that the de#ices ha#e
been found as in the e4am"le belo!
N de3ctl 6c 7cic'
de3ctl+ Eound scsibus1" tc,
To chec. !hat the current settin$s are and confirm the abo#e commandsA
enter Ndumpcon% N grep tcN !hich !ill "roduce out"ut similar to the
follo!in$:
tc, tc , ,;,,,,,,,, S scsi scsibus1
tc1 tc 1 ,;,,,,,,,, S scsi scsibus22
tc2 tc 2 ,;,,,,,,,, S scsi scsibus26
tc% tc % ,;,,,,,,,, S scsi scsibus%,
tcp-u; pseudo 6
If the ori$inal settin$s are sho!in$A the !or. has been com"leted
successfully
IfA on usin$ the @dect+ Hc 1cicnM command to reconfi$ureA only one ne!
de#ice sho!sA try a Duery on the found de#ice e$ Mdect+ Hc scsibus1,@
!hich may detect the other de#ice
Note+ I% the system responses dierge in any =ay %rom the e2amp+es gien
aboe; contact 0e1uent T00 -roup.
159esettin! the ;C;I extenders
15.1 vervie3
There is a reDuirement to use S3SI e4tenders to connect some ta"e dri#es
because there is a "hysical limit of 2,m on S3SIA !hich in most com"uter
halls is not enou$h to allo! for full use to be made of e4"ensi#e robotic
libraries
The use of S3SI e4tenders $reatly increases the number of "oints of failure
that can are "resent in the confi$uration It also com"licates "roblem
resolution
Page %& of 61
Contents
N7T48 the resettin! of ;C;I extenders is usually done by the 6ata Centre
Operations teams. <o1e"er the procedure is as follo1s8
1 0emo#e the co#ers at the bac. of the robot Identify the offendin$ dri#e
2 &n to" of the dri#e there is a small L3/ "anel connected #ia a cable
&n this "anelA usin$ the arro! .eysA scroll do!n to N97L&'/ /0I>EN and
"ress N0ET907 This !ill e;ect any ta"e !hich mi$ht be mounted
3 7e4tA $o to the P3 at the end of the silo and send a M?EEPM si$nal as
follo!s:
from the o"en !indo!A clic. on M3&--'7/SM then from the dro" do!n
menuA clic. on M?EEPM
mo#e the cursor to NS&903EN and enter dri#e number F e$ /21A must be
in u""er case
Then clic. on NEPE39TEN
6 8o bac. to the L3/ dis"lay "anel connected to the dri#e and from the
.ey"ad menuA "ress MEM then M2M
Lou should see M1I5LE7M on the dis"lay This indicates that you ha#e
carried out a Mfibre len$thM chec. Then "ress M3M to clear
Ho!e#erA if nothin$ ha""ens after enterin$ M2M then "o!er &11 the dri#e
The &7%&11 s!itch is at the bac. of the dri#e on the ri$ht hand side
Then "o!er bac. &7
, 0eset both S3SI e4tenders
117seful $edia Commands
11.1 &)ME%IA= Free>e? un/ree>e? suspen! or
unsuspen! me!ia
b"media :["arameter\ :e# [mediaGid\
e$
<bpmedia H%reeJe He o+ser >DT*216?
6freeDe 1ree@e s"ecified media id
6unfreeDe 9nfree@e s"ecified media id
6suspend Sus"end s"ecified media id
6unsuspend 9nsus"end s"ecified media id
6e3 S"ecify media id
Page %' of 61
Contents
6) s"ecify host
11.2 #ape Management
< bpadm
Re"orts
$edia
$edia Summary
11.2.1 ;illing )rocesses
< )i++ F* ]PI/ 7umber^
Ubpps Ha
root 12+) 1 + -ar 22 O +:32
%usr%o"en#%netbac.u"%bin%b"rd
root 121( 1 + -ar 22 O 1:3,
%usr%o"en#%netbac.u"%bin%b"dbm
11.2.2 @#155 Comman! "ine
1or "roducin$ readable lists from usually non:readable te4t files
< cd /usr/open/netbac)up/bin/admincmd
< ./bpc++ist 7RACL4 HL or H3
11.2.$ "ogs
These may be in #arious locations de"endant on the "latformA ho!e#er chec.
in the follo!in$ first
/usr/adm/)t+og/ H 0e1uent
/usr/spoo+/adm/
/usr/adm/sys+og/
/ar/adm/sys+og/ H 9*
/ar/adm/messages 5 0un
errpt 5a H AI:
/usr/open/netbac)up/+ogs/bpO
11.2.' <H Comman!
=ho Hb <Sho!s !hen system !as last re:booted=
Page %6 of 61
Contents
11.2.( M# 9SCSI "evel: Comman!s
'll VmtW commands in#ol#e the use of the full "athname of the dri#e de#ice
file They can normally only be issued from root and tal. to the '-9 on a
S3SI le#el
< mt H% /de/rmt/tc&c status
tc+: Waitin$ u" to *+ seconds for ta"e ready
tc+: /e#ice 7ot 0eady <chec. cartrid$e=
%de#%rmt%tc+c: I%& error
< mt H% /de/rmt/tc1c status
%de#%rmt%tc1c: I%& error
tc2: Waitin$ u" to *+ seconds for ta"e ready
tc2: /e#ice 7ot 0eady <chec. cartrid$e=
tc3: Waitin$ u" to *+ seconds for ta"e ready
tc3: /e#ice 7ot 0eady <chec. cartrid$e=
The abo#e VmtW commands !ill !or. only if a ta"e is loaded onto dri#e so
ta"es should be loaded usin$ das commands
129estore :uidance
12.1 &ac7groun!
This section is intended for use by $rou"s !ho may be reDuired to "erform
restores on 9ni4 Systems usin$ 7et5ac.u" There are a number of different
!ays to run a restore These are:
1 1rom the 'dmin 89I
2 1rom the command line interfaceA the b"adm or b" "anels
3 1rom the command line directly
12.1.1 Intro!uction
The restore facility for 7etbac.u" is e4tremely "o!erful Its use must only be
considered !ith sufficient ;ustification by the reDuestor It can be #ery easy to
destroy a 97IP bo4 !ith 7etbac.u"A "articularly if you are restorin$ files
in % or %usr If you are at all unsure about !hat is bein$ reDuested by the userA
Page %( of 61
Contents
then refuse to com"lete the reDuest In the "ast many unnecessary restore
reDuests ha#e been e4ecuted because the customer !as not com"letely sure
!hat !as reDuired to fi4 the "roblem
In short do not be a%raid to as) =hy they re1uire restores.
Where#er "ossibleA the file or files should be restored to an alternati#e "ath
If the user !ishes to restore to the ori$inal directoryA see if you need to
o#er!rite dataA doin$ a restore !ith o#er!rite not allo!ed is much safer and
should be considered as the normal method for "erformin$ restores 'l!ays
im"ress on the reDuestor the im"ortance of "erformin$ restores safelyA e#en if
it means the user has to do some !or.
12.1.2 +et&ac7up
7et5ac.u" bac.s u" data to cartrid$e To reco#er this data is strai$htfor!ard
as lon$ as the correct information is used to initiate the restore "rocess
It must be remembered that a file in 97IP does not al!ays e4istA as it could
be a hard or soft lin. to another real file If 7et5ac.u" is as.ed to bac.u" a
soft lin. it !ill not follo! itY So to ensure that the data is bac.ed u" the tar$et
of the lin. must be s"ecified This also holds true !hen attem"tin$ to restore
the file 1or a hard lin.s the situation is sli$htly different 5ut as hard lin.s
are not commonly used details are not included in this document
If you attem"t to restore a soft lin. then no data !ill be restored Therefore
"ro#idin$ the name of the soft lin. as the file to be restored is !orthless
12.2 *estore in/ormation
There are some "ieces of information that must be "ro#ided to allo! a restore
to be carried out These are as follo!s:
1 The host name of the bo4 from !hich the data !as bac.ed u"
2 The o"eratin$ system of the bo4 Is it a 97IP or an 7T client
3 The date and timeA or dates and times bet!een !hichA the bac.u" !as ta.en
The shorter the time bet!een the start and end the Duic.er the search throu$h
the 7et5ac.u" database !ill beA and hence the restore !ill ta.e less time
6 Should the data be "laced in an alternate locationA or can the data be
o#er!rittenO <Insist the reDuestor s"ecifies !hat is reDuired=
, Should the fileA files or directories be renamed !hen restoredO
) The host name of the tar$et bo4A if different to the source bo4
3ross client restores are only "ossible if both the tar$et and source bo4 are
connected to the same 7et5ac.u" ser#er
3ross client restores must be enabled !ithin 7et5ac.u"
2 The fully Dualified names of the files or directories to be restored
( ' contact number for !hen the !or. is com"lete
Page % of 61
Contents
* Ac1uiring a time+ine:
In order for 5rid$e &"erations to effecti#ely mana$e hi$h "riority restores
<P1 or P2Ms=A ensure !e ha#e a documented time:line "ro#ided by the
reDuester in order to be able to chec."oint our "ro$ress The onus is on
5rid$e &"erations to obtain from the reDuester this information before !e
ta.e on the res"onsibility of the restore Ensure this information is
documented in the 3larify case This !ill allo! oursel#es and the -IT team
to ha#e some #isibility of e4"ected "ro$ress and !ill allo! us to ma.e a
considered ;ud$ement for further escalationN This timeline can only be used
for #ery rou$h $uidance This !ill #ary de"endin$ on the si@e of the database
and !hether locally attached dri#es are used If locally attached dri#es are
used then the restore should ta.e no lon$er than the bac.u" &n a shared
ser#er this should be estimated at 2+J lon$er than the bac.u"
12.$ Me!ia relate! items
I!enti/ying the me!ia reAuire! an! its location
The Duic.est and easiest !ay to identify the media reDuired is to run the restore reDuest and
loo. in the lo$ file
'n alternati#e is to use the b"ima$elist command on the master ser#er
tpadsm'( ) %pimagelist -*
%pimagelist+ unre"ogni,ed option -*
-SA.+ %pimagelist /-media0 /-l1-L1--1-idonl&0
/-d mm/dd/&&&& ##+mm+ss0 /-e mm/dd/&&&& ##+mm+ss0 /-#oursago #ours0
/-2e&3ord 42e&3ord p#rase50
/-"lient "lient6name0 /-server server6name0
/-%a"2upid %a"2up6id0 /-option option6name0
/-"lass "lass6name0 /-"t "lass6t&pe0
/-rl retention6level0
/-sl s"#ed6la%el0 /-st s"#ed6t&pe0
/-M master6server7770 /-v0
The class nameA client nameA start and end dates <and "ossibly times= are the minimum
reDuirements to $et a com"rehensi#e media list If you can s"ecify more information it !ill be
a more accurate list
bpimage+ist 5media 5d &(/&1/2&&2 He &(/&'/2&&2 5c+ient tpedm&1H%e 5c+ass 7RACL4
This !ill "ro#ide a list of all the cartrid$es used for &0'3LE class bac.u" bet!een the dates
s"ecifie for the TPE/-+1 client
Page %! of 61
Contents
12.' *estore processes
If a restore is reDuired to resol#e a ser#ice affectin$ "roblem !ith a
"roduction system then a "roblem record should be raised and can be used to
initiate and document the restore
Note+ Please note that all &"erations run restores !ill use the o#er!rite H no o"tion
as a default This is to ensure that customer data is not inad#ertently o#er
!ritten Therefore all restore reDuests !here the data is to be restored to its
ori$inal location the files affected !ill ha#e to be deleted before the restore
can com"lete
If a restore is reDuired as "art of some testin$ or de#elo"ment !or. then a
chan$e record should be raised This !ill allo! the im"lementors to "lan
their !or. effecti#ely
The record !ill be used to trac. the "ro$ress of the restoreA to deal !ith any
unforeseen "roblemsA ie cartrid$es not a#ailableA and reDuest the a""ro"riate
access 0estores must be run from an account that has the correct access le#el
to the files and directoriesA usually the root account is used
If it is for a number of files or directories then a list file should be created
This list file should contain each fully Dualified file nameA one on each line
This file can then be used as an in"ut file for the restore and sa#e a lot of
!or. for the &"erations team
Warning: The +ist %i+e must not contain any b+an) +ines or e2tra b+an) spaces at the
end o% the %i+e names. I% it does the restore process =i++ %ai+ =ith an Pina+id +ine
+engthE message and an 42it 0tatus 1"" error code.
If the files need to be renamed indi#idually then a rename list file should be
created This file must contain the follo!in$ synta4:
change originalKfull*K7ualifiedKfileKna-e to ne/Kfull*K7ualifiedKfileKna-e
&ne line for each rename must be s"ecifiedI no !ildcards or substitution can
be done
Warning: The same restrictions app+y to this %i+e as to the +ist %i+e.
The dates bet!een !hich the bac.u"s !ere done must be su""liedA if
"ossible 5y s"ecifyin$ a s"ecific date ran$e the search time throu$h the
7et5ac.u" database !ill be reduced
12.'.1 )reparation /or *estore
5efore in#o.in$ the b" utilityA as. yourself the follo!in$ Duestions
/o I need root to "erform the restore
'm I sure that the host name is correct for the restore <"articularly im"ortant
for E1+. bo4es=
/o I need to be on the client to do the restore or can I do it from the ser#er
Page &, of 61
Contents
Is the client in a H' confi$uration
Is the restore reDuest for a ra! "artition or a filesystem file
Has the user reDuested a restore to an alternati#e "ath and if so !hat do you
need to do about lin.s
&nce you are sure you .no! the ans!er to each of the abo#eA then "roceed
12.'.2 )er/orming the *estore
Lo$on to the correct -aster bac.u" ser#er This assumes you are doin$ a
Ser#er directed restore
12.'.$ "ogon to *oot
7et5ac.u" uses standard 97IP file "ermissions to control access to files so
unless you ha#e read%!rite "ermissions to the file you !ill need to use root
0emember that you may not e#en see the file in 7et5ac.u" if you do not
ha#e the correct "ermissions 9sin$ root !ill ensure that you can al!ays see
the client
12.'.' Invo7e 0p
9se %usr%o"en#%netbac.u"%bin%b"
't this "oint you select the restore menuA and then you !ill be "resented !ith
o"tions to restore from different .inds of bac.u"s If you are restorin$ from
normal filesystem bac.u"sA then select restore from bac.u"s If you are
restorin$ from ra! "artitions then select restore from ra! If you are restorin$
7T then select restore ms:!indo!s
*rimary 7ptions $enu. Initiating a restore
't this screen you !ill be able to ma.e all the selections needed before
actually initiatin$ the restore
Lou can chec. all the o"tions are correct before you initiate the restore
Source client%/estination 3lient: both these entries must be the sameA if not
you may restore a file from one machine to another -a.e sure that the
clientname is as .no!n to 7et5ac.u" !ith all rele#ant suffi4es <ie 5%e=
/ate 0an$e: ensure that you ha#e the correct date ran$e and that it is as
narro! as "ossible When 7etbac.u" searches its database it could be
uncom"ressin$ lar$e amounts of data about bac.u"s &b#iously the !ider the
search the more time it !ill ta.e to do this Lou also run the ris. of fillin$ the
%usr%o"en# dis. "artition
1ile Path: s"ecify the file "ath you reDuire to search on
Page &1 of 61
Contents
S"ecify 'lternati#e Path : use this if you !ish to restore to an alternati#e
directoryA care must be used !ith this o"tion: : remember that the Vrestore
fromW "rom"t reDuires you to enter the "attern that !ill be used to match for
restores It must be the directory you !ish to restore or at a lo!er directory
"ath le#el than all the different "aths you !ish to restore E$ If you !ished
to restore %usr%o"en#%"atchesA and %usr%o"en#%#ersion The restore from
"rom"t may default to %usr%o"en#%"atches%A this !ould mean that you could
end u" restorin$ "atches to a different directoryA but #ersion !ould be
restored to the ori$inal location The correct location for the restore from
!ould be %usr%o"en# 'lso !hen you s"ecify an alternati#e "athA the restore
!ill create a "ath from that le#el onA so a restore usin$ restore from as
%usr%o"en# and restore to as %tm" !ill create %tm"%"atches directory
Set your directory de"th for searchin$A this defaults to 1A if you ha#e this at
too lo! a le#el it may seem that the file is not bac.ed u" 5ecause it !ill only
sho! files or directories to that le#el Settin$ the le#el to @ero means you !ill
see e#erythin$ do!n from the "athA but be careful !ith its useA as it can $i#e
you masses of data to loo. at
When you are selectin$ files and directories you are searchin$ the 7et5ac.u"
database so this may ta.e some time 'lso be a!are you can drill throu$h
directories usin$ @oom in and outA this can be useful if you are searchin$ for a
file and the user is unsure !here it !as
If you need to you can build u" a restore ;obA for instance you may select
some files for restoreA chan$e your "ath and select some more
9se edit%#ie! to see you selectionsA before you initiate a restore
When initiatin$ a restore read the "rom"ts carefully /ecide !here you !ish
to "lace the "ro$ress lo$ The out"ut from this lo$ can be si$nificant in si@e if
you are chan$in$ the "aths for a number of files
12.'.( *unning a restore
The "ro$ress of the restore must be ti$htly monitored in order to enable
successful com"letion in the minimum time "ossible 5e a!are of the
timelines s"ecified
1= 1irstly : has the ;ob actually started or is it DueuedO If the ;ob is Dueued
0ection 11.' >Reduction o% Netbac)up drie usage to a++o= a restore to
run? sho!s you ho! to ensure sufficient resources are made a#ailable to
allo! the restore to commence

2= &nce the restore has been initiated start monitorin$ the "rocess lo$ The
"rocess lo$s are in the format of b"lo$rest444 and unless you s"ecified
differently !hilst settin$ u" the restore are created in the root directory e$
tail :f %b"lo$rest++1
Page &2 of 61
Contents
3= Ensure the ta"es reDuested are successfully mounted If the lo$ states the
ta"es are not in the library or the restore ;ob a""ears to han$ then chec. the
"hysical location of the ta"es usin$ rele#ant dasadmin or mtlib commands as
documented in the res"ecti#e 0obotic Library sections 1or ta"es not in the
library inform the rele#ant Hard!are site of this and as. them to locate the
reDuested ta"es and insert them bac. into the library &nce the ta"es ha#e
arri#ed bac. onsiteA "laced in the libraryA ensure the ta"es are made onsite
Note in so-e circu-stances /)en 3aulting )as run" t)e re7uired tapes /ill be
p)*sicall* on dri3es but -arked as not a3ailable to t)e robot /)ic) /ill cause t)e
restore to )ang9 In t)ese circu-stances t)e status of t)e tape /ill need to be c)anged
back and an* pending re7uests eit)er Oresub-ittedP orP deniedP as appropriate9 In
order to c)eck for t)is" run "moprcmd on t)e rele3ant -aster/ -edia ser3er and
c)eck for an* Pending 5e7uests fro- t)e output9 5unning "moprcmd &resubmit =
re5uest id > /ill allo/ t)e restore to continue9
6= 9nder 7etbac.u" 6, you can chec. u" on the restore usin$ the acti#ity
monitor This !ill sho! you the throu$h"ut in ?5%sA 7umber of files restored
and also the "ercenta$e com"lete &ne other !ay to chec. u" on !hat is
ha""enin$ is #ia the 90L Rhtt":%%byadsm+3natbtcom%acti#ity%S
,= The b" "rocess created by the restore on the ser#er can be chec.ed to
ensure that the 3P9 "rocess is incrementin$
)= It is also !orth!hile chec.in$ that the files reDuested to be restored are
actually bein$ restored by cdMin$ to the rele#ant directory on the client and
usin$ the ls command to ensure "ro$ress is bein$ made
When a restore is runnin$ there are some "oints to bear in mind::
When you are loo.in$ at the restore ;ob it may ta.e a !hile before a restore
.ic.s in This is because the restore is searchin$ a lar$e 75 client database
for the filesI this may ta.e u" to 2+ minutes &n hea#ily used ser#ersA the
restore may time out If so ;ust reissue the restore reDuest
If the file is in a lar$e system sin$le stream bac.u" you may notice it ta.es a
lon$ time to restore relati#ely small filesA after the restore is "ositioned on the
ta"e This is because the restore has to read throu$h the bac.u" to find the
files on the ta"e
Lou may see !aitin$ for mount of ta"eA this could be because the dri#es are
in use for bac.u"sA or the ta"e is offsiteA you chec. for an offsite ta"e by
loo.in$ at the reDuested ta"e in media mana$er
'lternati#ely you may !ish to use the "anels
Issue b"adm
7et5ac.u" Ser#er: t"cds1
Page &% of 61
Contents
7et5ac.u" 'dministration
::::::::::::::::::::::::
s= Stora$e 9nit -ana$ement
c= 3lass -ana$ement
$= 8lobal 3onfi$uration
r= 0e"orts
m= -anual 5ac.u"s
4= S"ecial 'ctions
u= 9ser 5ac.u"%0estore
e= -edia -ana$ement
h= Hel"
D= Quit
E7TE0 3H&I3E:
Select u
-aster Ser#er: t"cds1
3lient: t"cds1
-ain -enu
:::: ::::
b= 5ac.u"
r= 0estore
h= Hel"
D= Quit
E7TE0 3H&I3E:
Select rS
-aster Ser#er: t"cds1
3lient: t"cds1
0estore -enu
::::::::::::
b= 0estore 1iles and /irectories from 5ac.u"s
a= 0estore 1iles and /irectories from 'rchi#es
r= 0estore 1rom 0a! Partition 5ac.u"s
f= 0estore 1rom 'us"e4 1ast5ac. 5ac.u"s
d= 0estore 1rom True Ima$e 5ac.u"s
o= 0estore 1rom &racle /5 5ac.u"s
i= 0estore 1rom Informi4 /5 5ac.u"s
s= 0estore 1rom Sybase /5 5ac.u"s
t= 0estore 1rom SQL:5ac.Trac. /5 5ac.u"s
"= 0estore 1rom S'P /5 5ac.u"s
2= 0estore 1rom /52 /5 5ac.u"s
m= 3han$e -aster Ser#er
h= Hel"
D= Quit -enu
Page && of 61
Contents
E7TE0 3H&I3E:
Select bS
Path: %usr%users%stora$e%
Start /ate: 11%2*%*( 22:11:3* -aster Ser#er: t"cds1
End /ate: 12%+1%** 23:,*:,* Source 3lient: t"cds1
1iles Selected: + /estination 3lient: t"cds1
/irectory /e"th: 1 le#el 3lass Ty"e: Standard
/is"lay -ode: 5rief ?ey!ord Phrase:
0estore 5ac.u"s
:::::::::::::::
s= Select 1iles and /irectories "= 3han$e Path
e= Edit%>ie! Selected 1iles d= 3han$e /ate 0an$e
i= Initiate 0estore c= 3han$e /irectory /e"th
4= 3han$e /is"lay -ode to >erbose m= 3han$e -aster Ser#er
l= List 5ac.u" Ima$es b= 3han$e Source 3lient
a= S"ecify 'lternate Path t= 3han$e /estination 3lient
D= Quit -enu y= 3han$e 3lass Ty"e
h= Hel" .= 3han$e ?ey!ord Phrase
E7TE0 3H&I3E:
Then "S to chan$e "ath
dS to chan$e date ran$e
cS directory de"th F + $i#es the most information
sS to select your files%directories
When you are satisfied !ith the abo#e
Enter IS to initiate restore
1inishin$ ' 0estore
Send the restore lo$ to the reDuestorA and as. them to #erify the restore
12.'.- )ro0lems "ocating a File.
If you cannot find the file you !ant to restore consider the follo!in$
Is the source and destination client name correct
Is the file in a directory !hich is a lin.ed directory name
're the date ran$es too narro!
Is the "ath name ri$ht
Is the client%ser#er a H' confi$uration
Is the restore a ra! "artition
Ha#e the directory le#el settin$s been too hi$h or too lo!
/o I need to be root
Page &' of 61
Contents
1inally it may be that the file !as not bac.ed u"A this should be chec.ed
a$ainst the "olicy to see if it co#ers the reDuested files%directories
12.( Failover *estores
The reason for failo#er restores is !hen a -edia Ser#er is unreachable and
your restore ;ob is tryin$ to use itCs ta"e dri#es The Bbpmedia 5moedbC
command is then reDuired to mo#e the 7et5ac.u" catalo$ entries from the
files you are restorin$ to an alternati#e -edia Ser#er
The restore attem"ts failed because 7et5ac.u" !as tryin$ to mount the
rele#ant ta"e into the nfm"ramb dri#e !hich !as ino"erable at the time The
restore reDuest itself !as resol#ed by co"yin$ an identical file across from the
nfm"rama ser#erA but the need to "ro#ide a means of mana$in$ this "roblem
in future remained
1ollo!in$ $uidance re failo#er restores in the manualsA 3%0 ,6++222 !as
raised to u"date the byadsm+1 b"conf in the ho"es that this !ould enable the
automatic s!itch to alternati#e dri#e<s= s"ecified in the b"conf fileA in the
e#ent of similar restore failures 9nfortunatelyA the restore tests !hich
follo!ed failed and case 16++126)1 has been raised !ith >eritas to loo. into
the causes
In the meantime ho!e#erA the follo!in$ command has no! been successfully
tested and can be used from root access once the s"ecific ta"e reDuired is
.no!n:
Q/usr/open3/netbackup/bin/ad-inc-d/bp-edia 6-o3edb 6e3 =-ediaKid>
6ne/ser3er =)ostna-e> 6oldser3er =)ostna-e>N
In this e4am"le the command !ould read:
N%usr%o"en#%netbac.u"%bin%admincmd%b"media :mo#edb :e# 55L++2
:ne!ser#er nfm"rama :oldser#er nfm"rambN
/etails of the ta"e<s= reDuired !ill al!ays be are sho!n in the rele#ant
b"lo$rest++n lo$A <usually in the root directory= from the failed restore
attem"t
&nce the restore has been com"leted successfullyA it !ould be ad#iseable to
run the same command in re#erse to ensure that confusion is a#oided at a
future date should further restores be reDuired usin$ the same ta"e '$ain in
this conte4tA the command !ould read:
N%usr%o"en#%netbac.u"%bin%admincmd%b"media :mo#edb :e# 55L++2
:ne!ser#er nfm"ramb :oldser#er nfm"ramaN
's this method only "ro#ides a #ery s"ecific solution and needs to be .eyed
inA efforts !ill be continued to "ro$ress the case !ith >eritasA in the ho"es
that !e !ill be able to confi$ure automatic restore failo#er functionality
&#er#ie!
Page &6 of 61
Contents
#p+ist is a command su""lied by >eritas to interro$ate the 7et5ac.u"
bac.u" ima$es database It is an e4tremely useful 7et5ac.u" function as it
allo!s users the ability to chec. the e4istence of bac.u"sA !ithout ha#in$ to
use the restore "anels This command can be easily incor"orated into scri"ts
that can be run on a re$ular basis There are a lar$e number of "arameters
that are associated !ith bp+istI these are documented in the 7et5ac.u"
systems administration $uide
12.(.1 Functional !escription.
The b"list binary !ill search the 7et5ac.u" master ser#er for bac.u"sI by
default the command !ill search usin$ client name and master name
s"ecified in your b"conf file The b"conf file resides in
%usr%o"en#%netbac.u" or you can s"ecify your o!n b"conf file in your home
directory that o#errides the $lobal b"conf
The netbac.u" database of bac.u"s is really a catalo$ueA !hich com"rised of
a series of directories and flat filesA on some of the hi$h #olume ser#ers the
ima$e information !ill be com"ressed When a b"list command is e4ecutedA
the ran$e of dates s"ecified should be considered carefullyA as you could end
u" searchin$ the entire set of bac.u"s for a ser#er
9sin$ #arious "arameters you !ill be able to $et listin$s similar to the 9ni4
command VlsWA and !here there are multi"le occurrences of a file in a listin$A
this !ill indicate that there are multi"le bac.u"s
12.(.2 Using &plist
The command /usr/open/netbac)up/bin/bp+ist can be used to find out if a
file or directory !as bac.ed u" on a s"ecific dateA e$
bp+ist 5s --/dd/** 5e --/dd/** full*K7ualifiedKfileKna-e
This command !ould ha#e to be run on the client bo4A usin$ the root account
or one that has access to the files or directories bein$ DueriedThe b"list
command has $lobal e4ecute "ermissionsA but it is im"ortant to realise that
7etbac.u" security is based u"on 9ni4 file "ermissions If you do not ha#e
"ermission to restore a fileA then you !ill be unable to "erform a b"list for the
file 'll that !ill ha""en is that you !ill be returned bac. to the command
line
The .ey!ord function allo!s bac.u"s to be associated !ith a .ey!ordA
!hich is inde4ed and should s"eed u" Duery times It can be s"ecified !hen
the b"bac.u" command is issuedA or by the 7et5ac.u" administrator !hen a
scheduled bac.u" is defined
5efore listin$ bac.u"sA confirm !hether the file is filesystemA or ra!
"artition
Please note that by limitin$ the date ran$e the database search time !ill be
reduced
Page &( of 61
Contents
It is also "ossible to do recursi#e searches throu$h a directory tree structure
9se the bp+ist HK 3ommand to $et a full list of the a#ailable .ey!ords
a#ailable
12.(.$ &plisting Filesystem &ac7ups
The follo!in$ e4am"les all are e4am"les !here the command is issued from
the clientA to find files in a directory called %usr%home
Listin$ filesystem file bac.u"s
b"list F0 :l :s mm%dd%yy %usr%home
The e4am"le abo#e !ill search for a filesystem bac.u" recursi#ely listin$ all
files and directories from %usr%home do!n!ards The F0 o"tion tells the
b"list to dis"lay all files and directories from %usr%home do!n!ards the F0
o"tion !ill allo! you to limit the de"th of the search E$ F0 2 !ill sho! 2
directory de"ths
The Fs o"tion tells b"list to start searchin$ from the s"ecified date on!ardsA
there is also a VFeW o"tion for end dateA and you can s"ecify the hours and
minutes <see the manual= 9sin$ an end:date !ill reduce the search time The
V:bW o"tion dis"lays the bac.u" date and time of each file
b"list F0 Fb Fl :s mm%dd%yy %usr%home
Will $i#e a listin$ !here the date and time of the bac.u"s are listed
12.(.' &plisting *a3 )artition 0ac7ups.
To list a ra! "artition bac.u" the V:rW "arameter must be s"ecifiedA this !ill
tell 7et5ac.u" to loo. for ra! "artition bac.u"s only
' ty"ical b"list command for a ra! "artition mi$ht be
b"list Fr Fs mm%dd%yy %ra!%#olume%name
Qey=ord .unction
If bac.u"s are "erformed usin$ the .ey!ord !ithin a bac.u" classA it is e#en
Duic.er to identify bac.u"s 5ac.u"s "erformed usin$ the standard orahot
and oracold scri"ts use this and a bac.u" can be identified usin$ the .ey!ord
E$
b"list F 0 F.ey!ord VdbaG_W Fl %
12.(.( HA Environments
If you are runnin$ b"list in a H' set:u" be a!are ho! the client !as bac.ed
u" Which !ould be either by the shared IP address or by the "hysical IP
address &nce you ha#e this ensure that you are usin$ the ri$ht client nameA
this uses the F3 client6na-e o"tion
Page & of 61
Contents
*rob+ems
If you do not $et a result but you thin. you shouldA "lease do not hesitate to
contact 7et5ac.u" su""ort
12.(.- Comman! line
The command line restore commandA bprestoreA is used !hen bac.ed u" or
archi#ed files or directories are to be restored If a directory is s"ecified all
files bac.ed u" !ill be restored
Caution6 7ote that by default a restore !ill "robably use the -P07 unless the client
name is the same as that used by 7et5ac.u" for the bac.u"
Warning: Aou =i++ need to inc+ude the P 5t 13 E parameter i% the c+ient is an NT bo2.
The de%au+t type is standard and suitab+e %or 3NI: c+ients on+y.
1or e4am"le the synta4 of the bprestore command to restore a list of files
and rename them is as follo!s:
bprestore HQ HL logKfileKna-e 5R rena-eKfile HC clientna-e 5s
--/dd/** 5e --/dd/** 5% listfile
Where :Q means that e4istin$ files !ill not be o#er!ritten
HL is the location of the lo$ file <these can be lar$e and must be
mana$ed=
HR is the file listin$ the renames to be done
HC is the client ser#er name Please ensure you s"ecify the same
name that 7et5ac.u" uses to bac.u" the dataA e$ dyfin+6:fe instead of
dyfin+6
Hs is the start date
He is the end date
H% is the list of files to be restored
'nother e4am"le of ho! to restore a s"ecific file bac. to its ori$inal location
and o#er:!rite any e4istin$ file is as follo!s:
bprestore 5L logKfileKna-e 5C clientna-e 5s --/dd/** 5e --/dd/**
full*K7ualifiedKfileKna-e
&ther o"tions and "arameters can be s"ecified The com"lete synta4 is:
%usr%o"en#%netbac.u"%bin%b"restore [:' Z :5\ [:?\ [:l Z :H Z :y\[:r\ [:T\ [:L
progressKlo$\ [:0 rena-eKfile\ [:3 client\ [:/ client\
[:S -asterKser3er\ [:t classKt*pe\ [:c class\ [:s --/dd/**
?))+--+ss\\ [:e --/dd/** ?))+--+ss\\ [:! ?))+--+ss\\ [:.
Qke*/ordKp)raseN\ :f listfile Z filena-es
Page &! of 61
Contents
12.- *e!uction o/ +et&ac7up !rive usage to allo3 a
restore to run
12.-.1 Intro!uction
The 7et5ac.u" infrastructure is in constant use This means that the
resources reDuired for a hi$h "riority restore mi$ht all be allocated and not
immediately a#ailable The follo!in$ document outlines the "rocedure to
follo! to ensure that sufficient resources are made a#ailable in a controlled
manner This !ill cause the least im"act on other customers as their bac.u"s
!ill not be cancelled but merely Dueued for the duration of the restore
12.-.2 I!enti/ication
When a restore is submitted it has a hi$her "riority than a bac.u" ;ob It !ill
therefore ta.e the first a#ailable dri#e and start runnin$ but due to the !ay
that 7et5ac.u" runs it !ill try to .ee" a cartrid$e loaded for as lon$ as there
are bac.u" ;obs that can !rite to it This is e#en in "reference to other
bac.u"s that may ha#e been Dueued for a lon$ time So a restore may be
Dueued because bac.u" ;obs are $ettin$ "reference as it ma.es more efficient
use of the cartrid$e dri#es
If there are no dri#es a#ailable then it is ad#isable to try and free one by
reducin$ the number of ;obs runnin$
12.-.$ Action
0educin$ the number of runnin$ ;obs can be achie#ed by the follo!in$
method
1rom the 7et5ac.u" administration 89IA o"en u" the P&LI3IES iconA select
the &0'3LE ;ob classA ma.e a note of the B-a4 Jobs%classC "arameter and
reduce it to 6 This !ill ha#e the effect of limitin$ the &0'3LE bac.u" to
one dri#e only 3arry out a similar "rocess for the &0'3LEG0E/& "olicy
Note+ It may be necessary to reduce the ma4imum number of ;obs e#en further if
there are limited cartrid$e dri#es a#ailable and if the restore is of a hi$h
"riority and deemed necessaryA .ill off any system%non:ur$ent bac.u"s from
the 'cti#ity -onitor
This !ill $radually reduce the number of acti#e ;obs and release some dri#es
7nce the restore has comp+eted you must reHinstate the P$a2 Robs/c+assE
parameters to their origina+ a+ues.
Page ', of 61
Contents
12.1 A!!itional in/ormation
12.1.1 *a3 volumes
0a! #olume restores are a bit more com"licated In $eneral s"ecific files can
not be restoredA only the com"lete ra! #olumeA for this reason restores are
rarely done If a restore is reDuired then 3W5'3?9P should be consulted
1$#ackups
It may be necessary on occasions to run test bac.u"s or manual bac.u"s as
"art of "roblem resolution Please follo! the actions listed belo!
9sin$ the 7etbac.u" 89I :
&n the main netbac.u" screen you !ill see a list of "oliciesA search for
sma++Ctest
Hi$hli$ht sma++Ctest then brin$ do!n the 'ctions toolbar and select manual
bac.u" and schedule and s"ecific client name
3hec. on Job -onitor that the ;ob becomes acti#e
If you recei#e an error !hen usin$ the 89I
9se b"adm <>T1++=
<bpadm
select mS manual bac.u"s
select bS bro!se classes for!ard <.ee" $oin$ until smallGtest comes u"=
then enter
iS initiate bac.u"
1'Net#ackup o!mon Error $essa!es
Please note: !ith status codes 6*A ,+A ,1A ,)A ,2A ,(A ,*A 2)A (6A *,A 1)6 and
2+,A the &mnibus Threshold 0ule "ro#ides an e4tra filter Such tra"s are
su""ressed until o#er 2+ tra"s ha#e been issued in a t!enty:minute "eriodA at
!hich "oint a tra" is then sent Please see section 161 belo! for fuller
details
The error codes are e2p+ained in more depth in section 1".
Page '1 of 61
Contents
1'.1 +et&ac7up )rocesses an! )rocmon Error
Messages
There are si4 "rocesses that need to be acti#e on 7et5ac.u" ser#ersA and
these consist of t!o 7et5ac.u" "rocesses and four -edia -ana$er "rocesses
as follo!s:
7et5ac.u" ::
bpdbm ? bprd
-edia -ana$er ::
ltid ? a"rd ? "md ? tlmd #can be tlhcd e9g9 so-e ICIP -ac)ines$
In the e#ent of any of these "rocesses dyin$A the follo!in$ messa$es !ill be
issued usin$ the follo!in$ synta4:
process died 8 =processname>@ formerly PI6 nnnn..
E.!. process died8 bpdbm@ formerly PI6 +44*A
Please note that a sin$le tra" may list se#eral "rocessesA since the loss of ltid
!ill .ill all other media mana$er "rocessesA as in the follo!in$ e4am"le:
+B0*B0** +C8BB8*3 cy"sis*a ;ystem Entity )ailed8 Processes died8
bpdbm@ formerly PI6 C2*4A? ltid@ formerly PI6 C2*D3? a"rd@ formerly
PI6 C2*44? "md@ formerly PI6 C2*4*? tlmd@ formerly PI6 C2*4D
The 7et5ac.u" "rocmon file has the chec.startu" command includedA !hich
as it im"liesA chec.s !hether a "rocess that has died restarts after!ards
/e"endin$ on the resultA either of the t!o follo!in$ commands !ill be
issued:
O;$)Eprocmon8 Process bpdbm has not started
;$$Eprocesses8 Process back up8 bpdbm@ ne1 PI6 4333
&n recei#in$ a 7et5ac.u" B"rocess diedC or M"rocess not startedM tra"A Tinsley
Par. &"erations should !ait for fi#e minutes <a""ro4= to see !hether a
M"rocess bac. u"M tra" is recei#ed Where no such tra" a""earsA contact should
be made !ith 3WHP&PS9P to ad#ise them of a "ossible "roblem
Page '2 of 61
Contents
1(Net#ackup ;TAT7; Exits & the bi! hitters
Here are some of the main status e4its that are re$ularly out"ut
1(.1Exit Status '1 B+et3or7 connection time! outC
9aise a P2 incident record and pro!ress fault resolution as follo1s8
This section aims to describe the "rocess of dealin$ !ith these tra"s from a
5rid$e "ers"ecti#e
Some definitions
E4it Status 61: Nnet!or. connection timed outN
E4it Status ,6: Ntimed out connectin$ to clientN
These are similar faults but resolution normally differs 61Ms are often due to
a failure to ma.e a connection to a client ,6Ms usually occur after a
connection has been made but the client fails to res"ond This document !ill
deal !ith these failures se"arately There are a number of considerations
!hen dealin$ !ith this fault Is for instance the machine sho!in$ as
unreachable on &mnibus and bein$ dealt !ith #ia a "roblem or chan$e
recordO If so then the case can be sim"ly transferred to 3W5'3?9P for
info If not then a method for dealin$ follo!s
8o to the Job -onitor !indo! and chec. !hich -edia Ser#er the ;ob !as
usin$ &ften a bac.u" may fail !hen made from one ser#er but not another
<ie a client may be successfully bac.u" from t"adsm+1 but not from
t"adsm+2= 5efore "roceedin$ it is !orth chec.in$ that other clients are not
e4"eriencin$ "roblems There ha#e been a number of faults recently !here
all bac.u"s to clients from the media ser#er ha#e failed !ith 61Ms but ha#e
been o.ay from the master 0esolution has been to reboot the media ser#er
61Ms are often client s"ecific "roblems but can also be due to net!or.
con$estion With the latter a number of bac.u"s for a client one mi$ht fail
but others are o.ay This is often a "roblem at 5letchley for clients usin$ a
#irtual "ri#ate net!or. <these ha#e names li.e as"asea1:#"n= !here the
bac.u" is actually $oin$ o#er the "ublic net!or. and can fail occasionally
due to con$estion
Ste" one is to lo$ into the rele#ant ser#er and attem"t to "in$ the client from
that ser#er I !ill use here the e4am"le of t"finder1 and assume that there
ha#e been bac.u" failures from t"adsm+1A t"adsm+2 and t"adsm+3
Page '% of 61
Contents
The error !ill re"ort as indicate a failure for t"finder1:fe
Lo$in to t"adsm+1 and "in$ t"finder1 /id it res"ondO Then "in$ t"finder1:
fe It is im"ortant that you "in$ the hostname as it is .no!n by 7etbac.u" as
this tests the s"ecific net!or. interface 7ormally our machines !ill ha#e a
se"arate net!or. card !hich is connected to the dedicated 7etbac.u" "ri#ate
net!or. This is ensures that connections can be made by users to a machine
on its "ublic net!or. interface !ithout de$radation due to #ery lar$e amounts
of data !hich bac.u"s often in#ol#e 'nother issue is that one should al!ays
use the hostname and not the IP address as this tests the name resolution
When telnettin$ bet!een the -aster%-edia Ser#er and the clientA run the
B=ho am IC command on both to ensure the hostnames reflect the %etc%hosts
and b"conf entries
If t"finder1 "in$s o.ay but t"finder1:fe did not then the "roblem lies
some!here on the "ri#ate net!or. <!hich includes the net!or. card and
net!or.in$ soft!are on the client=
Lo$ onto the client and chec. that the net!or. is u" usin$ Mifconfi$M <for
#endor s"ecific instructionA see the &' 7et!or. /ia$nostic "rocedures= If
the rele#ant interface is do!n then try bouncin$ the interface See also
instructions for MnetstatM in the &' "rocedures
-ore often than not caused by the 7et5ac.u" ser#er bein$ unable to
communicate !ith the client
3hec. that you can "in$ % telnet to the bac.u" interface on the client F if itCs
not contactableA for!ard case to TSS for "ro$ression
If you can telnet to the bac.u" interfaceA lo$on and chec. the entries in
%etc%hosts for the 7et5ac.u" ser#er IP addresses <includin$ media ser#ers=
and the clientCs o!n addresses <includin$ the name as re$istered % .no!n
!ithin 7et5ac.u" on the ser#er=
3hec. %usr%o"en%netbac.u"%b"conf to ensure the entries are correct
Ensure you are connected #ia the correct net!or. <ie ty"e !ho am I in
order to identify the connection you ha#e been routed #ia=
If the confi$ chec.s out o.A s!itch the multistreamin$ o"tion &7 !ithin the
7et5ac.u" class in order to see if the errors are occurrin$ under one
"articular mount "oint If soA then this may be one of t!o "roblems It may be
that the 9ni4 bo4 !as connected to a 71S <7et!or. 1ile system= that has
since been remo#edA therefore !hen listin$ the mount "ointsA the session
han$s 'lternati#elyA there may be a "roblem !ith some corru"t data filesA or
it may ;ust be the number of files that 7et5ac.u" has been as.ed to bac. u"
&n some "roblematic clientsA !e are bac.in$ u" in e4cess of 1million files
<7ote: thatCs files 7&T actual bytes of data=
Page '& of 61
Contents
1(.2 Exit Status 1 B&ac7up 3as partially success/ulC
No need to raise an incident record as the backup is deemed successful.
-ainly caused by files bein$ i$nored as a result of hot bac.u"sA ie !here
databases are acti#e durin$ the bac.u" and files are remo#ed before
7et5ac.u" bac.s them u" This occurs mainly on 7T clients 3hec. the
"roblems re"ort from the b"adm "anels on the master ser#er to identify the
offendin$ files and inform TSS
1(.$ Exit Status (2 B#ime! out 3aiting /or Me!ia
Manager to mount volumeC
3aused by a library failin$ to satisfy a reDuest for a ta"e mount before the
timeout e4"ired If all of the dri#es are do!n and 7et5ac.u" is still acti#ely
"rocessin$ bac.u" reDuestsA then e#entually they !ill timeout F hence the
status ,2 1irstlyA chec. that the library is u" It is also im"ortant to chec. that
all the dri#es are not in use by other ;obsA !hich !ill also be a cause of this
error 0esolution can be ensurin$ that there are not a lar$e number of
concurrent ;obs <"articularly !ith differin$ retention le#els= F chec. !ith
/5'CsA 'S8CsA TSS de"endin$ on the ;obs affected if "ersistent
1(.' Exit Status 11 B+one o/ the /iles in the /ile list existC
' number or all of the mount "oints as defined !ithin the 7etbac.u" bac.u"
class do not actually e4ist on the client !hen the bac.u" runs ' common
cause can be !here clients are clustered and therefore share mount "ointsA ie
the &'WS </ar!in= clients that ha#e master and sla#e bo4es They utilise
raid filesystemsA !hich only reside on one of the bo4es de"endin$ !here the
li#e ser#ice is In order to ensure that both bo4es are co#eredA !e !ould need
to maybe introduce a Bfloatin$C IP address method of clusterin$ bo4es It
!ould be !orth contactin$ the TSS "eo"le in order to discuss this
&ther causes of this error could be incorrect "ermissions on files % directories
bein$ bac.ed u"
1(.( Exit Status 214 B#he reAuire! storage unit is not
availa0leC
9aise a P2 incident record and pro!ress fault resolution as follo1s. If the
problem is NOT resol"ed 1ithin + hour@ please up!rade to PC.
Page '' of 61
Contents
9sually caused !hen there are no dri#es a#ailable for bac.u"s F this can
mean one of t!o thin$s
'll attached dri#es are do!n
'll dri#es are u"A but in use by other bac.u"s for a lon$ "eriod of time
3hec. the dri#e status <if the error occurred o#erni$htA chec. the dri#e usa$e
information on the 7et5ac.u" interacti#e re"ortin$ "a$e=A follo!in$ the
normal Bdo!n dri#esC "rocedure If you find that the "roblem is "ersistently
bein$ caused by !or.loadA then liaise !ith !ho is res"onsible for the bac.u"s
<normally &racle /5'Cs= and ad#ise them to reduce the number of
concurrent ;obs
1(.- Exit Status (' B#ime! out connecting to clientC
Problem occurs !hen a bac.u" is in#o.edA ho!e#erA connection from the
master ser#er to the client is bro.en as a result of it timin$ out
This "articular error is not normally caused by net!or. "roblemsI itCs usually
lin.ed !ith 7et5ac.u" confi$ anomalies -ore s"ecificallyA ensure that the
entries for the client and ser#er in /etc/)osts are correct on both bo4es This
can often occur !hen the bac.u"s are usin$ a media ser#er and the host
entries are incorrect for it 1ollo! the 7et!or. "roblem resolution document
for more details on chec.in$ this
1(.1 Exit Status 2' BMe!ia 3rite errorC
The main cause of this error is incom"atible media 3urrentlyA !e use LT&
and 3,*+ ta"es &f the 3,*+ ta"esA there are 2 different ty"es of 3,*+ ta"es F
there are 3,*+5 and 3,*+E 3,*+5 ta"es are com"atible !ith the normal
density dri#es and double density dri#esA !hereas the 3,*+E ta"es are only
com"atible in the double density <e4tended= dri#es When 3,*+E ta"es are
mounted in 3,*+5 dri#esA as soon as the bac.u" starts !ritin$ dataA it fails
!ith the media !rite error
This can occur !here you ha#e different media ser#ersA usin$ different dri#e %
ta"e ty"es but the same #olume "oolA alternati#elyA it is not uncommon for
someone to un.no!in$ly assi$n 3,*+E ta"es to a #olume "ool used by
3,*+5 dri#es If this is the caseA then the assi$ned ta"es must be re:assi$ned
else!here 'S'P
If a $enuinely faulty ta"e causes the "roblemA then you must follo! the
"rocedure for dealin$ !ith faulty ta"es
1(.2 Exit Status (1 BClient connection re/use!C
Page '6 of 61
Contents
3aused normally by the 7etbac.u" client code bein$ remo#ed from the client
bo4 F this often ha""ens as a result of an u"$rade to the &"eratin$ System
<9ni4 % Windo!s= 9nless !e are informed of such chan$es #ia 3han$e
3ontrol then these errors !ill "ersist Locate the TSS contact for the bo4 and
reDuest that a chan$e record be raised for a reinstall
To chec. that the 7et5ac.u" client code is resident on a clientA lo$ on to it
and see if you can access the %usr%o"en#%netbac.u" directory F if not then the
code has been remo#ed This error is normally caused by inacti#ity of the
b"cd "rocess on the client bo4 This is reDuired in order to allo! access from
the 7et5ac.u" ser#er to the client and must com"ly !ith the T3P%IP
"rotocol
The !ay to chec. this is to follo! the telnet hostname bpcd instructions
found in the su""ortin$ document
If you find that an error is $i#en !hen tryin$ thisA do the follo!in$ to ensure
it is !or.in$:
&n the client bo4:
ty"e $re" b"cd %etc%ser#ices
The follo!in$ line should a""ear:
b"cd 132(2%tc" b"cd
This indicates that the "ort numbered 132(2 is to be used only for 7et5ac.u"
connections and is $eneric across all 7et5ac.u" confi$s <ser#er and clients=
/o the same for the reDuest daemon:
$re" b"rd %etc%ser#ices
for the follo!in$ entry:
b"rd 1322+%tc" b"rd
'lsoA the follo!in$ entry must be in %etc%inetdconf:
grep bpcd /etc/inetd9conf F !hich !ill $i#e you the follo!in$:
b"cd stream tc" no!ait root %usr%o"en#%netbac.u"%bin%b"cd b"cd
&n system IPL <boot= the %etc%inetdconf file is read and !ill start:u" any
reDuired ser#ices found !ithin the %etc%ser#ices file
&n 7T bo4esA chec. for 7et5ac.u" ser#ices under 3ontrol PanelA Ser#ices
and ensure it is Started and set to 'uto-atic
1(.4 Exit Status 4- BUna0le to allocate ne3 me!ia /or
0ac7up? storage unit has none availa0leC
9aise a P2 incident record and pro!ress fault resolution as follo1s. If the
problem is NOT resol"ed 1ithin + hour@ please up!rade to PC.
Error caused by numerous factors The ob#ious one is nil scratch in the
s"ecified #olume "ool F in this instance the scratch le#els must be to""ed u"
'S'P &ne thin$ to be !ary of and that is the e4"iration date set !ithin the
Page '( of 61
Contents
7et5ac.u" 89I F this can be set on ta"es that are not time assi$ned <ie
scratch= 7et5ac.u" !ill i$nore these until manually un:e4"ired <unless they
ha#e been e4"ired for a "articular reason= F this method of e4"iration is used
to sto" ta"es from bein$ used e#en if they are scratch If there are e4"ired
ta"es that are a#ailable for useA sim"ly use the 89I to un:e4"ire them
'nother reason for these errors can be due to a ta"e bein$ stuc. in a dri#e If
a ;ob is allocated a dri#e in !hich to mount a ta"e that already has a ta"e
stuc. inA then 7et5ac.u" !ill a""ear to attem"t to mount all of the a#ailable
scratch ta"es as it is not intelli$ent enou$h to be able to identify !hether a
ta"e is stuc. or not It !ill send a si$nal to the robot re$ardless of !hat is
already in the dri#e 's suchA you !ill need to identify the dri#e !ith the
stuc. ta"e in and remo#e it 3hec. the 7et5ac.u" 'cti#ity -onitor for
details on !hether the ;ob is tryin$ unsuccessfully to mount numerous ta"es
1(.15 Exit Status 1$1 Client is not vali!ate! to use the
server
The client nameA as determined from the connection to the ser#erA did not
match any client name in the 7et5ac.u" confi$uration and there !as no
altnames confi$uration for this client on the master ser#er ' client and ser#er
that ha#e multi"le net!or. connections can encounter this "roblem if the
name by !hich the client is confi$ured is not the one by !hich its routin$
tables direct connections to the ser#er
Try the follo!in$
1 E4amine the 7et5ac.u" Problems re"ort
2 3reate a debu$ lo$ directory for b"rd and retry the o"eration 3hec. the
resultin$ debu$ lo$ to determine the connection and client names
/e"endin$ on the reDuest ty"e <restoreA bac.u"A and so on=A you may need
or !ant to:
_ 3han$e the clientMs confi$ured name
_ -odify the routin$ tables on the client
_ &n the master ser#erA set u" an altnames directory and file for this client
<see the 7et5ac.u" System 'dministratorMs 8uide for 97IP=
or
_ &n a 97IP master ser#erA create a soft lin. in the 7et5ac.u" ima$e
catalo$
3 0e#ie! >erifyin$ Host 7ames and Ser#ices Entries in the
Troubleshootin$ 8uide
0econfi$uration

1 'scertain an alternate connection <usually o#er the mana$ement L'7=
'll -edia ser#ers <includin$ the -aster= should ha#e connecti#ity to the
client <if this is not "ossible then the bac.u"s !ill ha#e to be restricted to
certain 7et5ac.u" -edia Ser#ers only=
2 -a.e co"ies of the clientMs %usr%o"en#%netbac.u"%b"conf and %etc%hosts
file
3 9"date the clientMs %usr%o"en#%netbac.u"%b"conf so that 3LIE7TG7'-E
reflects the Mne!M name
Page ' of 61
Contents
6 -odify the clientMs %etc%hosts file to reflect the client name
, -odify the the PolicyMs on 7et5ac.u" to include the Mne!M client name
This usually includes &0'3LEA &0'3LEG0E/& and any other
&S%'""lication bac.u" that is defined
) -a.e co"ies of %etc%hosts on all the 7et5ac.u" -edia Ser#ers that the
client can use
2 Ensure the 7et5ac.u" -edia Ser#erMs %etc%hosts file reflects the Mne!M
client name

3hec.s

1 Test both user and scheduled bac.u"s !or. as e4"ected
2 Test that a restore !or.s <this test is only "ossible if resources on the
7et5ac.u" -aster%-edia are free=
3 Test to ensure %usr%o"en#%netbac.u"%bin%b"list !or.s This should be
tested for both MrootM user and MoracleM user E4"ected returns:
a a file list of !hat has been bac.ed u"
b Mno entity foundM : this !ill be returned if the user does not ha#e the
correct userid%"ermissions to interro$ate the files e$ a userid of Mstora$eM
cannot interro$ate files belon$in$ to a userid of MoracleM
c If a 7et5ac.u" error code is returnedA then that error reDuires
in#esti$atin$ and resol#in$

Errors 0eturned

9suallyA the Mrenamin$M of a client !ill not cause any issues to bac.u"sA
restores or b"list commands Errors !ill only occur in unusual set:
u"s%confi$urations <e$ if static routin$ is bein$ used=
The most li.ely error is Status 3ode: 131 : client is not #alidated to use the
ser#er <7et5ac.u" Troubleshootin$ 8uide entry belo!= There are a fe!
!ays of resol#in$ this issue <the recommend resolution is detailed=:
1 run b"list statin$ the client name in the command ie b"list :3
clientna-e %
2 create an entry in the %usr%o"en#%netbac.u"%db%altnames directory <on the
7et5ac.u" -aster Ser#er= to allo! the MoldM client name to access data
bac.ed u" #ia the Mne!M client name
1(.11 Application *esource Alert
Tra"s !ill be issued for these alerts The follo!in$ alerts are ty"icalA for
e4am"le:
0cratch +ee+ be+o= minimum6
1'M in Netbac)up H Raise *3 I/R and pass to C8987*03:
App+ication Resource A+ert6 0cratch +ee+ be+o= minimum 6 'M in
Netbac)up H Raise I/R as a priority 2 and pass to the #ridge c+ass as
+isted be+o=. I% the prob+em is out o% hours you =i++ need to ino)e
ca++out.
Tra"s !ill be issued for these alertsA for e4am"le:
Page '! of 61
Contents
Dup+icate Tape>s? H Raise *3 I/R and pass to C8#ACQ3*.
Each ta"e should be e4amined on the ser#ers that ha#e it defined The ser#er
that has most recently !ritten to the ta"e should be allo!ed to .ee" the
definitionA the remainin$ ser#ers should ha#e all bac.u" ima$es that reside on
that ta"e manually e4"iredA and the ta"e should be com"letely deleted from
all 7etbac.u" databases on that ser#er
Netbac)up 8arning6 0ererHname has not processed any bac)ups %or 1'
minutes H Inestigation Re1uired H Raise *3 I/R and pass to
C89*7*03: in the %irst instance.
Chec) %or current actiity and identi%y any possib+e prob+ems. I% %urther
action is re1uired pass the #ridge record to C8#ACQ3* %or %urther
inestigation.
App+ication Resource A+ert6 Net#ac)up Drie 8arning6 sererHname H
Drie n 9as Remained Do=n H Inestigation Re1uired. Raise *3 I/R and
pass to C89*7*03: in the %irst instance.
1ollo!in$ the actions indicated for error status 21*A and section ( /ri#e
Testin$
App+ication Resource A+ert6 Net#ac)up Drie 8arning6 SsererH
nameL Dries are in ADR mode H Inestigation re1uired Sdate and
timeL
App+ication 4ntity .ai+ed6 Net#ac)up D$D 8arning6 SsererH
nameL Cannot connect to md daemon H Inestigation re1uired
Sdate and timeL
1(.12 Slo3 throughput o/ 0ac7ups
3hec. the bac.u" ;ob throu$h"ut #ia the 'cti#ity -onitor on the admin 89I
or by runnin$ Bb"db;obs Z $re" 'cti#eC on the -aster ser#er If all client
bac.u"s are slo! then this is li.ely to be either a net!or. "roblem or a
net!or. interface "roblem on the -aster%-edia ser#er runnin$ the bac.u"s
If the slo! throu$h"ut is on one client only then the "roblem is "robably on
the client end !ith the bac.u" interface card bein$ set to half:du"le4 Pass
the "roblem record to the rele#ant Ser#ice /eli#ery team for them to
chec.%amend the card settin$s
1-Netbackup Clients
1-.1 +et0ac7up %ocumentation
Page 6, of 61
Contents
1-.2 #rou0leshooting .ui!e
Re%er to Net#ac)up manua+s
1-.$ +et&ac7up reporting
1-.' Supportal
G3ollHI G5oo.HI GSTa$HI G90LHhtt":%%su""ortalnatbtcom%defaultas"I
htt":%%su""ortalnatbtcom%defaultas"
11APPEN6ICE;
4ND 7. D7C3$4NT
Page 61 of 61

Netbackup operational procedures document

Cargado por

Información del documento

Título original

Derechos de autor

Formatos disponibles

Compartir este documento

Compartir o incrustar documentos

Opciones para compartir

¿Le pareció útil este documento?

¿Este contenido es inapropiado?

Copyright:

Formatos disponibles

Netbackup operational procedures document

Cargado por

Copyright:

Formatos disponibles

ISL/OPS/A869

También podría gustarte