Step-By-Step Installation of RAC on IBM AIX (RS/6000)

Note: This note was !eate" fo! #i RAC$ The %0& '!a le "o ()entation p!o*i"es installation inst!( tions fo! %0& RAC$ These inst!( tions an +e fo(n" on 'TN: '!a le, Real Appli ation Cl(ste!s Installation an" Confi&(!ation -(i"e
%0& Release % (%0$%) fo! AIX-Base" Syste)s. hp /0-1X 0A-RISC (62-+it). hp T!(62 1NIX. 3in(4. Sola!is 'pe!atin& Syste) (S0ARC 62-+it)

0(!pose
This document will provide the reader with step-by-step instructions on how to install a cluster, install Oracle Real Application Clusters (RAC) and start a cluster database on IB AI! "AC #$%& (CR ) '('()( *or additional e)planation or in+ormation on any o+ these steps, please see the re+erences listed at the end o+ this document( This note does not cover IB &#, plat+orm( 5is lai)e!: If the!e a!e any e!!o!s o! iss(es p!io! to step 6$6. please onta t IBM S(ppo!t$ The info!)ation ontaine" he!e is as a (!ate as possi+le at the ti)e of w!itin&$ %$ Confi&(!in& the Cl(ste! /a!"wa!e .(. inimal "ardware list $ &ystem Re/uirements .(.(. "ardware .(.(, &o+tware .(.(0 #atches

.(, Installin1 2is3 Arrays .(0 Installin1 Cluster Interconnect and #ublic 4etwor3 "ardware ,(. "AC #$%& &o+tware Installation ,(, Con+i1urin1 Cluster Topolo1y ,(0 &ynchroni5in1 Cluster Topolo1y ,(' Con+i1urin1 Cluster Resources ,('(. Create volume 1roups to be shared concurrently on one node ,('(, Create &hared RA6 7o1ical 8olumes ,('(0 Import the 8olume 9roup on to the Other 4odes ,('(' Add a Concurrent Cluster Resource 9roup ,('(: Con+i1ure the Concurrent Cluster Resource 9roup ,('(; Creatin1 #arallel *ilesystems (9#*&)

7$ C!eatin& a l(ste!

,(: &ynchroni5in1 Cluster Resources ,(; <oinin1 4odes Into the Cluster

,(= Basic Cluster Administration 0(. Con+i1ure the shared dis3s and >4I! preinstallation tas3s 0(,(. Con+i1ure the shared dis3s 0(,(, >4I! preinstallation tas3s

6$ 0!epa!in& fo! the Installation of RAC

0(, >sin1 the Oracle >niversal Installer +or Real Application Clusters 0(0 Create a RAC 2atabase usin1 the Oracle 2atabase Con+i1uration Assistant

2$ A")iniste!in& Real Appli ation Cl(ste!s Instan es 8$ Refe!en es

%$ Confi&(!in& the Cl(ste!s /a!"wa!e

%$% Mini)al /a!"wa!e list / Syste) Re9(i!e)ents
*or a two node cluster the +ollowin1 would be a minimum recommended hardware list( Chec3 the RAC$IB AI! certi+ication matri) +or RAC updates on currently supported hardware$so+tware(

%$%$% /a!"wa!e
IBM se!*e!s - two IB servers capable o+ runnin1 AI! '(0(0 or :7 ;'bit :o! IBM o! thi!"-pa!ty sto!a&e p!o"( ts. Cl(ste! inte! onne ts. 0(+li netwo!;s. Swit h options. Me)o!y. swap < C01 !e9(i!e)ents ons(lt with the ope!atin& syste) *en"o! o! ha!"wa!e *en"o!$ Me)o!y. swap < C01 !e9(i!e)ents %ach server must have a minimum o+ :., b o+ memory, at least .9b swap space or twice the physical memory whichever is 1reater( To determine system memory use?$ /usr/sbin/lsattr -E -l sys0 -a realmem

To determine swap space use?$ /usr/sbin/lsps -a

;'-bit processors are re/uired( AI! '(0(0? '('()

%$%$7 Softwa!e
6hen usin1 IB "AC #$%& CR

Only RA6 7o1ical 8olumes (Raw 2evices) +or 2atabase *iles supported Oracle &erver %nterprise %dition @i Release . (@(A(.) or @i Release , (@(,(A) AI! :(. (:7)? *or 2atabase *iles residin1 on RA6 7o1ical 8olumes (Raw 2evices)?

6hen usin1 IB

"AC #$%& CR

'('() is not re/uired)

*or 2atabase +iles residin1 on #arallel *ilesystem (9#*&)? "AC #$%& '('() ("AC #$CR 9#*& .(: IB or IB #atch #T*., and IB #atch #T*.0 patch IB0'@.=

Oracle &erver %nterprise %dition @i Release , (@(,(A) Oracle &erver %nterprise %dition @i +or AI! '(0(0 and :7 are in separate C2 pac3s and include Real Application Cluster (RAC)

%$%$6 0at hes

The IB Cluster nodes mi1ht re/uire patches in the +ollowin1 areas? IB AI! Operatin1 %nvironment patches &tora1e +irmware patches or microcode updates a3e sure all cluster nodes have the same patch levels 2o not install any +irmware-related patches without /uali+ied assistance Always obtain the most current patch in+ormation Read all patch R%A2 % notes care+ully( *or a list o+ re/uired operatin1 system patches chec3 the sources in 4ote ,..:0=(. and contact IB corporation +or additional patch re/uirements(

#atchin1 considerations?

To see all currently installed patches use the +ollowin1 command? = /(s!/s+in/instfi4 -i To veri+y installation o+ a speci+ic patch use? = /(s!/s+in/instfi4 -i*; >0at hn()+e!? e(1(? = /(s!/s+in/instfi4 -i*; I@60#7A

%$7 Installin& 5is; A!!ays

*ollow the procedures +or an initial installation o+ the dis3 enclosures or arrays, prior to installin1 the IB AI! operatin1 system environment and "AC # so+tware( #er+orm this procedure in conCunction with the procedures in the "AC # +or AI! '(!(. Installation 9uide and your server hardware manual(

%$6 Installin& Cl(ste! Inte! onne t an" 0(+li Netwo!; /a!"wa!e

The cluster interconnect and public networ3 inter+aces do not need to be con+i1ured prior to the "AC # installation but must be con+i1ured and available be+ore the cluster can be con+i1ured( I+ not already installed, install host adapters in your cluster nodes( *or the procedure on installin1 host adapters, see the documentation that shipped with your host adapters and node hardware( Install the

transport cables (and optionally, transport Cunctions), dependin1 on how many nodes are in your cluster? A cluster with more than two nodes re/uires two cluster transport Cunctions( These transport Cunctions are %thernet-based switches (customer-supplied)(

Bou install the cluster so+tware and con+i1ure the interconnect a+ter you have installed all other hardware(

7$ C!eatin& a Cl(ste!
7$% IBM /ACM0/BS Softwa!e Installation
The "AC #$%& '(!(! installation and con+i1uration process is completed in several maCor steps( The 1eneral process is? install hardware install the IB AI! operatin1 system so+tware AI! maintenance level and re/uired patches install the latest IB

install "AC #$%& '(!(! on each node install "AC #$%& re/uired patches con+i1ure the cluster topolo1y synchroni5e the cluster topolo1y con+i1ure cluster resources synchroni5e cluster resources

*ollow the instructions in the "AC # +or AI! '(!(! Installation 9uide +or detailed instructions on insallin1 the re/uired "AC # pac3a1es( The re/uired$su11ested pac3a1es include the +ollowin1? cluster(adt(es(client(demos cluster(adt(es(client(include cluster(adt(es(server(demos cluster(clvm(rte "AC # +or AI! Concurrent cluster(cspoc(cmds "AC # C&#OC commands cluster(cspoc(dsh "AC # C&#OC dsh and perl cluster(cspoc(rte "AC # C&#OC Runtime Commands cluster(es(client(lib %& Client 7ibraries cluster(es(client(rte %& Client Runtime cluster(es(client(utils %& Client >tilities cluster(es(clvm(rte %& +or AI! Concurrent Access cluster(es(cspoc(cmds %& C&#OC CommandsD cluster(es(cspoc(dsh %& C&#OC dsh and perl cluster(es(cspoc(rte %& C&#OC Runtime Commands

cluster(es(hc(rte %& "C 2aemon cluster(es(server(dia1 %& &erver 2ia1s cluster(es(server(events %& &erver %vents cluster(es(server(rte %& Base &erver Runtime cluster(es(server(utils %& &erver >tilities cluster(hc(rte "AC # "C 2aemon cluster(ms1(%nE>&(cspoc "AC # C&#OC cluster(ms1(enE>&(cspoc "AC # C&#OC cluster(ms1(enE>&(es(client cluster(ms1(enE>&(es(server cluster(ms1(enE>&(haview "AC # "A8iew cluster(vsm(es %& 8& Con+i1uration >tility cluster(clvm(rte "AC # +or AI! Concurrent cluster(es(client(rte %& Client Runtime cluster(es(clvm(rte %& +or AI! Concurrent Access cluster(es(hc(rte %& "C 2aemon cluster(es(server(events %& &erver %vents cluster(es(server(rte %& Base &erver Runtime cluster(es(server(utils %& &erver >tilities cluster(hc(rte "AC # "C 2aemon cluster(man(enE>&(client(data cluster(man(enE>&(cspoc(data cluster(man(enE>&(es(data %& cluster(man(enE>&(server(data rsct(basic(hacmp R&$;AAA Cluster Technolo1y rsct(basic(rte R&$;AAA Cluster Technolo1y rsct(basic(sp R&$;AAA Cluster Technolo1y rsct(clients(hacmp R&$;AAA Cluster Technolo1y rsct(clients(rte R&$;AAA Cluster Technolo1y rsct(clients(sp R&$;AAA Cluster Technolo1y rsct(basic(rte R&$;AAA Cluster Technolo1y an #a1es - >(&( %n1lish essa1es - >(&( essa1es - >(&( essa1es - >(&(

Bou can veri+y the installed "AC # so+tware with the Fclveri+yF command( G $usr$sbin$cluster$dia1$clveri+y At the Fclveri+yDF prompt enter Fso+twareF then at the Fclveri+y(so+twareDF prompt enter FlppF( Bou should see a messa1e similar to?

Chec3in1 AI! +iles +or "AC # +or AI!-speci+ic modi+ications((( -$etc$inittab not con+i1ured +or "AC # +or AI!( I+ I# Address Ta3eover is con+i1ured, or the Cluster must contain the proper "AC # +or AI! entries( Command completed( --------- "it Return To Continue --------Contact IB support i+ there were any +ailure messa1es or problems e)ecutin1 the Fclveri+yF command( ana1er is to be started on boot, then $etc$inittab

7$7 Confi&(!in& the Cl(ste! Topolo&y

>sin1 the Fsmit hacmpF command? C s)it ha )p Note: The +ollowin1 is an e)ample o+ a 1eneric "AC # con+i1uration to be used as an e)ample only( &ee the "AC # installation and plannin1 documentation +or speci+ic e)amples( All /uestions concernin1 the con+i1uration o+ your cluster should be directed to IB &upport( This con+i1uration does not include an e)ample o+ a I# ta3eover networ3( FsmitF +astpaths are bein1 used to navi1ate the Fsmit hacmpF con+i1uration menus( %ach one o+ these con+i1uration screens are obtainable +rom Fsmit hacmpF( All con+i1uration is done +rom one node and then synchroni5ed to the other participatin1 nodes( Add the cluster de+inition? &mit "AC # -D Cluster Con+i1uration -D Cluster Topolo1y -D Con+i1ure Cluster -D Add a Cluster 2e+inintion *astpath? C s)it )D onfi&D l(ste!$a""
Add a Cluster 2e+inition Type or select values in entry +ields( #ress %nter A*T%R ma3in1 all desired chan1es( H%ntry *ieldsI --4OT%? Cluster ana1er >&T B% R%&TART%2 in order +or chan1es to be ac3nowled1ed(-E Cl(ste! I5 E Cl(ste! Na)e F0G F l(ste!%G

The FCl(ste! I5F and FCl(ste! Na)eF are arbitrary( The FCl(ste! I5F must be a valid number between A and @@@@@ and the FCl(ste! Na)eF can be any alpha strin1 up to 0, characters in len1th( Con+i1urin1 4odes? &mit "AC # -D Cluster Con+i1uration -D Cluster Topolo1y -D Con+i1ure 4odes -D Add Cluster 4odes *ast#ath? C s)it )D onfi&Dno"es$a""
Add Cluster Nodes Type or select values in entry fields. Press Enter AFTE ma!in" all desired

c#an"es.

$Entry Fields% * Node Names [node1 node2]

FNo"e Na)esF should be the hostnames o+ the nodes( They must be alpha numeric and contain no more than 0, characters( All nodes participatin1 in the cluster must be entered on this screen separated by a space( 4e)t to be con+i1ured is the networ3 adapters( This e)ample will utili5e two ethernet adapters on each node as well as one R&,0, serial port on each node +or heartbeat( No"e Na)e node. a""!ess .@,(.;J(A(. .@,(.;J(.(. $dev$ttyA node, .@,(.;J(A(, .@,(.;J(.(, $dev$ttyA node,srvc node,stby I0 3a+el (/et /hosts) node.srvc node.stby Type service standby serial service standby serial

The +ollowin1 screens are con+i1uration settin1s needed to con+i1ure the above networ3s into the cluster con+i1uration? &mit "AC # -D Cluster Con+i1uration -D Cluster Topolo1y -D Con+i1ure 4odes -D Add an Adapter *ast#ath? C s)it )D onf&Da"apte!s$a""
Add an Adapter Type or select values in entry fields. Press Enter AFTER ma in! all desired c"an!es.

[Entry Fields] * Adapter #P $a%el [node1srvc] * Net&or Type [et"er] * Net&or Name [ipa] * Net&or Attri%ute public * Adapter Function service Adapter &dentifier $%

' ' ' '

Adapter 'ard(are Address $% Node Name [node1] '

It is important to note that the FA"apte! I0 3a+elF must match what is in the F$etc$hostsF +ile otherwise the adapter will not map to a valid I# address and the cluster will not synchroni5e( The FNetwo!; Na)eF is an arbitrary name +or the networ3 con+i1uration( All the adapters in this ether con+i1uration should have the same FNetwo!; Na)eF( This name is used to determine what adapters will be used in the event o+ an adapter +ailure(
Add an Adapter Type or select values in entry fields. Press Enter AFTER ma in! all desired c"an!es.

[Entry Fields] * Adapter #P $a%el [node1st%y] * Net&or Type [et"er] ' * Net&or Name [ipa] ' * Net&or Attri%ute public ' * Adapter Function stand%y ' Adapter &dentifier $% Adapter 'ard(are Address $% Node Name [node1] '

A"" an A"apte! Type o! sele t *al(es in ent!y fiel"s$ 0!ess Bnte! A:TBR )a;in& all "esi!e" han&es$ FBnt!y :iel"sG Fno"e7s!* G Fethe!G FipaG public se!*i e HI HI Fno"e7G

E A"apte! I0 3a+el E Netwo!; Type E Netwo!; Na)e E Netwo!; Att!i+(te E A"apte! :(n tion Adapter Identi+ier Adapter "ardware Address No"e Na)e

H H H H H

A"" an A"apte! Type o! sele t *al(es in ent!y fiel"s$ 0!ess Bnte! A:TBR )a;in& all "esi!e" han&es$ FBnt!y :iel"sG Fno"e7st+yG Fethe!G FipaG public stan"+y HI HI Fno"e7G

E A"apte! I0 3a+el E Netwo!; Type E Netwo!; Na)e E Netwo!; Att!i+(te E A"apte! :(n tion Adapter Identi+ier Adapter "ardware Address No"e Na)e

H H H H H

The +ollowin1 is the serial con+i1uration?

Add an Adapter Type or select values in entry fields. Press Enter AFTER ma in! all desired c"an!es.

[Entry Fields] * Adapter #P $a%el [node1(tty] * Net&or Type [rs2)2] ' * Net&or Name [serial] ' * Net&or Attri%ute serial ' * Adapter Function service ' Adapter #dentifier [*dev*tty+] Adapter 'ard(are Address $% Node Name

[node1]

'

Add an Adapter Type or select values in entry fields. Press Enter AFTER ma in! all desired c"an!es.

[Entry Fields] * Adapter #P $a%el [node2(tty] * Net&or Type [rs2)2] ' * Net&or Name [serial] ' * Net&or Attri%ute serial ' * Adapter Function service ' Adapter #dentifier [*dev*tty+] Adapter 'ard(are Address $% Node Name [node2] '

&ince this is not on the same networ3 as the ethernet cards the FNetwo!; Na)eF is di++erent( The same name is used +or the networ3 name( >se Fsmit m3ttyF to con+i1ure the R&,0, adapters? C s)it );tty
Add a TT) Type or select values in entry fields. Press Enter AFTE ma!in" all desired c#an"es. $T*P% $Entry Fields% TT) type tty

TT) interface rs+,+ -escription Async#ronous Terminal Parent adapter sa0 * P,RT num%er [+] ' Ena%le $,-#N disa%le ' .A/- rate $0100% 2 PA &T) $none% 2 .&T3 per c#aracter $4% 2 Number of 3T*P .&T3 $5% 2 T&6E before advancin" to ne7t port settin" $0% 28 TE 6&NA9 type $dumb% F9*: C*NT *9 to be used $7on% 2 $6* E...,5%

Be sure that FBna+le 3'-INF is set to the de+ault o+ FdisableF( The F0'RT n()+e!F is the value that is to be used in the $dev$ttG where FCF is the port number( &o i+ you de+ined this as F0F the device would be F$dev$ttyAF(

7$6 Syn h!oniIin& the Cl(ste! Topolo&y

A+ter the topolo1y is con+i1ured it needs to be synchroni5ed( The synchroni5ation per+orms topolo1y sanity chec3s as well as pushes the con+i1uration data to each o+ the nodes in the cluster con+i1uration( *or the synchroni5ation to wor3 user e/uivalence must be con+i1ured +or the root user( There is several ways to do this( One way would be to create a F(rhostsF +ile on each node in the F$F directory( %)ample o+ a F(rhostsF +ile?
node1 root node2 root .e sure permissions on t#e ;/.r#osts; file is 100. . c"mod /++ *.r"osts /se a remote command suc# as ;rcp; to test e<uivalence from eac# node= From node5= . rcp *etc*!roup node20*tmp Frome node+= . rcp *etc*!roup node10*tmp

8iew your IB operatin1 system documentation +or more in+ormation or contact IB any /uestions or problems settin1 up user e/uivalence +or the root user( *ast#ath? C s)it onfi& h;$"ialo&
3ync#roni>e Cluster Topolo"y

support i+ you have

&mit "AC # -D Cluster Con+i1uration -D Cluster Topolo1y -D &ynchroni5e Cluster Topolo1y

Type or select values in entry fields. Press Enter AFTE ma!in" all desired c#an"es. $T*P% $Entry Fields% #!nore 1luster 2erification Errors3 [No] ' * Emulate or Actual3 [Actual] ' Note= *nly t#e local node?s default confi"uration files !eep t#e c#an"es you ma!e for topolo"y -A E emulation. *nce you run your emulation@ to restore t#e ori"inal confi"uration rat#er t#an runnin" an actual -A E@ run t#e 36&T command@ ; estore 3ystem -efault Confi"uration from Active Confi"uration.; :e recommend t#at you ma!e a snaps#ot before runnin" an emulation@ Aust in case uncontrolled cluster events #appen durin" emulation. N*TE= &f t#e Cluster 6ana"er is active on t#is node@ sync#roni>in" t#e Cluster Topolo"y (ill cause t#e Cluster 6ana"er to ma!e any c#an"es ta!e effect once t#e sync#roni>ation #as successfully completed. $.*TT*6%

7$2 Confi&(!in& Cl(ste! Reso(! es

In a RAC con+i1uration only one resource 1roup is re/uired( This resource 1roup is a concurrent 1roup +or the shared volume 1roup( The +ollowin1 are the steps to add a concurrent resource 1roup +or a shared volume 1roup? *irst there needs to be a volume 1roup that is shared between the nodes( S/ARB5 3'-ICA3 J'31MB MANA-BR . S/ARB5 C'NC1RRBNT 5ISKS ( N' JS5 ) The two instances o+ the same cluster database have a concurrent access on the same e)ternal dis3s( This is real concurrent access and not a shared one li3e in the 8&2 environment( Because several instances access at the same time the same +iles and data, loc3s have to be mana1ed( These loc3s, at the C78 layer (includin1 memory cache), are mana1ed by "AC #( .) Chec3 i+ the tar1et dis3s are physically lin3ed to the two machines o+ the cluster, and seen by both( Type the lsp* command on both machines( 4ote ? the hdis3 number can be di++erent, dependin1 on the others nodes dis3 con+i1urations( >se the second +ield o+ the output (#8id) o+ lspv to be sure you are dealin1 with the same physical dis3 +rom two hosts( Althou1h hdis3 inconsistency may not be a problem IB su11ests usin1 1host dis3s to ensure hdis3 numbers match between the nodes( Contact IB +or +urther in+ormation on this topic(

7$2$% C!eate *ol()e &!o(ps to +e sha!e" on (!!ently on one no"e

G s)it *& &elect FA"" a Jol()e -!o(pF Type or select values in entry +ields(
Add a Bolume Croup Type or select values in entry fields. Press Enter AFTE ma!in" all desired c#an"es.

$Entry Fields% 2,$45E -R,4P name [oracle(v!] P"ysical partition 6#7E in me!a%ytes )2 ' * P896#1A$ 2,$45E names ["dis :] ' Activate volume !roup A4T,5AT#1A$$9 no ' at system restart3 2olume -roup 5A;,R N45<ER [:=] '. 1reate 2- 1oncurrent 1apa%le3 yes ' Auto>varyon in 1oncurrent 5ode3 no '

The F0/@SICA3 J'31MB na)esF must be physical dis3s that are shared between the nodes( 6e do not want the volume 1roup automatically activated at system startup because "AC # activates it( Also FA(to*a!yon in Con (!!ent Mo"eLF should be set to FnoF because "AC # varies it on in concurrent mode( Bou must choose the maCor number to be sure the volume 1roups have the same maCor number in all the nodes (attention, be+ore choosin1 this number, you must be sure its +ree on all the nodes)( To chec3 all de+ined maCor number, type? = ls al /"e*/E crw-rw---- . root system :=, A Au1 A, .0?0@ $dev$oracleEv1 The maCor number +or oracleEv1 volume 1roup is :=( %nsure that := is available on all the other nodes and is not used by another device( I+ it is +ree then ma3e use o+ the same on all nodes( On this volume 1roup, create all the lo1ical volumes and +ile systems you need +or the cluster database(

7$2$7 C!eate Sha!e" RAM 3o&i al Jol()es if not (sin& -0:S$ See se tion 7$2$6 fo! "etails a+o(t -0:S$
m3lv -yKdbDnameDcntrl5D550mK -wKnK -sKnK -rKnK usupportEv1 ' hdis3: m3lv -yKdbDnameDcntrl+D550mK -wKnK -sKnK -rKnK usupportEv1 ' hdis3: m3lv -yKdbDnameDsystemDE00mK -wKnK -sKnK -rKnK usupportEv1 .0 hdis3: m3lv -yKdbDnameDusersD5+0mK -wKnK -sKnK -rKnK usupportEv1 ' hdis3: m3lv -yKdbDnameDdrsysD00mK -wKnK -sKnK -rKnK usupportEv1 0 hdis3: m3lv -yKdbDnameDtoolsD5+mK -wKnK -sKnK -rKnK usupportEv1 . hdis3:

m3lv -yKdbDnameDtempD500mK -wKnK -sKnK -rKnK usupportEv1 ' hdis3: m3lv -yKdbDnameDundotbs5D,5+mK -wKnK -sKnK -rKnK usupportEv1 .A hdis3: m3lv -yKdbDnameDundotbs+D,5+mK -wKnK -sKnK -rKnK usupportEv1 .A hdis3: m3lv -yKdbDnameDlo"55D5+0mK -wKnK -sKnK -rKnK usupportEv1 ' hdis3: m3lv -yKdbDnameDlo"5+D5+0mK -wKnK -sKnK -rKnK usupportEv1 ' hdis3: m3lv -yKdbDnameDlo"+5D5+0mK -wKnK -sKnK -rKnK usupportEv1 ' hdis3: m3lv -yKdbDnameDlo"++D5+0mK -wKnK -sKnK -rKnK usupportEv1 ' hdis3: m3lv -yKdbDnameDind7DF0mK -wKnK -sKnK -rKnK usupportEv1 0 hdis3: m3lv -yKdbDnameDc(mliteD500m? -wKnK -sKnK -rKnK usupportEv1 ' hdis3: m3lv -yKdbDnameDe7ampleD510mK -wKnK -sKnK -rKnK usupportEv1 : hdis3: m3lv -yKdbDnameDoemrepoD+0mK -wKnK -sKnK -rKnK usupportEv1 . hdis3: m3lv -yKdbDnameDspfileDGmK -wKnK -sKnK -rKnK usupportEv1 . hdis3: m3lv -yKdbDnameDsrvmconfD500mK -wKnK -sKnK -rKnK usupportEv1 ' hdis3: &ubstitute your database name in place o+ the FdbEnameF value( 6hen the volume 1roup was created a partition si5e o+ 0, me1abytes was used( The seventh +ield is the number o+ partitions that ma3e up the +ile so +or e)ample i+ FdbDnameDcntrl5D550m; needs to be 550 me"abytes (e (ould need E partitions.
T#e ra( partitions are created in t#e ;/dev; directory and it is t#e c#aracter devices t#at (ill be used. T#e ;);l* -yNd%(name(cntrl1(11+mN -wNnN -sNnN -!NnN (s(ppo!tD*& 2 h"is;8F

creates two +iles? /"e*/"+Dna)eD nt!l%D%%0) /"e*/!"+Dna)eD nt!l%D%%0) Chan1e the permissions on the character devices so the so+tware owner owns them? C hown o!a le:"+a /"e*/!"+Dna)eE

7$2$6 I)po!t the Jol()e -!o(p on to the 'the! No"es

>se Fimportv1F to import the oracleEv1 volume 1roup on all o+ the other nodes On the +irst machine, type? = *a!yoff*& o!a leD*& On the other nodes, import the de+inition o+ the volume 1roup usin1 Fs)it *&F ? &elect FI)po!t a Jol()e -!o(pF Type or select values in entry +ields( #ress %nter A*T%R ma3in1 all desired chan1es(
&mport a Bolume Croup Type or select values in entry fields. Press Enter AFTE ma!in" all desired c#an"es.

$Entry Fields% 2,$45E -R,4P name [oracle(v!] * P896#1A$ 2,$45E name ["dis :] ' 2olume -roup 5A;,R N45<ER

[:=] '. 5a e t"is 2- 1oncurrent 1apa%le3 no ' 5a e default varyon of 2- 1oncurrent3 no '

It is possible that the physical volume name (hdis3) could be di++erent on each node( Chec3 the #8I2 o+ the dis3 usin1 Flsp*F, and be sure to pic3 the hdis3 that has the same #8I2 as the dis3 used to create the volume 1roup on the +irst node( Also ma3e sure the same maCor number is used as well(( This number has to be unde+ined on all the nodes( The FMa;e "efa(lt *a!yon of J- Con (!!entLF option should be set to FnoF( The volume 1roup was created concurrent capable so the option FMa;e this J- Con (!!ent Capa+leLF can be le+t at FnoF( The command line +or importin1 the volume 1roup a+ter varyin1 it o++ on the node where the volume 1roup was or1inally created on would be? = i)po!t*& -J>)aOo! C? -y >*&na)e? h "is;C = h*& -an >*&na)e? = *a!yoff*& >*&na)e? A+ter importin1 the volume 1roup onto each node be sure to chan1e the ownership o+ the character devices to the so+tware owner? C hown o!a le:"+a /"e*/!"+Dna)eE

7$2$2 A"" a Con (!!ent Cl(ste! Reso(! e -!o(p

The shared resource in this e)ample is ForacleEv1F( To create the concurrent resource 1roup that will mana1e ForacleEv1F do the +ollowin1? &mit "AC # -D Cluster Con+i1uration -D Cluster Resources -D 2e+ine Resource 9roups -D Add a Resource 9roup *ast#ath? C s)it )Da""D&!p
Add a esource Croup

Type or select values in entry fields. Press Enter AFTE ma!in" all desired c#an"es.

$Entry Fields% * Resource -roup Name [s"ared(v!] * Node Relations"ip concurrent ' * Participatin! Node Names [node1 node2] '

The FReso(! e -!o(p Na)eF is arbitrary and is used when selectin1 the resource 1roup +or con+i1uration( Because we are con+i1urin1 a shared resources the FNo"e RelationshipF is FconcurrentF meanin1 a 1roup o+ nodes that will share the resource( F0a!ti ipatin& No"e Na)esF is a space separated list o+ the nodes that will be sharin1 the resource(

7$2$8 Confi&(!e the Con (!!ent Cl(ste! Reso(! e -!o(p

Once the resource 1roup is added it can then be con+i1ured with?

&mit "AC # -D Cluster Con+i1uration -D Cluster Resources -D Chan1e$&how Resources +or a Resource 9roup *ast#ath? C s)it )D f&D!es$sele t
Confi"ure esource Croup Type or select values in entry fields. Press Enter AFTE ma!in" all desired c#an"es. $T*P% $Entry Fields% Resource -roup Name concurrent(!roup Node Relations"ip concurrent Participatin! Node Names opc%ai?1 opc%ai?2 3ervice &P label Filesystems esources for a

$% $%

2 Filesystems Consistency C#ec! fsc! 2 Filesystems ecovery 6et#od se<uential 2 Filesystems to E7port $% 2 Filesystems to NF3 mount $% 2 Bolume Croups $% 2 1oncurrent 2olume !roups [oracle(v!] ' Ra& @is P2#@s [+++A1AB/e%C+e%%=] ' A&H Connections 3ervice $% 2 A&H Fast Connect 3ervices $% 2 Application 3ervers $% 2 'i"#ly Available Communication 9in!s $% 2 6iscellaneous -ata $% &nactive Ta!eover Activated false 2 0,,, -is! Fencin" Activated false 2 33A -is! Fencin" Activated false 2

Filesystems mounted before &P confi"ured false 2 $.*TT*6%

4ote that the settin1s +or FReso(! e -!o(p Na)eF, FNo"e RelationshipF and F0a!ti ipatin& No"e Na)esF comes +rom the data entered in the previous menu( FCon (!!ent Jol()e &!o(psF needs to be a pre-created volume 1roup on shared stora1e( The FRaw 5is; 0JI5sF are the physical volumes I2s +or each o+ the dis3s that ma3e up the FCon (!!ent Jol()e &!o(psF( It is important to note that you a resource 1roup mana1e multiple concurrent resources( In such a case separate each volume 1roup name with a space( Also, the FRaw 5is; 0JI5sF will be a space delimited list o+ all the physical volume I2s that ma3e up the concurrent volume 1roup list( Alternatively each volume 1roup can be con+i1ured in its own concurrent resource 1roup(

7$2$6 C!eatin& 0a!allel :ilesyste)s (-0:S)

6ith AI! :(. (:7) you can also place your +iles on 9#*& (RA6 7o1ical 8olumes are not a re/uirement o+ 9#*&)( In this case create 9#*& capable o+ holdin1 all re/uired 2atabase *iles, Control+iles and 7o1+iles(

7$8 Syn h!oniIin& the Cl(ste! Reso(! es

A+ter con+i1urin1 the resource 1roup a resource synchroni5ation is needed( &mit "AC # -D Cluster Con+i1uration -D Cluster Resources -D &ynchroni5e Cluster Resources *ast#ath? C s)it lsyn no"e$"ialo&
Type or select values in entry fields. Press Enter AFTE ma!in" all desired c#an"es. $T*P% $Entry Fields% &"nore Cluster Berification ErrorsI $No% 2 /n/Confi"ure Cluster esourcesI $)es% 2 J Emulate or ActualI $Actual% 2 Note= *nly t#e local node?s default confi"uration files !eep t#e c#an"es you ma!e for resource -A E emulation. *nce you run your emulation@ to restore t#e ori"inal confi"uration rat#er t#an runnin" an actual -A E@ run t#e 36&T command@ ; estore 3ystem -efault Confi"uration from Active Confi"uration.; :e recommend t#at you ma!e a snaps#ot before runnin" an emulation@ Aust in case

uncontrolled cluster events #appen durin" emulation. $.*TT*6%

<ust 3eep the de+aults(

7$6 Poinin& No"es Into the Cl(ste!

A+ter the cluster topolo1y and resources are con+i1ured the nodes can Coin the cluster( It is important to start one node at a time unless usin1 C-&#OC (Cluster-&in1le #oin1 o+ Control)( *or more in+ormation on usin1 C-&#OC consult IB Ks "AC # speci+ic documentation( The use o+ C-&#OC will not be covered in this document( &tart cluster services by doin1 the +ollowin1? &mit "AC # -D Cluster &ervices -D &tart Cluster &ervices *ast#ath? C s)it lsta!t$"ialo&
Type or select values in entry fields. Press Enter AFTE ma!in" all desired c#an"es.

$Entry Fields% * 6tart no&D on system restart or %ot" no& ' <R,A@1A6T messa!e at startup3 false ' 6tartup 1luster $oc 6ervices3 false ' 6tartup 1luster #nformation @aemon3 true '

&ettin1 FSta!t now. on syste) !esta!t o! +othF to FnowF will start the "AC # daemons immediately( F!esta!tF will update the F$etc$inittabF with an entry to start the daemons at reboot and F+othF will do e)actly that, update the F$etc$inittabF and start the daemons immediately( FBR'A5CAST )essa&e at sta!t(pL Q can either be Ft!(eF or FfalseF( I+ set to Ft!(eF wall type messa1e will be displayed when the node is Coinin1 the cluster( FSta!t(p Cl(ste! 3o ; Se!*i esLF should be set to FfalseF +or a RAC con+i1uration( &ettin1 this parameter to Ft!(eF will prevent the cluster +rom wor3in1 but the added daemon is not used( I+ FclstatF is 1oin1 to be used to to monitor the cluster the F&tartup Cluster In+ormation 2aemonLF will need to be set to Ft!(eF( 8iew the F$etc$hacmp(outF +ile +or startup messa1es( 6hen you see somethin1 similar to the +ollowin1 it is sa+e to start the cluster services on the other nodes?
May 76 0#:6%:26 BJBNT C'M03BTB5: no"eD(pD o)plete no"e%

6hen Coinin1 nodes into the cluster the other nodes will report a success+ul Coin in their F$tmp$hacmp(outF +iles?
May 76 0#:62:%% BJBNT C'M03BTB5: no"eD(pD o)plete no"e%

7$A Basi Cl(ste! A")inist!ation

The F$tmp$hacmp(outF is the best place to loo3 +or cluster in+ormation( FclstatF can also be used to veri+y cluster health( The FclstatF pro1ram can ta3e a while to update with the latest cluster in+ormation and at times does not wor3 at all( Also you must have the FSta!t(p Cl(ste! Info!)ation 5ae)onLF set to Ft!(eF when startin1 cluster services( >se the +ollowin1 command to start FclstatF? C /(s!/es/s+in/ l(ste!/ lstat
clstat - 'AC6P for A&H Cluster 3tatus 6onitor -------------------------------------------Cluster= cluster5 K0L Tue Mul + 04=,4=01 E-T +00+ 3tate= /P Nodes= + 3ub3tate= 3TA.9E Node= node5 3tate= /P &nterface= node5 K0L Address= 50+.514.0.5 3tate= /P

Node= node+ 3tate= /P &nterface= node+ K0L Address= 50+.514.0.+ 3tate= /P

One other way to chec3 the cluster status is by /ueryin1 the FsnmpdF daemon with Fsnmpin+oF? C /(s!/s+in/sn)pinfo -) &et -o /(s!/es/s+in/ l(ste!/ha )p$"efs -* Cl(ste!S(+state$0 This should return F0,F? l(ste!S(+State$0 R 67 I+ other values are returned +rom any node consult your IB support( &mit "AC # -D Cluster &ervices -D &how Cluster &ervices
C*66AN3TAT/3 Command= *N stderr= no stdout= yes

"AC # documentation or contact IB

Bou can 1et a /uic3 view o+ the "AC # speci+ic daemons with?

.efore command completion@ additional instructions may appear belo(. 3ubsystem P&3tatus clstrm"rE3 ++000 active Croup cluster

clinfoE3 +5,0E active clsmu7pdE3 5E,E+ active clloc!dE3 inoperative clresm"rdE3 +0F+0 active

cluster cluster loc!

Sta!tin& < Stoppin& Cl(ste! No"es To Ooin an" e*i t no"es f!o) the l(ste! (se: &mit "AC # -D Cluster &ervices -D &tart Cluster &ervices &ee section ,(; +or more in+ormation on Coinin1 a node into the cluster( >se the +ollowin1 to evict a node +rom the cluster? &mit "AC # -D Cluster &ervices -D &top Cluster &ervices *ast#ath? C s)it lstop$"ialo&
3top Cluster 3ervices Type or select values in entry fields. Press Enter AFTE ma!in" all desired c#an"es.

$Entry Fields% * 6top no&D on system restart or %ot" no& ' <R,A@1A6T cluster s"utdo&n3 true ' * 6"utdo&n mode !raceful ' K"raceful or "raceful (it# ta!eover@ forcedL

&ee section ,(; FPoinin& No"es Into the Cl(ste!Q +or and e)planation o+ FStop now. on syste) !esta!t o! +othF and FBR'A5CAST l(ste! sh(t"ownLF( The F&hutdown modeF determines whether or not resources are 1oin1 to move between nodes i+ a shutdown occurs( Ffo! e"F is new with '('(. o+ "AC # and will leave applications runnin1 that are controlled by "AC # events when the shutdown occurs( F&!a ef(lF will brin1 everythin1 down but cascadin1 and rotatin1 resources are not switched where as with F&!a ef(l with ta;eo*e!F these resources will be switched at shutdown( 3o& :iles fo! /ACM0/BS All cluster recon+i1uration in+ormation durin1 cluster startup and shutdown 1oes into the F$tmp$hacmp(outF(

6$0 0!epa!in& fo! the installation of RAC

The Real Application Clusters installation process includes +our maCor tas3s( Con+i1ure the shared dis3s and >4I! preinstallation tas3s( Run the Oracle >niversal Installer to install the Oracle@i %nterprise %dition and the Oracle@i Real Application Clusters so+tware( Create and con+i1ure your database(

6$% Confi&(!e the sha!e" "is;s an" 1NIX p!einstallation tas;s 6$%$% Confi&(!e the sha!e" "is;s
Real Application Clusters re/uires that all each instance be able to access a set o+ un+ormatted devices on a shared dis3 subsystem i+ 9#*& is not bein1 used( These shared dis3s are also re+erred to as raw devices( I+ your plat+orm supports an Oracle-certi+ied cluster +ile system, however, you can store the +iles that Real Application Clusters re/uires directly on the cluster +ile system( Note: I+ you are usin1 #arallel *ilesystem (9#*&), however, you can store the +iles that Real Application Clusters re/uires directly on the cluster +ile system M The Oracle instances in Real Application Clusters write data onto the raw devices to update the control +ile, server parameter +ile, each data+ile, and each redo lo1 +ile( All instances in the cluster share these +iles( The Oracle instances in the RAC con+i1uration write in+ormation to raw devices de+ined +or? The control +ile The sp+ile(ora %ach data+ile %ach O47I4% redo lo1 +ile &erver ana1er (&R8 ) con+i1uration in+ormation

It is there+ore necessary to de+ine raw devices +or each o+ these cate1ories o+ +ile( The Oracle 2atabase Con+i1uration Assistant (2BCA) will create a seed database e)pectin1 the +ollowin1 con+i1uration?

Raw Jol()e
&B&T% tablespace

:ile SiIe
'AA b .,A b .AA b 0., b .AA

Sa)ple :ile Na)e

dbDnameDra(DsystemDE00m

>&%R& tablespace T% # tablespace >42OTB& tablespace per instance C6 7IT% tablespace

dbDnameDra(DusersD5+0m

dbDnameDra(DtempD500m

dbDnameDra(Dundotbs7D,5+m dbDnameDra(Dc(mliteD500m

b %!A #7% O% R%#O I42! tablespace TOO7& tablespace 2RB&B& tablespace *irst control +ile &econd control +ile Two O47I4% redo lo1 +iles per instance sp+ile(ora srvmcon+i1 .;A b ,A =A ., @A ..A b ..A b b b b b
dbDnameDra(De7ampleD510m dbDnameDra(DoemrepoD+0m dbDnameDra(Dind7DF0m dbDnameDra(DtoolsD5+m dbDnameDra(DdrsysD00m dbDnameDra(Dcontrolfile5D5 50m dbDnameDra(Dcontrolfile+D5 50m

dbDnameDt#readDlo"numberD5 .,A b ) , +0m

dbDnameDra(DspfileDGm dbDnameDra(DsrvmconfD500m

.AA b

4ote? Automatic >ndo ana1ement re/uires an undo tablespace per instance there+ore you would re/uire a minimum o+ , tablespaces as described above( By +ollowin1 the namin1 convention described in the table above, raw partitions are identi+ied with the database and the raw volume type (the data contained in the raw volume)( Raw volume si5e is also identi+ied usin1 this method( 4ote? In the sample names listed in the table, the strin1 dbEname should be replaced with the actual database name, thread is the thread number o+ the instance, and lo1number is the lo1 number within a thread( On the node +rom which you run the Oracle >niversal Installer, create an A&CII +ile identi+yin1 the raw volume obCects as shown above( The 2BCA re/uires that these obCects e)ist durin1 installation and database creation( 6hen creatin1 the A&CII +ile content +or the obCects, name them usin1 the +ormat? databaseEobCectNrawEdeviceE+ileEpath 6hen you create the A&CII +ile, separate the database obCects +rom the paths with e/uals (N) si1ns as shown in the e)ample below?
system5O/dev/rdbDnameDsystemDE00m spfile5O/dev/rdbDnameDspfileDGm users5O/dev/rdbDnameDusersD5+0m temp5O/dev/rdbDnameDempD500m undotbs5O/dev/rdbDnameDundotbs5D,5+m undotbs+O/dev/rdbDnameDundotbs+D,5+m e7ample5O/dev/rdbDnameDe7ampleD510m

c(mlite5O/dev/rdbDnameDc(mliteD500m ind75O/dev/rdbDnameDind7DF0m tools5O/dev/rdbDnameDtoolsD5+m drsys5O/dev/rdbDnameDdrsysD00m control5O/dev/rdbDnameDcntrl5D550m control+O/dev/rdbDnameDcntrl+D550m redo5D5O/dev/rdbDnameDlo"55D5+0m redo5D+O/dev/rdbDnameDlo"5+D5+0m redo+D5O/dev/rdbDnameDlo"+5D5+0m redo+D+O/dev/rdbDnameDlo"++D5+0m

Bou must speci+y that Oracle should use this +ile to determine the raw device volume names by settin1 the +ollowin1 environment variable where +ilename is the name o+ the A&CII +ile that contains the entries shown in the e)ample above? csh?
setenv @<1A(RAE(1,NF#- filename !s#@ bas# or s#= @<1A(RAE(1,NF#-FfilenameG e?port @<1A(RAE(1,NF#-

6$%$7 1NIX 0!einstallation Steps

4ote? In addition, you can run the install#rep(sh script provided in 4ote .J@,:;(. which catches most uni) environment problems( A+ter con+i1urin1 the raw volumes, per+orm the +ollowin1 steps prior to installation as root user? A"" the '!a le 1SBR a3e sure you have an osdba 1roup de+ined in the /etc/"roup +ile on all nodes o+ your cluster( To desi1nate an osdba 1roup name and 1roup number and osoper 1roup durin1 installation, these 1roup names must be identical on all nodes o+ your >4I! cluster that will be part o+ the Real Application Clusters database( The de+ault >4I! 1roup name +or the osdba and osoper 1roups is dba( There also needs be an oinstall 1roup which the so+tware owner should have as its primary 1roup( A typical entry would there+ore loo3 li3e the +ollowin1?
d%a001+10oracle oinstall001+20rootDoracle

The +ollowin1 is an e)ample o+ the command used to create the FdbaF 1roup with a 1roup I2 o+ F.A.F? C );&!o(p -NAN i"RN%0%N (se!sRNo!a leN "+a Create an oracle account on each node so that the account? Is a member o+ the osdba 1roup (dba in this e)ample) "as oinstall as its primary 1roup Is used only to install and update Oracle so+tware "as write permissions on remote directories

The +ollowin1 is an e)ample o+ the smit command used to create the ForacleF user? &mit -D &ecurity O >sers -D >sers -D Add a >ser

*astpath? C s)it );(se!

Type or select values in entry fields. Press Enter AFTE ma!in" all desired c#an"es. $T*P% $Entry Fields% * 4ser NA5E [oracle] 4ser #@ [1+1] . A@5#N#6TRAT#2E 46ER3 false ' Primary -R,4P [oinstall] ' Croup 3ET $% 2 A-6&N&3T AT&BE C */P3 $% 2 *9E3 $% 2 Anot#er user can 3/ T* /3E I true 2 3/ C */P3 $A99% 2 8,5E directory [*"ome*oracle] #nitial PR,-RA5 [*%in* s"] /ser &NF* 6AT&*N $% EHP& AT&*N date K66--##mmyyL $0%

4ote that the primary 1roup is not FdbaF( The FuseF o+ FoinstallF is optional but recommended( *or more in+ormation on the use o+ the FoinstallF 1roup see the ? Oracle@i Installation 9uide Release , (@(!(!(!(A) +or >4I! &ystems? AI!-Based &ystems, Compa/ Tru;' >4I!, "# @AAA &eries "#->!, 7inu) Intel and &un &#ARC &olaris documentation( Create a mount point directory on each node to serve as the top o+ your Oracle so+tware directory structure so that? The name o+ the mount point on each node is identical to that on the initial node The oracle account has read, write, and e)ecute privile1es

On the node +rom which you will run the Oracle >niversal Installer, set up user e/uivalence by addin1 entries +or all nodes in the cluster, includin1 the local node, to the (rhosts +ile o+ the oracle account, or the /etc/#osts.e<uiv +ile( As oracle account user, chec3 +or user e/uivalence +or the oracle account by per+ormin1 a remote lo1in (rlo"in) to each node in the cluster( As oracle account user, i+ you are prompted +or a password, you have not 1iven the oracle account the same attributes on all nodes( Bou must

correct this because the Oracle >niversal Installer cannot use the rcp command to copy Oracle products to the remote nodeKs directories without user e/uivalence( Bsta+lish syste) en*i!on)ent *a!ia+les &et a local %in directory in the userKs #AT", such as /usr/local/bin, or /opt/bin( It is necessary to have e)ecute permissions on this directory( &et the -&3P9A) variable to point to the systemKs (+rom where you will run O>I) I# address, or name, ! server, and screen( &et a temporary directory path +or T #2IR with at least ,A write permission( b o+ +ree space to which the O>I has

Bsta+lish '!a le en*i!on)ent *a!ia+les? &et the +ollowin1 Oracle environment variables? Bn*i!on)ent Ja!ia+le ORAC7%EBA&% ORAC7%E"O % ORAC7%ET%R 47&E7A49 ORAE47&00 #AT" C7A&&#AT" S(&&este" *al(e e1 /u05/app/oracle e1 /u05/app/oracle/product/005 )term A %RICA4-A %RICA(>T*J +or e)ample
$* AC9ED'*6E/ocommon/nls/admin/data

&hould contain $* AC9ED'*6E/bin

$* AC9ED'*6E/M E=$* AC9ED'*6E/Alib P $* AC9ED'*6E/rdbms/Alib= P $* AC9ED'*6E/net(or!/Alib

Create the directory /var/opt/oracle and set ownership to the oracle user(

4ote? There is a veri+ication script Install#rep(sh available which may be downloaded and run prior to the installation o+ Oracle Real Application Clusters( This script veri+ies that the system is con+i1ured correctly accordin1 to the Installation 9uide( The output o+ the script will report any +urther tas3s that need to be per+ormed be+ore success+ully installin1 Oracle @() 2ata&erver (R2B &)( This script per+orms the +ollowin1 veri+ications?ORAC7%E"O % 2irectory 8eri+ication >4I! >ser$umas3 8eri+ication >4I! 9roup 8eri+ication emory$&wap 8eri+ication T # &pace 8eri+ication Real Application Cluster Option 8eri+ication >ni) Pernel 8eri+ication ($Install#rep(sh Bou are currently lo11ed on as oracle

Is oracle the uni) user that will be installin1 Oracle &o+twareL y or n y %nter the uni) 1roup that will be used durin1 the installation 2e+ault? dba dba %nter 7ocation where you will be installin1 Oracle 2e+ault? $uA.$app$oracle$product$oracle@i $uA.$app$oracle$product$@(,(A(. Bour Operatin1 &ystem is AI! 9atherin1 in+ormation((( #lease wait Chec3in1 uni) user ((( user test passed Chec3in1 uni) umas3 ((( umas3 test passed Chec3in1 uni) 1roup ((( >ni) 9roup test passed Chec3in1 emory O &wap((( emory test passed $tmp test passed Chec3in1 +or a cluster((( AI! Cluster test Cluster has been detected Bou have , cluster members con+i1ured and , are curently up 4o cluster warnin1s detected #rocessin1 3ernel parameters((( #lease wait Runnin1 Pernel #arameter Report((( Chec3 the report +or Pernel parameter veri+ication Completed( $tmp$OracleEInstall#repEReport has been 1enerated #lease review this report and resolve all issues be+ore attemptin1 to install the Oracle 2atabase &o+tware

6$7 1sin& the '!a le 1ni*e!sal Installe! fo! Real Appli ation Cl(ste!s
*ollow these procedures to use the Oracle >niversal Installer to install the Oracle %nterprise %dition and the Real Application Clusters so+tware( Oracle@i is supplied on multiple C2-RO dis3s( 2urin1 the installation process it is necessary to switch between the C2-RO &( O>I will mana1e the switchin1 between C2s( *or the latest RAC$IB certi+ication matri) see here(

To install the Oracle &o+tware, per+orm the +ollowin1?( 7o1in as the root user and mount the +irst C2-RO
# mount -rv cdrfs /dev/cd0 /cdrom

i+ installin1 +rom C2-RO

%)ecute the Frootpre(shF shell script on the C2-RO mount point or the location o+ 2is3. i+ installin1 +rom a dis3 sta1e( &ee the Oracle@i Installation 9uide Release , (@(!(!(!(A) +or >4I! &ystems? AI!-Based &ystems, Compa/ Tru;' >4I!, "# @AAA &eries "#->!, 7inu) Intel and &un &#ARC &olaris documentation +or more in+ormation on creatin1 dis3 sta1es(
# /<Location_Of_Install_Media>/rootpre.sh

7o1in as the oracle user and e)ecute the FrunInstallerF( &ee Q4ote?.:0@;A(.D i+ you e)perience problems startin1 the runInstaller(
$ /<Location_Of_Install_Media>/runInstaller

At the O>I 6elcome screen, clic3 Ne4t( A prompt will appear +or the Inventory 7ocation (i+ this is the +irst time that O>I has been run on this system)( This is the base directory into which O>I will install +iles( The Oracle Inventory de+inition can be +ound in the +ile /et /o!aInst$lo ( Clic3 'K( 8eri+y the >4I! 1roup name o+ the user who controls the installation o+ the Oracle@i so+tware( I+ an instruction to run /tmp/orainst oot.s# appears, the pre-installation steps were not completed success+ully( Typically, the /var/opt/oracle directory does not e)ist or is not writeable by oracle( Run /tmp/orainst oot.s# to correct this, +orcin1 Oracle Inventory +iles, and others, to be written to the * AC9ED'*6E directory( Once a1ain this screen only appears the +irst time Oracle@i products are installed on the system( Clic3 Ne4t( The *ile 7ocation window will appear( 5o N'T chan1e the &ource +ield( The 2estination +ield de+aults to the * AC9ED'*6E environment variable( Clic3 Ne4t( &elect the #roducts to install( In this e)ample, select the '!a le#i Se!*e! then clic3 Ne4t( &elect the installation type( Choose the Bnte!p!ise B"ition option( The selection on this screen re+ers to the installation operation, not the database con+i1uration( The ne)t screen allows +or a customi5ed database con+i1uration to be chosen( Clic3 Ne4t( &elect the con+i1uration type( In this e)ample you choose the Advanced Con+i1uration as this option provides a database that you can customi5e, and con+i1ures the selected server products( &elect C(sto)iIe" and clic3 Ne4t( &elect the other nodes on to which the Oracle R2B & so+tware will be installed( It is not necessary to select the node on which the O>I is currently runnin1( Clic3 Ne4t( Identi+y the raw partition in to which the Oracle@i Real Application Clusters (RAC) con+i1uration in+ormation will be written( It is recommended that this raw partition is a minimum o+ .AA B in si5e( An option to >p1rade or i1rate an e)istin1 database is presented( 2o N'T select the radio button( The Oracle i1ration utility is not able to up1rade a RAC database, and will error i+ selected to do so( The &ummary screen will be presented( Con+irm that the RAC database so+tware will be installed and then clic3 Install( The O>I will install the Oracle@i so+tware on to the local node, and then copy this in+ormation to the other nodes selected(

Once Install is selected, the O>I will install the Oracle RAC so+tware on to the local node, and then copy so+tware to the other nodes selected earlier( This will ta3e some time( 2urin1 the installation process, the O>I does not display messa1es indicatin1 that components are bein1 installed on other nodes - I$O activity may be the only indication that the process is continuin1(

6$6 C!eate a RAC 5ata+ase (sin& the '!a le 5ata+ase Confi&(!ation Assistant
The Oracle 2atabase Con+i1uration Assistant (2BCA) will create a database +or you (+or an e)ample o+ manual database creation see 2atabase Creation in Oracle@i RAC)( The 2BCA creates your database usin1 the optimal +le)ible architecture (O*A)( This means the 2BCA creates your database +iles, includin1 the de+ault server parameter +ile, usin1 standard +ile namin1 and +ile placement practices( The primary phases o+ 2BCA processin1 are?8eri+y that you correctly con+i1ured the shared dis3s +or each tablespace (+or non-cluster +ile system plat+orms) Create the database Con+i1ure the Oracle networ3 services &tart the database instances and listeners

Oracle Corporation recommends that you use the 2BCA to create your database( This is because the 2BCA precon+i1ured databases optimi5e your environment to ta3e advanta1e o+ Oracle@i +eatures such as the server parameter +ile and automatic undo mana1ement( The 2BCA also enables you to de+ine arbitrary tablespaces as part o+ the database creation process( &o even i+ you have data+ile re/uirements that di++er +rom those o++ered in one o+ the 2BCA templates, use the 2BCA( Bou can also e)ecute user-speci+ied scripts as part o+ the database creation process( The 2BCA and the Oracle 4et Con+i1uration Assistant also accurately con+i1ure your Real Application Clusters environment +or various Oracle hi1h availability +eatures and cluster administration tools( 2BCA will launch as part o+ the installation process, but can be run manually by e)ecutin1 the command dbca +rom the $* AC9ED'*6E/bin directory on >4I! plat+orms( The RAC 6elcome #a1e displays( Choose '!a le Cl(ste! 5ata+ase option and select Ne4t( The Operations pa1e is displayed( Choose the option C!eate a 5ata+ase and clic3 Ne4t( The 4ode &election pa1e appears( &elect the nodes that you want to con+i1ure as part o+ the RAC database and clic3 Ne4t( I+ nodes are missin1 +rom the 4ode &election then per+orm clusterware dia1nostics by e)ecutin1 the $* AC9ED'*6E/bin/lsnodes -v command and analy5in1 its output( Re+er to your vendorKs clusterware documentation i+ the output indicates that your clusterware is not properly installed( Resolve the problem and then restart the 2BCA( The 2atabase Templates pa1e is displayed( The templates other than 4ew 2atabase include data+iles( Choose New 5ata+ase and then clic3 Ne4t( The Show 5etails button provides in+ormation on the database template selected( 2BCA now displays the 2atabase Identi+ication pa1e( %nter the -lo+al 5ata+ase Na)e and '!a le Syste) I"entifie! (SI5)( The 9lobal 2atabase 4ame is typically o+ the +orm name(domain, +or e)ample )y"+$(s$o!a le$ o) while the &I2 is used to uni/uely identi+y an instance (2BCA should insert a su11ested &I2, e/uivalent to name. where name was entered in the 2atabase 4ame +ield)( In the RAC case the &I2 speci+ied will be used as a pre+i) +or the instance number( *or e)ample, M@5B, would become M@5B%. M@5B7 +or instance . and , respectively(

The 2atabase Options pa1e is displayed( &elect the options you wish to con+i1ure and then choose Ne4t( Note? I+ you did not choose 4ew 2atabase +rom the 2atabase Template pa1e, you will not see this screen( The Additional database Con+i1urations button displays additional database +eatures( are chec3ed and clic3 'K( a3e sure both

&elect the connection options desired +rom the 2atabase Connection Options pa1e( Note? I+ you did not choose 4ew 2atabase +rom the 2atabase Template pa1e, you will not see this screen( Clic3 Ne4t( 2BCA now displays the Initiali5ation #arameters pa1e( This pa1e comprises a number o+ Tab +ields( odi+y the Me)o!y settin&s i+ desired and then select the :ile 3o ations tab to update in+ormation on the Initiali5ation #arameters +ilename and location( Then clic3 Ne4t( The option C!eate pe!sistent initialiIation pa!a)ete! file is selected by de+ault( I+ you have a cluster +ile system, then enter a file syste) na)e, otherwise a !aw "e*i e na)e +or the location o+ the server parameter +ile (sp+ile) must be entered( Then clic3 Ne4t( The button :ile 3o ation Ja!ia+les displays variable in+ormation( Clic3 'K( The button All InitialiIation 0a!a)ete!s displays the Initiali5ation #arameters dialo1 bo)( This bo) presents values +or all initiali5ation parameters and indicates whether they are to be included in the sp+ile to be created throu1h the chec3 bo), included (B$4)( Instance speci+ic parameters have an instance value in the instance column( Complete entries in the All InitialiIation 0a!a)ete!s pa1e and select Close( Note? There are a +ew e)ceptions to what can be altered via this screen( %nsure all entries in the Initiali5ation #arameters pa1e are complete and select Ne4t( 2BCA now displays the 5ata+ase Sto!a&e 6indow( This pa1e allows you to enter +ile names +or each tablespace in your database( The +ile names are displayed in the 5atafiles +older, but are entered by selectin1 the Ta+lespa es icon, and then selectin1 the tablespace obCect +rom the e)panded tree( Any names displayed here can be chan1ed( A con+i1uration +ile can be used, see section 0(,(., (pointed to by the environment variable -.CAD A:DC*NF&C)( Complete the database stora1e in+ormation and clic3 Ne4t( The 5ata+ase C!eation 'ptions pa1e is displayed( %nsure that the option C!eate 5ata+ase is chec3ed and clic3 :inish( The 5BCA S())a!y window is displayed( Review this in+ormation and then clic3 'K( Once the &ummary screen is closed usin1 the OP option, 2BCA be1ins to create the database accordin1 to the values speci+ied(

A new database now e)ists( It can be accessed via Oracle &R7-#7>& or other applications desi1ned to wor3 with an Oracle RAC database(

2$0 A")iniste!in& Real Appli ation Cl(ste!s Instan es

Oracle Corporation recommends that you use &R8CT7 to administer your Real Application Clusters database environment( &R8CT7 mana1es con+i1uration in+ormation that is used by several Oracle tools( *or e)ample, Oracle %nterprise ana1er and the Intelli1ent A1ent use the con+i1uration in+ormation that &R8CT7 1enerates to discover and monitor nodes in your cluster( Be+ore usin1 &R8CT7, ensure that your 9lobal &ervices 2aemon (9&2) is runnin1 a+ter you con+i1ure your database( To use &R8CT7, you must

have already created the con+i1uration in+ormation +or the database that you want to administer( Bou must have done this either by usin1 the Oracle 2atabase Con+i1uration Assistant (2BCA), or by usin1 the srvctl add command as described below( I+ this is the +irst Oracle@i database created on this cluster, then you must initiali5e the clusterwide &R8 con+i1uration( *irstly, create or edit the +ile /var/opt/oracle/srvConfi".loc +ile and add the entry srvconfi"DlocOpat#Dname(where the path name is a small cluster-shared raw volume e1
$ vi *var*opt*oracle*srv1onfi!.loc srvconfi"DlocO/dev/rracDsrvconfi"D500m

Then e)ecute the +ollowin1 command to initiali5e this raw volume (4ote? This cannot be run while the 1sd is runnin1( #rior to @i Release , you will need to 3ill the ((($Cre$.(.(J$bin$((( process to stop the 1sd +rom runnin1( *rom @i Release , use the 1sdctl stop command)?
$ srvconfi! >init

The +irst time you use the &R8CT7 >tility to create the con+i1uration, start the 9lobal &ervices 2aemon (9&2) on all nodes so that &R8CT7 can access your clusterKs con+i1uration in+ormation( Then e)ecute the srvctl add command so that Real Application Clusters 3nows what instances belon1 to your cluster usin1 the +ollowin1 synta)?

For Oracle RAC v9.0.1:

S &s" &uccess+ully started the daemon on the local node( S s!* tl a"" "+ -p "+Dna)e -o o!a leDho)e Then +or each instance enter the command +rom either node? S s!* tl a"" instan e -p "+Dna)e -i si" -n no"e To display the con+i1uration details +or, e)ample, databases racdb.$,, on nodes racnode.$, with instances racinst.$, run?S s!* tl onfi& racdb. racdb, S s!* tl onfi& -p !a "+% racnode. racinst. racnode, racinst, S s!* tl onfi& -p !a "+% -n !a no"e% racnode. racinst. %)amples o+ startin1 and stoppin1 RAC +ollow?-

S s!* tl sta!t -p !a "+% Instance success+ully started on node? racnode, 7isteners success+ully started on node? racnode, Instance success+ully started on node? racnode. 7isteners success+ully started on node? racnode. S s!* tl stop -p !a "+7 Instance success+ully stopped on node? racnode, Instance success+ully stopped on node? racnode. 7istener success+ully stopped on node? racnode, 7istener success+ully stopped on node? racnode. S s!* tl stop -p !a "+% -i !a inst7 -s inst Instance success+ully stopped on node? racnode, S s!* tl stop -p !a "+% -s inst #RPO-,A0: ? Instance is already stopped on node? racnode, Instance success+ully stopped on node? racnode.

For Oracle RAC v9.2.0+:

S &s" tl sta!t &uccess+ully started the daemon on the local node( S s!* tl a"" "ata+ase -" "+Dna)e -o o!a leDho)e F-) "o)ainDna)eG F-s spfileG Then +or each instance enter the command? S s!* tl a"" instan e -" "+Dna)e -i si" -n no"e To display the con+i1uration details +or, e)ample, databases racdb.$,, on nodes racnode.$, with instances racinst.$, run?S s!* tl onfi& racdb. racdb, S s!* tl onfi& -p !a "+% -n !a no"e% racnode. racinst. $uA.$app$oracle$product$@(,(A(. S s!* tl stat(s "ata+ase -" !a "+% Instance racinst. is runnin1 on node racnode. Instance racinst, is runnin1 on node racnode,

%)amples o+ startin1 and stoppin1 RAC +ollow?S s!* tl sta!t "ata+ase -" !a "+7 S s!* tl stop "ata+ase -" !a "+7 S s!* tl stop instan e -" !a "+% -i !a inst7 S s!* tl sta!t instan e -" !a "+% -i !a inst7 S &s" tl stat 9&2 is runnin1 on local node S &s" tl stop *or +urther in+ormation on srvctl and 1sdctl see the Oracle@i Real Application Clusters Administration manual(

8$0 Refe!en es
4ote? .J,A0=(. - AI!? Ruic3 &tart 9uide - @(A(. R2B & Installation 4ote? ,A.A.@(. - AI!? Ruic3 &tart 9uide - @(,(A R2B & Installation 4ote? ==0';(. - Overview o+ "AC # Classic and $ or "AC #$%& 4ote?.0=,JJ(. - 2atabase Creation in Oracle@i RAC 4ote?.J0'AJ(. - Raw 2evices and Cluster *ilesystems 6ith Real Application Clusters RAC$IB AI! certi+ication matri) Oracle@i Real Application Clusters Installation and Con+i1uration Release . (@(A(.) Oracle@i Real Application Clusters Concepts Oracle@i Real Application Clusters Administration Oracle@i Real Application Clusters 2eployment and #er+ormance Oracle@i Installation 9uide +or Compa/ Tru;', "ewlett-#ac3ard "#>!, IB -AI!, 7inu), and &un &olaris-based systems( Oracle@i Release 4otes