Documentos de Académico
Documentos de Profesional
Documentos de Cultura
2
SQL to check database before Shutdown database ..................................................................................2
Oracle.Metalink.com document ...............................................................................................................2
1076161.6:-Shutdown Normal or Shutdown Immediate Hangs. SMON disabling TX Recovery
......................................................................................................................................................3
375935.1:- What To Do and Not To Do When 'shutdown immediate' Hangs.............................5
Applies to: ...............................................................................................................................6
Goal..........................................................................................................................................6
Solution....................................................................................................................................6
References................................................................................................................................8
Errors........................................................................................................................................8
Bug:5057695: Shutdown Immediate Very Slow To Close Database...........................................8
Applies to: ...............................................................................................................................8
Symptoms................................................................................................................................8
Changes....................................................................................................................................8
Cause........................................................................................................................................9
Solution....................................................................................................................................9
61997.1: SMON - Temporary Segment Cleanup and Free Space Coalescing...........................10
100054.1: Transaction Rollback after a failed operation or during Database Shutdown..........15
Conversation between me and Oracle.Metalink.com document ...........................................................16
More information....................................................................................................................................45
1
Issue
TO SHUTDOWN DATABASE IT IS TAKING LONG TIME, SOMETIME MORE THAN 2
HOURS. Frequently we are facing this problem. Some time to take shutdown it takes 10 min
.One days we have got one error that is " ORA-01013: user requested cancel of current
operation" time of shutdown database
select sid, serial#, username, status, schemaname, logon_time from v$session where status='ACTIVE' and
username is not null
Select f.R "Recovered", u.nr "Need Recovered" from (select count(block#) R , 1 ch from sys.fet$ ) f,(select
count(block#) NR, 1 ch from sys.uet$) u where f.ch=u.ch
4. Large Transactions
========================
select sum(used_ublk) from v$transaction
Oracle.Metalink.com document
1 1076161.6:-Shutdown Normal or Shutdown Immediate Hangs. SMON disabling
TX Recovery
2 Note 375935.1 - What To Do and Not To Do When 'shutdown
immediate' Hangs
3 Note 428688.1 Bug 5057695: Shutdown Immediate Very Slow
To Close Database.
2
4 Note 61997.1 SMON Temporary Segment Cleanup and Free
Space Coalescing
5 100054.1 Subject: Transaction Rollback after a failed operation or during
Database Shutdown
1076161.6:-Shutdown
Normal or Shutdown Immediate Hangs.
SMON disabling TX Recovery
Subject: Shutdown Normal or Shutdown Immediate Hangs. SMON disabling TX
Recovery
Doc ID: Note:1076161.6 Type: BULLETIN
Last Revision Date: Status:
21-DEC-2007 PUBLISHED
Description
===========
SHUTDOWN NORMAL or SHUTDOWN IMMEDIATE hangs. In the alert.log, you see only
the following:
Shutting down instance (immediate)
License high water mark = 12
Thu Dec 8 18:43:16 1994
alter database close normal
Thu Dec 8 18:43:17 1994
SMON: disabling tx recovery
SMON: disabling cache recovery
or
waiting for smon to disable tx recovery
There are no ORA errors or trace files.
Scope & Application
===================
Informational
During a SHUTDOWN IMMEDIATE and SHUTDOWN NORMAL, SMON is cleaning up extents
which are no longer needed and marking them as freed.
Either wait for SMON to clean up the free extents in the database as it shuts
down or perform a SHUTDOWN ABORT to shutdown the instance. A SHUTDOWN ABORT
will not perform a clean shutdown.
Verify that temporary segments are decreasing
To verify that the temporary segments are decreasing have an active session
available in Server Manager or SQLPLUS during the SHUTDOWN IMMEDIATE. Issue
3
the following query to ensure the database is not hanging, but is actually
perform extent
cleanup:
SVRMGR/SQL> select count(block#) from fet$;
COUNT(BLOC
7
SVRMGR/SQL> select count(block#) from uet$;
COUNT(BLOC
402
After some time has elapsed, reissue the query and see that the values for
fet$ have increased while the values or uet$ have decreased:
SVRMGR/SQL> select count(block#) from fet$;
COUNT(BLOC
10
SVRMGR/SQL> select count(block#) from uet$;
COUNT(BLOC
399
select f.R "Recovered", u.nr "Need Recovered" from (select count(block#) R ,
1 ch from fet$ ) f,
(select count(block#) NR, 1 ch from uet$) u where f.ch=u.ch
During shutdown the SMON process is cleaning up extents and updating the data
dictionary tables with the marked free extents. As the extents are marked as
freed, they are removed from the table for used extents, UET$ and placed on
the table for free extents, FET$.
How to Avoid creating many Temporary Extents
Once the database has shutdown cleanly, to avoid creating many temporary
extents change the initial and next extent sizes on temporary tablespaces to a
more appropriate size:
ALTER TABLESPACE <temp> DEFAULT STORAGE (INITIAL <size>M/K NEXT
<size>M/K);
Note: If the temporary tablespace is of type TEMPORARY, then this change will
only affect temporary segments created after issuing the above command. Any
existing temporary segments already in the TEMPORARY tablespace will not be
affected till the instance is restarted. On shutdown, existing temporary
segments are dropped. If the TEMPORARY TABLESPACE is of type PERMANENT, then
cleanup is performed by SMON after completion of the process using it.
4
Increasing the initial and next extent size will decrease the number of
extents that are allocated to temporary segments. Since there are fewer
extents to deallocate, the database should shutdown more speedily.
Take the following scenario:
A database was subject to large sorts with the following sort parameter in
the "init.ora" file:
sort_area_size=1000000
The temporary tablespaces for this database were all created with initial and
next extents sized at 50k and the total database size was about 300mb.
Database sorts will utilize memory as much as possible based on the "init.ora"
parameter "sort_area_size". Once this memorybased sort area is filled, the
database will utilize the temporary table space associated with the database
user to complete the sort operation. During a shutdown normal, the database
will attempt to clean up the temporary tablespaces.
If a small extent size is used, then a large number of extents will be created
for a large sort. The cleanup of the temporary tablespace takes much longer
with a large number of extents.
Note:
=====
You have to do a shutdown abort and then bring the database back up to run the
suggested queries.
For other reasons for slow/hung shutdown see also these notes:
Note 375935.1 What To Do and Not To Do When 'shutdown immediate' Hangs
Note 428688.1 Bug 5057695: Shutdown Immediate Very Slow To Close Database.
References:
===========
Note 61997.1 SMON Temporary Segment Cleanup and Free Space Coalescing
Search Words:
=============
hanging
shutdown
5
Last Revision Status:
02-JUN-2007 MODERATED
Date:
This document is being delivered to you via Oracle Support's Rapid Visibility (RaV) Rapid Visibility (RaV) process,
and therefore has not been subject to an independent technical review.
Applies to:
Oracle Server - Enterprise Edition - Version: 8.1.7 to 10.2
Information in this document applies to any platform.
Goal
What to do when shutdown immediate appears to hang. Sometimes, the message 'Waiting for
smon to disable tx recovery' is posted in the alert log. This note only addresses situations when
the apparent hang occurs when the database is going from OPEN to MOUNT, which is actually
the most common situation. If the apparent hang occurs at a different step, then this note does not
apply.
Solution
The big problem in these situations is that it is noticed only after the shutdown immediate has
been issued.
This kind of situation is mostly caused by 2 things:
1. a large query running at the shutdown moment.
2. a large transaction running at the shutdown moment.
Both have to complete in order for the database to be brought down when shutdown immediate is
issued. Actually, the files cannot be closed consistently because of one of the 2 possibilities
above and, as such, the transition from OPEN to MOUNT is postponed until the files are closed,
which means that either the large query completes or the large transaction is rolled back. This is
not a hang, it is the expected behavior.
So, before issuing the shutdown immediate, it would be recommended to check the following
views, especially when the database needs to be brought down for a very short period of time:
1. for large queries:
select count(*) from v$session_longops where time_remaining>0;
2. for large transactions:
select sum(used_ublk) from v$transaction;
A result greater than 0 for the first query and a large value returned for the second one would
mean a relatively long time to wait until the shutdown immediate completes.
For the second situation, please also check step 9 in Note 117316.1 to "guestimate" the time to
rollback the transactions.
-----------------------------------------------------------
Step 9 from Note 117316.1
Check for rollback:
6
select used_ublk from v$transaction where ADDR=<value from
TADDR in v$session>;
If there is a value there, this is the number of undo blocks used by
the transaction. Wait one minute and again select "used_ublk" from
"v$transaction" where ADDR=<value from TADDR in v$session>; . Note the
value. If it is decreasing, a rollback is occuring and based on the
difference between these values, you can "guesstimate" the time required to
complete the rollback. For example, if the first query returns a value of
80000 and the second one returns 70000, it took 1 minute to rollback 10000
blocks. Based on this number,you can guestimate the time to complete the
rollback. In this case, it would be 7 minutes.
--------------------------------------------------------------------------
1. For the large queries situation, when the shutdown immediate is hanging, you can just bring
down the database using: shutdown abort, as the database could be easily brought to a
consistent state by:
startup restrict followed by shutdown immediate.
One should take the backup and/or do whatever else need to be done after the shutdown
immediate.
2. For the second situation, the workaround cannot be applied, especially when it's needed to
take a cold backup. The database must be closed in a consistent state in order to do this and the
consistent state cannot be achieved until all the transactions have completed one way or another
(commit/rollback).
As such, it's up to the local personnel to decide what to do, depending on the local needs.
It is very important to realize that: BY SHUTTING DOWN A DATABASE YOU DO NOT
SOLVE A PERFORMANCE PROBLEM CAUSED BY A LARGE TRANSACTION. You are
only making things worse.
There are situations when the database is brought down even when a large transaction/large
recovery is taking place. Then it's brought up again and a new shutodwn is tried. Again, the
shutdown immediate is hanging, for a very simple reason - the large recovery is still going on.
At this moment, the v$transaction view is not displaying anything.
Hoever, it is still possible to check the recovery operation by checking the:
select * from v$fast_start_transactions;
and/or
select * from v$fast_start_servers;
views. They are the ones that display the recovery status.
As such, when a large transaction is taking place, do not try successive shutdown aborts, startups
and shutdown immediate. The hang will reoccur. The database must be consistent when the
database is dismounted - performing successive shutdowns/startups is not helping at all, it's only
making the recovery even more lengthy.
You should prevent these situations by notifying the users a shutdown will be done and no large
operations should be started.
If a large operation has already started at the moment when you want to shutdown immedate,
7
assess what would be faster - rollback the current situation or allow it to complete.
References
Note 117316.1 - ORA-0054: When Dropping or Truncating Table, When Creating or Rebuilding
Index
Errors
ORA-54 "resource busy and acquire with NOWAIT specified"
Applies to:
Oracle Server - Enterprise Edition - Version: 9.2.0.8 to 10.2.0.3 This problem can occur on any
platform. Oracle Database Server
Symptoms
Database with 150 or so connections take a very long time to shutdown, compared
to the same system when using Oracle 9i (9.2.0.6 and prior) where shutdown was very fast.
Given the fact there is no activity on the connections in question, so rollbacks aren't an
issue.
There are no trace files and no indication of a problem in the alert log.
Changes
8
Cause
Previously the shutdown immediate, the ksukia function would kill each individual
user session and check in intervals of 5 seconds that it had died before progressing
to the next user.
With fix for bug 5080775 (Unpublished) this time was reduced so that instead
of waiting for 5 seconds it now waited 0.05 seconds and this would be repeated
40 times at intervals of 0.05 seconds then it would increase to checking every
5 seconds.
Note this code is relatively new. It was introduced in 9.2.0.7 and 10.2. It was
a design change where previously it issues a kill (skgpkill) and then proceeds
onto the next process.
Solution
Workaround :
Wait for the shutdown to finish.
Solution :
9
61997.1: SMON - Temporary Segment Cleanup and
Free Space Coalescing
Subject: SMON - Temporary Segment Cleanup and Free Space
Coalescing
Doc ID: Note:61997.1 Type: FAQ
Last Revision Date:22-APR-2004 Status: PUBLISHED
SMON Temporary Segment Cleanup and Free Space Coalescing in Oracle 7.3 and
Higher
PURPOSE
~~~~~~~
Since the introduction of the unlimited extents feature in Oracle 7.3, it is
possible for SMON to have to either clean up a large number of temporary
extents, or to coalesce a large number of free extents. This can manifest
itself by SMON appearing to spin, consuming a high percentage of CPU for long
periods. This article explains what is happening, and what (if anything) can
be done.
The discussion concentrates mainly on nonTEMPORARY type tablespaces. There
is however a section at the end of the article which discusses possible
issues with tablespaces of type TEMPORARY.
SCOPE & APPLICATION
~~~~~~~~~~~~~~~~~~~
This article is intended to assist DBAs encountering SMON appearing to spin
and consume high percentages of CPU by providing an understanding of the
issues.
RELATED DOCUMENTS
~~~~~~~~~~~~~~~~~
Note 35513.1 Removing `stray` TEMPORARY Segments
Note 50592.1 Extent Sizes for Sort, Direct Load and Parallel Operations
Note 65973.1 Temporary Tablespaces and the Sort Extent Pool
Note 68836.1 How to efficiently drop a table with many extents
Note 47400.1 EVENT: DROP_SEGMENTS Forcing cleanup of TEMPORARY segments
What to look for
~~~~~~~~~~~~~~~~
The most common indicator is the SMON process consuming large amounts
of CPU for a long period. UNIX O/S utilities sar or vmstat will show how
busy
CPU(s) are; ps will show which process is using the CPU.
What is SMON doing
~~~~~~~~~~~~~~~~~~
Once you have identified that SMON is using lots of CPU, you need to
identify whether it is performing temporary segment (extent) cleanup, or
free space coalescing.
10
Free space coalescing
~~~~~~~~~~~~~~~~~~~~~
When does SMON coalesce?
o. SMON wakes itself every 5 minutes and checks for tablespaces with
default pctincrease != 0.
How to identify whether SMON is coalescing
o. Check whether there are a large number of free extents that might
be being coalesced by running the following query a few times:
SELECT COUNT(*) FROM DBA_FREE_SPACE;
If the count returned is dropping while SMON is working, it is
likely that SMON is coalescing free space.
What are the effects on the database?
o. Because SMON acquires the Space Transaction (ST) enqueue in
exclusive mode, other processes requiring the enqueue will be
blocked. This is typically manifested by multiple <oerr:ORA1575>
errors.
o. SMON sits in a very tight loop while coalescing, and consumes close
to 100% CPU. If the system is CPUbound, the run queue will increase
as other processes try to get onto CPU.
Can anything be done to stop SMON grabbing CPU?
o. If there is no CPU contention, and no processes being blocked because
of failure to acquire the ST enqueue, DO NOT DO ANYTHING. Leave SMON
to complete the coalescing.
THE DATABASE CAN BE SHUTDOWN CLEANLY WITH UNCOALESCED EXTENTS. If SMON
is performing the coalesce, a shutdown will NOT undo the work completed
so far.
o. Use the 'alter tablespace <tbs name> coalesce' command. This is quicker
than SMON, and the work is performed in in fewer space transactions,
and
therefore makes fewer enqeueue acquisitions. HOWEVER, IF THE COMMAND IS
INTERRUPTED, ALL ITS COALESCING WORK WILL BE LOST.
o. It is possible to force a user session to coalesce free extents. See
Note 35513.1 "Removing `stray` TEMPORARY Segments" for details. Again,
quicker than SMON. HOWEVER, IF THIS OPERATION IS INTERRUPTED, ALL IT'S
COALESCING WORK WILL BE LOST.
o. Offlining the tablespace/datafiles containing the extents to be
coalesced has NO effect.
Temporary segment cleanup
11
~~~~~~~~~~~~~~~~~~~~~~~~~
When does SMON cleanup temporary segments?
o. Typically a user process allocates a temporary segment (multiple
extents) and then dies before cleaning them up, or the user process
receives an error causing the statement to fail. SMON is posted to do
the cleanup. SMON also might get tied up cleaning uncommitted
transactions though, and be too busy to process requests to grow an
existing sort segment. Starting with Oracle 8i, playing around with
fast_start_parallel_rollback might workaround that.
In addition, actions like CREATE INDEX create a temporary segment for
the index, and only convert it to permanent once the index has been
created. Also, DROP <object> converts the segment to temporary and then
cleans up the temporary segment.
o. During normal operations, user processes that create temporary segments
are responsible for cleanup.
How to identify whether SMON is cleaning up temporary extents
o. Check whether there are a large number of temporary extents that might
be being cleaned up by running the following query a few times:
SELECT COUNT(*) FROM DBA_EXTENTS WHERE SEGMENT_TYPE='TEMPORARY';
If the count returned by the above query is dropping while SMON is
working, it is likely that SMON is performing temp segment cleanup.
See section 'Tablespaces of type TEMPORARY' for more details on
this.
What are the effects on the database?
o. Again, SMON will continually acquire and then release the ST enqueue
in exclusive mode. This can cause contention with other processes and
lead to <oerr:ORA1575> errors.
o. CPU utilization is not exceptionally high. During tests, SMON
consumed between 10% and 20% CPU during cleanup, and so this operation
has less impact than coalescing, as far as SMON is concerned.
Furthermore, SMON performed the cleanup in 'chunks', cleaning up a
subset of the extents at a time.
Can anything be done to stop SMON grabbing CPU?
o. Not a great deal. As with coalescing, if there is no CPU contention,
and no processes being blocked because of failure to acquire the ST
enqueue, DO NOT DO ANYTHING. However because SMON does not work as hard
cleaning up temporary extents, it should not be a big issue.
Note: If you are using TEMPORARY type temporary tablespaces then
SMONs cleanup of the segment can be a problem as it will not
12
service sort segment requests while performing cleanup.
See below (TEMPORARY tablespaces) for more information.
It should be noted that a normal/immediate shutdown will not complete
until all temporary segments have been cleaned up. Shutdown will
'kick' SMON to complete cleanup.
o. Offlining the tablespace/datafiles in which the extents reside has NO
effect.
o. DROP_SEGMENTS event could be set set to force the cleanup of
temporary segments, see Note 47400.1 "EVENT: DROP_SEGMENTS Forcing
cleanup of TEMPORARY segments".
Avoidance
~~~~~~~~~
With a little forethought and care, the above situations can be avoided:
o. Do not create temporary tablespaces with small initial and next default
storage parameters. Also beware of unlimited maxextents on temporary
tablespaces.
Note, TEMPORARY type tablespaces set maxextents unlimited automatically.
Furthermore, the NEXT AND INITIAL extent sizes are determined from
the default NEXT size (default INITIAL is ignored). For more details on
temporary extent sizes, see Note 50592.1 "Extent Sizes for Sort, Direct
Load and Parallel Operations (PCTAS & PDML)".
o. Use tablespaces of type TEMPORARY. Sort segments in these tablespaces
are not cleaned up. This reduces contention on the ST enqueue and also
reduces CPU usage by SMON **UNLESS** the database is shutdown and
restarted. If TEMPORARY type tablespaces are in use then SMON will
clean up its segments after startup following a shutdown. In this case
large numbers of extents can be a severe problem as SMON will not
service user "sort segment requests" until the cleanup is complete.
If the cleanup is to take a long time users will not be able to perform
sort operations.
In this scenario you can point users at a PERMANENT temporary tablespace
while SMON cleans up the TEMPORARY temporary tablespace. This is likely
to
cause ST enqueue contention but will allow users sessions to sort on
disk
when necessary rather then them just blocking.
Eg:
If SMON is busy cleaning up a TEMP segment containing a lot
of extents it cannot service 'sort segment requests' from other
sessions. Pointing the users at a PERMANENT tablespace as
their temporary tablespace can help keep the system running
until SMON is free again:
CREATE TABLESPACE NEWTEMP .... (your own specification here)
13
(DO NOT CREATE IT AS TYPE TEMPORARY)
Move the users over to this:
select username from dba_users
where temporary_tablespace='TEMP';
For each user in this list:
alter user XXXXX temporary tablespace NEWTEMP;
Once SMON has cleaned up the extents reset the storage clause
on each tablespace to sensible values and you can then point
users back at a TEMPORARY temp tablespace.
Starting with Oracle8i, rather than reverting back to a PERMANENT
tablespace if SMON is cleaning up an old sort segment at startup,
you can potentially drop and recreate the tempfiles of the existing
TEMPORARY tablespace. The cleanup should be faster anyway since by rule
a TEMPORARY tablespace made of tempfiles need to be LOCALLY MANAGED.
You can remove tempfiles from TEMPORARY tablespaces and keep the logical
structure empty.
o. Beware of creating large objects with inappropriate (small) extents. If
the creation of the object fails, SMON cleans up. Also, dropping such an
object will create a lot of cleanup work for the user process.
Oracle8 ONLY. Make use of the tablespace MINIMUM EXTENT size to help
minimise the risk of mistakes in scripts causing small extent sizes.
This parameter ensures that every used and/or free extent size in a
tablespace is at least as large as, and is a multiple of, this value.
Oracle8i ONLY: It is worth considering the use of a locally managed
temporary tablespace. This has the benefit of faster temporary segment
cleanup after the instance has been aborted.
Note, locally managed temporary tablespaces must be created using
tempfile(s). Any attempt to create a locally managed temporary
tablespace using a datafile will result in the error:
ORA25144: invalid option for CREATE TABLESPACE with TEMPORARY contents
Tablespaces of type TEMPORARY
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
TEMPORARYtype tablespaces were introduced in Oracle 7.3 (see Note 65973.1
"Temporary Tablespace, the Sort Extent Pool, and OPS"). In summary:
o. The first disk sort (after instance startup) creates a sort segment in
the TEMPORARY tablespace.
o. Free extents in the sort segment are reused as required by sessions.
14
o. The sort segment grows to a steadystate.
o. Sort extents are not deallocated whilst the instance is running.
o. Permanent objects cannot be created in TEMPORARY tablespaces.
o. There is a maximum of one sort segment per TEMPORARY tablespace.
Thus, contention on the ST enqueue is reduced as user sessions are
allocating
and deallocating fewer extents. Even if a user session dies, SMON will not
deallocate extents that the session was using for sorting.
SMON actually deallocates the sort segment after the instance has been
started and the database has been opened. Thus, after the database has been
opened, SMON may be seen to consume large amounts of CPU as it first
deallocates the (extents from the) temporary segment, and then when it is
requested to perform free space coalescing of the free extents created by
the temporary segment cleanup. Again, this behaviour will be exaggerated if
the TEMPORARY tablespace in which the sort segment resides has
inappropriate
(small) default NEXT storage parameters (see 'Avoidance' above).
Permanent object cleanup
~~~~~~~~~~~~~~~~~~~~~~~~
If a permanent object is made up of many extents, and the object is to be
dropped, the user process dropping the object will consume large amounts
of CPU in the same way as SMON does when cleaning up a temporary segment.
Please see Note 68836.1 "How To Efficiently Drop A Table With Many Extents"
for a discussion of this.
References
~~~~~~~~~~
Problem Description
To explain what happens to uncommitted transactions after a failed or aborted
operation (i.e. long running transaction fails) or caused by a Database
Shutdown
command being issued.
15
Problem Explanation
If a long running transaction (indeed any transaction) is cancelled or fails,
then Oracle needs to cleanup the uncommitted work that was done by this
transaction
so that other transactions can progress. This cleanup involves rolling back
the uncommitted work.
If a database shutdown is issued with outstanding transactions to be rolled
back, then the database will not be shutdown until these transactions have
completed or have been rolled back.
For example on a shutdown immediate we send a message to all the shadow
processes
asking them to complete this rollback and cleanup. We will not shutdown until
all of them have completed this cleanup. This means that because we have
shutdown
cleanly no redo needs to be applied on startup.
If the database is aborted with outstanding transactions then these
transactions
must be rolled back after the database has been restarted. Post 7.3 this
transaction recovery is applied by SMON after the database has been opened.
If
another transaction needs to access blocks that were modified by the
transaction
that is rolling back then the new session will rollback the blocks that it
requires itself. This can cause these transactions to perform slower than
normal.
The activity of SMON has been designed so as to minimize the impact on other
processes by only rolling back a small proportion of rollback entries at a
time.
References
Bug 517917 SHUTDOWN IMMEDIATE HANGS SMON DOING TX RECOVERY
Note 61997.1 SMON Temporary Segment Cleanup and Free Space Coalescing
16
SR Number 6779932.992 Open Date 19-FEB-08 04:30:01
Support Identifier 15272898 Name mamun seraji
Severity 3 Last Update 17-APR-08 07:02:57
Product Oracle Server - Enterprise Product Version 9.2.0.8.0
Edition
Platform AIX5L Based Systems (64-bit) Detailed Status Soft Close
SR Reference n/a BUG Reference n/a
Abstract
TO SHUTDOWN DATABASE IT IS TAKING LONG TIME, SOMETIME MORE THAN 2
HOURS.
Resolution History
### When was the last time you performed a tuning process to your database? ###
Frequently we are facing this problem. some time to take shutdown it take 10 min.
17
### Please list all files that you plan to upload: ###
if need we will send
Can you easily recover from, bypass or work around the problem?
Yes
Does your system or application continue normally after the problem occurs?
Yes
Are the standard features of the system or application still available; is the loss of service
minor?
No
Error : ORA-01013
Contact me via : MetaLink
UPDATE
======
Hi Mamun,
Thank you for using MetaLink. We are currently reviewing/researching the situation and will
update the Service Request (SR) or call you
as soon as we have relevant information. Thank you for your patience.
Best Regards,
Bharath
Global Customer Services
STATUS
=======
.
UPDATE
=======
Hi Mamun,
18
Can you please upload the Alert_log file which contains the last 5-6 shutdown information
logged
1) Were there any transactions that were cancelled by doing shutdown immediate.
Regards,
Bharath Sunku
STATUS
=======
The customer : ruhul.amin@bracbank.com : has uploaded the following file via MetaLink:
Alert_log.txt
DATA COLLECTED
19
============
20
Wed Feb 20 00:33:51 2008
Shutting down instance (immediate)
License high water mark = 1905
Wed Feb 20 00:59:31 2008
ALTER DATABASE CLOSE NORMAL
Wed Feb 20 00:59:31 2008
SMON: disabling tx recovery
SMON: disabling cache recovery
Wed Feb 20 00:59:33 2008
Shutting down archive processes
Archiving is disabled
21
Sun Feb 24 00:46:37 2008
Shutting down archive processes
Archiving is disabled
.
UPDATE
========
Hi Mamun,
As you have also specified that there are some Batch Jobs which are running in the Night and
we
suspect they might have got cancelled due to "Shut immediate" which is consuming
long time for transaction recovery
Please let me know if you need any other clarifications on this issue
Regards
Bharath Sunku
1) How you came to know that Transaction recovery is consuming 99% of the time ?
2) is there any way to shutdown database at short time ?
Regards
22
Md Ruhul Amin
.
UPDATE
=======
Hi Ruhul,
From the Above we see that the Shutdown was performed at 00:56 and 02:52 tx recovery was
completed..
Please check my Data Collected Tag.. you will find that the 99% of the time is consumed
at this step and after that its hardly taking 3-5 minutes to shutdown the databa
se
~~~~~~~
Thu Feb 21 17:45:07 2008
Shutting down instance: further logons disabled
Shutting down instance (immediate)
License high water mark = 376
Thu Feb 21 17:46:18 2008
ALTER DATABASE CLOSE NORMAL
Thu Feb 21 17:46:18 2008
SMON: disabling tx recovery
SMON: disabling cache recovery
Thu Feb 21 17:46:18 2008
23
Shutting down archive processes
~~~~~~~~~~~~
Shutdown was executed at 17:45 and 17:46 is the tx recovery completed and immediatly
database got shutdown
Please let me know if you need any other clarifications on this issue
Regards
Bharath Sunku
STATUS
=======
can u please tell me how can i came to know that today shutdown will take more time. Please
arrange sometime to talk over ch
at.
regards
Md Ruhul Amin
24
UPDATE
=======
Hi Ruhul,
Time to shutdown depends on the operations being performed on the database before you do a
Shutdown
If there are any transaction happening on the database and if the SHUT IMMEDIATE is
performed which is going to
terminate the transactions and then do the Rollback, in this case depending on t
he time of transaction recovery your shoudown will be depended
Please let me know if you need any other clarifications on this issue
http://www.oracle.com/corporate/contact/index.html
Regards,
Bharath Sunku
STATUS
=======
25
Regards
md Ruhul Amin
.
UPDATE
=======
Hi Ruhul,
Will update the SR / Call you so that we can further progress the issue
Thanks,
Bharath
STATUS
=======
26
.
UPDATE
=======
Hi Ruhul,
Thanks,
Bharath
STATUS
=======
Thanks
Md Ruhul Amin
.
ISSUE CLARIFICATION
====================
During night so many batch job running in our database. That time we have checked alert log
27
file
and found "ORA-01555: caused by SQL statement below (Query Duration=8000)". Then we
have ch
anged
undo_retention parameter value. The old value was 6500 and the new value is 8000.
.
ISSUE VERIFICATION
===================
Verified by seeing the Alert_log file
.
CAUSE DETERMINATION
====================
Tx recovery is taking lot of time
CAUSE JUSTIFICATION
====================
.
POTENTIAL SOLUTION(S)
======================
Apply the patch
same as above
.
SOLUTION / ACTION PLAN
=======================
Hi Mamun,
28
happened before 9.2.0.8
If there are any active sessions in your database which has to be killed while "shutdown
immediate" of the database, th
en oracle is waiting for 5 seconds on each session and this is consuming more ti
me when there are many databases.. With the BUG FIX applied on the database.. Or
acle is going to wait only for .05 seconds so that the Killing Active sessions w
ill be completed farstly..
Since this is an Generic BUG and implies to all the 9.2.0.8 databases on all the platforms.. we
recommand you to apply the patch and
see whether you are still having the same issue
Regards
Bharath Sunku
Regards,
Bharath Sunku
STATUS
=======
one clarification is need how to find whether this patch is applied in our database or n
ot
please check
Regards
md Ruhul Amin
29
26-FEB-08 10:29:21 GMT
.
UPDATE
=======
Hi Ruhul,
1) Go to Oracle_Home
++ This contains the information about the patches applied on the database
With regarding to the one-off patch, please answer the below question so that we can goahead
and request for a one-off
patch
~~~~~~~~~~
1) Platform and the database version
uname -a
Unix:
$ORACLE_HOME/inventory/ContentsXML
Windows:
$ORACLE_HOME\inventory\ContentsXML
2) List of one-off patches applied since applying the current patch set.
opatch lsinventory -detail
30
Regards,
Bharath Sunku
STATUS
=======
The customer : ruhul.amin@bracbank.com : has uploaded the following file via MetaLink:
Config.zip
[][DB][FINBRAC]/oracle/app/product/9.2.0/inventory/ContentsXML> uname -a
AIX DBServer 3 5 00C6F01D4C00
Regards
Md Ruhul Amin
31
27-FEB-08 04:33:03 GMT
.
UPDATE
=======
Hi Mamun,
Can you please answer the rest of the questions so that we can goahead and raise the backport
request
Answers to all teh questions are mandatory since based on that the Backport request priority
will be decided
Regards,
Bharath Sunku
STATUS
=======
32
Regards
md Ruhul Amin
.
UPDATE
=======
Hi Mamun,
Regards,
Bharath Sunku
STATUS
=======
.
UPDATE
=======
Hi Mamun,
33
Since the problem is intermitant so you can goahead and shutdown the database to take the
cold backups.
Meanwhile we will raise an backport request for the BUG and will try to get the one-off patch
as soon as pos
sible so that the issue can be resolved..
For this please provide me the Answers for rest of the questions so that we can goahead with
raising the backport req
uest
Regards,
Bharath Sunku
STATUS
=======
34
4 q. Does this affect a major Milestone?
========================================
What are you trying to mean by the "major Milestone"? please describe in details
Regards
Md Ruhul Amin
.
UPDATE
=======
Hi Mamun,
"Major Milestone" means is there any specific Activity on your database ( Ex migration ,
Upgradation.. etc ) which is getting blocke
d because of this BUG..
Regards,
Bharath Sunku
STATUS
=======
.
FOLLOW UP
==========
Hi Mamun,
Please update the SR as soon as possible with the current status and/or requested information
so we can continue working the i
35
ssue.
Thank You,
Bharath
STATUS
=======
8) Business Impact:
=====================
It is a every day process and there is a time limit for it. we have to open our business activities
before 8.00 am.
Regards,
Md Ruhul Amin
36
05-MAR-08 09:37:07 GMT
.
UPDATE
=======
Hi Ruhul,
One off patch ( Patch Patch 5057695 ) for your BUG is ready and can be downloaded from
metalink..
Please download the patch and apply on the database abnd let us know if the issue is resolved
or not
Regards,
Bharath Sunku
STATUS
=======
we do not want to kill session manually. Is your patch will make shutdown faster if we are not
following our manual process( kill sessio
n)?.
Regards
Md Ruhul Amin
37
06-MAR-08 02:26:14 GMT
.
UPDATE
=======
Hi Ruhul,
Firstly the Patch that we have requeste dyou to download is for an identified BUG in 9.2.0.8.0
which is causing problem while Shutdo
wn Immediate of the database
As specified in the BUG, it is also closing the Active sessions bit more faster than before
patchset and version releases and so
this patch should solve your problem..
We resommand you to apply the patch and monitor the database for a week so that we can
progress the issue further
Regards,
Bharath Sunku
STATUS
=======
TAR has been in CUS status for 11 days -- Sending First Close Alert email.
.
FOLLOW UP
==========
Hi Mamun,
Please update the SR as soon as possible with the current status and/or requested information
so we can continue working the i
38
ssue.
Thank You,
Preetham
STATUS
=======
.
FOLLOW UP
==========
Hi Mamun,
Please update the SR as soon as possible with the current status and/or requested information
so we can continue working the i
ssue.
Thank You,
Preetham
STATUS
=======
TAR has been in CUS status for 11 days -- Sending First Close Alert email.
TAR has been in CUS status for 14 days -- Setting status to ACL -- Sending Second Close
Alert email.
39
09-APR-08 17:59:25 GMT
.
FOLLOW UP
==========
Hi Mamun,
Please update the SR as soon as possible with the current status and/or requested information
so we can continue working the i
ssue.
Thank You,
Preetham
STATUS
=======
CALL SUMMARY
=============
Called Rahul and spoke about the issue
Rahul told that he will start working on this issue from tommarow
Rahul asked me steps for applying the patch.. Told that Readme file has all the instructions and
asked him
to follow and if problem asked him to update the SR
Call Summary
==========
40
Called Rahul and spoke about the issue
Explained Rahul on how to apply the patch
Found that Opatch is not installed in the Ho,e..
instruncted on how to download and install the opatch and asked him to apply the pat
ch
Told Rahul to update the SR if he needs any clarifications on how to apply the patch
Again i heve tried to apply patch, but again error found. here is error information
==================================================================
[FINBRAC]/home/oracle/5057695> ls -lart
total 24
drwxr-xr-x 3 oracle orainv 256 Jun 16 2007 files
drwxr-xr-x 4 oracle orainv 256 Jun 16 2007 etc
-rw-rw-rw- 1 oracle orainv 4968 Jun 16 2007 README.txt
drwxr-xr-x 4 oracle orainv 256 Jun 16 2007 .
drwxr-xr-x 8 oracle orainv 4096 Apr 16 14:00 ..
[FINBRAC]/home/oracle/5057695> /oracle/oraInventory/OPatch/opatch apply
Invoking OPatch 10.2.0.1.6
41
ApplySession failed:
--------------------------------------------------------------------------------
The Oracle Home does not meet OUI version requirement.
This OPatch (version 10.2.0.1.6) detects OUI version 10.1.0.5.0 in the home.
It requires OUI version 10.2 or above.
===============================================
regards
Md Ruhul Amin
The customer : ruhul.amin@bracbank.com : has uploaded the following file via MetaLink:
C:\Documents_and_Settings\User\Desktop\opatch2008-04-16_14-16-17PM.log.txt
.
UPDATE
=======
Hi Rahul,
Regards,
Bharath Sunku
STATUS
42
=======
The customer : ruhul.amin@bracbank.com : has uploaded the following file via MetaLink:
C:\Documents_and_Settings\User\Desktop\opatch2008-04-16_14-16-17PM.log.txt
.
UPDATE
=======
Hi Rahul,
We have created a Spin-off ( SR:18780616.6 ) with t h einstall team to resolve opatch errors
that you are facing.
Please upload the latest log files that you have uploaded in this SR to the New SR so that the
Ins
tall Analyst can progress the issue further
Regards
Bharath Sunku
regards
md Ruhul Amin
43
17-APR-08 06:10:28 GMT
.
UPDATE
=======
Hi Rahul,
++ The above command will provide the list of patches installed on your database after the
patchset installed on your database
Please let us know whether we can keep the SR in the Soft Closure ( Monitoring ) status for 2
weeks so that yo
u can monitor the database for few days to check whether the same issue is reocc
urring or not
Regards,
Bharath Sunku
STATUS
=======
Regards,
44
Ruhul
.
UPDATE
=======
Hi Rahul,
This means that the TAR will remain in inactive mode(extended sleep mode) for another 14
days, during which period
you can directly update the TAR via Metalink to re-activate it and I will be gl
ad to assist you further. Otherwise, no update is necessary and the TAR will aut
omatically close after two weeks.
For further new issues/errors, if any, please open a new SR so that it gets best handled by
appropriate Team
Regards,
Bharath Sunku
STATUS
=======
More information
To get more information, login to this web side
www.dbasolve.com
45