White Paper

Transactional Data Management for

Eliminating Database Downtime When Upgrading or Migrating from Oracle 8i or 9i to Oracle 10g

Publication Date: April 2007 Author: Alok Pareek, Vice President of Technology, GoldenGate Software, Inc.

Abstract: Eliminating database downtime poses a significant challenge for IT organizations that need to upgrade or migrate mission-critical database environments running Oracle database software versions 8i or 9i to the latest Oracle 10g release. That is particularly true for critical applications that must provide continuous or near-continuous operations to users who increasingly expect uninterrupted availability of online service. Any outage of an application or website, even if that outage is scheduled or planned, has an impact on the revenue and reputation of the business. The purpose of this paper is to present a solution that tackles the specific challenge of upgrading or migrating an Oracle 8i or 9i database to Oracle software version 10.x without taking any database downtime. Key technology components used in this solution are GoldenGate Softwares Transactional Data Management (TDM) software and Oracles Cross Platform Transportable Tablespace feature. The details within this paper will showcase key technical strengths of the solution, including how to achieve a rolling upgrade/migration for this scenario; using a clone database to offload instantiation and conversions; keeping transactions in synch across the databases; managing partial or phased migrations or upgrades; conducting data verification post-upgrade/migration; and implementing an easy and reliable failover strategy.

White Paper: GoldenGate TDM for Eliminating Downtime when Upgrading or Migrating from Oracle 8i or 9i to 10g

April 16, 2007

Table of Contents
Section 1. Introduction 2. Goals 3. Concepts and Terminology Transactional Data Management (TDM) and Oracle Database Upgrade Database Migration Standby Database Transportable Tablespace Cross Platform Transportable Tablespace Clone Database RMAN (Recovery Manager) 4. Zero-Downtime Approach Using TDM and Oracle Upgrade vs. Migration 5. Upgrade vs. Migration Challenges Business Challenges Technical Challenges 6. Available Solutions: 8i or 9i 10g Upgrade or Migration Database Upgrade Assistant Export/Import SQL Plus As Copy Transportable Tablespaces SQL*Loader Oracle Dataguard, Oracle Streams Transactional Data Management (TDM) 7. Overview of GoldenGates TDM Technology Components Capture On Disk Queues aka Trail files Delivery (Apply) GoldenGate Veridata 8. Overview of Cross Platform Transportable Tablespaces Feature 9. Detailed Steps Using a Platform Migration Case Example Migration without Failback Migration with Failback Failback Steps Partial Migration Using Bi-Directional Synchronization 10. In Summary About the Author References Contact Information Page Number 1 1 2 2 3 3 3 4 4 4 4 4 5 5 5 6 9 9 9 10 10 10 10 11 12 12 12 12 13 13 13 14 15 16 16 16 17 17 18

White Paper: GoldenGate TDM for Eliminating Downtime when Upgrading or Migrating from Oracle 8i or 9i to 10g

April 16, 2007

1.

INTRODUCTION
Today, mission-critical applications must provide continuous operations to users who increasingly expect uninterrupted availability of online services. Any outage of an application or website, even if that outage is scheduled, has an impact on the revenue and reputation of the business. In fact, many e-commerce businesses report that a half-day outage would incur millions of dollars in financial losses. For databases that host the data store for these mission-critical applications, availability requirements have become stringent. From that high availability standpoint, there are essential events that still require application downtime: modifying hardware or database software, upgrading applications, applying software patches, migrating to different computing architectures, etc. Since such events are not conceived via system or data failures, they are aptly classified as planned outages. Empirical data from non-trivial applications deployed in very large database (VLDB) environments demonstrates that minimizing downtime to handle planned outages is a complex, time consuming, error prone, and costly process. For all of those reasons, many organizations choose to postpone those essential IT events as long as possible. If that future event is to move to Oracle 10g, there is a better solution available today. The purpose of this paper is to present a solution that takes the specific problem of either upgrading or migrating an Oracle 8i or 9i database to Oracle software version 10.x without taking any database downtime (excluding application switchover time). Eliminating database downtime for upgrades or migrations in mission-critical Oracle database environments running Oracle database software versions 8i or 9i poses a significant challenge for IT organizations. Oracle Corp. does not support the database rolling upgrade solution when upgrading the database from Oracle version 8i or 9i to Oracle version 10 (or higher). As such, the solution described and illustrated in this paper is the only method that limits database downtime to mere seconds during the migration or upgrade to Oracle 10.x. In the case of applications running on 8i, theres a strong sense of urgency and concern to upgrade to Oracle version 10 as the 8i product has reached end-of-life support from Oracle. The key technology components used in this solution to achieve the zero-database downtime for a migration are GoldenGate Software, Inc.s Transactional Data Management (TDM) software and Oracles Cross Platform Transportable Tablespace feature. Oracles Cross Platform Transportable Tablespace feature facilitates high-speed data movement of Oracle data from one machine/platform to another; it is a recommended though not mandatory method during instantiation of a target database that eventually participates in a rolling upgrade.

2.

GOALS
This paper is created primarily for advanced database users. The main goal of this paper is to increase awareness of a solution for minimizing database downtime during a database upgrade or migration under the circumstances outlined in the introduction. As such, resource issues such as disk space, software-licensing costs, etc., are not discussed not because such issues are unimportant, but because the need to minimize downtime or completely avoid it outweighs other considerations in demanding high availability environments. Furthermore, this document is not intended to repeat the upgrade steps, typical warnings, and preparatory details associated with an Oracle database upgrade. There are many articles and notes that exist on the OracleMetaLink customer support portal, in various books, and on the web that are helpful towards understanding the operational procedures and complexities associated with upgrades and migrations. As one of the lead developers of the Cross Platform Transportable Tablespace feature in 10g at Oracle, I struggled with minimizing downtime, when migrating very large 8i or 9i databases from one platform to

White Paper: GoldenGate TDM for Eliminating Downtime when Upgrading or Migrating from Oracle 8i or 9i to 10g

April 16, 2007

another. My aim therefore is to present a novel solution that allows for a database upgrade/migration from 8i or 9i to 10g without taking database downtime or having to make tablespaces read only, or take any locks. The solution presented leverages the benefits of real-time synchronization, clone databases, rolling upgrades, transportable and cross-platform transportable tablespaces, and failback. Every effort is made to avoid overhead at the primary database to ensure application availability (activities such as instantiation, file conversion for cross platform transport, etc., are offloaded to a clone). A secondary goal is also to explain how to minimize the wall clock time required to perform the entire upgrade. Experienced users will realize that wall clock time can be to the order of days or weeks, yet the downtime to the application can still be near zero. The key here, as will be explained in the following pages, is to offload work processing to additional database copies.

3.

CONCEPTS AND TERMINOLOGY

Throughout this document, we refer to the production database as the SOURCE and the secondary copy as the TARGET database. The following concepts and terminology deserve review to fully comprehend the solution proposed in this white paper:

3.1

Transactional Data Management (TDM) and Oracle Transactional data management (TDM) is a technology platform that guarantees capture, routing, transformation, delivery, and verification of data across heterogeneous business systems in real time. GoldenGate Software implements TDM to capture transactions non-intrusively from a source database, transform when needed, propagate, and apply those transactions with guaranteed integrity to another database (target) in real time. GoldenGate TDM processes run continuously (and can even be bi-directional) and support high-volume, rapidly changing environments, moving thousands of transactions per second with very low impact. The target database is a transactional replica at a logical level; it can be functionally leveraged for multiple applications including a rolling database upgrade. Aside from the differentiating ability of TDM technologies to handle heterogeneous database environments, readers familiar with Oracle data movement technologies may ruminate on the perceived resemblance between Oracle Streams and GoldenGates TDM. This collation, however, is purely conceptual. The underlying architectures are significantly different. For example, Streams in Oracle 9i has problems scaling in high-throughput environments. In addition, it is limited in its support for data types (e.g. LONG) and for certain kernel functionality (e.g. log_parallelism). These issues are not present in GoldenGates TDM platform. GoldenGates TDM is designed for managing high transaction volumes in real time without impacting throughput or commit time latencies on the source database. The transaction capture component does not rely on the shared memory of the database instance to stage transactions or associated metadata, thereby avoiding any resource contention with the ongoing application workload. Not relevant to this specific topic, but noteworthy for this audience nevertheless, is that TDM also can be used when the target and source database software come from different vendors.

White Paper: GoldenGate TDM for Eliminating Downtime when Upgrading or Migrating from Oracle 8i or 9i to 10g

April 16, 2007

3.2

Database Upgrade A database upgrade advances the Oracle software release number from one version to another. There are two primary ways to perform an upgrade: 3.2.1 In place An in-place upgrade renders the database inaccessible for business applications while the database software is being upgraded. This procedure entails running an upgrade script (e.g. for u0902000.sql), recompiling invalid PL/SQL etc., and experiencing downtime that is usually not acceptable in most mission-critical environments. The intent of this paper is to not to address in place upgrades.

3.2.2

Rolling Upgrade The term rolling upgrade refers to upgrading different databases or different instances of the same database (in a Real Application Clusters environment) one at a time, without stopping the database. Oracle doesnt allow a rolling database upgrade in Oracle 9i. A rolling upgrade (database) consists of the following high-level steps: 1. 2. 3. 4. 5. 6. 7. The application points to the production database running software version VOLD A secondary logical database copy is constructed running software version VOLD The secondary database copy is upgraded to the next database version VNEW The secondary and primary databases are synchronized The primary database is shut down The application is pointed to the secondary database The primary is kept in the same version VOLD for failback reasons

3.3

Database Migration A migration allows for the underlying operating system or hardware platform to be changed. In Oracle, on disk file formats are not homogeneous across platforms. Under 10g compatibility, the on disk structures for platforms that appear in v$TRANSPORTABLE_PLATFORM are identical, but the endian format could differ. Moving an Oracle database across different operating systems is a common requirement in many computing environments.

3.4

Standby Database A standby database is a copy of the main production database that is maintained for high availability or disaster tolerance purposes. The standby database can be physical or logical. In the physical standby copy, the database is kept on recovery mode (i.e. the redo logs of the production database are applied to a mounted copy of the production database the hardware/OS limitations here are obvious since redo across Oracle platforms is not compatible). A logical standby is a copy of the production database that contains the same objects, but doesnt physically match the structure of the production database copy. For example, a standalone table EMP in the physical database could be represented as obj# 3143 whereas its logical equivalent at

White Paper: GoldenGate TDM for Eliminating Downtime when Upgrading or Migrating from Oracle 8i or 9i to 10g

April 16, 2007

the standby site could be represented as obj# 5931. It is maintained by instantiating a logical equivalent of the production database and replaying the SQL that modifies the production database at the standby site.

3.5

Transportable Tablespace Transportable tablespace was a feature introduced in Oracle 8i that allows for non-system tablespaces to be moved from one database to another by physically grafting the tablespaces datafiles into the target databases control file and importing object metadata into the target databases dictionary. It has three main phases: 1. 2. 3. Exporting the metadata (dictionary data for the objects) Transferring the datafiles from one database to another Importing the metadata and datafiles

3.6

Cross Platform Transportable Tablespaces Cross Platform Transportable tablespaces is an extension of the transportable tablespaces feature that allows for tablespaces to be transported across database platforms. This feature can only be used after the database compatibility has been advanced to 10.0.0.0 or higher. This whitepaper makes use of this feature along with GoldenGates TDM software to migrate an 8i or 9i database to a 10g database.

3.7

Clone Database A clone database is a database constructed using a restored backup of an existing database, recovered to a point in time, and opened.

3.8

RMAN (Recovery Manager) RMAN (Recovery Manager) is an Oracle tool that manages the process of making backups and managing the process of restoring and recovering from them. It is also used for conversion of endian systems during a cross platform CONVERT.

4.

ZERO-DOWNTIME APPROACH USING TDM AND ORACLE

Since an in place upgrade requires application downtime during the upgrade process, it is not feasible in critical HA environments. Therefore we consider only rolling upgrades henceforth. This solution is not available using standalone Oracle tools to upgrade/migrate from 8i or 9i onto 10g. Using Transactional Data Management in conjunction with Oracle technologies, a rolling upgrade or a rolling migration can be performed without taking any application downtime other than the very minimal time required for application switchover, typically less than one minute and in most cases, only seconds.

White Paper: GoldenGate TDM for Eliminating Downtime when Upgrading or Migrating from Oracle 8i or 9i to 10g

April 16, 2007

4.1

Upgrade vs. Migration In general, a migration is more complex than an upgrade given that in addition to the software upgrade, the platforms are different and there are on disk incompatibilities across the datafiles/logfiles/controlfiles of the source and target database platforms. As such, this paper will take a real-life case study of migrating an existing Oracle database on the Sun (Solaris OE (32-bit)) platform running on 9i and migrate it to Linux (Linux IA (64-bit)) on 10g. Readers who are interested only in an upgrade should skip the cross platform transport steps covered in Section 8.

5.

UPGRADE AND MIGRATION CHALLENGES

Such upgrade and migration projects pose challenges for mitigating business risk and navigating technology issues. Some of the more common and critical challenges include:

5.1

Business Challenges 5.1.1 Impact on Revenue Downtime equates to lost revenue. Many businesses (e.g. retail, travel, banking, etc.) no longer have the extended downtime windows for planning upgrades/migrations. Many database environments, for scalability or cost saving reasons, would like to take advantage of recent trends in technology such as deploying low cost storage (new hardware), or other advances in clusterware, and software. The following table (Figure 1) illustrates the downtime costs associated with various businesses.

Figure 1: Average Cost of Downtime for Business Applications

5.1.2

Customer Expectations/Loyalty The Internet era posits that businesses that continue to support online processing beyond conventional business hours. For the user, the cost of switching over to the competition is non existent or minimal. For example, lets look at what happened when eBay took a 22hour outage several years ago on a Thursday. In a combined study by Nielsen Media

White Paper: GoldenGate TDM for Eliminating Downtime when Upgrading or Migrating from Oracle 8i or 9i to 10g

April 16, 2007

Research and NetRatings Inc., which measures website usage, it was found that during eBays downtime, the average time spent per person at Yahoo's auction site the next day increased from 7 minutes to 18 minutes and the number of people using Yahoos site skyrocketed from 62,000 on Thursday to 135,000 on Saturday. Financial losses to eBay were estimated at $3 million to $5 million.4 5.1.3 Interdependencies, Business Reputation E-commerce has resulted in digital business partnerships where non-availability of critical data at one site may also result in loss of business services at multiple sites (via data access from either web services, direct querying, or pre-configured batch loading). Extended downtimes are becoming difficult to plan for since significant coordination is required with business partners. This results in postponement of upgrades/migrations, which has associated indirect costs (e.g. running the existing software with complex workarounds for bugs, not being able to take advantage of new software functionality in the later releases, etc.).

5.2

Technical Challenges The following presents some of the serious issues that need to be dealt with when considering a zero database downtime upgrade/migration. Once these issues are well understood, one gains clarity into why a unified solution using TDM and Cross Platform Transportable Tablespace provides the best way to perform the upgrade or migration for HA environments.

5.2.1

Instantiation The first problem that needs to be addressed when performing a rolling upgrade is the method in which the target database is instantiated. In other words, how do you create a first version of the target database? The complexity of the problem magnifies when the target database is on a different platform, since physical backups cannot simply be restored at the target and converted into a clone. To handle multi-terabyte sized databases, procedures such as export/import are tedious and error prone. Failures during the instantiation or global data consistency are not adequately addressed with methods that are time consuming, unless some form of locking is used. Extracting data from the production database over a period of time results in a performance impact on the source database that is unacceptable. Theres also interference with the normal mechanics of the database cache manager affecting buffer allocation and flushing algorithms, resource management algorithms, etc., since the typical OLTP database is not optimized to handle long running queries, table scans, etc. A good instantiation solution therefore must properly address: o o o o Global data consistency Efficiency (speed) Performance (i.e. low impact on the production database) Manageability

5.2.2

Incremental Data Movement The next post-instantiation challenges are to determine: i) The point that demarcates the last committed transaction picked up by the instantiation process with the subsequent transactions submitted against the production database; and

White Paper: GoldenGate TDM for Eliminating Downtime when Upgrading or Migrating from Oracle 8i or 9i to 10g

April 16, 2007

ii)

The method in which the subsequent transactions get propagated to the target database.

Depending on the instantiation method, one or both problems may be pertinent. The following example depicts this challenge with the aid of the Figure 2.

Figure 2: Illustrating high-level instantiation of a new target database for upgrades/migrations.

Example: 1. Tables MASTER and SLAVE have a parent-child relationship via a foreign key constraint. 2. The instantiation method (e.g. export, SQL copy, etc.) picks up a read consistent version of table MASTER as of SCN 23.12360. 3. The instantiation method picks up a read consistent version of table SLAVE as of SCN 23.12777 4. Transaction TM inserts new row R1 into table MASTER at SCN 24.12890

There are several problems with this method. First, how are transactions handled that are active as of SCN 23.12360 but commit after SCN 23.98745? Second, the MASTER and SLAVE tables are from different points in time, and as a result inconsistent with each other from a referential integrity point of view. And third, the data submitted subsequent to SCN 23.12360 for the MASTER table (in this case row R1) still needs to be moved. Thus, a good solution that addresses incremental data movement must: i) ii) iii) iv) Ensure global consistency Identify a clean instantiation termination point Allow transactions from that point onward to be propagated to the target database Handle any failures during incremental data propagation (e.g. loss of network, media failure, etc.)

5.2.3

Performance Impact Another major concern during the upgrade is to understand what kind of impact is experienced by the application running on the production database. The upgrade steps should not degrade performance on the production database such that the application service levels (SLAs) are compromised.

White Paper: GoldenGate TDM for Eliminating Downtime when Upgrading or Migrating from Oracle 8i or 9i to 10g

April 16, 2007

5.2.4

Staging Areas Adequate disk resources need to be configured and managed when addressing rolling upgrades or migrations. Especially when a no-downtime migration is required proper consideration must be given to handling the disk requirements for the clone database.

5.2.5

Change Management Changes other than those made by DML need to be addressed during the upgrade/migration process. Examples of these are datafile additions, PL/SQL package creations, etc. In general if the overall solution is fast (i.e. O(hours) ) such changes can be restricted since they are infrequent operations.

5.2.6

Special Handling of Legacy Data or Certain Datatypes Any datatypes that are not supported by the solution must be identified ahead of time and dealt with separately. These are usually objects that Export/Import doesnt work with, such as certain application tables in the SYS schema and typically are fairly small in size. These simply can be recreated at the target. Additional handling is required to support certain datatypes when upgrading/migrating from Oracle 8i environments.

5.2.7

Verification Testing that the source and target database are synchronized needs to be conducted post upgrade/migration any out-of-synch conditions at the new source and/or failback database could pose risks to the business and should be identified and acted upon. Conducting this verification should be possible even as ongoing transactions are being processed at the source database. A good solution must therefore address fast and efficient comparison of data across the two databases, i.e. the database should not acquire any locks during the verification period. In-flight transactions must be accounted for during the comparison.

5.2.8

Failback Once the upgrade/migration takes place, a failback strategy must be in place to avoid any downtime and data loss in case the new environment is not stable. Therefore, all transactions that have been processed at the target (the new production database) need to be propagated to the original production database in a consistent and manageable manner for a successful failback.

White Paper: GoldenGate TDM for Eliminating Downtime when Upgrading or Migrating from Oracle 8i or 9i to 10g

April 16, 2007

6.

AVAILABLE SOLUTIONS: 8i or 9i

10g UPGRADE OR MIGRATION

Figure 3: An overview of downtime categories for different solutions, given various Oracle 10g upgrade and migration scenarios.

There are a number of possible solutions that may be evaluated to help with a project to move onto 10g. The figure above (Figure 3) depicts whether these solutions support different possible migration or upgrade scenarios, along with associated time categoriesfrom extended downtime to real-time switchover. Those and several other common methods and solutions are summarized below with their respective advantages and disadvantages, illustrating where significant shortfalls exist for environments seeking minimal downtime and high availability:

6.1

Database Upgrade Assistant This method only works for an in place upgrade. This doesnt solve the rolling upgrade scenario, nor does it address platform migration.

6.2

Export/Import A full export can be done from an 8i or 9i database, followed by an import into an existing vanilla 10g database. Oracles Export utility extracts data from the database and writes it to a proprietary file (called the export dump file). Import processes the data and recreates all the objects (tables, indexes, packages, etc.). Thus the time required for an export/import upgrade is high. An export in FULL mode does have advantages in that it moves all database objects; it also supports both upgrades and migrations. However, in a high availability environment, there are three significant disadvantages to using Export/Import: 1. Volume-Time Relationship: The time required to extract the data is dependent on the size of the data. Extracting and reinserting high volumes of data can take to the order of days or even weeks for large databases.

White Paper: GoldenGate TDM for Eliminating Downtime when Upgrading or Migrating from Oracle 8i or 9i to 10g

April 16, 2007

2.

Ongoing Transactions: Ongoing changes are not captured by Export, nor does there exist a way to demarcate the boundaries across SCNS that allow the incremental data to be reexported. This effectively implies application downtime during the export/import process. Recoverability: Failures during export and/or import are not easy to recover from.

3.

6.3

SQLPlus As Copy Using this method, all schemas/tables can be captured from one database and recreated on another database over a database link. An example of schema copy is as follows: COPY FROM SCOTT/TIGER@LOCALDB TO SCOTT/TIGER@REMOTEDB This method doesnt scale nor can it be viewed as serious to move high volumes of data. It has a high database impact, is unmanageable, and doesnt support incremental data movement.

6.4

Transportable Tablespaces Transportable tablespaces have been used effectively to minimize planned outages, since the time to transport avoids extracting data and instead relies on integrating datafiles directly into the target database. Its not a complete solution, because it doesnt address incremental data movement and also imposes restrictions on the production application by rendering the user data to read-only workloads. Also, the method cannot be used for migrations. The high level flow of the upgrade process using transportable tablespaces is as follows: 1. Create target database in 10g using install utilities. Also install any components and applications as necessary. Pre-create all users, roles, java packages, views, and additional objects not picked up by transport. Extract any unsupported objects using export or recreate object commands. Make all non SYSTEM tablespaces read only. Unplug the non SYSTEM tablespaces using the export utility this extracts only the dictionary metadata associated with the non SYSTEM tablespaces. Transport the datafiles in the non SYSTEM tablespaces to the target database. Plug the datafiles into the target database using the import utility in transportable mode. Run manual verification tests to ensure that the data is synchronized with the primary.

2. 3. 4. 5. 6. 7.

This method can achieve the upgrade at disk speeds of file transfer; but the drawback is that the application is inaccessible -- so the downtime could be extended and as previously noted, does not support migrations.

6.5

SQL*Loader This method involves scanning all the tables and writing the data into flat files, and using the SQL loader utility to reload the data into the target database. It is tedious, slow, and doesnt address all database object movement. It suffers from the same disadvantages as Export/Import and SQL Copy.

6.6

Oracle DataGuard, Oracle Streams The above two solutions do not support a rolling upgrade from 8i or 9i to 10g. (See references).

10

White Paper: GoldenGate TDM for Eliminating Downtime when Upgrading or Migrating from Oracle 8i or 9i to 10g

April 16, 2007

6.7

Transactional Data Management (TDM) Transactional data management (TDM) solutions are best suited to address a minimal database downtime from 8i or 9i to 10g. In particular, GoldenGates TDM platform offers real-time, heterogeneous data movement that easily supports upgrades and migration scenarios. The upgrade using TDM consists of the following high-level steps: 1. 2. 3. 4. 5. 6. Initial Setup of a standby database running 8i or 9i software using an existing database backup. Upgrade the standby database to 10g. Synchronize the standby database with the production database. Test in active/live mode. Switchover the application to the standby database. Upgrade the primary database to 10g after comprehensive application testing at standby.

For a migration scenario, the TDM solutions main high-level steps are as follows: 1. 2. 3. 4. 5. 6. 7. 8. Initial Setup of a clone database running 8i or 9i software using an existing database backup on the same platform. Upgrade the clone database to 10g, advance compatibility to 10.0.0.0. Create a vanilla 10g database on the new platform with 10.0.0.0 compatibility. Move the data from the clone database to the new target database using Oracles Cross Platform Transportable tablespaces feature. Synchronize the new target database with the original 8i or 9i production database. Test in active/live mode. Switchover the application to the standby database. Upgrade the primary database to 10g after comprehensive application testing at standby.

From here, Sections 7-8 describe the technologies that will be used to conduct the zero-downtime upgrade or migration. Section 9 discusses the steps in greater detail. Since a migration is a more complicated procedure than an upgrade, the detailed steps take a case study of migrating a 9i database on one platform to a 10g database on another platform.

11

White Paper: GoldenGate TDM for Eliminating Downtime when Upgrading or Migrating from Oracle 8i or 9i to 10g

April 16, 2007

7.

OVERVIEW OF GOLDENGATES TDM TECHNOLOGY COMPONENTS

There are a number of key technology components that comprise GoldenGates implementation of Transactional Data Management and are relevant to the subject at hand.

Figure 4: High-level architecture diagram of GoldenGates TDM software products, running bi-directionally and verifying data between two databases.

7.1

Capture GoldenGate Capture is a process that runs on the source database system. It is multithreaded in an Oracle RAC environment. The process mines transactions from the Oracle redo log and propagates transactions over the network to an on-disk queue. Only committed transactions are written to the queues. 7.1.1 Initialization Capture can be used either to initialize the target database directly, or to start propagating transactions against an existing target database from a given point. Change Capture Only table DML operations are captured from the source database, i.e. updates to indexes, system tables, etc. are not captured.

7.1.2

7.2

On Disk Queues or Trail Files Trail Files can be conceptualized as a persistent ordered set of committed transactions generated by the GoldenGate Capture process. GoldenGates Trail records describe DML operations (inserts, updates, and deletes) along with transactional context as captured from the source database.

7.3

Delivery (Apply) The Apply process runs on the target system. It reads the captured data from the Trail Files, and applies the captured transactions at the target using dynamic SQL. To maintain synchronization between the source and target databases, GoldenGate applies data changes to the target tables using native database calls, statement caches, and local database access. To ensure data and

12

White Paper: GoldenGate TDM for Eliminating Downtime when Upgrading or Migrating from Oracle 8i or 9i to 10g

April 16, 2007

referential integrity, GoldenGate applies data changes in the transaction commit order that occurred at the source database.

7.4

GoldenGate Veridata Veridata is a stand-alone, high-speed comparison solution that identifies and reports data discrepancies between two databases without interrupting ongoing business processes. Veridata allows data discrepancies to be isolated for testing and troubleshooting. Veridata is ideal for conducting data validation after the rolling upgrade, once the source and target are fully operational and running different versions of Oracle. It can also help to determine if a failback is needed, in case of any risky data anomalies.

8.

OVERVIEW OF THE CROSS PLATFORM TRANSPORTABLE TABLESPACES FEATURE

The cross platform tablespace procedure to transport all non SYSTEM tablespaces is described below, since it will be used as part of the rolling upgrade/migration process: 1. 2. 3. 4. Ensure compatibility of source and target database is at 10.0.0.0 or higher. Make all non SYSTEM tablespaces read only. Unplug the non SYSTEM tablespaces using the export utility this extracts only the dictionary metadata associated with the non SYSTEM tablespaces. Convert the datafiles associated with the non SYSTEM tablespaces using the RMAN CONVERT functionality. If the cross platform transport is across the same endian system, then conversion may not be required. Transport the converted datafiles in the non SYSTEM tablespaces to the target database, Plug the datafiles into the target database using the import utility in transportable mode.

5. 6.

9.

DETAILED STEPS USING A PLATFORM MIGRATION CASE EXAMPLE

We use a case study of moving an Oracle 9i database on Sun Solaris to Oracle 10g on Linux and discuss the detailed implementation steps. These steps can be simplified and adopted to accomplish a rolling upgrade. Two scenarios are described next: A migration without the addition of a failback solution, and the same migration with the addition of a failback.

13

White Paper: GoldenGate TDM for Eliminating Downtime when Upgrading or Migrating from Oracle 8i or 9i to 10g

April 16, 2007

9.1

Migration without Failback

Figure 5: Cross-Platform Migration (without failback)

1. 2. 3. 4. 5. 6. 7.

Address change management by restricting creation of newer packages, tablespaces, etc. during the migration process. (Not depicted) Start GoldenGate TDM Capture process (captures consistent scn = Qscn) at the production database Dprod. Do a point-in-time recovery of an existing backup of Dprod until Qscn in a separate staging area. Call this database Dpitr. Upgrade Dpitr to 10g on Solaris. Advance compatibility to 10.0 or higher. Set up a vanilla 10g database on Linux. Call this database Dtarget. Take a full export without rows at Dpitr. Call the generated export exp_norows.dmp. This will pick up any objects that are not picked up as part of the transportable tablespace export. Unplug the user tablespaces from Dpitr using the Oracle Cross Platform Transportable tablespaces feature using source side endian conversion. (Note the conversion would not be required if the endian systems were the same.) This is the step that avoids any performance degradation and does not require any quiescing at Dprod. This step will create a small export dump file. Call this exp_xtts.dmp. Plug the set of tablespaces from Dpitr into Dtarget using the Cross Platform Transportable tablespace feature. Use the exp_xtts.dmp file created from Step 7. Note that the plugged in tablespaces are in read-only mode. Make the set of user tablespaces in Dtarget Read Write. (Not depicted)

8.

9.

10. Do a NOROWS import with IGNORE=Y option at Dtarget using the exp_norows.dmp dump file. 11. Start GoldenGate Apply process at Dtarget and synchronize up to the changes generated since Qscn. 12. If any data types are not supported by Transportable tablespace or GoldenGates TDM, then do a special export/import of these objects from Dprod to Dtarget. 13. Use GoldenGate Veridata to verify that the data at Dprod and Dtarget is synchronized.

14

White Paper: GoldenGate TDM for Eliminating Downtime when Upgrading or Migrating from Oracle 8i or 9i to 10g

April 16, 2007

14. Switchover the application from Dprod to Dtarget. (Not depicted) The above procedure offloads any quiescing, conversion work to a clone database, and takes advantage of TDMs incremental real-time data capture and delivery to eliminate the downtime to zero (excluding the application switchover time).

9.2

Migration with Failback

Figure 6: Cross-Platform Migration with failback

1. 2. 3. 4. 5. 6. 7.

Address change management by restricting creation of newer packages, tablespaces, etc., during the migration process. Start GoldenGate TDM Capture process (captures consistent scn = Qscn) at the production database Dprod. Do a point-in-time recovery of an existing backup of Dprod until Qscn in a separate staging area. Call this database Dpitr. Upgrade Dpitr to 10g on Solaris. Advance compatibility to 10.0 or higher. Set up a vanilla 10g database on Linux. Call this database Dtarget. Take a full export without rows at Dpitr. Call the generated export exp_norows.dmp. This will pick up any objects that are not picked up as part of the transportable tablespace export. Unplug the user tablespaces from Dpitr using the Oracle Cross Platform Transportable tablespaces feature using source side endian conversion. (Note the conversion would not be required if the endian systems were the same.) This is the step that avoids any performance degradation and does not require any quiescing at Dprod. This step will create a small export dump file. Call this exp_xtts.dmp. Plug the set of tablespaces from Dpitr into Dtarget using the Cross Platform Transportable tablespace feature. Use the exp_xtts.dmp file created from Step 7. Note that the plugged in

8.

15

White Paper: GoldenGate TDM for Eliminating Downtime when Upgrading or Migrating from Oracle 8i or 9i to 10g

April 16, 2007

tablespaces are in read-only mode. 9. Make the set of user tablespaces in Dtarget Read Write.

10. Do a NOROWS import with IGNORE=Y option at Dtarget using the exp_norows.dmp dump file. 11. Start GoldenGate Apply process at Dtarget and synchronize up to the changes generated since Qscn. 12. If any data types are not supported by Transportable tablespace or GoldenGates TDM then do a special export/import of these objects from Dprod to Dtarget. 13. After GoldenGate eliminated the lag between Dprod and Dtarget , use Veridata to verify that the data at Dprod and Dtarget is synchronized. 14. Start GoldenGate Capture process at Dtarget 15. Switchover the application from Dprod to Dtarget. (Not depicted) 16. Start GoldenGate Apply on Dprod.

9.3

Failback Steps Transactional Data Management guarantees that the two databases are running and able to support transaction processing, while real-time bi-directional data movement is occurring. This makes failback a trivial process, if the new database is not stable: 1. 2. 3. Stop the application at Dtarget (the new primary) running software version 10.0 or higher Once GoldenGate has applied all transactions from Dtarget to Dprod, switch application to Dprod Declare Dprod as the new primary

9.4

Partial Migration Using Bi-Directional Synchronization In certain environments, it may be possible to migrate a limited number of applications over to the target database on 10g while running the rest of the applications against the 8i or 9i source database. This can be done based on logical application partitioning or geographical partitioning. For e.g., if the application supports 100 bank branches, only one branch might be switched over to the new database until the new 10g environment is deemed stable after appropriate testing and validation. GoldenGate TDM can be run bi-directionally to synchronize the data across the source and the target systems, thus supporting multi-master database environments.

10.

IN SUMMARY
Using GoldenGates Transactional Data Management (TDM) software platform and Oracles Cross Platform Transportable Tablespaces feature, a rolling upgrade or even a migration can be done with zero database downtime and only very minimal application switchover downtime. To summarize, the key technical strengths of the solution are as follows: 1. 2. 3. A rolling upgrade/migration using two databases No instantiation using primary database by offloading to a clone database Conversions offloaded to a clone staging database

16

White Paper: GoldenGate TDM for Eliminating Downtime when Upgrading or Migrating from Oracle 8i or 9i to 10g

April 16, 2007

4. 5. 6.

Synchronize transactions across databases Verify data replication and transactional integrity using GoldenGate Veridata Easy failover strategy using a bi-directional TDM configuration

As the leader in Transactional Data Management, GoldenGate Software enables the worlds largest enterprises to improve the availability, performance, and accessibility of the transactional data that drives mission-critical business processes. GoldenGates solutions for High Availability and Disaster Tolerance include: Zero-Downtime Operations (migrations, upgrades, and maintenance); Live Standby for immediate failover; Active-Active for real-time transaction sharing and balancing and Database Tiering for unlimited transaction processing scalability. GoldenGates technology is found in more than 1500 solution implementations at 300+ customers worldwide, including U.S. Bancorp, Merrill Lynch & Co., Bank of America, Overstock.com, Southwest.com, Sabre Holdings, Mayo Clinic, Montefiore Medical Center, LINK Interchange Network Ltd., and Federated Investors. Established in 1995, GoldenGate is headquartered in San Francisco, CA.

ABOUT THE AUTHOR

Alok Pareek is the Vice President of Technology for GoldenGate Software, previously working as its Software Architect for High Availability solutions. Prior to joining GoldenGate, he had over 10 years of server development experience at Oracle in the Recovery/High Availability area. His significant (patent-filed) contributions at Oracle include development of Cross Platform Transportable Tablespaces (10g), Multi threaded redo generation (9i), Multiple block size cache support (9i), and Whole database transport (10.2). He was responsible for the redo generation component of the database from 8i to 10.2. He led the technical team responsible for high-speed data movement across platforms as part of Oracle's cost-cutting measures. He has presented at major industry events, numerous regional Oracle user groups, and various user conferences on the topics of recovery and high availability. Alok holds a graduate degree in Computer Science from Stanford University, where his research area was Database Systems.

REFERENCES
1. 10.10 Upgrade with Logical Standby Database (http://otn.oracle.com/pls/db10g/db10g.show_toc?which=main&partno=b10763&maxlevel=2&section=&expand=46875) Using a logical standby database enables you to accomplish upgrades for database software and patch sets with almost no downtime. Note: This capability is not available for upgrading from Oracle9i to Oracle Database 10g. 2. 3. 4. Devirtualizable Virtual Machines Enabling General, Single-Node, Online Maintenance, by David E. Lowell, Yasushi Saito, and Eileen J. Samberg (david.lowell@hp.com, yasushi.saito@hp.com, eileen.samberg@hp.com) GoldenGate 8.0 Operations Guide for Windows and UNIX Yahoo cashes in on Ebays outage, by Chet Dembeck.. E-commerce Times, June 18, 1999 (http://www.ecommercetimes.com/perl/story/545.html)

17

White Paper: GoldenGate TDM for Eliminating Downtime when Upgrading or Migrating from Oracle 8i or 9i to 10g

April 16, 2007

CONTACT INFORMATION
GoldenGate Software, Inc. 301 Howard Street, San Francisco, CA 94105 USA Tel: +1 415-777-0200 Email: USsales@goldengate.com For a list of worldwide offices and for more company information, please visit: www.goldengate.com

2007 GoldenGate Software, Inc.

18