ODIDemo Script Final

Demo Script – Oracle Data
Integrator Product Walkthrough

Created by : Matt Dahlman & Nick Malfroy
Modified by :
Creation Date : 29-Jan-2007
Last Updated :
Version : 0.1
Change Record
Date Editor Version Change Reference
29-Jun-2007 Derrick Cameron 1.1D Revised to reflect BIC2g Demo VM

TBD Matt and Nick 1.0D Initial version – Draft
Oracle Data Integrator Product Walkthrough Script
Date: 5/23/2008 Page #: 1

Contents
Contents......................................................................................................................2
Introduction..................................................................................................................3
Demonstration.............................................................................................................4
Getting Started............................................................................................................5
Section 1: Models and Metadata..................................................................................8
Section 2: Projects.....................................................................................................18
Section 3: Operator....................................................................................................49
Section 4: Metadata Navigator...................................................................................51
Section 5: Lightweight Designer.................................................................................56
Section 6: Common Format Designer.........................................................................57
Conclusion: Overall Questions....................................................................................64
Date: 5/23/2008 Page #: 2

Introduction
Oracle Data Integrator is a comprehensive data integration platform that covers all
data integration requirements—from high-volume, high-performance batches, to
event-driven, trickle-feed integration processes, to SOA-enabled data services.
Oracle Data Integrator addresses multiple enterprise data integration needs.

• Data Warehousing and Business Intelligence
• Service-Oriented Architecture
• Master Data Management (MDM)
• Migration
Architecture
Design Time
Development
ODI Design-Time Environment Development Servers and
Applications
User Interfaces
Topology/Security Data
Code Execution CRM
Administrators Warehouse
Agent
Design-time Execution Log Data Flow Return Codes Legacy
Metadata/Rules Repositories Conductor
Designers ERP
ESB Files /
XML
Scenarios
and Projects
Releases
Production
ODI Runtime Environment Production Servers and

User Interfaces Applications
Topology/Security
Administrators Code Execution
Runtime Agent CRM Data
Execution Log Repository Execution Log Data Flow Return Codes Warehouse
Operators Conductor
Legacy
ERP
Thin Client
Metadata Lineage ESB Files /
Metadata XML
Data Stewarts
Navigator
Date: 5/23/2008 Page #: 3

Demonstration
The Demo is a loading process, taking data from an Excel spreadsheet, a SQL Server
source table, and an Oracle dimension table and loads to an Oracle target table.
In the walkthrough, we’ll examine:

• Obtaining and extending metadata
• Creating, running transformations
• Tracking your running transformations
• Reporting against metadata
• Using the web to make slight changes to mappings
• Designing a warehouse/data hub using existing tables from sourcing
applications
Note: Oracle Data Integrator was designed at the outset to support a wide range of
technologies, including Oracle and non-Oracle. Some components were configured
to demonstrate how ODI plays into a non-Oracle technology stack, but do not
function due to missing third party software. These include:
• MySQL: There are several objects defined in the MySQL – Sales db model
which are not used in the working part of the demonstration. MySQL has been
installed and tested, but has been removed prior to distribution of this VM
image.
• In folder Demo > Packages > Main Flow > Steps > Error Email we have a
notification step. Not mail client has been configured on this VM image.
Replace the mail details with your own.
• Web Services: In folder Demo > Packages > Main Flow > Steps >
GetStockQuote has an example of calling a web service. This does not
function, and is only an example. Substitute your own web service to see how
this works.
• Web Services: Further, data sources can be exposed as a web service,
although a web services engine is required. This demonstration was originally
configured with Apache Axis2, however this was removed prior to distribution.
The configuration remains as an example.
Date: 5/23/2008 Page #: 4

Getting Started
Date: 5/23/2008 Page #: 5

Step What you should Do What you should See
1. Start up the VMWare image. You will see

this screen. Hope an bowser on your host
(desktop/laptop) and go to the URL
highlighted in red.
Follow the instructions on the web page.
2. Go back to this screen (Vmware window)

and log in as userid/pw: oracle / oracle.
Then enter startx
3. You will come to the image desktop.
Go to Start>All
Programs>Oracle>Oracle Data
Integrator and run Designer
4. As login options, choose OracleDI Local

Date: 5/23/2008 Page #: 6

Date: 5/23/2008 Page #: 7

Section 1: Models and Metadata
How to setup Technical Metadata in ODI
Reverse Engineering
Step What you should do What you should say What you should see
1. Show the Designer What you see on the screen is the

Window ODI Designer. This is where, as a
developer, you will do all of your
development work. The window
layout is fully customizable. You’ll
notice that we’ve chosen to separate
out Models and Projects. This is a
logical way to view the layout.
Models are the technical metadata
used to describe the structures you
will be using in your data integration
projects. These in turn are defined as
Projects. We’ll start by creating a
model for our target database.
Date: 5/23/2008 Page #: 8

2. Create a new Model. To We’ll start by creating a model for our target
do this, click on the add Oracle Enterprise Data Warehouse schema.
model button. To do this, we’ll add a model.
Since this model will point to an Oracle

database, let’s select Oracle from the list of
supported technologies.
Name the model
MYOracle - AS EDW Now we can select a logical schema for this
(the code will fill itself model. This is the logical connection to the
out automatically). database. This logical connection will point
LEAVE THE EXISTING to a physical connection based on a context
MODEL Oracle – that is chosen when a job is run. For
Analytic Server EDW, instance, here we have a development, qa
and use as a and production context. If I were to run a
completed example. job that uses objects from this model using
the production context, the tables on my
Select Oracle from the production connection would be used.
Technology dropdown
menu. Presenter Note: Contexts are defined in
the Topology. Users can define as many
Select XCRM_ORACLE contexts as they see fit.
from the Logical
Schema menu.
Date: 5/23/2008 Page #: 9

3. Click on the Reverse We’ll then need to determine what context

tab and select to use to reverse engineer our model. Here,
Development from the we’ll use the development context and thus
Context dropdown connect to the development database.
menu.
4. Click on the Selective We can selectively choose which tables we

Reverse tab. Click on want to reverse engineer into ODI. By using
the Selective Reverse this method, we can pick from the table list
then the Objects to in the Select Reverse tab using checkboxes.
Reverse checkboxes.
This will return all

tables in the database
so it may take a few
seconds to load.
Date: 5/23/2008 Page #: 10

5. Click back to the Note that we can use a filter to reduce the
Reverse tab and number of table names that are retrieved by
change the % in the using the Mask option. For instance here,
Mask field to W_AFFL_ we tell ODI that we only want tables that
%. Then, click back to begin with W_AFFL_. Clicking back to the
the Selective Reverse Selective Reverse tab, we see that only a
tab. subset of the tables are shown.
6. Click the Reverse By clicking the Reverse button, ODI starts

button pulling in all metadata information into its
repository including table, column, and
Click Cancel to close constraint information. Note that at no point
the model is that actual data loaded into the repository.
Date: 5/23/2008 Page #: 11

7. The method just used for reverse

engineering used JDBC connectivity to the
database. In cases where this is not an
option, ODI gives you the option of using
reverse engineering Knowledge Modules.
Example1 Excel: the definition of flat files
may be stored in a spreadsheet. ODI
includes a sample KM that reads the files
names, column names, datatypes,
comments, etc. from an Excel spreadsheet
and creates the appropriate Data Stores.
Example2 BAM: No JDBC driver exists to
get metadata from the Oracle BAM Active
Data Cache. But there is a Java API that
returns relevant metadata information. A KM
ships with ODI that allows users to use the
API to get this metadata and then create the
submodels and Data Stores in ODI.
Date: 5/23/2008 Page #: 12

Enhancing Metadata
1. Expand the Oracle- In ODI, it is possible to enhance the

Staging model. Expand metadata. We can add constraint
the SRC_CITY table. information in the models without it
Expand Constraints. actually being deployed on the RDBMS
We have a PK, but no FKs. in question. This information can be
Logically this table ought used to perform quality control
to have a link to the table opereations on the data automatically
SRC_REGION, so we can as it is loaded. A common use for this
add a link showing that is to enforce foreign key constraints
SRC_CITY.REGION_ID = on the database without actually
SRC_REGION.REGION_ID. having them defined on the database.
Date: 5/23/2008 Page #: 13

To do this, right click In this case, we’d like to add a foreign

Constraints and select key constraint to the MySQL model.
Insert Reference. In Again, it is important to note that we
the Window that appears, are NOT altering the database in
select SRC_REGION from question
the Table dropdown
menu. Then, in the
Columns tab, click the
add button:
Make sure both values in

the row are REGION_ID.
Date: 5/23/2008 Page #: 14

2. From the Control tab, Checking the data immediately is

click Check (click OK in great during development to get
the window that may instant feedback on the quality of the
appear). This does an data. With large volumes of data it can
immediate data integrity be more practical to launch a session
check on the data that runs independently. Later we’ll
currently in the table. In look at the details of the code that was
addition, you can right- generated to check the data.
click on the SRC_CITY
data store and choose
Control -> Check. This
launches a session that
runs in the background
and checks the integrity
of the data. This is
particularly useful if you
want to see the actual
bad record (*these are
accessed by right-clicking
on the SRC_CITY data
store and choosing
Control -> Errors)
* If you choose to check the data the second way, you

should get a data view with 3 records.
Date: 5/23/2008 Page #: 15

3. Expand the Used to Some metadata is automatically

Populate branch under reversed. Some is added manually by
SRC_CITY. the developer.
And some metadata is generated
Note: These are automatically by ODI. We can
cascading so if a immediately see what targets are
customer asks, this view populated by this table and what other
could keep on going. sources are used to populate this
table. Metadata like this is invaluable
when trying to understand and
document the ETL processes.
Date: 5/23/2008 Page #: 16

Printing Documentation
1. Right-click the model With all of this metadata, we can A PDF report of your model.
Oracle - Staging and easily generate PDF documentation
choose Print -> Print for all of your data integration
Model... objects. Here, we are generating a
Oracle Data Integrator PDF report of the model.
will automatically
generate a PDF report
including all of our
metadata
Date: 5/23/2008 Page #: 17

Section 2: Projects
Building Transformations in ODI
Create an Interface
1. Now let’s jump ahead a bit. We have

defined all of the sources and
targets that we want to use in our
ETL processes. Next we want to
build the transformations and
workflows that will perform the jobs
that we need.
2. Go to the Projects The basic building block used to
tab in the tree, perform transformations in ODI is
expand the ODI called an Interface. We will build an
Demo project and interface now to combine data from
expand the folder several different sources, and insert
named Demo. Right- it into one of the AS EDW tables
click the Interfaces after transforming the data.
node and select
Insert Interface…
Name the interface

load
MYW_AFFL_CA_H
Leave the existing
interface
W_AFFL_CA_H as an
example.
Date: 5/23/2008 Page #: 18

3. Click on the Diagram We now need to associate the

tab. In this tab, we’ll datastores that will be used for this
drag in the transformation with the Interface.
appropriate tables as We do this from the diagram tab.
a target and as a Here we can drag a target table to
source. the right and any number of source
tables from any number of different
From the Oracle – technologies to the left.
Analytic Server
EDW model (not the In our example, we are loading from
one we created the SRC_DA which is an Oracle table
earlier), drag the and DIRECT_LIST which represents
W_AFFL_CA_H table data in a flat file. We’ll take data
into the Target from these sources and load them
Datastore area. into the W_AFFL_CA_H table which is
on an Oracle database.
From the Oracle-
Staging model, drag
the SRC_CA table into
the Source area (on
the left hand side of
the current Interface
tab). Hit Yes on the
window that appears.
From the File –

Source model, drag
the DIRECT_LIST
table into the Source
area.
Date: 5/23/2008 Page #: 19

4. The join between the Note that the join between the Excel
two tables will be and MSSQL tables was created
created automatically: automatically. This is because of a
but we need to modify known relationship between the
it. tables.
Must be a right outer
join and performed on We do need to modify this join
the staging area. however. It needs to be on the
Staging Area we are joining data
To change this, click from a non-relational source.
on the join icon. In Remember that the ODI architecture
the bottom panel, takes advantage of the database
you’ll want the engines already in use, so the join
following (Note, stay will need to happen on an
in the implementation appropriate database server.
tab):
<text>: By default, the target database is
DIRECT_LIST.FROM_C used as the staging area. So in this
ON_WID=CA.FROM_C case, the data necessary data from
ON_WID (Note: should the two tables will be loaded into the
be there already) Oracle database where they will be
Execute On: Staging then joined.
Area
Check the right check

box for a right outer
join.
Date: 5/23/2008 Page #: 20

Note: Customers may ask about the staging area:
Q. Where are temp tables created?
A. They are created in a temporary schema, not in the actual data schema.
Q. Do we have to create them?
A. No. ODI does this automatically for you with its Knowledge Modules.
Q. Will that have an impact on my target server?
A. Minimal. As a rule of thumb, ODI will scale with the warehouse db. If a database is properly sized, you will little to no impact on
your queries. Also note that in warehousing environments (where this question is more commonly asked due to data volumes), more
often than not, ETL jobs are run during low traffic hours where the staging area server would have a lot of available capacity.
Q. What if my target is not a relational DB?
A. You have the option of moving your staging area to a different database. This is typically done when your target is a flat file, MQ or
other non-relational type.
5. We’ll now start going down the list of

target columns to specify our
mapping logic for each.
Date: 5/23/2008 Page #: 21

6. Start by mapping the These first 2 columns should be

top 2 columns. These loaded with a constant value
are both hard coded corresponding to this source system.
values. To map, click Note that some columns are mapped
on the target column automatically based on matching the
you wish to define a source column name to the target
mapping for and, in column name. We can modify these
the mapping panel (on if desired and we'll need to define
the bottom), modify the other mappings ourselves.
the text.
Click the
DATASOURCE_NUM_
ID column and:
- set value to 1
- scroll down in the
panel and uncheck the
Check Not Null
checkbox
Click the
ETL_PROC_WID
column and:
- set value to 2
- scroll down in the
panel and uncheck the
Check Not Null
checkbox
Date: 5/23/2008 Page #: 22

7. We’re going to use the We can also use an expression Expression Editor:
expression editor editor to help us with the building of
here. Click on the the mappings. For instance, I’d like
DIRECT_FLG field and to build the mapping for the
in the panel, click the DIRECT_FLG field. Clicking on the
expression editor button brings me a
button. contextual editor. (a) For instance,
here I can see available standard
Select Oracle in the functions for the Oracle database
drop down and click since I am going to execute this
OK. expression in Oracle.
In this window, (a) (b) I can use the different items in

expand Date and the expression editor to help me
Time in the bottom build me expression.
middle panel.
Then (b):
- Type “CASE WHEN
” in the text window
- Double-click
INTEGRATION_ID
under DIRECT_LIST in
the left panel.
- Type “ IS NOT
NULL THEN '1' ELSE
'0' END”
- Click OK
- Change Execute On
radio button to
Staging Area
Date: 5/23/2008 Page #: 23

8. Click on the ROW_WID Rather than bringing up the

field. In the panel, do expression editor, in this case I can
the following: just type in the short mapping that I
- Type want.
XCRM.CA_ROW_WID
.NEXTVAL Notice that I can change the options
- Check the Active here so that this mapping, which
Mapping checkbox creates a new sequence number for
- Set the Execute On this field, is only performed when
radio button to there is an update operation for this
Target particular row.
- Scroll down and
uncheck the Update
checkbox
9. From the Oracle - Two remaining columns are not
Analytic Server mapped yet. These columns should
EDW model (not the be filled with values that are already
one we created in another table on the target
earlier), drag the system. Sometimes these are
W_AFFL_AA_H table referred to as lookup tables. A key
into the Source area. benefit of the ODI architecture is
(Click OK in the that we can just add the relevant
window that appears) table as one of our sources for this
interface. ODI knows that it doesn't
need to unload the data from the
source--it just joins to it as needed.
Date: 5/23/2008 Page #: 24

10. Drag from Notice that the final two columns are
SRC_CA.TO_ACCNT_I now mapped because the column
D names matched. Next we want to
to define a join between the new table
W_AFFL_AA_H .TO_A and the existing tables.
CCNT_ID
The resulting join

clause should be
SRC_CA.TO_ACCNT_ID
=AA_H.TO_ACCNT_ID
Scroll to the bottom of

the mapping panel
and click the Ordered
Join check box. We
need to do this since
the first join was an
outer join. This will
cause ODI to use
INNER JOIN as it’s join
syntax.
Date: 5/23/2008 Page #: 25

11. Click on the Flow tab The work that I just did in the Details
tab of the interface was simply to
define at a functional level how data
will be transformed. Notice that at
no time did I worry about how data
would physically be moved and
integrated. This is all set up from
the Flow tab. In effect we are
separating the WHAT in the Details
tab from the HOW in the Flow tab.
What you see here are the different

data areas used in the Interface.
Single table from FILE_GENERIC and
staging tables are loaded into the
Oracle database which acts as both
my target. Note the yellow gears in
the staging area/target. These
represent temp tables where data
from the sources will be loaded and
then joined. Also note that the third
table in our join is simply joined
“where it lives”. Again, it does not
need to be unloaded from the
server. This provides a significant
performance benefit.
Date: 5/23/2008 Page #: 26

12. (a) Click on the SS_3 The purpose of this Flow tab is to
(-1 – FILE_GENERIC) select which Knowledge Modules we
source box. want to use to generate our flow.
The KMs are completely open and
(b) Click on the LKM customizable flow templates. They
dropdown menu to will combine with the metadata that
show different KMs. we’ve defined up to now and
generate scripts and code that the
database can understand such as
SQL and loader scripts.
There are different types of KMs that

are used to perform different types
of tasks. For example (a) to load
from the flat file, we are using the
LKM File to Oracle (SQLLDR).
The choice of KMs is different based

on technologies used. For instance,
if I were loading from a SQL Server
database into Oracle, we would see
KMs that use LKM SQL to Oracle.
Again, keep in mind that a huge

benefit to these KMs is that they are
completely customizable. Though
we provide about 100 out of the box,
you can modify as you see fit.
Date: 5/23/2008 Page #: 27

13. (a) Click on the (a)The purpose of the LKMs is to

Target+Staging box. consolidate, or stage, the data into a
Then click on the IKM single location. The next step is to
dropdown menu to integrate that data into the target.
show different KMs. The strategy used to do this is
determined with an Integration
(b) Select the IKM Knowledge Module. There are
Oracle Incremental different types of strategies
Update. available. Anything from a simple
Append which simply inserts all data
to the target – sometimes used for
full loads, to an incremental update
which will merge data based on a
user specified unique key, to a
loading to a table that has slowly
changing dimensions.
Notice again that there are different

technology specific KMs.
(b)
You’ll see that there are many
different options available to control
how and what code gets generated.
For instance, you can have the job
create the target table for you if it
does not already exist. Note the
Flow Control option. Setting this to
yes will enable Flow Control for this
interface. This process will look at
all constraints defined for the target
table, both in the database and
simply defined in ODI, and identify
and trap any record that violates any
of these.
Date: 5/23/2008 Page #: 28

14. Click on the Control Flow Control can be parametrized

tab from this tab. Here we can specify
how many errors – either as a set
number or percentage of total
loaded records – are acceptable
before the process returns a failure.
We can also choose a Control

Knowledge Module which generates
the code to control the data.
Finally, we can specify which

constraints we would like ODI to
check.
Date: 5/23/2008 Page #: 29

15. Click the Execute We are now ready to execute our

button. Click OK in job! Clicking execute will have ODI
the 3 windows that generate the code and start the
follow (Save, Runtime process of sending it to the different
Options, Session databases.
Started)
We can now go to the Operator to
track our job’s progress.
Date: 5/23/2008 Page #: 30

16. (a)Click the Operator (a)The Operator console lets us track

button on top of the how jobs are behaving. Which ones
window to go to are running, which ones were
operator. successful and which ones failed,
which ones are queued for
execution. If we go to today’s
executions, we can see the job that
we just ran.
Once there, go to the
(b)This view will give us runtime
Sessions tab and
information. For instance, here, we
expand Date ->
see that we had 444 inserts, no
Today(<date)
updates or deletes and 5 records
that were flagged as errors. These
This shows you jobs
bad records violated one of the
that have run today.
constraints on the table and were
The topmost session is
written to an error table instead of to
the one we just ran. It
the target. These records can be
should be called load
fixed in the table and recycled on
W_AFFL_CA_H. The
the next run. The table can also e
session icon should
automatically extracted and emailed
look like
to owners of the source data.
or if it’s still
running.
Note: The actual number that you
see may be different depending on
(b)Expand the session
your data
and double click the
single node beneath
it. Go to the
Execution tab
Date: 5/23/2008 Page #: 31

17. Expand the node you By expanding this step, you can then
just double clicked in see every single step that was
the tree. executed. In every one of these
steps is stored the code that was
Double click task 13 sent to the database. Everything is
(13 – Integration - visible to you. This makes it very
load W_AFFL_CA_H - easy to audit every single thing that
Insert flow into I$ was executed.
table)
Has this step failed, you would also
Click on the se the error message and number
Description tab returned by the database.
Date: 5/23/2008 Page #: 32

Setup Changed Data Capture
1. Return to the Designer In our interface we moved all of the

by clicking the button data from the source to the target,
on the top of the but often we only want to move the
window. new or changed data. ODI uses the
Changed Data Capture (CDC) to
describe its method of tracking
changes on the source database.
Reducing the source data flow to this
relevant amount of data is useful in
many fields, such as data
Close the Interface if it synchronization and replication as
is still open. well as simply trying to reduce the
volume of data that needs to be
loaded.
CDC can be implemented via
triggers, logs, or filters. With triggers
the changed data will be saved into
a journal table. Logs follow a similar
structure, while filters simply limit
the selected data by using additional
criteria in the SQL “where clause” to
get only the data that we are
interested in.
Let’s take a look at implementing
CDC by using triggers on our source
database.
Date: 5/23/2008 Page #: 33

2. Expand the Oracle - As you see, setting up CDC in ODI is

Stagin model if it isn’t a matter of just a few clicks. I first
already. Right-click need to add this table to our list of
the SRC_CA table and CDC tables and then kick off the
select Changed Data journal for this table. This uses
Capture -> Add to another type of Knowledge Module –
CDC. a Journalizing Knowledge Module – to
create the appropriate objects on
Click OK on the the source database. In this case, it
window that appears. is creating a journal table to track
which records have changed, a few
Right-click the views to see the actual data that has
SRC_CA table and changed and a trigger on the table
select Changed Data we want to monitor.
Capture -> Start
Journal.
Click OK in the three

windows that follow.
If you minimize and

re-expand the model,
you should see a
green clock icon on
the table.
Date: 5/23/2008 Page #: 34

Note: Potential Questions:
Q. Will the trigger affect database performance?
A. Not significantly. The trigger is only tracking the PK, type of operation - additive (insert/update) or subtractive (delete), and
timestamp.
Q. What if I don’t have access to create objects on my source?
A. If there are timestamp or transaction IDs that are maintained by your source application, you can leverage these as well. That
method requires no modification to the source.
Q. What if my source is a Mainframe?
A. We have a partnership with Attunity. They provide a CDC gateway that we can leverage for such cases.
Q. What if my source is non relational?
A. CDC only works with Relational Sources. Here is a very important point: not using CDC does not preclude you from merging data.
You can still run the full load and the result will be the same. It just might take you a little longer to run the job.
3. In the Oracle - To see this in action, let’s make a

Staging model, right- change to the source data. I can do
click the SRC_CA this from within ODI.
table and select
Data... I’ve changed the data in this record
to <user or unspecified> and now I
In the window that can save my changes.
appears, find the
PROF_ROLE_CD By making these changes, the
column, double-click journal data has been populated.
to edit in the first row
and change the value
from
Unspecified to User
or vice versa.
Click Apply to save

changes. Then click
OK to close
Date: 5/23/2008 Page #: 35

4. In the Oracle - Now that the change has been

Staging model, right- made, I can see the contents of my
click the SRC_CA journal from ODI.
table and select
Changed Data Remember that these are
Capture -> lightweight triggers. Only the PK is
Journalized Data... saved. The rest of the row is
recovered through a view.
Date: 5/23/2008 Page #: 36

5. Go to the Projects Now that we’ve started

tab in the tree, journalization, we need to tell ODI to
expand the ODI use this journal instead of the full
Demo project, expand amount of data. We can do this
the folder named directly within the necessary
Demo and expand the interface. By clicking the checkbox,
Interfaces node. I put an automatic filter on the
Double-click the load source data.
MYW_AFFL_CA_H
interface that you I can also view the changed data
created earlier. directly from the interface.
Go to the Diagram When I run this job, only this one row
tab. Click on the title will be loaded and if the job is
of the SRC_CA table successful, that record will be
in the source. In the automatically deleted from the
bottom panel, check journal.
the Journalized Data
Only checkbox. A
filer will appear
Right click the SRC_CA

table and select Data.
This will show you the
journal data.
Date: 5/23/2008 Page #: 37

Use Web Services
*** THIS IS NOT A WORKING DEMONSTRATION. IT IS ONLY USED AS AN EXAMPLE OF HOW THIS COULD BE
DONE ***
1. Data Integrator leverage Web

Service technology in different ways.
We’ll talk briefly about the different
methods.
2. The Data Integrator Public Web

Services are web services that
enable users to leverage Data
Integrator features in a service-
oriented architecture (SOA). An
external tool like BPEL or BAM or any
third party tool supporting Web
Services can invoke an ODI scenario
using this web service.
Date: 5/23/2008 Page #: 38

3. Go to the Projects Another way the ODI uses this

tab in the tree, technology is that a step in an ODI
expand the ODI job can invoke an outside Web
Demo project, expand Service. The tool
the folder named Web OdiInvokeWebService has been
Service and expand provided to make this simple.
the Packages node.
Double-click the This tool provides a graphical
Invoke ODI Scenario interface to parse the WSDL file and
via Web Service populate the appropriate parameters
package. for the call.
Go to the diagram tab

and highlight the one
icon in there.
In the bottom panel,

click the Advanced…
button.
In the window that

appears, enter the
following URL into the
URL field (it should
already be there):
http://localhost:
8888/axis2/services/O
diInvoke?wsdl and
click the button.
You should see the
parsed WSDL file.
Date: 5/23/2008 Page #: 39

4. Cancel out of the Web ODI also has the ability of

Service window and automatically generating and
the package. deploying a Web Service based on
the metadata that is stored in ODI.
5. The Web Service that we will
generate and deploy here will then
be completely accessible outside of
ODI since it will be available on the
application server that we choose to
deploy it to. The Java code
generated to create the Data
Services is based on the template
provided in the Service Knowledge
Module (SKM). ODI comes packaged
with SKMs for all major database
platforms, and, as with any other
type of KM, the SKM may be
customized if desired.
Date: 5/23/2008 Page #: 40

6. Double-click the Default names for the Data Service,

Oracle - Stating Namespace, and Package are
model and go to the already entered. We can modify
Services tab. these if desired. The data source
that is referred to is something that
Make sure that the is defined on the application server
following is true: itself (not from within ODI).
- Application Server =
FILL IN
- Namespace=
DemoNamespace
- Package name=
com.odi_demo.ws
- Name of data
source=
java:/comp/env/jdbc/M
SSQL/SourceDB
- Name of data
service=
Mssqlserversalesdb
- SKM= SKM SQL.ODI
Demo
Click the Deployed

datastores tab and
make sure that
SRC_CA is checked
with DS name and
Published entity
filled.
Finally, hit the Apply

button to save any
changes then the
Generate and
Deploy button. Click
OK in the window that
appears.
Date: 5/23/2008 Page #: 41

7. (a) In the Oracle - (a)We can then go in and view the

Staging model, deployed Web Services. First, we
expand the SRC_CA can quickly see from the tree which
table (minimize firs, if services have been deployed.
necessary). You will
now see a node called To get a deeper view of the Web
WebService. Expand Services, we can open the Web
this node. Service calling console and see the
available commands in more detail.
(b) Right-click the
SRC_CA table and
select Test Web
Service. This will
parse our WSDL. You
can click through
some of the
Operations.
Date: 5/23/2008 Page #: 42

8. In the Web Service An example of a useful Web Service

window, highlight the to call is the GetChangedSrcCa
GetChangedSrcCa operation. This ties in to the CDC
operation. mechanism that we say earlier. We
can invoke this service and read or
In the Soap Editor consume a list of changed records.
part of the window,
set the Value for All of these methods are good
SubscriberName to examples of Web Service methods
SUNOPSIS. Hit the that lots of other applications might
Invoke button. take advantage of. The other
This will invoke the methods provide a wide range of
Web Service. You ways to query, insert, or update data
should see the in the database from applications
changed row from that are strictly Web Services
earlier in the right applications and would have no
hand pane. direct database access.
Date: 5/23/2008 Page #: 43

Workflows
1. Go to the Projects We’ve now seen lots of useful pieces

tab in the tree, that will be incorporated into a
expand the ODI larger workflow to create complete
Demo project, expand transformation processes. ODI uses
the folder named the term “Package” to denote the
Demo and expand the object that contains one complete
Packages node. workflow.
Double-click the Main This sample Package contains
Flow package. several pieces:
1) It invokes an external Web
Go to the diagram tab. Service to get some data
2) It launches a child workflow that
performs some other tasks in
parallel
3) It runs 3 interfaces
4) It waits for the child process that
is running in parallel before ending
5) If any of the steps has a problem
then it sends an email to the
administrator advising them to look
into it.
2. Highlight the As we saw earlier, ODI can invoke

GetStockQuote step external Web Services. This can be
a case of fire-and-forget, or it can be
returning data that we want to use
elsewhere in the workflow.
Date: 5/23/2008 Page #: 44

3. Highlight the Parallel We have a set of steps that are

Scenario step independent of the ones in our main
workflow, so we have encapsulated
them into a Scenario (This is
basically a compiled job). In some
cases we might need to wait for this
set of steps to complete before we
continue with the steps in our flow,
but in this case the two are able to
run independently, so we have
chosen to run them in parallel.
Date: 5/23/2008 Page #: 45

4. Expand the Let’s now add the interface that we

Interfaces node in created earlier into this workflow.
the current folder and Remember that this transformation
drag the load depends on an Excel spreadsheet to
MYW_AFL_CA_H be present. We can add a step to
Interface that we wait for this file to arrive before
created earlier into attempting to run the transformation
the Package diagram. that depends on it.
Add a step that waits

for the Excel file to be
present.
Do this by going to the
Event Detection
area in the Toolbox in
the ODI package and
selecting the
OdiFileWait tool.
Then click somewhere
on the Diagram where
you want to place this
command. You may
want to scroll down in
the General tab in the
panel at the bottom of
the diagram and
briefly talk about
options you could pass
in such as what
directory to look for
and polling interval.
Link up the steps

using the OK tool
(click the button
on the top of the
diagram) and
dragging from the
WaitForFile to the
Date: 5/23/2008 Page #: 46

5. Highlight the Error In a more complex workflow we

email step might choose to perform other
processing after a step fails or we
might retry the step that failed. In
this case we have chosen to use the
ODI tool that will send an email
letting someone know which session
failed. There are lots of other tools
You may want to scroll available to make it easy to develop
down in the General workflows. We can wait for data to
tab in the panel at the appear in a table, receive an email,
bottom of the diagram clear out the ODI log tables, etc. Of
and briefly talk about course we can also invoke any
options you could pass system commands or shell scripts
in such as what email that are needed in the workflow.
server, subject,
message,
attachments, etc.
Date: 5/23/2008 Page #: 47

Other Interface Examples
1. Go to the Projects ODI is fantastic for all kinds of data

tab in the tree, integrity checks, but it does not
expand the ODI directly include support for some
Demo project, expand data cleansing features. Name &
the folder named Address cleansing is a specialized
Demo and expand the task that we provide via
Interfaces node. partnerships.
Double-click the
address_cleansing For instance, here we have an
interface. example of loading a flat file into an
Oracle database using an IKM
Go to the Flow tab specifically designed to leverage
and highlight the Trillium Software’s data quality
target box. software.
2. Highlight the two While most interfaces are created

interfaces in the manually, many simple interfaces
Interfaces node: may be automatically generated by
load W_AFFL_CC_H ODI. load W_AFFL_CC_H and load
and load W_AFFL_CC_HS are examples of
W_AFFL_CC_HS. interfaces that were automatically
created by the tool. We will look at
the details of this process later when
we learn about the Common Format
Designer.
Date: 5/23/2008 Page #: 48

Section 3: Operator
Tracking your Jobs
Walking through the Operator
1. Click the Operator We saw the Operator briefly earlier.

button on top of the This is the tool that the operations
window to go to team can use to monitor all of the
operator. ODI jobs. Here we can see all of the
scenarios that have been developed
in this system. Scenarios are pre-
generated ODI jobs - built from
Packages, Interfaces, Procedures or
Variables - that area ready to be
Once there, go to the
scheduled. The operations team
Scenarios tab.
won’t use the Designer GUI, so it will
be right in this module that they can
schedule jobs to run automatically,
or manually choose to launch a job.
2. Click to the Sessions Here we can see all of the current

tab. Expand the All and past sessions that have run. For
Executions node. example, we can see that a previous
Find the example of package launched a child scenario
Main Flow with a and now both are running.
running icon ( ). For the sessions that have failed we
can see what step they failed on,
and we see the error message from
the database or operating system.
We have the option to restart a
session here. A key benefit is the
ability to restart the session from the
point of failure.
Date: 5/23/2008 Page #: 49

3. In the same session ODI can also automatically restart at

tab, find a failed the point a session failed. Note that
session and right-click in certain cases it may only make
it. Show the restart sense to rerun the entire session,
session option. but in other cases it’s a great
advantage to just restart the
stopped session.
Date: 5/23/2008 Page #: 50

Section 4: Metadata Navigator
Web Front End to Browse Metadata
Browse Metadata
1. Showing this is Metadata Navigator (MN) provides a

optional. Really for browser interface to see all of the
more technical metadata in our repository.
audience. Remember that this is looking
*** BE SURE TO directly at the repository--there's no
START OC4J ODI separate step involved like
FROM THE PROCESS generating a report, so there's no
CONTROL PANEL danger of seeing old information.
BEFORE YOU RUN We are basically looking at real-time
THIS DEMO *** reports on our metadata.
2. Go to Internet Explorer We log in to the application using

URL: the same credentials as the client
http://oracle2go.us.ora software. In fact, the security is
cle.com/oracledimn controlled from the same rules as
the application so you can specify
Log in using the what users are allowed to log into
dropdow: the system.
User: SUPERVISOR
Pass: SUNOPSIS
Date: 5/23/2008 Page #: 51

3. In the main panel, From this browser, you can drill

click ODI Demo. down to find information about any
Then, in the Folders packages that have been created.
tab, click Demo.
Then, in the
Packages tab, click
on Main Flow. Go to
the Package
Diagram tab.
4. Click on one of the From this package view, you can drill
Interface icons. Then, down to view information on the
click on the link next objects that make up the workflow.
to where it says For instance, here, we can drill down
Interface. The link to view more detailed information on
will be the name of this particular interface. We can see
the interface you information on all the mappings that
clicked in the previous were build to populate this particular
screen. Click on the table.
Mappings tab. This
will show all mappings
for this interface.
Date: 5/23/2008 Page #: 52

5. Expand Metadata in You can also get a quick snapshot of

the tree and click on how data objects feed each other.
Flow Map. For instance, here we want to see
how data flows between all of the
In the screen, select tables in our repository.
(All) for Models and
Projects. Set map
details to Datastores
and Projects. Click
View. This will give
you a picture of how
data flows between
tables.
Date: 5/23/2008 Page #: 53

6. Expand the From an operations stand point,

Execution node in users can track the execution of jobs
the tree. Click on All form this interface. Here we can see
Sessions. Find the the history of sessions. By clicking
session we ran earlier through, we can get more details on
(names load the individual jobs.
W_AFFL_CA_H). Click
down through the
details (click the
pictures). This will
take you through a
few screens. You
should be able to find
the same Insert flow
into I$ table step
from earlier. In there,
scroll down to find the
generated SQL.
Date: 5/23/2008 Page #: 54

7. Under Executions, Yet another use for the Metadata

click on Execute a Navigator is that of running or
Scenario. (Note, you restarting jobs. From here, a user
can show the Restart can specify what job to run, what
node as well, though agent to run it on, pass in any
it’s not really a runtime parameters, etc. In fact,
necessity). this can help quite a bit from a job
support standpoint. Jobs can be set
up to email in case of failure. So
image a case where a job fails
because a database was offline. An
email is sent to support at which
point, they log into the website and
pick the failed session from a list of
failed jobs in order to restart.
Date: 5/23/2008 Page #: 55

Section 5: Lightweight Designer
Web Front End to Modify Interfaces
*** LIGHTWEIGHT DESIGNER HAS NOT BEEN CONFIGURED ON THIS DEMONSTRATION IMAGE ***
Update Interfaces
1. Showing this is Lightweight Designer (LWD) was

optional. Really for created to provide more flexibility to
more technical customers that want to develop or
audience. review their transformation
processes via intranet.
Date: 5/23/2008 Page #: 56

Section 6: Common Format Designer
Automatically Manage Warehouse
Automatically Generate Objects
1. Common Format Designer (CFD) is a

set of ODI features that enables a
developer to automatically generate
DDL to create tables based on tables
that already exist in other schemas
and to automatically generate
interfaces to transport data into or
out of these tables.
Date: 5/23/2008 Page #: 57

2. Go back to the The three tables in this diagram are

Designer client. Go based on tables in a different model.
to the Models tab and Let’s add another table to show the
expand the Oracle - process.
Staging model.
Under the Diagrams
node open the Active
Hub diagram and go
to the Diagram tab.
Date: 5/23/2008 Page #: 58

3. Expand the MS SQL Next, we can create a foreign key to

Server - Sales DB define the relationship between this
model and drag the table and the rest of the warehouse.
SRC_ORDERS table
into the diagram. Now the table exists in the model,
Click OK in the but we need to generate the DDL to
message box that create this table and FK in the
appears. Change its database.
Name and Resource
name to
HUB_ORDERS. Go to
the Columns tab in
the new window and
add the column
SOURCE_SYSTEM
(VARCHAR2(35)) using
the add button:
Click OK.
Click the Add
Reference icon:
and create a reference

from HUB_ORDERS
to HUB_CUSTOMER:
Drag from
HUB_ORDERS to
HUB_CUSTOMER. In
the window that
appears, go to the
Columns tab. Hit the
add button:
Make sure that it says

CUST_ID in the left
column and CUSTID
in the right column.
Click OK.
Date: 5/23/2008 Page #: 59

4. Right-click the Oracle With a few clicks, ODI will detect the
- Staging model and difference between its version of the
choose Generate metadata and the metadata on the
DDL. In the Generate actual server and generate the DDL
DDL window, choose to propagate its version.
the ODI
Demo.Generated The DDL is generated as an ODI
Automatically Procedure. It’s possible now to view
Generation Folder: it and execute it.
Then check the

HUB_ORDERS table.
Click OK to generate
the DDL.
This will open a

procedure with the
DDL statements. Click
on the Details tab to
see the steps that
were generated. If
you’d like (though it’s
not necessary) you
can double click the
create table step to
show the actual code.
Click OK to close.
Date: 5/23/2008 Page #: 60

5. After the tables are created, we’ll

need to create Interfaces that load
data into them. Of course this can be
done manually, but to speed
development it’s possible to get a
head start by automatically
generating them.
Date: 5/23/2008 Page #: 61

6. Right-click the Oracle We can see a list of all candidate

- Staging model and Interfaces to create. Here we’ll only
choose Generate generate the Interface to load the
Interfaces IN. In the HUB_ORDERS table.
Generate Interfaces
IN window, choose the Notice that the new interface
ODI automatically has all columns that
Demo.Generated came from the source automatically
Automatically mapped. Now we can add mappings
Generation Folder: for any additional columns or modify
mappings as desired. Note that it will
even seamlessly handle more
complex cases when the target table
includes columns from multiple
sources: all source tables will
automatically appear as sources.
Additionally, we can generate
interfaces that load data back out to
the source if we want to have bi-
directional transfer of data. Another
Select only the common next step would be to
HUB_ORDERS implement Changed Data Capture
Datastore (to speed on the hub. This way we can easily
things up, click the replicate all new or modified data to
button to uncheck all any number of downstream
interfaces and simply applications.
check the
HUB_ORDERS
checkbox). Click OK to
generate the new
interface.
This will open a brand

new Interface. Click
on the Diagram tab.
Date: 5/23/2008 Page #: 62

Date: 5/23/2008 Page #: 63

Conclusion: Overall Questions
Potential Questions Prospects Might Ask.
To Do…
Date: 5/23/2008 Page #: 64

ODIDemo Script Final

Cargado por

Información del documento

Descripción original:

Título original

Derechos de autor

Formatos disponibles

Compartir este documento

Compartir o incrustar documentos

Opciones para compartir

¿Le pareció útil este documento?

¿Este contenido es inapropiado?

Copyright:

Formatos disponibles

ODIDemo Script Final

Cargado por

Copyright:

Formatos disponibles

Demo Script – Oracle Data

Integrator Product Walkthrough

Date Editor Version Change Reference

29-Jun-2007 Derrick Cameron 1.1D Revised to reflect BIC2g Demo VM

Oracle Data Integrator Product Walkthrough Script

Date: 5/23/2008 Page #: 1

Oracle Data Integrator Product Walkthrough Script

Date: 5/23/2008 Page #: 2

Oracle Data Integrator addresses multiple enterprise data integration needs.

ODI Runtime Environment Production Servers and

Oracle Data Integrator Product Walkthrough Script

Date: 5/23/2008 Page #: 3

In the walkthrough, we’ll examine:

Oracle Data Integrator Product Walkthrough Script

Date: 5/23/2008 Page #: 4

Oracle Data Integrator Product Walkthrough Script

Date: 5/23/2008 Page #: 5

1. Start up the VMWare image. You will see

Follow the instructions on the web page.

2. Go back to this screen (Vmware window)

3. You will come to the image desktop.

4. As login options, choose OracleDI Local

Date: 5/23/2008 Page #: 6

Date: 5/23/2008 Page #: 7

1. Show the Designer What you see on the screen is the

Oracle Data Integrator Product Walkthrough Script

Date: 5/23/2008 Page #: 8

Since this model will point to an Oracle

Oracle Data Integrator Product Walkthrough Script

Date: 5/23/2008 Page #: 9

3. Click on the Reverse We’ll then need to determine what context

4. Click on the Selective We can selectively choose which tables we

This will return all

Oracle Data Integrator Product Walkthrough Script

Date: 5/23/2008 Page #: 10

6. Click the Reverse By clicking the Reverse button, ODI starts

Oracle Data Integrator Product Walkthrough Script

Date: 5/23/2008 Page #: 11

7. The method just used for reverse

Oracle Data Integrator Product Walkthrough Script

Date: 5/23/2008 Page #: 12

1. Expand the Oracle- In ODI, it is possible to enhance the

Oracle Data Integrator Product Walkthrough Script

Date: 5/23/2008 Page #: 13

To do this, right click In this case, we’d like to add a foreign

Make sure both values in

Oracle Data Integrator Product Walkthrough Script

Date: 5/23/2008 Page #: 14

2. From the Control tab, Checking the data immediately is

* If you choose to check the data the second way, you

Oracle Data Integrator Product Walkthrough Script

Date: 5/23/2008 Page #: 15

3. Expand the Used to Some metadata is automatically

Oracle Data Integrator Product Walkthrough Script

Date: 5/23/2008 Page #: 16

Oracle Data Integrator Product Walkthrough Script

Date: 5/23/2008 Page #: 17

1. Now let’s jump ahead a bit. We have

Name the interface

Oracle Data Integrator Product Walkthrough Script

Date: 5/23/2008 Page #: 18

3. Click on the Diagram We now need to associate the