Table Index Compression

An Insider's Look at the
Next Release of DB2 :

New Kernel Features
Presented by:
•Camp Hill – Dwaine Snow
•Dallas – Bill Minor
•Austin - Matt Huras
•Minneapolis – Paul Zikopolous
•Kansas City – Matt Huras
Presentation prepared by:

•Matt Huras - IBM
Agenda
• Compression Advancements
• Indexes
• Temp Data
• LOBs
• Storage Management
• Automatic Storage : Rebalance Improvements
• Automatic Storage : Easier Adoption
• Reclaimable Storage
• Table Maintenance
• Sparse MDC Tables
• Online Table Move
• Online (Inplace) Table Reorganization Improvements
• Table Partitioning
• Partitioned Indexes
• Currently Committed Isolation
• High Availability Disaster Recovery
• Active Standby
2
Disclaimer
• This presentation describes technology IBM may, or
may not, deliver in a future deliverable(s) of DB2 LUW
• Presentation of this material is not a commitment that

the technology will be delivered in the future
• This information is dynamic in nature due to the state

of development. It is possible that information may be
modified or added prior to presentation at IDUG.
3
Agenda
• Indexes
• Temp Data
• LOBs
• Online Table Reorganization Improvements
• Active Standby
4
Index Compression : Overview
• Three different compression techniques applies to index pages
1. Slot directory compression
2. “RIDlist” compression
3. Prefix key compression
• Index pages stored compressed on disk and in bufferpool

• As with table compression
• New INDEX compression attribute

• CREATE INDEX … COMPRESS YES
• ALTER INDEX … COMPRESS YES
• An INDEX will be compressed when created/reorg’d/loaded, if:

• The index compress attribute is YES, or,
• The index compress attribute is not set, and the table compression attribute is YES
5
Index Compression : Existing Page Format
Page Header
Fixed Slot Directory (maximum size reserved)
COLE, F, KIM 1055, 1056
COLE, F, KIRA 3011, 3025, 3026, 3027, 3029, 3033, 3035, 3036, 3037
DALY, M, AL 3009, 3012, 3013, 3015, 3016, 3017, 3109
DALY, M, ANTHONY 6008, 6009, 6010, 6011
An index key RIDlist (the IDs of the records in the table with this key)
6
Index Compression : Variable Slot Directory
Page Header
Var Slot Dir

Saved Space
COLE, F, KIM 1055, 1056
COLE, F, KIRA 3011, 3025, 3026, 3027, 3029, 3033, 3035, 3036, 3037
DALY, M, AL 3009, 3012, 3013, 3015, 3016, 3017, 3109
DALY, M, ANTHONY 6008, 6009, 6010, 6011
An index key RIDlist (the IDs of the records in the table with this key)
7
Index Compression : RIDlist Compression
Page Header
Var Slot Dir
COLE, F, KIM 1055, 1
COLE, F, KIRA 3011, 14, 1, 1, 2, 4, 2, 1, 1
DALY, M, AL 3009, 3, 1, 2, 1, 1, 92
DALY, M, ANTHONY 6008, 1, 1, 1
An index key Compressed RIDlist
8
Index Compression : Prefix Compression
Page Header
Var Slot Dir
COLE, F, KIM 1055, 1
COLE, F, KIRA 3011, 14, 1, 1, 2, 4, 2, 1, 1
DALY, M, AL 3009, 3, 1, 2, 1, 1, 92
DALY, M, ANTHONY 6008, 1, 1, 1
An index key Compressed RIDlist
9
Index Compression : Prefix Compression
Page Header
Var Slot Dir COLE, F, KI DALY, M, A 0, 2
M 1055, 1
RA 3011, 14, 1, 1, 2, 4, 2, 1, 1
L 3009, 3, 1, 2, 1, 1, 92
NTHONY 6008, 1, 1, 1
Compressed Compressed
Key RIDlist
10
Index Compression : Sample Data
Compression Ratio (%)

(higher is better)
TPC-DS
80 SAP BW (1st cust.)
70 SAP BW (2nd cust.)
60 SAP R/3
50
40
30
20 Index compression typically most
10 effective with
• Non-unique indexes (more
0 •
effective RID list compression)
Larger keys (more effective key
compression)
11
Index Compression : Monitoring
• How much space could I save by compressing the

indexes on table T1 ?
SELECT index_name, pages_saved_percent, compress_attr, index_compressed

FROM TABLE SYSPROC.ADMIN_GET_INDEX_COMPRESS_INFO
(‘T’, ‘myschema’, ‘T1’, ‘’, ‘’)) AS T
INDEX_NAME PERCENT_PAGES_SAVED COMPRESS_ATTR INDEX_COMPRESSED

------------------- -------------------------------------- --------------------------- -------------------------------
INDEX1 57 N N
12
Index Compression : Monitoring
• How much space did I save by compressing INDEX1?
ALTER INDEX index1 COMPRESS YES

REORG INDEXES ON TABLE t1
RUNSTATS ON TABLE t1
SELECT index_name, pages_saved_percent, compress_attr FROM SELECT
index_name, pages_saved_percent, compress_attr, index_compressed
FROM TABLE SYSPROC.ADMIN_GET_INDEX_COMPRESS_INFO
(‘T’, ‘myschema’, ‘T1’, ‘’, ‘’)) AS T
INDEX_NAME PERCENT_PAGES_SAVED COMPRESS_ATTR INDEX_COMPRESSED

------------------- -------------------------------------- --------------------------- -------------------------------
INDEX1 58 Y Y
Actual savings
13
Index Compression : Performance Attributes
• Fewer index levels

• Fewer logical and physical I/Os for key search (insert, delete,
select)
• Better bufferpool hit ratio
Relative Performance
(% Elapsed Time)
• Fewer index leaf pages
• Fewer logical and physical I/Os for index scans
(lower is better)
• Fewer splits 100
• Better bufferpool hit ratio
80
• Tradeoff 60
• Some additional CPU cycles needed for compress / 40
decompress
• 0-10% in early measurements 20
• Typically outweighed by reduction in I/O resulting in higher 0 INSERTS QUERY 1 QUERY 2
overall throughput
Uncompressed
Compressed 14
Temp Compression : Overview
• Compression techniques closely modeled after existing table

compression
• Dictionary based compression
• Temp tables will be compressed automatically, if DB2 installation

is licensed for data compression
• No need to turn on explicitly
• Applies to both “internal” and “external” temps
15
Temp Compression : Details
• Internal temp tables

• These are temp tables automatically created by DB2 in order to run
an access plan
• Examples:
• ‘System’ temps (often used for temporary results)
• Spilling sorts
• Eligible for compression if the optimizer estimates a large enough
• Data size (eg. 100 MB)
• Row size (eg. 20 bytes)
• Actually compressed at run-time when
• ‘System’ temps : after enough data has been inserted to create compression
dictionary, subsequent rows inserted into the temp will be compressed
• Spilling sorts : dictionary created on first spill; all rows will be compressed
16
Temp Compression : Details
• External temp tables

• These are temp tables explicitly created by a user or application
• Examples:
• Declared Global Temporary Table
• Created Global Temporary Table
• Use an algorithm very similar to automatic dictionary creation for
regular tables
1) CREATE
2) INSERT..
Internal threshold designed as Note that, unlike regular

best compromise b/w tables, the compression
compression ratio, and dictionary is not stored in the
dictionary build speed. table (it is only stored in
Default ~1 MB memory for temp tables, since
it does’nt need to be
persisted).
17
Temp Compression : Sample Data (TPC-DS)
Temp Space (GB) Elapsed Time (min)

(lower is better) (lower is better)
100 56%
185
reduction
5% better
Not Compressed
180
50 Compessed
175
0 170
18
LOB Compression
• Enabled through LOB “Inlining”

• Normally LOBs are not stored in the base row
• They are stored in a separate storage object
• An “inlined” LOB is one that is stored directly in the base row
CREATE TABLE (NAME CHAR(20), AGE INT, PICTURE BLOB(10MB), SERVICE INT)
Non-inlined LOB Case Inlined LOB Case
Actual LOB John 15 Actual

LOB
Descriptor Data (eg. LOB Data
JPG) (eg. JPG) 5
John 15 xyz 5
Row Row
Data Page in Segment in Data Page in

Table Object LOB Object Table Object
19
LOB Inlining : Advantages
• Compression
• Inlined LOBs are in the base data row, and therefore can be compressed with
DB2’s row compression
• Performance
• Inlined LOBs require do not require an extra I/O to be accessed
• Usage Recommendations
• Consider inlining LOBs that have one or more of the following characteristics:
• Frequently accessed
• Often small enough to fit entirely on a data page
• Compressible (i.e. not random data, not already compressed)
• Keep in mind:
• Potential growth in table size
20
LOB Inlining : Details
• Specify maximum INLINE LENGTH on CREATE or ALTER TABLE
• CREATE TABLE … PICTURE BLOB(10MB) INLINE LENGTH 10000
• ALTER TABLE … ALTER COLUMN PICTURE SET INLINE LENGTH 10000
• LOBs <= than inline length will be stored directly in the row in the data page
• DB2 will search for a page with sufficient space
• There can be a mixture of inlined and non-inlined LOBs within a single page and within a
single row
• INLINE LENGTH limits
Page size Row size limit INLINE LENGTH limit
4k 4005 4001
8k 8101 8097
16k 16293 16289
32k 32677 32673
• Implicit Inlining
• LOBs can be inlined even if there is no INLINE LENGTH set !
• Occurs when a LOB’s actual length is <= it’s maximum descriptor size based on the declared
maximum length of the LOB (see speaker notes)
• Otherwise space would be consumed both in data page (for descriptor) and LOB object (actual LOB)
21
Implicit Inlining
Non-inlined LOB in 9.5 Implicitly Inlined LOB
LOB
Descriptor
LOB
John 15 xyz 5 John 15 xyz 5

Row Row
Data Page in Segment in Data Page in

Table Object LOB Object Table Object
22
LOB Inlining : Example Savings
Total physical storage (KBs)
70000
60000
50000
40000
30000
20000
10000
0
No Compression Data Compression LOBs Inlined LOBs Inlined plus Data
Compression
No Compression Data No Compression Compression

Compression LOBs Inlined plus Inlined
LOBs
Base table (KBs) 24320 5248 22144 4736 All LOBs in this table
LOB storage 30336 30336 128 128 are smaller than the
(KBs) default inline length for
Indexes (KBs) 10880 10752 10880 10752 the LOB columns and
Total (KBs) 65536 46336 33152 15616
therefore all are inlined
23
Agenda
• Indexes
• Temp Data
• LOBs
• Active Standby
24
Automatic Storage Rebalance : Motivation
• Let’s review what occurs when paths are added to your automatic
storage pool,…
Two storage paths New storage paths p1 and p2 become full DB2 automatically
and a table space has not used by the creates a new stripe
a container on each table space set on p3 and p4
immediately
p1 p2 p1 p2 p3 p4 p1 p2 p3 p4 p1 p2 p3 p4
C0 C1 C0 C1
ALTER DB TS TS
ADD p3, p4 grows C0 C1 grows C0 C1
C2 C3
• Works fine, and is automatic,… but,…

• What if you wanted to stripe your data across all 4 devices immediately
(perhaps to gain more I/O parallelism) ?
25
Automatic Storage Rebalance : REBALANCE
ALTER TABLESPACE myts REBALANCE
High Water Mark
Two storage paths New storage paths REBALANCE causes DB2 If table space is not growing
and a table space has not used by the to create equal-sized rapidly, consider REDUCing
a container on each table space containers in new paths, and it to make space available
immediately redistribute extents to them for other table spaces
p1 p2 p1 p2 p3 p4 p1 p2 p3 p4 p1 p2 p3 p4
C0 C1 C2 C3
C0 C1 C0 C1 C0 C1 C2 C3
ALTER DATABASE … ALTER TABLESPACE ... ALTER TABLESPACE ...

ADD p3, p4 REBALANCE REDUCE
(optional)
High Water Mark
26
Dropping Storage Paths
ALTER DATABASE DROP STORAGE PATH ON ‘path1’,…
• If storage path is “Not In Use” then it is immediately removed from

database
• If storage path is “In Use” then:
• Storage path state is changed to “Drop Pending”

• Affected table spaces are marked as “An
Underlying Storage Path(s) has been Dropped”
• A “Drop Pending” storage path is removed when no automatic

storage table space is using it:
• Requires REBALANCE 27
Dropping Storage Paths : Example
p1 p2 p3 p1 p2 p3 p1 p3
C1 C2 C3 C1 C2 C3
C1 C3
ALTER DATABASE … ALTER TABLESPACE ..

DROP p2 REBALANCE
Drop Pending
28
Monitoring
New database snapshot field
Automatic storage path = /storagePath1
Node number = 0
State = In Use | Not In Use | Drop Pending
File system ID = 64768
Storage path free space (bytes) = 66725142528
File system used space (bytes) = 50100731904
File system total space (bytes) = 317068410880
New tablespace snapshot field

Tablespace name = USERSPACE1
Tablespace ID = 2
...
Rebalancer Mode = No Rebalancing
Underlying storage path has been dropped = No | Yes
Minimum Recovery Time =
...
SQL to find tablespaces using dropped paths

SELECT DISTINCT(A.TBSP_NAME)
FROM SYSIBMADM.SNAPTBSP A, SYSIBMADM.SNAPTBSP_PART B
WHERE A.TBSP_ID = B.TBSP_ID AND
A.TBSP_CONTENT_TYPE IN ('ANY','LARGE') AND
B.PATH_DROPPED = 1
29
Agenda
• Indexes
• Temp Data
• LOBs
• Active Standby
30
Adopting Automatic Storage : Motivation
• DB2 automatic storage …
• Collapses storage management tasks for any number of tablespaces down to
a single point of management
• Significantly reduces human costs
• Is IBM’s strategic direction going forward for DB2 LUW
• However, adopting the technology was not simple

• Required database to be created as an automatic storage database
CREATE DATABASE mydb CREATE DATABASE mydb AUTOMATIC

STORAGE YES on D:, E:, F:
CONNECT to mydb
CONNECT to mydb
CREATE TABLESPACE myts
MANAGED BY AUTOMATIC STORAGE CREATE TABLESPACE myts
> Error ! MANAGED BY AUTOMATIC STORAGE
> Success
31
Adopting Automatic Storage : Made Simpler
1. Can convert non-AS database to AS

• ALTER DATABASE ADD STORAGE now supported on a non-AS database
• Creates a storage pool for the database
• Allows creation of new automatic storage tablespaces
• Also allows existing DMS tablespaces to be converted to automatic storage (next bullet)
2. ALTER TABLESPACE enhanced to convert a DMS tablespace to an

automatic storage tablespace
• ALTER TABLESPACE … MANAGED BY AUTOMATIC STORAGE
• Causes all new growth to come from the automatic storage paths
• Old (file or raw) containers can be removed via the DROP and REBALANCE enhancements
previously covered
3. REDIRECTED RESTORE enhanced to allow restoration from DMS

table spaces to AUTOMATIC STORAGE table spaces
• SET TABLESPACE CONTAINERS … USING AUTOMATIC STORAGE
32
Agenda
• Indexes
• Temp Data
• LOBs
• Active Standby
33
Reclaimable Storage : Motivation
• CREATE DATABASE … ON /p1, /p2 /p1 /p2
• CREATE TABLESPACE ts1
• CREATE TABLE t1 IN ts1 …

• LOAD t1 …

• LOAD t2 …

• LOAD t3 …
• DROP TABLE t1
Trapped Free
• LOAD t3 …
Space
34
Reclaimable Storage : Overview
ALTER TABLESPACE <tsname> REDUCE --+-----------------------------+---
'-- <size> --+-------------+--+--'
| +----- K -----+ |
| +----- M -----+ |
| +----- G -----+ |
| '-- PERCENT --' |
'--- MAX ---------------------'
'--- STOP --------------------'
• ALTER TABLESPACE .. REDUCE enhanced to free “trapped free

space”
• Works by:
• Moving used extents from higher addresses in the tablespace to unused lower addresses
• Lowering high water mark accordingly
• Shrinking/removing containers to return space back to the Automatic Storage paths
• MAX frees as much space as possible

• Can also specify a target size to free up
• Runs in the background

• Works in batches, committing free extents as it progresses 35
• STOP option terminates a background REDUCE operation
Reclaimable Storage : Benefit
• CREATE DATABASE … ON /p1, /p2 /p1 /p2
• LOAD t1 …
• LOAD t2 …

• LOAD t3 …
• DROP TABLE t1
• ALTER TABLESPACE ts2 REDUCE MAX
• LOAD t3 … Trapped Free

Space
36
Reclaimable Storage : Examples
ALTER
DROP TABLE 2 TABLESPACE …
DROP TABLE 3 REDUCE MAX
Internal tablespace metadata extents

Table 1
Table 2
Table 3
Extent that is allocated to tablespace, but not to a table
37
Reclaimable Storage : Examples
ALTER
DROP TABLE 2 TABLESPACE …
DROP TABLE 3 REDUCE nn
Internal tablespace metadata extents

Table 1
Table 2
Table 3
Extent that is allocated to tablespace, but not to a table
38
Reclaimable Storage : Monitoring
• Tablespace snapshot
Tablespace Type = Database managed space
Using automatic storage = Yes
Using reclaimable storage = Yes
...
Tablespace State = 0x'00000010‘
Detailed explanation: Extent movement in progress
• New MON_GET_EXTENT_MOVEMENT_STATUS()
TBSP_NAME TBSP_ID .... NUM_EXTENTS_MOVED NUM_EXTENTS_LEFT TOTAL_MOVE_TIME
--------- ------- ----------------- ---------------- ---------------
USERSPACE1 2 -1 -1 -1
TS1 3 4000 2000 60000
• Very rough estimate for remaining time as (LEFT/MOVED) * TIME

• 30000 milli-seconds (30 secs) in this example for TS1
39
Agenda
• Indexes
• Temp Data
• LOBs
• Active Standby
40
Sparse MDC Tables : Motivation
DELETE FROM Sales

WHERE Region = SW
Region AND Year = 2004 Region
NW,2004 SW,2004 SW,2005 NW,2004 SW,2005
Year Year
Table Object
• These pages and storage are still assigned to the MDC table
• How can this storage be reused elsewhere in the table space?
• If you have a maintenance window : Classic REORG
• If not,…. ?
41
Sparse MDC Tables
REORG TABLE Sales
RECLAIM EXTENTS ONLY
Region Region
NW,2004 SW,2005 NW,2004 SW,2005
Extent is freed
Year Year back to table
space & can be
Table Object used by other
tables !
What’s Going at the Table Space Level ?
REORG TABLE Sales

RECLAIM EXTENTS ONLY
42
Sparse MDC Tables
REORG TABLE <mdc table name> RECLAIM EXTENTS ONLY

[ ALLOW { WRITE | READ | NO } ACCESS ]
• Very fast !
• Not really a reorg
• No copy of the table created, no copy phase, etc
• Done in-place with no data movement, minimal

logging
• Find the empty blocks in block map
• Mark them as unallocated in the MDC table’s block map
• Mark them as unallocated in the table space SMPs
43
Sparse MDC Tables
• When should you run RECLAIM EXTENTS ONLY ?

• Use new RECLAIMABLE_SPACE column
returned from ADMIN_GET_TAB_INFO()
function
Get reclaimable space for a specific table
SELECT reclaimable_space as R FROM TABLE
SYSPROC.ADMIN_GET_TAB_INFO_V97 ( ‘db2user1’, ‘emp’)) AS T
Get tables with over 10 MB of reclaimable space

SELECT tabschema, tabname, reclaimable_space
FROM sysibmadm.admintabinfo
WHERE reclaimable_space > 10,000,000
GROUP BY tabschema, tabname
44
Online Table Move : Motivation
• Initial definition
CONNECT to mydb
CREATE TABLESPACE myts … PAGESIZE 4K
CREATE TABLE mytable IN myts (C1 CHAR(8), C2 CHAR(8), C3 CHAR(8),…

C500 CHAR(8))
• Six months later you need to add a column …

ALTER TABLE mytable ADD (C501 CHAR(8))
> Error !
• Now what ?
• If you have a large enough maintenance window
• UNLOAD, re CREATE, re LOAD
• But what if you have no maintenance windows ?
45
Online Table Move : Data Flow
Online table move control table
1 INIT PHASE SYSTOOLS.ADMIN_MOVE_TABLE
Create triggers,target, tabschema tabname key value
staging tables
SOURCE TARGET
TABLE 2 TABLE
COPY PHASE
c1 c2 … cn c1 c2 … cn
3
REPLAY PHASE
INSERT INSERT
c1 c2 … cn
Rows with
DELETE DELETE keys present
UPDATE UPDATE in staging
table are
re-copied
Online Keys of
from source
Workload row changed
table
by online
workoad 4 SWAP PHASE
captured via
triggers STAGING Rename Target -> Source
TABLE
46
Online Table Move : Example Use Cases
• Online REORG or Redistribute
• Online table compression
• Online change pagesize
• Online conversion to LARGE tablespaces
• Move data/index/long data to new/different tablespaces
• Support for schema change:

• Add or remove columns, change column
datatypes, enlarge columns
• Add/change MDC dimensions, range partitioning
or partitioning key
47
Online Table Reorg : Review
• Space-reclaiming inplace TIME
reorg does simultaneous

backward and forward Move rows from end of table, filling up holes at
the start
table scans
• Moves rows from end of
table to ‘holes’ found
starting at beginning of
table
• When the scans meet, a
table S lock is taken and
one final pass is taken to
pick up any newly inserted
rows at the end of the
table, followed by a
truncate
48
Online Table Reorg Concurrency : Motivation
Online Drain Move rows to re-establish clustering … if Truncate requested …

Reorg Existing and/or reclaim free space Wait Drain Trun- S
Issued (Wait for for S Existing cate Lock
existing
accesses to lock Rel-
end, while eased
letting new
accesses
start)
No new write
Time period dependent on … accesses
- Duration of existing write transactions during this
- Time taken for final row movement phase and truncate phase
period
49
Online Table Reorg Concurrency :
Improvements
• Final pass of row movement made significantly more efficient

• Up to twice as fast
• REORG … PAUSE enhanced when invoked during truncate

phase
• Truncates pages it can – i.e. the pages freed up ‘so far’ in final pass
• Releases S lock
• REORG … RESUME will re-acquire S lock and continue final pass
• REORG … STOP will terminate reorg
• Usage recommendations:
• If you find the S lock is holding up your workload, use REORG… PAUSE to
reclaim currently available space, and resume full table access
• At a later time, check current table size via ADMIN_GET_TAB_INFO()
• If satisfied with space reclaimed:
• REORG … STOP
• Else
• REORG … RESUME
50
Agenda
• Indexes
• Temp Data
• LOBs
• Active Standby
51
Table Partitioning Improvements : Motivation
SQL
Automatic SET INTEGRITY
Asynchronous
Index Cleanup COMMIT
Global Index (order_id)
CREATE
TABLE
JUNE
Jan Feb Mar Apr May June
ALTER TABLE ALTER TABLE

LOAD
DETACH ATTACH
PARTITION JUNE
JAN
52 © 2009 IBM Corporation
Key Improvement : Partitioned Indexes
tbsp4 tbsp4 tbsp5 tbsp6
i1 i1
tbsp1 tbsp2 tbsp3 tbsp1 tbsp2 tbsp3
t1.p1 t1.p2 t1.p3 t1.p1 t1.p2 t1.p3
CREATE TABLE t1(c1 INT) CREATE TABLE t1(c1 INT)

INDEX IN tbsp4 PARTITION BY RANGE(a)
PARTITION BY RANGE(a) (STARTING FROM (1) ENDING(34) IN tbsp1
(STARTING FROM (1) ENDING(34) IN tbsp1, INDEX IN tbsp4,
ENDING(67) IN tbsp2, ENDING(67) IN tbsp2
ENDING(100) IN tbsp3) INDEX IN tbsp5,
ENDING(100) IN tbsp3
CREATE INDEX i1(c1) INDEX IN tbsp6)
CREATE INDEX i1(c1) PARTITIONED
53
Partitioned Indexes : Benefits
• Storage
• Partitioned indexes do not store partition identifier in each key
• 2 byte savings per RID
• Each partitioned index will be a fraction of the size of an alternative non-partitiioned index
• In some cases, may be able to use REGULAR (small RID) tablespaces for indexes
• Another 2 byte savings per RID
• Performance
• Storage savings typically leads to performance savings
• Less I/O bandwidth and memory consumption
• However, be aware that partitioned indexes will (in general) not provide ‘order’, eg:
SELECT * … ORDER BY partitioned_index_key // can’t use partitioned index for order; may need sort
// (unless of course part. index key is prefixed by table part key)
SELECT * … WHERE table_part_key=X … // can use partitioned index instead of sort

ORDER BY partitioned_index_key
•
• Several advantages in Utility and DDL processing
54
Partitioned Indexes : Utilities / DDL
• Partition-level index REORG
REORG INDEXES ALL FOR TABLE xyz … ON DATA PARTITION jan … ALLOW WRITE ACCESS
• Reorganizes indexes on specified partition only

• All partitions (including target partition) available for read/write access
• Partition-level table REORG

REORG TABLE xyz … ON DATA PARTITION jan … ALLOW READ ACCESS
• Reorganizes specified partition and it’s partitioned indexes
• In the above example, target partition available for read access
• All other partitions available for read/write access
• Important note: if non-partititoned indexes also exist, they will also be reorganized (which will
restrict availability of all partitions)
• Statistics
• New SYSINDEXPARTITIONS catalog table added to contain per-index-partition statistics
• Also, new statistics added to SYSDATAPARTITIONS to help determine when/if partitioning level
table reorg would be useful
• RUNSTATS continues to operate on table level
55
Partitioned Indexes : Utilities / DDL (cont’d)
• ATTACH
• If required partitioned indexes have already been created on the table about to be attached
• They are just quickly “linked in” as the index partition for this new data partition
• SET INTEGRITY will not perform index maintenance operations for such indexes (eliminates
associated logging) !
• Otherwise, they will be created automatically
• Hint: use the REQUIRE MATCHING INDEXES to ensure automatic creation does not occur
ALTER TABLE … ATTACH PARTITION…. REQUIRE MATCHING INDEXES
• DETACH
• Partitioned indexes for the detached partition quickly “unlinked” from the range partitioned
table, and become regular indexes on the detached partition (now just a regular table)
• No Async Index Cleanup necessary for Partitioned indexes
56
Partitioned Indexes : Streamlined Roll-in
CREATE TABLE
JUNE
CREATE INDEX
Part’d Part’d Part’d Part’d Part’d

Index Index Index Index Index
Feb Mar Apr May June
ALTER TABLE
LOAD
ATTACH
JUNE
No Global Index to Maintain SET INTEGRITY

Partitioned Indexes : Streamlined Roll-out
SQL
Z lock removed
Part’d Part’d Part’d Part’d Part’d

Index Index Index Index Index
Jan Feb Mar Apr May
ALTER TABLE
DETACH
PARTITION
JAN
Agenda
• Indexes
• Temp Data
• LOBs
• Active Standby
59
Currently Committed Isolation : Motivation
SELECT * FROM EMP
EMPID NAME OFFICE SALARY
6354 Smith A1/21 43

EMP
> wait rowid empid name office salary
48 6354 Smith A1/21 43
77 4245 Chan Y2/11 11
96 7836 Jones AA/00 C3/46 21
104 1325 Tata X1/03 33
205 5456 Baum D2/18 22
Uncommitted insert
Uncommitted update
Uncommitted delete
60
Currently Committed Isolation : Output
SELECT * FROM EMP DB2 returns currently

committed data without
waiting for locks !
(Delete and Update undone; Insert skipped.)
EMPID NAME OFFICE SALARY
6354 Smith A1/21 43

7836 Jones AA/00 21 EMP
1325 Tata X1/03 33 rowid empid name office salary
5456 Baum D2/18 22 48 6354 Smith A1/21 43
77 4245 Chan Y2/11 11

> SUCCESS
96 7836 Jones AA/00 C3/46 21
104 1325 Tata X1/03 33
205 5456 Baum D2/18 22
Uncommitted insert
Uncommitted update
Uncommitted delete
61
Currently Committed Isolation : How it Works
Locklist Log Buffer

EMP
rowid lock log ref UPD: Emp,3,7836,Chan,AA/00
C3/46
rowid empid name office salary 77 X(I) -
48 6354 Smith A1/21 43 96 X(U)
Active Log Files
77 4245 Chan Y2/11 11 205 X(D)
96 7836 Jones AA/00 C3/46 21 DEL: Emp,5,5456,Baum,D2/18
INS: Emp,1,4245,Chan,Y2/11,11
104 1325 Tata X1/03 33
205 5456 Baum D2/18 22
Log Archive
INS:Emp,1,6354,Smith,A1/21, 43
INS:Emp,4,1325,Tata,X1/03,33
Uncommitted INSERTed data is skipped.
For uncommitted DELETEs and UPDATEs, when encountering a lock which would
otherwise conflict, DB2 uses new information in the lock manager to reconstruct and
return the previously committed data from the log buffer or log file.
62
Currently Committed : Internals & Usage Notes
• Log-based implementation : simple & fast
• No need for rollback segments !
• Currently committed data typically reconstructed from memory (log buffer)
• Exception: updates/deletes from mass update transactions that spill log buffer
• Active logs read from storage in this case
• Fallback to traditional locking

• If the currently committed data is unavailable (or not available quickly), DB2 will fall back to the
traditional locking behavior
• Examples
• Currently committed data is only available from an archived log (as may be the case with infinite logging)
• Updater held table lock (not row lock)
• Usage hints & tips

• Consider increasing your log buffer size
• Options for evaluating log buffer size
• New Cur Commit Total Log Reads, Cur Commit Disk Log Reads outputs from db2pd -logs
• Existing num_log_data_found_in_buffer, num_log_read_io monitor elements
• Consider increasing lock list size (or using AUTOMATIC setting)
• To avoid escalation to table locks (disables currently committed behavior for the table)
• Be aware of potential for increase in log space consumption if CC enabled
• First update to a given row in a transaction logs entire row image
63
Currently Committed Isolation : Externals
CUR_COMMIT database configuration parameter
ON : cursor stability scans return currently committed data (default for new dbs)
AVAILABLE : cursor stability scans return currently committed data if explicitly requested
DISABLED : disabled (default for upgraded databases)
BIND option
>--+-------------------------------------------------------------+-->
'--CONCURRENTACCESSRESOLUTION--+--USE CURRENTLY COMMITTED--+--'
'--WAIT FOR OUTCOME---------‘
CLI, JCC
SQL_ATTR_CONCURRENT_ACCESS_RESOLOUTION = 1
concurrentAccessResolution property
Notes
• CC does not apply to target table of write, catalogs, RCT, “integrity scans”
• DB2 needs to hold the right to update the row in this case
• Target table of write still exhibit “skip uncommitted inserts” aspect of CC
• CC extends “skip uncommitted inserts” to Read Stability isolation as well
64
Currently Committed Isolation : Example
APPLICATION A APPLICATION B
UPDATE T1 SET COL1=? WHERE COL2=? UPDATE T2 SET COL1=? WHERE COL2=?
SELECT COL1, COL3 FROM T2 SELECT COL1, COL5 FROM T1

WHERE COL2 > ? WHERE COL5=? AND COL2=?
• Typical Result without Currently Committed : ?

• Deadlock possible : dependent on
• Timing of execution of each application
• Values being selected at any point in time
• The number of rows affected (selecttivity) of selects and updates
• The access plan chosen (eg. Index vs tasble scan)
• Result without Currently Committed : Success

• Deadlock no longer possible
65
Currently Committed Isolation : Semantics
Isolation Dirty Reads Non- Phantoms ?

? repeatable
Reads ?
Can a query return Can rows be Can additional rows
uncommitted data ? dropped from the be included in the
result set if a query result set if a query is
is run a 2nd time in a run a 2nd time in a
transaction ? transaction ?
Uncommitted Read Y Y Y
Cursor Stability N Y Y
with CC N Y Y
Read Stability N N Y
with CC N N Y
Repeatable Read N N N
66
Agenda
• Indexes
• Temp Data
• LOBs
• Active Standby
67
HADR Reads on Standby : Motivation
Read enabled Standby in
OLTP Clients perpetual Rollforward
Clients Primary Database Logs Standby
HADR Communication
Clients
Reporting/DSS/BI
workloads
Resource Utilization
• Standby provides High Availability, Disaster Recovery & allows read-only workloads
• Improve resource utilization
• Decrease total cost of ownership
68
HADR Reads on Standby : Overview
• Standby can be opened for read access

• Enabled via : DB2_HADR_ROS = ON
• Supports all synchronization modes

• ASYNC
• NEAR SYNC
• FULL SYNC
• Isolation level is Uncommitted Read

• Can force applications to use UR : DB2_STANDBY_ISO = UR
• If this registry variable is not set, an application requesting an isolation level other
than UR will fail with SQL1773N
• Writers blocked
• Error SQL1773N
69
Matt Huras
IBM
huras@ca.ibm.com
dwsnow@us.ibm.com
paulz@ca.ibm.com
bminor@ca.ibm.com

Table Index Compression

Cargado por

Información del documento

Descripción original:

Derechos de autor

Formatos disponibles

Compartir este documento

Compartir o incrustar documentos

Opciones para compartir

¿Le pareció útil este documento?

¿Este contenido es inapropiado?

Copyright:

Formatos disponibles

Table Index Compression

Cargado por

Copyright:

Formatos disponibles

An Insider's Look at the

Next Release of DB2 :

Presentation prepared by:

• Presentation of this material is not a commitment that

• This information is dynamic in nature due to the state

• Index pages stored compressed on disk and in bufferpool

• New INDEX compression attribute

• An INDEX will be compressed when created/reorg’d/loaded, if:

Fixed Slot Directory (maximum size reserved)

COLE, F, KIM 1055, 1056

DALY, M, AL 3009, 3012, 3013, 3015, 3016, 3017, 3109

DALY, M, ANTHONY 6008, 6009, 6010, 6011

Var Slot Dir

COLE, F, KIM 1055, 1056

DALY, M, AL 3009, 3012, 3013, 3015, 3016, 3017, 3109

DALY, M, ANTHONY 6008, 6009, 6010, 6011

Var Slot Dir

COLE, F, KIM 1055, 1

COLE, F, KIRA 3011, 14, 1, 1, 2, 4, 2, 1, 1

DALY, M, ANTHONY 6008, 1, 1, 1

An index key Compressed RIDlist

Var Slot Dir

COLE, F, KIM 1055, 1

COLE, F, KIRA 3011, 14, 1, 1, 2, 4, 2, 1, 1

DALY, M, ANTHONY 6008, 1, 1, 1

An index key Compressed RIDlist

Var Slot Dir COLE, F, KI DALY, M, A 0, 2

Compression Ratio (%)

• How much space could I save by compressing the

SELECT index_name, pages_saved_percent, compress_attr, index_compressed

INDEX_NAME PERCENT_PAGES_SAVED COMPRESS_ATTR INDEX_COMPRESSED

• How much space did I save by compressing INDEX1?

ALTER INDEX index1 COMPRESS YES

INDEX_NAME PERCENT_PAGES_SAVED COMPRESS_ATTR INDEX_COMPRESSED

• Fewer index levels

• Compression techniques closely modeled after existing table

• Temp tables will be compressed automatically, if DB2 installation

• Applies to both “internal” and “external” temps

• Internal temp tables

• External temp tables

Internal threshold designed as Note that, unlike regular

Temp Space (GB) Elapsed Time (min)

• Enabled through LOB “Inlining”

Non-inlined LOB Case Inlined LOB Case

Actual LOB John 15 Actual

Data Page in Segment in Data Page in

Non-inlined LOB in 9.5 Implicitly Inlined LOB

John 15 xyz 5 John 15 xyz 5

Data Page in Segment in Data Page in

No Compression Data No Compression Compression

• Works fine, and is automatic,… but,…

ALTER TABLESPACE myts REBALANCE

High Water Mark

ALTER DATABASE … ALTER TABLESPACE ... ALTER TABLESPACE ...

High Water Mark

ALTER DATABASE DROP STORAGE PATH ON ‘path1’,…

• If storage path is “Not In Use” then it is immediately removed from

• If storage path is “In Use” then:

• Storage path state is changed to “Drop Pending”

• A “Drop Pending” storage path is removed when no automatic

ALTER DATABASE … ALTER TABLESPACE ..

New tablespace snapshot field

SQL to find tablespaces using dropped paths