DB2 For z/OS: Utilities and Application Development

TRAINING & CONSULTING
DB2 for z/OS: Utilities and

Application Development
ABIS Training & Consulting
www.abis.be
training@abis.be
2005, 2006
Document number: 0092_03b.fm

11 January 2006
Address comments concerning the contents of this publication to:
ABIS Training & Consulting, P.O. Box 220, B-3000 Leuven, Belgium
Tel.: (+32)-16-245610, Fax: (+32)-16-245691
Copyright ABIS N.V.
TABLE OF CONTENTS
UTILITIES & APPLICATION DEVELOPMENT
1 ____ Introduction ____________________________________________________6

1.1
1.2
1.3
1.4
Utilities and application-related functionality

Utility processing versus SQL processing
Usability
Alternative: dropping indexes
6
7
8
9
2 ____ Supported functionality __________________________________________10

2.1
2.2
2.3
2.4
UNLOAD utility for READ activities

LOAD utility for WRITE activities
REORG utility for REMOVE activities
REORG utility for REWRITE activities
10
13
15
16
3 ____ Examples of simple scenarios_____________________________________17

3.1
3.2
3.3
Adding many rows

Copying many rows
Deleting many rows
17
19
22
4 ____ Examples of mixed scenarios _____________________________________24

4.1
4.2
4.3
4.4
Deleting many parent rows

Use of INCURSOR
INSERT after complex control
Utilities and remote access
DB2 for z/OS: Utilities and Application Development
24
26
29
33
iii
iv
Utilities & Application

Development
Objectives :
to introduce the main difference between SQL processing and the execution of utilities
to give an overview of utility functions that can replace SQL processing
to show with simple and more complex examples how and when you can
take advantage from this functionality
ABIS Training & Consulting
Introduction
Utilities and application-related functionality
1.1

Development
1. Introduction
2. Supported functionality
3. Examples of simple scenarios
4. Examples of mixed scenarios
UNLOAD:
for READ actions
LOAD:
for WRITE actions
REORG:
for REMOVE actions
REORG PAUSE or UNLOAD - (RE)LOAD:
for REWRITE or COMPLEX actions
ABIS
Utility processing versus SQL processing
1.2

Development
1. Introduction
SQL modifications:
a lot of overhead:
- logging
- locking
- on the fly:
index maintenance
referential integrity control (and/or actions: cascade/nullifies)
In addition, if large amount of modifications:
---> often, a REORGANISATION needed
For simple actions on 1 table:
---> UTILITIES are much cheaper
most actions directly on VSAM files
ABIS
Usability
1.3

Development
1. Introduction
In general:
- for simple actions on high volumes
stand-alone tables and activities
- if application activity is always (or often) combined with utility
processing
e.g. batch job followed by a REORG
- if asynchronous execution is allowed
e.g action is not a part of a larger transaction
ABIS
Alternative: dropping indexes
1.4

Development
1. Introduction
If SQL is needed for the modifications

more tables involved in the process
SQL functions needed
...
using utilities is no option any more
Alternative:
- DROP the INDEXES that cause overhead, before processing
indexes that are maintained in flight
and not needed for the process itself
- CREATE them again after processing
BE prepared for INVALID application plans and packages
- Ask a REBIND for these plans/packages:
check the related access paths (EXPLAIN)
ABIS
Supported functionality
UNLOAD utility for READ activities
2.1
Example of a Select of many rows

Development
1. Introduction
UNLOAD
TABLESPACE TBD7971.TBSSTMOD
FROM TABLE STOCKMODIFICATIONS
(SMID, SM_PRNO, SMTIMESTAMP, SMQUANTITY)
WHEN (SMTIMESTAMP BETWEEN 01.01.2004 AND 31.12.2004)
SHRLEVEL CHANGE ISOLATION UR
- one scan of the input data set

(no indexes are used)
- selected rows are written as records in an output file
- base table stays intact and available (see SHRLEVEL)
Remark:
UNLOAD utility = COPY of a formatted SUBSET
COPY utility = exact replication (backup)
ABIS
10
UNLOAD utility for READ activities (I)

Development
1. Introduction
Additional possibilities:
UNLOAD
FROMCOPYDDN DB2COPY1
SAMPLE 5 LIMIT 2500
(SMID,
SMTIMESTAMP TIMESTAMP EXTERNAL,
SMCOMMENT VARCHAR STRIP TRAILING)
WHEN (SMTIMESTAMP BETWEEN 01.01.2004 AND 31.12.2004)
PUNCHDDN DB2PUNCH
DELIMITED COLDEL ,
SHRLEVEL CHANGE ISOLATION UR
ABIS
11
UNLOAD utility for READ activities (II)

Development
1. Introduction
- FROMCOPYDDN
to use an IMAGE COPY as input
- PUNCHDDN
to create corresponding LOAD utility statements
- SAMPLE - LIMIT
to limit the number of output records
- TIMESTAMP EXTERNAL - VARCHAR STRIP TRAILING
to format output fields
- DELIMITED COLDEL ,
to format output records
ABIS
12
LOAD utility for WRITE activities
2.2

Development
1. Introduction
Example of an Insert of many rows

LOAD DATA
RESUME YES
SHRLEVEL NONE
INTO TABLE STOCKMODIFICATIONS
FORMAT UNLOAD or <FIELD-specifications>
- input records are added as rows to the table(space)

- new index entries are sorted
- they are merged with the existing entries
- corrections are done for:
duplicate keys
R.I. violations
ABIS
13
LOAD utility for WRITE activities (II)

Development
1. Introduction
Additional possibilities:
LOAD DATA
LOG NO NOCOPYPEND
ENFORCE NO
RESUME YES
SHRLEVEL CHANGE
FORMAT UNLOAD or <FIELD-specifications>
- LOG NO NOCOPYPEND - ENFORCE NO

less overhead - faster processing
only for controlled data
- SHRLEVEL CHANGE:
no solution for performance
(= mass insert with in flight maintenance of indexes)
ABIS
14
REORG utility for REMOVE activities
2.3

Development
1. Introduction
Example of a Delete of many rows

REORG TABLESPACE TBD7971.TBSSTMOD
LOG NO SORTKEYS COPYDDN(SYSCOPY)
STATISTICS TABLE ALL INDEX ALL
SHRLEVEL REFERENCE
DISCARD FROM TABLE STOCKMODIFICATIONS
WHEN (SMTIMESTAMP < CURRENT DATE - 3 YEAR)
- one scan of the input data set (tablespace)

(clustering index may be used)
- non-discarded rows are written as records in an output file
- they are rewritten in clustering sequence into the table(space)
- remaining index entries are sorted and used to rebuild indexes
ABIS
15
REORG utility for REWRITE activities
2.4

Development
1. Introduction
Example of an Update of many rows

UNLOAD PAUSE
Do while
READ from <unload-file>
modify record
WRITE to <reload-file>
End
RESTART REORG TABLESPACE ...
- unload-file has no indexes

- indexes are REBUILT after restart
- remarks:
constraints (check and R.I.) are not checked
there will be no CHKP-status
ABIS
16
Examples of simple scenarios

Adding many rows
3.1

Development
1. Introduction
Do while
READ input-record
...
INSERT INTO StockModifications
VALUES(:WS-StockModifications:WS-SMInd)
End
Problem:
- large input-files and 4 indexes
--> long execution time
- locking:
from exclusive table lock and low commit frequency to page locks
and high commit frequency:
--> more concurrency
--> +/- doubled elapsed time
ABIS
17
Adding many rows (II)

Development
1. Introduction
Alternative
LOAD DATA
RESUME YES
SHRLEVEL NONE
<FIELD-specifications>
- Plus:
less execution time (between 25 and 50%)
no programming effort
- Minus:
FIELD-specifications
SHRLEVEL NONE
(SHRLEVEL CHANGE = INSERT)
only KEY and R.I. control
no TRIGGER support
ABIS
18
Copying many rows
3.2

Development
1. Introduction
DECLARE CopyStM CURSOR FOR

SELECT *
FROM StockModifications
WHERE SMTimestamp = CURRENT DATE - 1 DAY
LOCK TABLE StockModifications IN SHARE MODE
OPEN CopyStM
Do while
FETCH CopyStM INTO :WS-StockModifications:WS-SMInd
...
WRITE uotput-record
end
Problem:
- time that the table is unavailable (exclusively locked) is too long
- lock is needed: concurrent updates can give a wrong result
ABIS
19
Copying many rows (II)

Development
1. Introduction
Alternative 1:
UNLOAD DATA
WHEN (SMTIMESTAMP = CURRENT DATE - 1 DAY)
SHRLEVEL REFERENCE
- Plus:
faster than SQL (typical value: 3 times faster)
format of output records is controllable
- Minus:
no complex conditions
ABIS
20
Copying many rows (III)

Development
1. Introduction
Alternative 2:
COPY
SHRLEVEL REFERENCE
UNLOAD
FROMCOPYDDN ...
WHEN (SMTIMESTAMP = CURRENT DATE - 1 DAY)
- Plus:
COPY +/- 2 times faster than UNLOAD (of alternative 1)
- Minus:
second job (UNLOAD)
(but without impact on the base table)
ABIS
21
Deleting many rows
3.3

Development
1. Introduction
DELETE
FROM StockModifications
WHERE SMTimestamp < CURRENT DATE - 2 DAYS
AND SMStatus = 7
Problem:
- +/- 50% of the rows are deleted
- table has 4 indexes
--> very long execution time (60 minutes)
- space management
--> REORG needed after execution
- remarks:
locking: exclusive table lock
--> low CPU-usage, many time-outs
R.I. is no problem (dependent table)
ABIS
22
Deleting many rows (II)

Development
1. Introduction
Alternative
LOG NO SORTKEYS COPYDDN(SYSCOPY)
STATISTICS TABLE ALL INDEX ALL
SHRLEVEL REFERENCE
DISCARD FROM TABLE STOCKMODIFICATIONS
WHEN (SMTIMESTAMP = CURRENT DATE - 2 DAYS
AND SMSTATUS = 7)
- Plus:
much faster than SQL (few minutes)
REORG and DISCARD: one job
- Minus:
no complex conditions
if parent table: CHKP for dependent tables
ABIS
23
Examples of mixed scenarios
Deleting many parent rows
4.1
Process:

Development
1. Introduction
- parent rows with a given status must be deleted (+/- 75%)

- delete rule is cascade
- on average 500 dependents for a parent row
SQL solution
DELETE
FROM LastOrders
WHERE LOStatus = OK
- Problem:
index maintenance on LastStockModifications (dependent table)
(index maintenance on LastOrders)
ABIS
24
Deleting many parent rows (II)

Development
1. Introduction
Alternative
UPDATE LastStockModifications
SET LSMStatus = OK
WHERE EXISTS (SELECT 1 FROM LastOrders
WHERE LOStatus = OK
AND LO_PrNo = LSM_PrNo)
efficient UPDATE process if no indexes on LSMStatus

REORG TABLESPACE TBD7971.TBSLSTSM
DISCARD FROM TABLE LASTSTOCKMODIFICATIONS
WHEN (LSMSTATUS = OK)
REORG TABLESPACE TBD7971.TBSLSTOR
DISCARD FROM TABLE LASTORDERS
WHEN (LOSTATUS = OK)
DISCARD of parent rows --> dependent table: CHKP
ABIS
25
Use of INCURSOR
4.2
Process:
- information must be copied from a source table to a target table

Development
1. Introduction
- availability of the target table is critical

- complex selection criteria for the source table
SQL solution
SELECT * FROM LastStockModifications T
WHERE NOT EXISTS (SELECT 1 FROM Products
WHERE PrNo = T.LSM_PrNo
AND PrStatus = 7)
- Problem:
index maintenance on StockModifications
ABIS
26
Use of INCURSOR (II)

Development
1. Introduction
Alternative 1:
EXEC SQL
DECLARE GETSTMOD CURSOR FOR
SELECT * FROM LASTSTOCKMODIFICATIONS T
WHERE NOT EXISTS (SELECT 1 FROM PRODUCTS
WHERE PRNO = T.LSM_PRNO
AND PRSTATUS = 7)
ORDER BY LSMTIMESTAMP
ENDEXEC
LOAD DATA
INCURSOR GETSTMOD
RESUME YES
SHRLEVEL NONE
ABIS
27
Use of INCURSOR (III)

Development
1. Introduction
- Alternative 2:
INSERT INTO TempStockModifications
SELECT * FROM LastStockModifications T
WHERE NOT EXISTS (SELECT 1 FROM Products
WHERE PrNo = T.LSM_PrNo
AND PRStatus = 7)
efficient SQL if no indexes on TempStockModifications

UNLOAD DATA
SHRLEVEL NONE
FROM TABLE TEMPSTOCKMODIFICATIONS
LOAD DATA
RESUME YES SHRLEVEL NONE
ABIS
28
INSERT after complex control
4.3

Development
1. Introduction
Process:
- uncontrolled information (but correct format) must be copied
from an input-file into a target table
- availability of the target table is critical
- complex control criteria for the input information
SQL solution
Do while
READ input-record
<control activities>
If OK
VALUES(:WS-StockModifications:WS-SMInd)
End
- Problem:
again index maintenance on StockModifications
ABIS
29
INSERT after complex control (II)

Development
1. Introduction
Alternative 1:
LOAD DATA
RESUME YES
SHRLEVEL NONE
<FIELD>-specifications
by default: SMStatus = NULL (i.e. not yet available)

--> additional VIEW (SMStatus IS NOT NULL)?
DECLARE NewStMod CURSOR FOR
SELECT * FROM StockModifications
WHERE SMStatus IS NULL
Do while
FETCH NewStMod INTO :WS-StockModifications:WS-SMInd
If NOT-OK
<correction>
End
ABIS
30
INSERT after complex control (III)

Development
1. Introduction
Alternative 2:
LOAD DATA
RESUME YES
SHRLEVEL NONE
INTO TABLE NEWSTOCKMODIFICATIONS
<FIELD>-specifications
no impact on the base table

DECLARE NewStMod CURSOR FOR
SELECT * FROM NewStockModifications
Do while
FETCH NewStMod INTO :WS-StockModifications:WS-SMInd
If NOT-OK
<correction>
End
no impact on the base table

ABIS
31
INSERT after complex control (IV)

Development
1. Introduction
- Alternative 2 (cont.):
EXEC SQL
DECLARE GETSTMOD CURSOR FOR
SELECT * FROM NEWSTOCKMODIFICATIONS
WHERE NSMSTATUS IS NOT NULL
ORDER BY NSMTIMESTAMP
ENDEXEC
LOAD DATA
INCURSOR GETSTMOD
RESUME YES
SHRLEVEL NONE
efficient index manipulation of StockModifications
ABIS
32
Utilities and remote access
4.4

Development
1. Introduction
Process:
- refresh of a replicated table inside DB2
- at runtime initiated by a remote Java application
Standard solution using SQL:
DELETE FROM LastStockModifications
INSERT INTO LastStockModifications
SELECT *
FROM STOCKMODIFICATIONS
WHERE SMTIMESTAMP = CURRENT DATE
Problem:
- in flight index maintenance
- how to use utilities as alternative?
ABIS
33
Stored Procedure: DSNUTILS

Development
1. Introduction
St Proc to execute utilities:

- allocates dynamically data sets
- creates the utility statements
- invokes DB2 utilities using program DSNUTILB
- inserts the result (SYSPRINT) into a temporary table
(SYSIBM.SYSPRINT)
- declares a cursor:
DECLARE Sysprint
CURSOR WITH RETURN FOR
SELECT seqno, text
FROM sysibm.sysprint
ORDER BY seqno
- OPENs the cursor and RETURNs
ABIS
34
Stored Procedure: DSNUTILS (II)

Development
1. Introduction
Calling DSNUTILS:
CALL DSNUTILS
(<utility-id>,
<restart-indicator>,
<utility-statements>,
<return-code>,
<ANY / <utility-name>,
<dataset-name1>,
.....)
Utility statements
UNLOAD DATA
WHEN (SMTIMESTAMP = CURRENT DATE)
LOAD DATA REPLACE
INTO TABLE LASTSTOCKMODIFICATIONS
ABIS
35
Stored Procedure: DSNUTILS (III)

Development
1. Introduction
Checking the execution

- SQLCODE (in calling program):
466: PROCEDURE ... RETURNED ... QUERY RESULTS SETS
- return-code (out parameter from St Proc)
utility RETURN CODE (0, 4, 8 ...)
- utility report:
can be fetched from SYSIBM.SYSPRINT
ABIS
36

DB2 For z/OS: Utilities and Application Development

Uploaded by

Copyright:

Available Formats

DB2 For z/OS: Utilities and Application Development

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

DB2 For z/OS: Utilities and Application Development

Uploaded by

Copyright:

Available Formats

What are the main DB2 utilities discussed in the document?

What are the main DB2 utilities discussed in the document?

What are some examples of when utilities can be used instead of SQL?

What are some examples of when utilities can be used instead of SQL?

TRAINING & CONSULTING

DB2 for z/OS: Utilities and

Document number: 0092_03b.fm

1 ____ Introduction ____________________________________________________6

Utilities and application-related functionality

2 ____ Supported functionality __________________________________________10

UNLOAD utility for READ activities

3 ____ Examples of simple scenarios_____________________________________17

Adding many rows

4 ____ Examples of mixed scenarios _____________________________________24

Deleting many parent rows

DB2 for z/OS: Utilities and Application Development

DB2 for z/OS: Utilities and Application Development

Utilities & Application

ABIS Training & Consulting

Utilities & Application

DB2 for z/OS: Utilities and Application Development

Utility processing versus SQL processing

Utilities & Application

DB2 for z/OS: Utilities and Application Development

Utilities & Application

DB2 for z/OS: Utilities and Application Development

Alternative: dropping indexes

Utilities & Application

If SQL is needed for the modifications

DB2 for z/OS: Utilities and Application Development

Example of a Select of many rows

Utilities & Application

- one scan of the input data set

UNLOAD utility for READ activities (I)

Utilities & Application

DB2 for z/OS: Utilities and Application Development

UNLOAD utility for READ activities (II)

Utilities & Application

DB2 for z/OS: Utilities and Application Development

LOAD utility for WRITE activities

Utilities & Application

Example of an Insert of many rows

- input records are added as rows to the table(space)

DB2 for z/OS: Utilities and Application Development

LOAD utility for WRITE activities (II)

Utilities & Application

- LOG NO NOCOPYPEND - ENFORCE NO

DB2 for z/OS: Utilities and Application Development

REORG utility for REMOVE activities

Utilities & Application

Example of a Delete of many rows

- one scan of the input data set (tablespace)

DB2 for z/OS: Utilities and Application Development

REORG utility for REWRITE activities

Utilities & Application

Example of an Update of many rows

- unload-file has no indexes

DB2 for z/OS: Utilities and Application Development

Examples of simple scenarios

1 Introduction ________________________________________________6

2 Supported functionality ______________________________________10

3 Examples of simple scenarios_________________________________17

4 Examples of mixed scenarios _________________________________24