CDA Release Notes
CDA Release Notes
5
Release Notes and Administration Manual
Contents
Introduction ..................................................................................................................................... 2
Previous Enhancements and Fixes............................................................................................... 3
Enhancements and Fixes in this Version (R2003.3.1.5).............................................................. 6
Installation....................................................................................................................................... 7
Running the CDA Installation Script.............................................................................................. 9
Overall Workflow .......................................................................................................................... 11
Archive Generation from the GUI................................................................................................12
Archive Generation from Command Line (ExtractXXArchive)................................................... 20
Integrating with WOW .................................................................................................................. 22
Preparing for Tape Cutting: The Staging Process...................................................................... 24
CDA Administrator Web Interface ............................................................................................... 28
CDA User Web Interface ............................................................................................................. 30
Appendix 1: Summary of Important Files and Variables............................................................ 31
Appendix 2: The Archive Database (ADB) ................................................................................. 33
Appendix 3: Background on the Devkit Shell lmksh ................................................................. 38
Appendix 4: Manual Installation................................................................................................... 40
Appendix 5: Moving Archiver Data into a new OpenWorks Project .......................................... 42
July 2004
Introduction
Corporate Data Archiver1 (CDA) creates metadata-rich reports on subsurface technical
project data, for subsequent archival or for general project documentation purposes. CDA
covers OpenWorks and SeisWorks projects, general UNIX disks (including but not limited
to Z-MAP Plus, SEG-Y and LAS data), GeoProbe projects and VIP VDB files. CDA is part
of Landmarks WebApps2 technology framework.
An archive is more than a project backup. Systems on the market today focus on the IT
aspects of copying data from disk to tape. What makes CDA unique is the metadata-rich
description of the projects contents that remains online post-archive. An innovative
metadata extraction process produces an online stub of the project contents. This stub
remains online, so that a project need never be restored 'just to see what's there'.
In addition to the static html stub, the Archiver also writes object-level metadata to an
Oracle database, residing within an OpenWorks project schema. This means that the
project can be documented post-archive, even after it has been removed from the system.
The Archiver also allows for a simple yet pragmatic approach for creating the physical
archive, utilising standard UNIX utilities.
The list below summarizes some of the uses of the Corporate Data Archiver:
1. Google for E&P: use CDA's stub generator on all data, which allows a powerful
cross-project cross-data type search for technical data
2. Project familiarization: save a new interpreter months getting to know a new area by
pre-generating project stubs. The stub is portable and can be copied to a laptop.
3. Project reporting: use CDA to auto-document technical project data for inclusion in an
OpenJournal or other project report.
4. Knowledge capture: document a project using the stub, even after project has
been taken offline.
5. Proactive disk management: disk is NOT cheap. Use CDA to flatten the rate of new
disk acquisition, because disk management costs over its lifetime add up.
6. Project cleanup: use stubs to aid a project cleanup exercise pre-archive.
7. Data rooms: show acreage/data to potential partners in a simple, secure way.
8. Project snapshots for regulatory purposes: record what was submitted to the
regulatory agency and when.
9. Non-operated partnership management: send snapshots to partners at critical points
in the project lifecycle.
10. Version control: record the state of a project at regular intervals for audit purposes.
11. Disaster recovery: make daily snapshots of hot projects for disaster recovery.
12. Archival: CDA helps you cut a tape to archive the project.
Page 2 of 43
July 2004
Page 3 of 43
July 2004
Version R2003.3.1
1. Removed incorrect 'generating grid image' message when extracting metadata on
ZMAP non-grid datasets.
2. ZMAP non-grid datasets are now handled properly so that large pointsets can still be
displayed without memory issues.
3. Archive project names with commas in the name are now handled correctly
(#149596).
4. PrepArchive now does include 2D Master Project Files (#150632).
5. Linux port of all CDA components
6. Added metadata extraction for SEG-Y, LAS and shapefiles.
7. Various metadata additions matching WOW enhancements.
Version R2003.3.1.1
1. Fixed error in control file editor if variable COLOR_SOURCE1 = project (#155936)
2. Fixed error in Unix extractor if .dbf files exist without .shp files
3. Fixed error on ZGFs with pictures deleted and recreated with same name
4. Fixed error if only OW project selected with no external directories (#157471)
5. Fixed incorrect PrepArchive flag for gtar includes: -T not I (#157432)
6. Changed default PrepArchive handling from 'tar' to 'gtar'
7. Fixed error on ZGFs with xscale different to yscale
Version R2003.3.1.2
1. WOW is now a prerequisite for CDA. The CDA web interfaces now check out a WOW
feature, rather than a CDAWEB feature. Clients with CDAWEB licenses will obtain a
WOW license for every CDAWEB license.
2. An Archives option has been added to the WOW left frame menu which will display
the CDA lite web interface, allowing users to browse archives alongside live projects.
3. It is now possible to create control files and execute the archive creation from the
WOW OpenWorks and SeisWorks project summary pages, provided the site has a
CDARCHIVER license. This executes the same functions as the Unix archiver GUI
4. Thumbnails generated in a throwaway project called 'Stubs' are reused in the WOW
horizon and seismic documentation pages, simplifying the documentation process.
The large images are reused in the seismic and horizon detail pages.
5. It is now possible to create horizon and seismic lists in WOW, via a paginated GUI
which displays a user-modifiable number of objects at a time. These page views will
also reuse thumbnails from an existing Stubs project.
6. Horizon and seismic list-deletion command-line utilities are provided to delete all
objects in lists created using the above list builders.
7. The ZGF extraction functions have been improved to more robustly handle various
data scenarios. SeisWorks seismic and horizon extractions also have improved
performance and robustness.
CDA R2003.3.1.5 Release Notes
Page 4 of 43
July 2004
8. The CDA administrator web interface has been changed to allow manual deletion of
archive projects, sets and objects. This helps 'fix' archives where the user has made
an error during creation.
9. The CDA lite web interface has been streamlined by merging multiple data sets into
an application project summary. Project users and boundary have been dropped from
the lite display. These changes make the project summary page more manageable.
10. A link is provided from the lite to the admin interface, and from the archive project
summary page to the staging area, allowing browsing of the staging area.
11. The CDA generator for SeisWorks now supports sub-selection using horizon and
seismic lists, enabling partial project archives. Lists can be created in SeisWorks
(horizons only), WOW or using a text editor. These partial project archives also allow
user control on exactly which SeisWorks file extensions should be archived, allowing
filtering/cleanup on archival.
12. SeisWorks metadata extractions for horizons and seismic now contain a separate
table of WOW documenter records.
13. A script is provided for moving CDA records from an old to a new archive OW project.
14. Support for a neutral format archive added, allowing archival in long-lasting formats
for a subset of data: SEG-Y for seismic, ASCII for navigation, horizons and faults,
ASCII for well headers, deviation data, time-depth tables and picks, and LAS for logs.
15. A GeoProbe Information Manager module has been added, which provides
functionality for browsing, searching, documenting, cleaning up and archiving
GeoProbe project data. The archival component supports rich metadata extraction
both in the Archiver GUI and from within the WOW browser.
16. An ArcGIS .dll has been added, providing substantially the same functionality as the
current WOW ArcView extension. This allows users of ArcGIS to launch WOW/CDA
in context by clicking on objects in shapefiles created by OpenExplorer and CDA. See
$OWHOME/WebApps/gis/arcgis.pdf for further information.
Other enhancements and bug fixes:
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
Page 5 of 43
July 2004
Version R2003.3.1.3
1. Fixed SeisWorks timeslice image creation failure
2. Fixed SeisWorks time slice label and added line/trace axis labels
3. Fixed seismic trace annotation bug when internal X = line
4. Flipped seismic images color mapping to match SeisWorks
5. Fixed SeisWorks neutral format creation bug when invoking bcm2d
6. Fixed error processing SEGY files > 2Gb
7. Improved robustness processing corrupt MFDs/ZGFs
8. Improved 'noclobber' logic on ZMAP extractions
9. Improved PrepArchive 'tarfile' option to separate directories
10. Handle 3D volumes that comprises a single crossline
11. Allow 3D volumes with master grid mismatches to process
12. Detect and skip recursive links in Unix extractor
13. Trap error when wells have no lat/lon coordinates
14. Fixed GeoProbe pointset processing error
Version R2003.3.1.4
1. Fix of GeoProbe event thumbnail sizes
2. Fixed link on OW/SW archives generated through WOW
3. Added line major keyword to 3D neutral format SEG-Y export
4. Fixed bricked float format in 3D neutral format SEG-Y export
Page 6 of 43
July 2004
Pre-Installation Planning
Corporate Data Archiver (CDA) requires WOW R2003.3.1. WOW contains the
configuration for the Apache Web Server as well as shared data access code. WOW also
provides Web browsing and administration of archive projects. Before attempting to install
CDA, you must first install WOW R2003.3.1, as described overleaf.
Important note: WOW and CDA share the same WebApps media. If you have installed
WOW, you already have the CDA media, and need only run the CDA install script.
You need to select an install user. If WOW is already installed, chose the same user as
owns the WOW installation. This should be a user with write access to most of the
OpenWorks and SeisWorks projects typically a data loader or administrator. Do NOT
use root or oracle for security reasons. You should run the install scripts as this user,
from within a Landmark environment.
You must also designate an existing or create a new OpenWorks project to store Archiver
metadata. The Archiver uses OpenWorks tables for leases, fields, basins, counties,
stratigraphy etc. See appendix 2 for more details on the Archiver data model.
Minimum Requirements
The minimum requirements for the CDA generator are identical to WOW:
Server Hardware: Sun Blade 100 or Pentium PC, 256Mb RAM, 100Mb free disk space
Operating System: Solaris 8 or Redhat Linux 7.2, patches as per R2003
Application Software: OpenWorks R2003, WOW R2003.3.1
The CDA browser has the following client requirements:
Client Hardware: not restricted
Operating System: not restricted
Web Browser: version 5.5 and above of Internet Explorer and Netscape.
Installation
The Corporate Data Archiver software is provided on CD in Landmark Release Manager
(CD Installer) format, or via ftp from isite.lgc.com as a compressed tar file. The
following installation steps are required:
1. Designate an existing or create a new OpenWorks project to store Archiver metadata
(e.g. ARCHIVES). Landmark advises creating a new OpenWorks database that is not
used for interpretation purposes.
2. Obtain and extract the WebApps (WOW and CDA) media.
3. Configure WOW R2003.3.1 by running the WOW installation script.
4. Configure CDA R2003.3.1 by running the CDA installation script.
5. Apply any patches as required.
Two methods are provided for executing these steps:
Page 7 of 43
July 2004
Manual Method
Run the WOW installation script if not already configured (see the WOW Release
Notes for installation instructions). The WOW installation script provides the option to
configure CDA immediately thereafter:
cd install
./WOWInstall o $OWHOME
If WOW has already been configured, run the CDA installation script as described in
the next section:
./CDAInstall o $OWHOME
CD Installer Method
Change directory to the CD install directory and execute the setup script:
cd /cdrom/cdrom0/install
./setup
Enter value for $OWHOME if requested. The Release Manager user interface will
appear. Choose Install for WebApps. Accept the default location of
$OWHOME/WebApps. Click on Start to extract the media.
After files are extracted, continue with the WOW and CDA installation scripts as
documented below.
Page 8 of 43
July 2004
Archive OW Project: the name of the designated OpenWorks project for storing
Archive database records. The project must exist, and the install user must be a
manager of the project with OW_ADMINISTRATOR privileges.
Stub Directory: the parent directory to which extracted metadata is written, e.g.
/data/archive_stub.
Stage Directory: the directory from which the actual archive tape is created, e.g.
/data/archive_stage.
Default Tape Device: the default tape device for the Archiver. Defaults to
/dev/rmt/0n.
Tape Device Capacity: the capacity of the default tape device in Mb. Defaults to
40000.
Review: check the options specified; exit and re-run if there are any errors.
During this stage the $OWHOME/WebApps/conf/wow.env file is updated with all the
variables required by the Archiver. See Appendix 1 for a list of all CDA configuration files.
Configuring the Archiver
During this stage the installation script edits environmental variables, sets up the Archiver
scripts as links in $OWHOME/bin, creates stub and staging directories; and creates the
ADB extension on the designated OpenWorks project. See Appendix 3 for background on
the devkit shells used by the Archiver.
Configuring the Stub Search Engine
CDA uses the Swish search engine for indexing and searching the static HTML stub. See
http://www.swish-e.org for further details on Swish. The installation script creates a default
configuration file stub.conf in $OWHOME/WebApps/dat/archive and prompts the
installer to set up the cron job to refresh the index every night, e.g.:
setenv EDITOR vi
crontab e
0 0 * * * /apps/ow/WebApps/bin/swish-e -c /apps/ow/WebApps/dat/archive/stub.conf
Page 9 of 43
July 2004
To test the CDA generator GUI type archiver in an OpenWorks xterm. See
Appendix 4 for manual installation instructions in the event that errors are encountered.
To add CDA to the OpenWorks launcher.dat file, you must launch via an xterm, e.g.:
"Archive Generator"
"xterm -e $OWHOME/WebApps/bin/archiver &"
Multiple Oracle Instances / OW_PMPATHs
The CDA web interfaces use the WOW mechanism for determining Oracle instance: see
Configuring Multiple Oracle Instances in the WOW Release Notes. The GUI interface obeys
standard UNIX OWSYSSID and OW_PMPATH environmentals. To use multiple instances with
CDA, you will need to create the ARCHIVES project with the same name on each instance.
See Appendix 4 for instructions on creating these projects.
Licensing
CDA R2003 uses regular FlexLM licensing. The archive generator uses the
CDARCHIVER feature, and the Web interfaces use the WOW feature. See the R2003
installation instructions and Release Notes for how to install licenses, and/or email
license@lgc.com to obtain a valid license key.
Patching the Installation
CDA patches are designed as simple compatible additions or replacements of existing
files in the software directories. CDA and WOW share a common architecture and are
therefore patched in parallel. Patches are provided as compressed tar files, available from
the Landmark ftp server isite.lgc.com:/products/CDA/Supported_Patches.
Contact your customer support representative for isite login details.
The current patch level is R2003.3.1.5.
Since R2003.3.1, a new rolling patch has been introduced. The objective is to fix bugs
with days and update a continuous rolling patch, so customers do not have to wait
months for the next official patch. This patch is on isite, directory
/products/CDA/Unsupported_Patches.
To install a patch, place it in your installation location, typically $OWHOME/WebApps. See
the associated README for detailed installation instructions.
Patch history:
R2003.0.3.0 foundation release (July 2002)
R2003.0.3.1 patch release; requires R2003.0.3.0 (October 2002)
R2003.3.1.0 point product release, includes all previous versions (January 2003)
R2003.3.1.1 patch release, requires R2003.3.1.0 (May 2003)
R2003.3.1.2 patch release, requires R2003.3.1.0 (August 2003)
R2003.3.1.3 patch release, requires R2003.3.1.0 (December 2003)
R2003.3.1.4 patch release, requires R2003.3.1.0 (April 2004)
R2003.3.1.5 patch release, requires R2003.3.1.0 (July 2004)
CDA R2003.3.1.5 Release Notes
Page 10 of 43
July 2004
Overall Workflow
The Corporate Data Archiver is designed to separate end-user tasks (e.g. identifying data
to archive/delete) from data administrator tasks (e.g. capturing automatic metadata) from
IT tasks (cutting and verifying the tape). See illustration below.
Archive Workflow
User tasks
Request
archive
Identify
data to
archive
Capture
manual
metadata
Identify
data to
clean up
User
Approval
Prearchive
cleanup
DM tasks
Capture
automatic
metadata
Load
metadata
to ADM
Extract
data
IT tasks
Request
archive
Delete
data
Create
archive
Verify
archive
Extract: the archiver is a UNIX application with a graphical user interface for
extracting Web-format metadata, writing object-level metadata data to Oracle and
configuring the staging directory for the physical archive.
The Archiver can be run from within the WOW browser interface.
Pre-archival Cleanup: WOW provides options for building lists of objects for
subsequent deletion prior to archival
Prepare: The Archiver does NOT create a tape: rather, it writes configuration
information into a staging directory, which is then prepared via an additional step for
writing to tape, near-line system, backup device, disk etc.
Page 11 of 43
July 2004
Users select a job type of OpenWorks, SeisWorks or UNIX, or choose to open an existing
control file.
Click on Open next to any job brings up the appropriate control file editor, as illustrated
overleaf for SeisWorks.
Click on Run to execute the extraction using the control file as saved.
Click on Save Log to save the log output from the bottom portion of the GUI to file.
Click on Clear Log to clear the log window prior to subsequent runs.
Page 12 of 43
July 2004
Note: the open existing control file and save as options default to $HOME. To change
this location in order to manage control files better, add the following line to
$OWHOME/WebApps/conf/wow.env:
ARCHIVE_CTL_LOC=/full/path/to/ctlfiles; export ARCHIVE_CTL_LOC
This allows control files to be managed centrally, rather than in multiple users home
directories.
OpenWorks Control File Editor
The OpenWorks control file editor GUI is illustrated below:
Notes:
The control file can be modified to allow either complete archival, or just stub
generation to an arbitrary disk location. Changing Write to ADB model from yes to
no will activate the Stubs dialog box.
Page 13 of 43
July 2004
Change Archive project type to Partial (native) to allow sub-selection by well list.
This will enable the Extract only specified well list option, which restricts the
extraction to the specified list only. However, with native format archival, the
OpenWorks project backup will always contain all data.
The Overwrite archive stub parameter applies to the metadata extraction, and has
no effect on the archive. Archive sets are always replaced in the CDA data model.
Data type selection parameters, e.g. Extract wells? (yes/no) apply to both metadata
extraction and archive set creation, although the project backup contains all the data.
Metadata extraction for well data is slow. Changing Extract well detail to no will
considerably improve performance, by not drilling down to every object within every
well.
Page 14 of 43
July 2004
Notes:
The control file can be modified to allow either complete archival, or just stub
generation to an arbitrary disk location. Changing Write to ADB model from yes to
no will activate the Stubs dialog box.
Changing Archive project type to Partial also enables the List of extra file
extensions option, providing fine control over what project files are added to a partial
archive. Critical project files are always archived, with extensions: .pdf .pds .ps2
Page 15 of 43
July 2004
.pd2 .merge_cat .sm .line_cat .hrz_cat and .hrz. Also excluded from
user control are all seismic and horizon files, with extensions: .hzd .hts
.hzh_glb .hzd_glb .3dv .3dh .bri .cmp and .2v2_glb. This option allows
the selective inclusion on non-critical files such as processing control files, colormaps
etc. If no file extensions are specified, the default list is used as stored in
$OWHOME/WebApps/conf/sw_file_extensions.dat.
The Overwrite archive stub parameter applies to the metadata extraction, and has
no effect on the archive. Archive sets in the CDA data model and the archive staging
area are always replaced.
Data type selection parameters, e.g. Extract horizons? (yes/no) apply to both
metadata extraction and archive set creation.
The parameter Extract seismic? provides an extra option of stub only, which will
create the stub, but not include the seismic files in the staging area. This can reduce
archive size by 80-90% where seismic is mastered elsewhere, e.g. Petrobank.
Metadata extraction for large 2D projects can be slow. Changing Extract 2D seismic
detail to no will improve performance by skipping the creation of live trace outlines.
Page 16 of 43
July 2004
Notes:
The control file can be modified to allow either complete archival, or just stub
generation to an arbitrary disk location. Changing Write to ADB model from yes to
no will activate the Stubs dialog box.
The Overwrite archive stub parameter applies to the metadata extraction, and has
no effect on the archive. Archive sets in the CDA data model and the archive staging
area are always replaced.
Data type selection parameters, e.g. Extract LAS files? (yes/no) apply to both
metadata extraction and archive set creation.
Page 17 of 43
July 2004
The Application parameter is important and must match a value as set in the
Archiver data model. When set to Z-MAPPlus R2003 special extraction routines are
run against MFD and ZGF files.
The UNIX extraction will recursively walk through any subdirectories of the start
directory specified in the control file.
The Extract ascii files? parameter will create html stubs of the first Number of data
records lines in the file. To identify a file as type ASCII, add in to
ascii_file_extensions.dat in $OWHOME/WebApps/conf. If it does not exist,
cd $OWHOME/WebApps/conf
cp ascii_file_extensions.dat.default ascii_file_extensions.dat
The Extract web files? parameter will copy web files directly into the stub: these are
mime types that are universally understood by web browsers, e.g. html, gif, jpeg etc.
To identify a file as a web file, add in to web_file_extensions.dat in
$OWHOME/WebApps/conf. If it does not exist,
cd $OWHOME/WebApps/conf
cp web_file_extensions.dat.default web_file_extensions.dat
Page 18 of 43
July 2004
surfaces yes
colormaps yes
points yes
faults yes
wells yes
states yes
scolour RedWhiteBlue
hcolour Rainbow1
#
#
#
#
#
#
#
#
Extract surfaces?
Extract colormaps?
Extract points?
Extract faults?
Extract wells?
Extract state files?
Color map for volumes
Color map for surfaces
Notes:
The control file can be modified to allow either complete archival, or just stub
generation to an arbitrary disk location. Changing Write to ADB model from yes to
no will activate the Stubs dialog box.
The Overwrite archive stub parameter applies to the metadata extraction, and has
no effect on the archive. Archive sets in the CDA data model and the archive staging
area are always replaced.
Data type selection parameters, e.g. Extract volumes? (yes/no) apply to both
metadata extraction and archive set creation.
Page 19 of 43
July 2004
Page 20 of 43
July 2004
Important note: we recommend not modifying the default control files. Rather, make a
copy of the files, e.g.
cp $OWHOME/WebApps/dat/archive/swextract.ctl ~/dan3d.ctl
vi ~/dan3d.ctl
Control files can be edited by hand and submitted on the command line, as described in
the following section. When editing control files by hand, note that the parameter name (1st
value) cannot be modified. The value (2nd parameter) is under user control. Any text after
the # is treated as a comment. Values with spaces should be quoted, e.g. Q23 well list.
Independent Stub Creation
It is entirely possible to run the metadata extraction without reference to archival. For
example, there may be a requirement to produce an html appendix to an OpenJournal
project which details the data used in an analysis.
To do this, set the control file parameters as shown:
stub
stage
model
yes
no
no
no
yes
no
Page 21 of 43
July 2004
This brings up a simplified control file editor. Once created, the stub can be previewed
directly.
Important Note: Use an archive project name of Stubs to create thumbnails in a fixed
location, for later re-use in the horizon and seismic list builders.
Important note: for sites not wishing to allow archive generation through the browser, add
the following line to $OWHOME/WebApps/conf/wow.env:
ARCHIVE_BROWSER_FLAG=REQUEST; export ARCHIVE_BROWSER_FLAG
Page 22 of 43
July 2004
This will replace the link to generate archive with a link to request archive generation,
which sends email to the CDA administrator.
The SeisWorks horizon and seismic list builders are accessible from the page links on the
SeisWorks project summary page. They allow browsing horizons and seismic a page at a
time, drilling down to view details or selecting objects to save or append a list.
The web interface is illustrated overleaf:
Thumbnails generated in the 'Stubs' projects will also appear in the horizon and seismic
documentation pages in WOW, making the process considerable easier.
List-based Deletion
Command-line scripts are provided to delete lists of horizons and seismic created using
the WOW page view list builders.
Important note: these routines are inherently destructive. Make sure you have a current
project backup before executing.
To configure batch deletion scripts (one-off operation):
cd $OWHOME/WebApps/bin
ln s lmksh_wrapper HrzListDelete
ln s lmksh_wrapper SeisListDelete
To execute the deletion:
HrzListDelete <project> <listname>
SeisListDelete <project> <listname>
Page 23 of 43
July 2004
An empty <project>.swp file with the name of the SeisWorks project. If neutralformat, the convention is <project>.swn.
A .dir file per directory in the SeisWorks project, with the system directory alone
annotated differently. Each of these files contains a project directory name, the name
of the relative file listing for that directory (including any subdirectories), and a
computed directory size in Mb.
A .lst file matching each .dir file, containing a relative file listing for that directory
(including subdirectories). This file is designed to be used with the tar command with
the include from file option.
For neutral-format archives, there is one .dir file for each neutral-format subdirectory
in the staging area. The .lst file contains a relative file listing for that directory as
before.
For 2D native-format archives, the process in addition also creates the .dir and
.lst files for only those files in the associated master project that are used by the
working project.
A stub.loc file containing the location of the archive project metadata stub directory.
OpenWorks Configuration
For OpenWorks, the following files completely describe the project contents for the
subsequent prepare step.
Page 24 of 43
July 2004
An empty <project>.owp file with the name of the OpenWorks project. If neutralformat, the convention is <project>.own.
A file OW_SYS_DATA.loc containing a single line with the location of the OpenWorks
system data directory.
A .dir file per OpenWorks project external directory. Each of these files contains a
directory name, the name of the relative file listing for that directory (including any
subdirectories), and a computed directory size in Mb.
A .lst file matching each .dir file, containing a relative file listing for that directory
(including subdirectories). This file can be used with the tar command with the include
(-I) flag.
For neutral-format archives, there is one .dir file for each neutral-format subdirectory
in the staging area. The .lst file contains a relative file listing for that directory as
before.
A stub.loc file containing the location of the archive project metadata stub directory.
UNIX Configuration
An empty <unixdir>.und file with the name of the UNIX directory (/ and other
special characters are replaced by _).
A .dir file for the UNIX directory. The file contains a directory name, the name of the
relative file listing for that directory (including any subdirectories), and a computed
directory size in Mb.
A .lst file matching the .dir file, containing a relative file listing for that directory
(including subdirectories). This file can be used with the tar command with the include
(-I) flag.
A stub.loc file containing the location of the archive project metadata stub directory.
Page 25 of 43
July 2004
Device is a no-rewind tape device or a file. Capacity is tape limit in Mb, i.e. 8000 for an
8mm Exabyte, 40000 for a DLT 7000. Tape device and capacity will default to the
variables $ARCHIVE_TDEVICE and $ARCHIVE_CAPACITY is set.
You may set ARCHIVE_CAPACITY to a high number when using gtar or tarfile, since in
both cases (multi-volume gtar or file) the tape length error message is not relevant.
For example, to emulate the TARME from the previous version of CDA (with gtar):
cd <archive stage directory>
PrepArchive (no further arguments required)
To archive from /archive_stage/ProjX to a tar file in the same directory:
PrepArchive /archive_stage/ProjX tarfile /archive_stage/ProjX.tar 100000
To archive in tar format to a new tape device /dev/rmt/1n with capacity 40Gb:
PrepArchive /archive_stage/ProjX tar /dev/rmt/1n 40000
The PrepArchive script (and any custom alternative) should perform the following:
Creates a tar file of the archive stub directory using file stub.loc
Builds lists of OpenWorks, SeisWorks and UNIX projects using files .swp, .owp and
.und files)
Definition of ARCHIVE_TDEVICE
Definition of ARCHIVE_CAPACITY
A check for total archive size not exceeding the value of $ARCHIVE_CAPACITY
Command to write the README, stub and other table of contents info to tape
Page 26 of 43
July 2004
Note that the first file on tape contains the README itself and the file lists. This means that
whoever restores the tape then knows exactly how many tar files to expect, and the size of
each.
Page 27 of 43
July 2004
Important note: changing archive project class from pending to complete disables
addition and deletion, in order to protect the archive metadata. The class must be set to
pending in order to do further addition or deletion.
Manage Lists of Values
The manage list of values option allows an administrator to add and delete entries in the
lookup tables used in the ADB data model. The default values are:
Archive format: native, neutral
Application: EDMS, Eclipse, GeoProbe, Geolog, Hampson-Russell, IrapRMS, MSOffice
2000, OpenWorks R2003, Petrel, SeisWorks R2003, StratiMagic, VIP, Z-MAPPlus R2003
Datatype: 2D Nav, 2D Seismic, 3D Model, 3D Nav, 3D Seismic, AVO, Basemap, Basins,
Documents, Fault Polygons, Faults, Fields, Grids, Horizons, Inversion, Leases, MS
Access, MS Excel, MS PowerPoint, MS Word, Pointsets, Project Snapshot, Simulation
Model, UNIX Files, Velocity, Wavelets, Wells
Media type: 3480, 3490, 3590, DLT 4000, DLT 7000, Exabyte 8200, Exabyte 8500
CDA R2003.3.1.5 Release Notes
Page 28 of 43
July 2004
Page 29 of 43
July 2004
Page 30 of 43
July 2004
Description
$OWHOME/WebApps/dat/archive:
owextract.ctl
swextract.ctl
unextract.ctl
gpextracl.ctl
prj_replaceme.html
set_replaceme.html
stub.conf
*.css
$OWHOME/WebApps/conf:
wow.env
wow_sidlist.dat
wow_distlist.dat
ascii_file_extensions.dat
sw_file_extensions.dat
web_file_extensions.dat
$OWHOME/WebApps/archive:
$OWHOME/WebApps/archive/
sqlscripts:
The list below provides an example of the environmental variables which can be set in wow.env. See
the WOW release notes for an example of a complete wow.env file.
#============================================================================
# Corporate Data Archiver environmentals
# Change value of all variables here
#---------------------------------------------------------------------------# Name of OW project to store Archive database records
ARCHIVE_OW_PROJECT=EMPTY; export ARCHIVE_OW_PROJECT
# Email address of the Archive administrator
ARCHIVE_ADMIN_EMAIL=${MAIL_TO}; export ARCHIVE_ADMIN_EMAIL
# Stub directory
ARCHIVE_STUB_DIR=/data1/archive_stub; export ARCHIVE_STUB_DIR
# Stub URL
ARCHIVE_STUB_URL=http://exprotech/wow/archive_stub; export ARCHIVE_STUB_URL
# Staging directory
CDA R2003.3.1.5 Release Notes
Page 31 of 43
July 2004
Page 32 of 43
July 2004
The model is designed for incorporation into a single OpenWorks project database. All
new tables are displayed in lowercase with prefix of adb_. New reference tables have a
prefix of adb_r_. Existing OpenWorks tables are shown in uppercase. For simplicity only
relevant attributes of OpenWorks tables are shown. For the main tables, keys are annoted
as follows: PK primary key; AK alternate key; FK foreign key.
Project table
A project is defined in a business sense rather than in physical terms, i.e. it is a collection
of data across multiple applications, databases and data types. The main adb_project
table contains attributes pertaining to the entire project and does not presuppose any
archival activity.
The first set of attributes describe the project:
project_id
internal identifier
project_name
full text name
project_description long text description of the project
project_start
start interpretation date
project_end
end interpretation date
Page 33 of 43
July 2004
project_type
project_class
project_status
project_source
proprietary_flag
edms_id
as before
validated, e.g. well, deviation, checkshot, pick, log, seismic
validated, native or neutral
validated, e.g. SeisWorks, Hampson-Russell
project name for the specified application
the date on which the set was created
Page 34 of 43
July 2004
Object Table
Each data set is composed of multiple objects. The purpose of the adb_object table is to
store object-level archive metadata, e.g. every horizon in a SeisWorks project, seismic file
in a Hampson-Russell project etc.
Attributes are:
set_id
object_id
object_name
object_description
object_orig_location
object_data_source
edms_id
strat_unit_name_id
post_archive_delete
Page 35 of 43
July 2004
media_type
media_id
Reference Tables
The ADB uses 8 new reference tables in addition to 9 existing OpenWorks tables for
validation: adb_r_archive_fmt , adb_r_application, adb_r_datatype, adb_r_app_datatype,
adb_r_media_type, adb_r_project_class, adb_r_project_status, adb_r_project_type. All of
these are simple single-column validation lists, with the exception of adb_r_app_datatype,
which is an intersection table between adb_r_application and adb_r_datatype.
Views
The following views will be used to simplify data access and query:
adb_v_project
adb_project + vc tables
adb_v_archive
adb_project + adb_archive
adb_v_set
adb_project + adb_archive + adb_set
adb_v_object
adb_set + adb_object
Data Access
All Oracle data access is via the Tcl/Tk oratcl extension. This provides scripted access to
the Oracle Call Interface (OCI). OCI provides stable and rapid select, insert, update and
delete functionality to the ADB tables stored in Oracle. This mechanism is identical to the
direct Oracle access mechanisms used in WOW.
This Oracle data model provides the level 1 metadata. See the diagram below for how
this links with the level 2 online remnant or stub.
Level 1 metadata
structured archive metadata
relational model in Oracle
project
set
object
archive
$ARCHIVE_STUB_DIR
prj1
prj2
prj3
prj4
prj5
prj_n
Level 2 metadata
Archive staging
area
Tape creation
offline archive to tape
prj1
prj2
prj3
prj4
prj5
prj_n
a1s1
a1s2
a1s3
a2s1
a2s2
data
data
data
data
data
$ARCHIVE_STAGING_DIR/prj_n
Page 36 of 43
TAPE
July 2004
The link between levels 1 and 2 is provided by a pointer in the Oracle data model to a
location of an index.html file. This file in turn will reference all detailed metadata, to any
level of detail, as determined by the relevant application. Locations are relative to a system
environmental variable $ARCHIVE_STUB_DIR, to facilitate the IT aspects of managing
large disk arrays.
The advantage of this approach is that it removes the need to model the universe of all
applications and objects to a consistent level of detail. Application archive metadata can
be at varying levels of detail and may be upgraded or improved without impacting other
applications. With a smaller core relational database, data model changes and hence
maintenance over time is considerably simplified. Lock-in to the developing vendor is
reduced. In addition, storing metadata in Web format opens up a host of opportunities for
implementing horizontal search tools, integrating with EDMS etc.
Page 37 of 43
July 2004
Practical Programming in Tcl and Tk (4th Edition), by Brent Welch, Jeffrey Hobbs &
Ken Jones, published by Prentice-Hall 2003, ISBN 0130385603
Tcl/Tk in a Nutshell, by Paul Raines & Jeff Tranter, OReilly 1999, ISBN 1565924339.
Running lmksh
To run lmksh you require a .lmkrc file in your home directory and to have sourced the
required environment:
prompt> cp $OWHOME/WebApps/templates/dotlmkrc ~/.lmkrc
(1-off)
(C-shell)
prompt> source $OWHOME/WebApps/templates/dotlogin*
prompt> . $OWHOME/WebApps/templates/dotprofile
(Bourne shell)
prompt> lmksh
lmksh{1}% OWSetup
(configures OpenWorks environment)
lmksh{2}% SeisSetUp
(configures SeisWorks environment)
lmksh{3}% info commands
(lists all available commands)
lmksh{4}% info commands sdl*
(lists commands beginning with sdl)
lmksh{5}% info procs
(lists all available procedures)
lmksh{6}% info args hrzinfo
(lists arguments for specified proc)
Alternatively, a wrapper script is provided and linked to $OWHOME/bin during installation:
prompt> rlmksh
Page 38 of 43
July 2004
Notes on .lmkrc: you will need to copy the .lmkrc into the home directory of every user
that wishes to run lmksh interactively:
prompt> cp $OWHOME/WebApps/templates/dotlmkrc $HOME/.lmkrc
Add or uncomment the commands OWSetup and SeisSetUp in .lmkrc so you do not
need to type these during every interactive session. Add or uncomment
SourceAllProcs to load all available commands.
The lmksh Commands
There are three classes of commands:
Tcl built-in commands: This is the core Tcl language. See the Tcl man pages or html
help at $OWHOME/WebApps/tcltk/htmldocs/tcltk.html for information on built-in
commands. To access the man pages:
prompt> source $OWHOME/WebApps/templates/dotlogin
prompt> man <cmd>
Extended commands: These are added to lmksh to provide scripted access to devkit
functions. The extended commands are usually low-level with little error trapping. Most
begin with sdl_ (SeisWorks), ow_ (OpenWorks), sil_ (MFD), zgf_ (ZGF), sgy_
(SEGY) and gen_ (LAS and others). To see the arguments for an extended command,
type the command name:
lmksh{7}% sdl_hrzinfo
Procedures: These are additional wrappers to the extended commands, including more
robust error trapping and other business logic to make the low-level commands more
useful. To see the arguments for any procedure:
lmksh{8}% info args hrzinfo
OpenWorks database access procedures usually start with a lowercase letter to denote
that they require an Oracle database connection to be established first, e.g.:
prompt> rlmksh
lmksh{9}% set lda [OWoralogon]
lmksh{10}% getWellLists TESTDATA
lmksh{11}% oralogoff $lda
Exposing lmksh Commands
It is sometimes more convenient to run lmksh procedures through a regular Bourne- or Cshell. A wrapper script is provided in $OWHOME/WebApps/bin for this purpose. To
expose any lmksh procedure on the command line, create a link as illustrated below:
prompt> cd $OWHOME/WebApps/bin
prompt> ln s lmksh_wrapper <proc>
The procedure can then be run on the command line, with any required parameters
passed through to the underlying lmksh.
Health Warning
WOW and CDA are largely read-only applications. But the underlying lmksh has
additional destructive options, e.g. the ability to create, rename and delete
SeisWorks horizons. Use of these options is entirely at your own risk, and you
should always ensure you have a current backup before proceeding.
Page 39 of 43
July 2004
Add Archiver environment to the wow.env configuration file (use your values below):
cd $OWHOME/WebApps/conf
vi wow.env
# Archiver definitions
ARCHIVE_OW_PROJECT=ARCHIVES; export ARCHIVE_OW_PROJECT
ARCHIVE_ADMIN_EMAIL=xxx@yyy.com; export ARCHIVE_ADMIN_EMAIL
ARCHIVE_STUB_DIR=/data/archive_stub; export ARCHIVE_STUB_DIR
ARCHIVE_STUB_URL=http://wow.oilco.com/wow/archive_stub; export ARCHIVE_STUB_URL
ARCHIVE_STAGE_DIR=/data/archive_stage; export ARCHIVE_STAGE_DIR
ARCHIVE_TDEVICE=/dev/rmt/0n; export ARCHIVE_TDEVICE
ARCHIVE_CAPACITY=40000; export ARCHIVE_CAPACITY
#ARCHIVE_BROWSER_FLAG=REQUEST; export ARCHIVE_BROWSER_FLAG
#ARCHIVE_TIMESTAMPS=1; export ARCHIVE_TIMESTAMPS
#ARCHIVE_CTL_LOC=${ARCHIVE_STUB_DIR}; export ARCHIVE_CTL_LOC
Page 40 of 43
July 2004
7. Create links in $OWHOME/bin to the archive builder and metadata extraction routines.
This means that the archive creation scripts can be run from any location, provided
the OpenWorks environment has been sourced.
cd $OWHOME/bin
ln -s $OWHOME/WebApps/bin/lmksh_wrapper ExtractOWArchive
ln -s $OWHOME/WebApps/bin/lmksh_wrapper ExtractSWArchive
ln -s $OWHOME/WebApps/bin/lmksh_wrapper ExtractUNArchive
ln -s $OWHOME/WebApps/bin/lmksh_wrapper ExtractGPArchive
ln -s $OWHOME/WebApps/bin/archiver archiver
ln s $OWHOME/WebApps/bin/lmksh_wrapper PrepArchive
8. Edit the Archive generator wrapper script in $OWHOME/WebApps/bin to refer to the
correct value of $OWHOME and $WOW_HOME:
cd $OWHOME/WebApps/bin
vi archiver
check settings of OWHOME and WOW_HOME on lines 32/33.
To test the installation, type archiver from an OpenWorks xterm.
9. Configure the Swish search engine used for indexing and searching the static HTML
stub. See http://www.swish-e.org for further details on Swish. To configure the search,
you will need to make a copy of and modify the default configuration file stub.conf
in $OWHOME/WebApps/dat/archive:
cp $OWHOME/WebApps/dat/archive/stub.conf ~/stub.conf
vi ~/stub.conf
Change the values of IndexDir and IndexFile to the correct values for your
site, i.e. with the correct $OWHOME.
###############################################################################
# Modified version of swish.conf used by the Corporate Data Archiver
###############################################################################
# Parent directory to index; MUST be below the web root
IndexDir /apps/ow/WebApps/htdocs/archive_stub
# Name of index file
IndexFile /apps/ow/WebApps/htdocs/archive_stub/stub.index
# Important swish parameters
IndexOnly .html
NoContents .gif .xbm .au .mov .mpg .pdf .ps .jpg .png
###############################################################################
# Other swish parameters (change if you know what you're doing)
###############################################################################
Page 41 of 43
July 2004
Page 42 of 43
July 2004
Landmark/Asia Pacific
Landmark/EAME
Landmark/The Americas
Springfield Drive
Houston, TX 77042
Malaysia
England
U.S.A.
Tel: 60-3-2164-1121
Tel: 44-1372-868-600
Tel: 1-713-839-2000
Fax: 60-3-2164-1135
Fax: 44-1372-868-601
Fax: 1-713-839-2168
Email: apsupport@lgc.com
Email: eame_helpdesk@lgc.com
Email: support@lgc.com
Trademarks
Landmark, the Landmark logo, 3D Drill View, 3D Drill View KM, 3DVIEW, Active Field Surveillance, Active
Reservoir Surveillance, ARIES, Automate, BLITZ, BLITZPAK, CasingSeat, COMPASS, Contouring Assistant,
DataStar, DBPlot, Decision Suite, Decisionarium, DecisionDesktop, DecisionSpace, DepthTeam, DepthTeam
Explorer, DepthTeam Express, DepthTeam Extreme, DepthTeam Interpreter, DESKTOP-PVT, DESKTOP-VIP,
DEX, DFW, DIMS, Discovery, Drillability Suite, DrillModel, DrillVision, DSS, Dynamic Surveillance System,
EarthCube, EdgeCa$h, eLandmark, EPM, e-workspace, FastTrack, FZAP!, GeoDataLoad, GeoGraphix,
GeoGraphix Exploration System, GeoLink, GES, GESXplorer, GMAplus, GrandBasin, GRIDGENR, I2 Enterprise,
iDims, IsoMap, LandScape, LeaseMap, LMK Resources, LogEdit, LogM, LogPrep, Make Great Decisions,
MathPack, Model Builder, MyLandmark, MyWorkspace, OpenBooks, OpenExplorer, OpenJournal, OpenSGM,
OpenTutor, OpenVision, OpenWorks, OpenWorks Well File, PAL, Parallel-VIP, PetroBank, PetroWorks, PlotView,
Point Gridding Plus, Pointing Dispatcher, PostStack, PostStack ESP, PRIZM, PROFILE, ProMAX, ProMAX 2D,
ProMAX 3D, ProMAX 3DPSDM, ProMAX MVA, ProMAX VSP, pStaX, QUICKDIF, RAVE, Real Freedom,
Reservoir Framework Builder, RESev, ResMap, RMS, SafeStart, SCAN, SeisCube, SeisMap, SeisModel,
SeisSpace, SeisVision, SeisWell, SeisWorks, SeisXchange, SigmaView, SpecDecomp, StrataMap, Stratamodel,
StratAmp, StrataSim, StratWorks, StressCheck, STRUCT, SynTool, SystemStart, T2B, TDQ, TERAS, Total
Drilling Performance, TOW/cs, TOW/cs The Oilfield Workstation, Trend Form Gridding, Turbo Synthetics, VIP,
VIP-COMP, VIP-CORE, VIP-DUAL, VIP-ENCORE, VIP-EXECUTIVE, VIP-Local Grid Refinement, VIPPOLYMER, VIP-THERM, WavX, Web OpenWorks, Well Editor, Wellbase, Wellbore Planner, WELLCAT,
WELLPLAN, WellXchange, WOW, Xsection, ZAP!, Z-MAP Plus are trademarks, registered trademarks or service
marks of Landmark Graphics Corporation.
All other trademarks are the property of their respective owners.
Page 43 of 43
July 2004