Fast, automated DB2 cloning and refresh operations to boost results
17th November 4pm UTC
There are many reasons to clone DB2 systems and refresh table spaces; create test environments, stage data-warehouse loads and offload business processed from production to other environments. Traditional methods are time-consuming, use large amounts of host processing resources and, when copying DB2 systems, require a separate LPAR to house the copy. Because manual effort is required to research and clone data, you risk impacting your systems availability.
Today’s storage-based fast replication technologies provide services to copy DB2 data instantaneously and without using host CPU or I/O resources. Join us for this complimentary teleconference and learn how the IBM DB2 Cloning Tool for z/OS copies entire DB2 systems within the same or shared LPAR using storage-based fast replication and how volume identifiers and DB2 metadata are managed to allow the data to be used quickly by a cloned DB2 system.
This in-depth look at the DB2 cloning Tool for z/OS will also cover how the product:
Automatically translates the object IDs that differ between source and target subsystems
Quickly clones DB2 subsystems, DB2 table spaces or index spaces to create up-to-date test environments
Clones a DB2 subsystem by renaming and cataloging the data sets, fixing the volume internals and updating the DB2 internal control information
I had to set up 10 TWS batch schedules this week, and baulked a bit at the idea of going through the admin tool and clicking them all in. Following a discusion with a colleague, I came across a TWS utility called “composer” which I hadn’t previously known about. This allows you to export job and schedule definitions to a file, and then re-import them again. Using this, I was able to create the first set of jobs and schedule and then export to a file using:
composer create file.txt from job=WORKSTATION#JOBNAME
and
composer create file2.txt from sched=WORKSTATION#SCHEDNAME
Then, using some dodgy shell, sed and awk scripting, I cloned all of these for each of the environments and reloaded them using:
composer replace file.txt
What could be simpler? The text files produced are fairly self explanatory, although – no doubt – with a little rummaging there’s bound to be a manual somewhere about the syntax.
Updates have been applied to snapper (data collector), snapsave (recorder) and snapmon (web based front end) to allow snapsave and snapmon to support multiple collectors.
New test data was generated, and it is now apparent that some summarisation of the data will have to take place, as the javascript widgets providing spreadsheet and charting will not perform to acceptable levels with 500 entries to manage!
Next actions:
1. Add summarisation
2. Make snapsave multi-threaded to support multiple data colelctor sources more effectively.
The snapshot monitoring tool now has a data recording component for DBM snapshots that records the event and all of the details into a DB2 UDB database.
Work has started on a web-based front end (Apache / PHP / JScript) which includes user login and management and the initial tabular presentation of the snapshot data.
This will be extended this week to include some basic charting of the numeric columns using an open source JScript toolkit called Dojo, which seems to be gaining a lot of interest on the internet. This includes many “look and feel” aspects, but of particular interest is the tooltip, charting and dialog box functionality.
I hope to have a basic demonstration in place by the end of the week.
The data capture aspects (“snapper”) of the snapshot monitoring tool are now complete. Thread scheduling has been verified, along with the timer thread operation and the communications thread.
A small command line utility has also been written to tell snapper to shutdown.
Work has started on the receiver (“snapsave”), which will interpret the DB2 monitor data stream passed to it by snapper. The framework has been written (database connectivity, tcp/ip connection to snapper, monitor stream retrieval) and had some minimalistic testing.
Have just started work on the code to interpret the stream and save the results to DB2. The header record (first element) – which will be the parent data for any of the snapshot types is just about done.
Once this is working, I’ll start on the DATABASE MANAGER snapshot stream.
Once this is all tested and working nicely, I’ll do a little work on a Apache/PHP/Web-browser based front end. Just to prove it all hangs together, I’ll sort something out on this once the DBM snapshot decode is done.
Once we’ve proved the architecture, we can start work on the other streams. Current focus is on (all database specific):
The snapshot server / daemon is just about finished – I need to build in some way of driving the shutdown event (already built into all of the threads) and put some additional events in to make sure that all of the snapshots threads are fully started before engaging the timer.
I’m planning to just use telnet to check that a data stream is being produced, but it’s all quite interesting at the moment.
Next on the agenda is the receiver, which will interpret the data streams and save the results off into a set of DB2 tables.
The current transmission method is over TCP/IP, but I’m thinking of externalising the comms to a replacable DLL, so that we can deliver data through a variety of different methods including MQ (probably).
I’ve started work on a UDB snapshot monitoring tool to gather and retain information from database servers. Only minimal thought has gone into reporting at the moment, but this will probably be web based.
The current proof of concept code can set the required monitor switches, gather all of the key snapshot data from the Admin API (db2GetSnapshot) and produce formatted reports. This represents most of the tedious grind work, so now it’s time to move onto the interesting bits and pieces – specifically:
Multi-threaded snapshot gatherer
TCP/IP communications between gatherer threads and the data store engine.
Data store engine.
(Probably) Apache / PHP front end with self refreshing pages showing status information
Hey everyone. I’ve started to think about the various tools which I’ve put together in the last few years. This is stuff that I’ve written to make my life easier – well, nobody likes to work too hard, do they?! It has done everything from allowing me to automate significant work items on customer sites, to just presenting information in a more managable way.
I’m now thinking about ideas for other tooling that would help make tedious or fiddly tasks simpler. Most of the current tooling is Windows based. Some of the tools have been ported to AIX and Linux (premig and some driver DLLs) but the others could make it over there as well.
One of my favourites is Restore, which does the whole redirected restore, including the tablespace container work. It’s able to do this because it maps out the tablespace data in the backup, so knows what it needs to reapply. This can then be done with specific targetted paths or files in the config file, or with generic pathing. Works with backups on the filesystem or in TSM. Quite neat, really!
