Ansible on z/OS – 2 – z/OS Open Automation (ZOA) Utilities
Introduction
In the second of this series, we look at the installation and execution of the second ansible pre-req: IBM z/OS Open Automation (ZOA) Utilities. These provide a raft of additional functionality to the Python world, include z/OS dataset handling, batch job submission and tracking and z/OS console use. These features are in turn used by ansible.
Installing ZOA Utilities
These utilities are installed via CBPDO and SMP/E as with other z/OS software. They can be ordered separately (Program ID is 5698-PA1) or as part of other tooling – e.g. Dependency Based Builder (DBB).
The end result should be a new path in USS:
/usr/lpp/IBM/zoautil
Configuration
To make use of these utilities, we will need to configure them into the USS environment. We did this by updating a single userid profile, by adding the following:
export PYTHON_HOME=/usr/lpp/IBM/cyp/v3r8/pyz export ZOA=/usr/lpp/IBM/zoautil export _BPXK_AUTOCVT=ON export PATH=$PATH:$PYTHON_HOME/bin export LIBPATH=$LIBPATH:$PYTHON_HOME/lib # ZOA Utilities export PATH=$PATH:$ZOA/bin export LIBPATH=$LIBPATH:$ZOA/lib export MANPATH=$MANPATH:$ZOA/docs/%L # Python support for ZOA Utilities export PYTHONPATH=$ZOA/lib
Note that the MANPATH export adds the ZOA Utilities into the scope of the “man” (manual) Unix command which provides syntax and parameters for each of the commands supplied by this feature.
Note also that the last entry (PYTHONPATH) is in place for ansible.
Documentation
The documentation for ZOA Utilities is all online and can be found here:
https://www.ibm.com/support/knowledgecenter/SSKFYE_1.0.3/zoautil_overview.html
The Python doc is here (for 1.0.3):
https://www.ibm.com/support/knowledgecenter/SSKFYE_1.0.3/python_doc_zoautil/index.html?view=embed
The product is updated (so far) by PTFs, so make sure that you’re looking at the correct version (selectable with the “Change version or product” pull down on the top left of the web page).
Example Commands
As well as Python modules and Java interfaces, the ZOA Utilities are implemented as shell commands as well. These provide extensions to the USS shell environment. Here are some examples:
- List PDS members
-
- mls “ZPDT.SYSTEM.JCL”
-
IBMUSER:/u/ibmuser: >mls "ZPDT.SYSTEM.JCL" ACSTEST : DSNTEP2 : ZFSNEW
- List a PDS member – this method uses the dtail (cf Linux tail command) starting with line 1 (-n +1)
-
- dtail -n +1 “ZPDT.SYSTEM.JCL(DSNTEP2)”
-
IBMUSER:/u/ibmuser: >dtail -n +1 "ZPDT.SYSTEM.JCL(DSNTEP2)" //IBMUSERJ JOB 'DB2 REST',CLASS=A,MSGCLASS=H,NOTIFY=&SYSUID //* //DSNTEP2 EXEC PGM=IKJEFT01,COND=(4,LT) //STEPLIB DD DISP=SHR,DSN=SDB2T.SDSNEXIT // DD DISP=SHR,DSN=SDB2T.SDSNLOAD //SYSTSPRT DD SYSOUT=* //SYSPRINT DD SYSOUT=* //SYSTSIN DD *,SYMBOLS=EXECSYS DSN S(DB2T) RUN PROGRAM(DSNTEP2) PLAN(DSNTEP12) + LIBRARY('SDB2T.RUNLIB.LOAD') END /* //SYSIN DD * SELECT CURRENT SERVER,CURRENT MEMBER,CURRENT TIMESTAMP FROM SYSIBM.SYSDUMMY1 ; /*
- Submit a batch job – this returns (in the job_id variable) the JOBnnnnn ID – e.g. JOB00637.
-
- job_id=$( jsub “ZPDT.SYSTEM.JCL(DSNTEP2)” )
-
- Check batch job status. BEWARE! If you issue this command without parameters, it will attempt to list every job in the system – active and completed. Make sure that you at least restrict it to the job owner (IBMUSER in this case), and better is to include a job name profile (as you would in SDSF). The columnar output headers are: Owner, Job/TSU/STC name, Job/TSU/STC id, status (AC = active, CC = finished, ABEND = abended, JCLERR = JCL error) and resulting code (active = ? (not yet set), ABEND = System or User Abend code, CC = max condition code):
-
- jls /IBMUSER/IBMUSER*
-
IBMUSER:/u/ibmuser: >jls /IBMUSER IBMUSER IBMUSER TSU00618 AC ? IBMUSER IBMUSERZ JOB00445 CC 0008 IBMUSER IBMUSERA JOB00447 CC 0000 : IBMUSER IBMUSERJ JOB00601 CC 0000 IBMUSER IBMUSER TSU00595 ABEND S222 IBMUSER IBMUSERJ JOB00637 CC 0000 ISF767I Request completed.
- Check a specific batch job status:
-
- jls JOB00637
-
IBMUSER:/u/ibmuser: >jls JOB00637 IBMUSER IBMUSERJ JOB00637 CC 0000 ISF767I Request completed.
- List batch job output DD names
-
- ddls JOB00637. This gives a columnar list – the headings could have been something like step name, proc step name, DD name, record format, logical record length, number of records.
-
IBMUSER:/u/ibmuser: >ddls JOB00637 JES2 - JESMSGLG UA 133 20 JES2 - JESJCL V 136 10 JES2 - JESYSMSG VA 137 28 DSNTEP2 - SYSTSPRT VBA 137 8 DSNTEP2 - SYSPRINT FBA 133 12 ISF767I Request completed.
- Retrieve batch job output
-
- pjdd JOB00601 DSNTEP2 SYSPRINT
-
IBMUSER:/u/ibmuser: >pjdd JOB00637 DSNTEP2 SYSPRINT 1 1PAGE 1 ***INPUT STATEMENT: SELECT CURRENT SERVER,CURRENT MEMBER,CURRENT TIMESTAMP FROM SYSIBM.SYSDUMMY1 ; +----------------------------------------------------------+ | | | | +----------------------------------------------------------+ 1_| DB2T | | 2021-01-22-11.46.33.446875 | +----------------------------------------------------------+ 0SUCCESSFUL RETRIEVAL OF 1 ROW(S)
- Get console output. There are a lot of different options, but the shortest about of time that it will retrieve is 10 minutes. This could be quite a lot of data in a busy service.
-
- pcon -r
-
- Issue a console command. Note that at the time of writing, a command prefixed with a hyphen (“-“) will fail. This can be a problem for customers who use hyphen-db2 as the subsystem command prefix – e.g. -DB2T DIS GROUP.
-
- opercmd “d prog,apf”
-
Example Python Scripts
Example 1 – Working With Jobs
The following is a short Python script which brings together most of the commands from above. Things to note:
- We checked that the dataset member with the JCL exists before we submit it
- The Jobs.list() returns one entry for the job submitted because we used the JOBxxxxx form returned by Jobs.submit().
- The ‘status’ dictionary entry is the current status of the batch job. ‘AC’ = active, ‘CC’ = finished with condition code set in ‘return’, ‘JCLERR’ = JCL error and ‘ABEND’ = system or user abend (see ‘return’)
import time from zoautil_py import Datasets,Jobs,OperatorCmd,ZSystem # PDS and member to submit the JCL from jclpds = "ZPDT.SYSTEM.JCL" mem = "DSNTEP2" jclmem = '%s(%s)' % (jclpds,mem) chkmem = Datasets.list_members(jclmem, True) if (chkmem == mem): print("Submitting the following JCL:") print(Datasets.read(jclmem)) job_id = Jobs.submit(dataset=jclmem) job_finished = False job_failed = False while (not(job_finished)): jlist = Jobs.list(job_id) state = jlist[0]['status'] if (state != 'AC'): if (state == 'CC'): print("Job finished %s=%s" % (state,jlist[0]['return'])) elif (state == 'JCLERR'): print("Job failed with JCL ERROR") job_failed = True else: print("Job status now %s (%s)" % (state,jlist[0]['return'])) job_finished = True else: time.sleep(1) if (not job_failed): print("Job %s output:" % job_id) print(Jobs.read_output(job_id,"DSNTEP2","SYSPRINT")) else: print('%s NOT FOUND in %s' % (mem,jclpds))
Here’s what it looks like when run:
IBMUSER:/u/ibmuser: >python3 prCurrent.py Submitting the following JCL: //IBMUSERJ JOB 'DB2 REST',CLASS=A,MSGCLASS=H,NOTIFY=&SYSUID //* //DSNTEP2 EXEC PGM=IKJEFT01,COND=(4,LT) //STEPLIB DD DISP=SHR,DSN=SDB2T.SDSNEXIT // DD DISP=SHR,DSN=SDB2T.SDSNLOAD //SYSTSPRT DD SYSOUT=* //SYSPRINT DD SYSOUT=* //SYSTSIN DD * DSN S(DB2T) RUN PROGRAM(DSNTEP2) PLAN(DSNTEP12) + LIBRARY('SDB2T.RUNLIB.LOAD') END /* //SYSIN DD * SELECT CURRENT SERVER,CATALOG_LEVEL,CURRENT TIMESTAMP FROM SYSIBM.SYSDUMMY1 ; /* Job finished CC=0000 Job JOB00688 output: 1 1PAGE 1 ***INPUT STATEMENT: SELECT CURRENT SERVER,CATALOG_LEVEL,CURRENT TIMESTAMP FROM SYSIBM.SYSDUMMY1 ; +----------------------------------------------------------------------+ | | CATALOG_LEVEL | | +----------------------------------------------------------------------+ 1_| DB2T | V12R1M500 | 2021-01-23-11.18.19.059917 | +----------------------------------------------------------------------+ 0SUCCESSFUL RETRIEVAL OF 1 ROW(S)
Example 2 – Issuing Console Commands
This next example issues a dynamic APF display command and then reports the output:
import sys from zoautil_py import Datasets,Jobs,OperatorCmd,ZSystem resp = OperatorCmd.execute("D PROG,APF") rlines = resp['message'].split("\n") for line in rlines: print(line)
Here’s what the output looks like when run:
IBMUSER:/u/ibmuser: >python3 prAPF.py S0W1 2021024 12:30:37.23 ISF041I CONSOLE NAME IBMUSER MODIFIED S0W1 2021024 12:30:37.23 ISF031I CONSOLE IBMUSER$ ACTIVATED S0W1 2021024 12:30:37.23 -D PROG,APF S0W1 2021024 12:30:37.27 CSV450I 12.30.37 PROG,APF DISPLAY 656 FORMAT=DYNAMIC ENTRY VOLUME DSNAME 1 C4RES1 SYS1.LINKLIB 2 C4RES1 SYS1.SVCLIB 3 C4RES1 SYS1.SHASLNKE : 56 ZPDT01 ZPDT.VTAMLIB 57 *SMS* SDB2T.RUNLIB.LOAD
Note that we don’t have the same challenge with the OperatorCmd.execute() as with “opercmd” implementation – i.e. if we change the command issued in the above example to “-DB2T DIS DDF DETAIL” we get the following:
IBMUSER:/u/ibmuser: >python3 prDDF.py S0W1 2021024 12:40:37.53 ISF041I CONSOLE NAME IBMUSER MODIFIED S0W1 2021024 12:40:37.53 ISF031I CONSOLE IBMUSER$ ACTIVATED S0W1 2021024 12:40:37.53 --DB2T DIS DDF DETAIL S0W1 2021024 12:40:37.55 STC00464 DSNL080I -DB2T DSNLTDDF DISPLAY DDF REPORT FOLLOWS: DSNL081I STATUS=STARTD DSNL082I LOCATION LUNAME GENERICLU DSNL083I DB2T -NONE -NONE DSNL084I TCPPORT=5000 SECPORT=0 RESPORT=5001 IPNAME=DB2T DSNL085I IPADDR=::192.168.2.25 DSNL086I SQL DOMAIN=s0w1.zpdt.local DSNL090I DT=I CONDBAT= 10000 MDBAT= 200 DSNL092I ADBAT= 0 QUEDBAT= 0 INADBAT= 0 CONQUED= 0 DSNL093I DSCDBAT= 0 INACONN= 0 IUDBAT= 0 DSNL105I CURRENT DDF OPTIONS ARE: DSNL106I PKGREL = COMMIT DSNL106I SESSIDLE = 001440 DSNL099I DSNLTDDF DISPLAY DDF REPORT COMPLETE
Observations
This evolving set of utilities provides a whole set of additional capabilities into the USS environment that have not previously been available to scripting languages like Java and Python, let alone shell scripts. As an enabling technology, it provides a big uplift in capability for the environment and could make something like a Jenkins node running in USS significantly more capable.
We haven’t investigated driving these capabilities from Jenkins and Groovy, as our main focus has been on ansible. We will try and add this in to the end of the current blog run.
Next Time
In the next blog in this series, we’ll get Ansible installed and run some simple playbooks using the z/OS core and zOSMF collections.
View all blogs in James Gill’s Ansible series.
Hi James,
At our site we have python on z installed but it is not a regular tool yet. Think your example 1 is ‘sub’ and ‘sd:st;pre IBMUS*’ combined in a single screen output. Nice! Inspired me to rewrite it a bit and to practice these ZOAU modules. Hope you like it.
Kind regards,
Richard Groot
from zoautil_py.jobs import submit, read_output, list_dds
from zoautil_py.datasets import read
from time import sleep
def printjob(jobid):
stepds = list_dds(jobid)
for item in stepds:
print(read_output(jobid, item[“stepname”], item[“dataset”]))
def subsd(fname):
print(120 * ‘*’)
print(read(fname))
print(120 * ‘*’)
joppie = submit(fname, wait=True, timeout=20)
print(“Job name: “, joppie.name, ” Job ID: “, joppie.id, ” Job owner: “, joppie.owner)
print(120 * ‘*’)
i=1
while joppie.status == ‘AC’:
sleep(0.1)
print(‘\r Running job’, i * ‘.’, end =”, flush=True)
i+=1
joppie.refresh()
printjob(joppie.id)
print(120 * ‘*’)
subsd(‘GROOTRR.TOOLS.JCL(DB2DISP)’)