Feb
04
2021 Posted by James Gill

Ansible on z/OS – 2 – z/OS Open Automation (ZOA) Utilities

Introduction

In the second of this series, we look at the installation and execution of the second ansible pre-req: IBM z/OS Open Automation (ZOA) Utilities. These provide a raft of additional functionality to the Python world, include z/OS dataset handling, batch job submission and tracking and z/OS console use. These features are in turn used by ansible.

 

Installing ZOA Utilities

These utilities are installed via CBPDO and SMP/E as with other z/OS software. They can be ordered separately (Program ID is 5698-PA1) or as part of other tooling – e.g. Dependency Based Builder (DBB).

The end result should be a new path in USS:

/usr/lpp/IBM/zoautil

 

Configuration

To make use of these utilities, we will need to configure them into the USS environment. We did this by updating a single userid profile, by adding the following:

export PYTHON_HOME=/usr/lpp/IBM/cyp/v3r8/pyz
export ZOA=/usr/lpp/IBM/zoautil
export _BPXK_AUTOCVT=ON
export PATH=$PATH:$PYTHON_HOME/bin
export LIBPATH=$LIBPATH:$PYTHON_HOME/lib
# ZOA Utilities
export PATH=$PATH:$ZOA/bin
export LIBPATH=$LIBPATH:$ZOA/lib
export MANPATH=$MANPATH:$ZOA/docs/%L
# Python support for ZOA Utilities
export PYTHONPATH=$ZOA/lib

Note that the MANPATH export adds the ZOA Utilities into the scope of the “man” (manual) Unix command which provides syntax and parameters for each of the commands supplied by this feature.

Note also that the last entry (PYTHONPATH) is in place for ansible.

 

Documentation

The documentation for ZOA Utilities is all online and can be found here:

https://www.ibm.com/support/knowledgecenter/SSKFYE_1.0.3/zoautil_overview.html

The Python doc is here (for 1.0.3):

https://www.ibm.com/support/knowledgecenter/SSKFYE_1.0.3/python_doc_zoautil/index.html?view=embed

The product is updated (so far) by PTFs, so make sure that you’re looking at the correct version (selectable with the “Change version or product” pull down on the top left of the  web page).

 

Example Commands

As well as Python modules and Java interfaces, the ZOA Utilities are implemented as shell commands as well. These provide extensions to the USS shell environment. Here are some examples:

 

  1. List PDS members
      1. mls “ZPDT.SYSTEM.JCL”
IBMUSER:/u/ibmuser: >mls "ZPDT.SYSTEM.JCL"
ACSTEST
   :
DSNTEP2
   :
ZFSNEW
  1. List a PDS member – this method uses the dtail (cf Linux tail command) starting with line 1 (-n +1)
      1. dtail -n +1 “ZPDT.SYSTEM.JCL(DSNTEP2)”
IBMUSER:/u/ibmuser: >dtail -n +1 "ZPDT.SYSTEM.JCL(DSNTEP2)"
//IBMUSERJ JOB 'DB2 REST',CLASS=A,MSGCLASS=H,NOTIFY=&SYSUID
//*
//DSNTEP2 EXEC PGM=IKJEFT01,COND=(4,LT)
//STEPLIB  DD  DISP=SHR,DSN=SDB2T.SDSNEXIT
//         DD  DISP=SHR,DSN=SDB2T.SDSNLOAD
//SYSTSPRT DD  SYSOUT=*
//SYSPRINT DD  SYSOUT=*
//SYSTSIN  DD  *,SYMBOLS=EXECSYS
  DSN S(DB2T)
    RUN PROGRAM(DSNTEP2) PLAN(DSNTEP12) +
        LIBRARY('SDB2T.RUNLIB.LOAD')
    END
/*
//SYSIN    DD  *
  SELECT CURRENT SERVER,CURRENT MEMBER,CURRENT TIMESTAMP
  FROM SYSIBM.SYSDUMMY1
  ;
/*
  1. Submit a batch job – this returns (in the job_id variable) the JOBnnnnn ID – e.g. JOB00637.
      1. job_id=$( jsub “ZPDT.SYSTEM.JCL(DSNTEP2)” )
  2. Check batch job status. BEWARE! If you issue this command without parameters, it will attempt to list every job in the system – active and completed. Make sure that you at least restrict it to the job owner (IBMUSER in this case), and better is to include a job name profile (as you would in SDSF). The columnar output headers are: Owner, Job/TSU/STC name, Job/TSU/STC id, status (AC = active, CC = finished, ABEND = abended, JCLERR = JCL error) and resulting code (active = ? (not yet set), ABEND = System or User Abend code, CC = max condition code):
      1. jls /IBMUSER/IBMUSER*
IBMUSER:/u/ibmuser: >jls /IBMUSER
IBMUSER  IBMUSER  TSU00618 AC       ?
IBMUSER  IBMUSERZ JOB00445 CC       0008
IBMUSER  IBMUSERA JOB00447 CC       0000
   :
IBMUSER  IBMUSERJ JOB00601 CC       0000
IBMUSER  IBMUSER  TSU00595 ABEND    S222
IBMUSER  IBMUSERJ JOB00637 CC       0000
ISF767I Request completed.
  1. Check a specific batch job status:
      1. jls JOB00637
IBMUSER:/u/ibmuser: >jls JOB00637
IBMUSER  IBMUSERJ JOB00637 CC       0000
ISF767I Request completed.
  1. List batch job output DD names
      1. ddls JOB00637. This gives a columnar list – the headings could have been something like step name, proc step name, DD name, record format, logical record length, number of records.
IBMUSER:/u/ibmuser: >ddls JOB00637
JES2     -        JESMSGLG UA         133       20
JES2     -        JESJCL   V          136       10
JES2     -        JESYSMSG VA         137       28
DSNTEP2  -        SYSTSPRT VBA        137        8
DSNTEP2  -        SYSPRINT FBA        133       12
ISF767I Request completed.
  1. Retrieve batch job output
      1. pjdd JOB00601 DSNTEP2 SYSPRINT
IBMUSER:/u/ibmuser: >pjdd JOB00637 DSNTEP2 SYSPRINT
1
1PAGE    1
 ***INPUT STATEMENT:
    SELECT CURRENT SERVER,CURRENT MEMBER,CURRENT TIMESTAMP
    FROM SYSIBM.SYSDUMMY1
    ;
     +----------------------------------------------------------+
     |                  |          |                            |
     +----------------------------------------------------------+
   1_| DB2T             |          | 2021-01-22-11.46.33.446875 |
     +----------------------------------------------------------+
0SUCCESSFUL RETRIEVAL OF          1 ROW(S)
  1. Get console output. There are a lot of different options, but the shortest about of time that it will retrieve is 10 minutes. This could be quite a lot of data in a busy service.
      1. pcon -r
  2. Issue a console command. Note that at the time of writing, a command prefixed with a hyphen (“-“) will fail. This can be a problem for customers who use hyphen-db2 as the subsystem command prefix – e.g. -DB2T DIS GROUP.
      1. opercmd “d prog,apf”

 

Example Python Scripts

Example 1 – Working With Jobs

The following is a short Python script which brings together most of the commands from above. Things to note:

  • We checked that the dataset member with the JCL exists before we submit it
  • The Jobs.list() returns one entry for the job submitted because we used the JOBxxxxx form returned by Jobs.submit().
  • The ‘status’ dictionary entry is the current status of the batch job. ‘AC’ = active, ‘CC’ = finished with condition code set in ‘return’, ‘JCLERR’ = JCL error and ‘ABEND’ = system or user abend (see ‘return’)
import time
from zoautil_py import Datasets,Jobs,OperatorCmd,ZSystem
# PDS and member to submit the JCL from
jclpds = "ZPDT.SYSTEM.JCL"
mem = "DSNTEP2"
jclmem = '%s(%s)' % (jclpds,mem)
chkmem = Datasets.list_members(jclmem, True)
if (chkmem == mem):
      print("Submitting the following JCL:")
      print(Datasets.read(jclmem))
      job_id = Jobs.submit(dataset=jclmem)
      job_finished = False
      job_failed = False
      while (not(job_finished)):
            jlist = Jobs.list(job_id)
            state = jlist[0]['status']
            if (state != 'AC'):
                  if (state == 'CC'):
                        print("Job finished %s=%s" % (state,jlist[0]['return']))
                  elif (state == 'JCLERR'):
                        print("Job failed with JCL ERROR")
                        job_failed = True
                  else:
                        print("Job status now %s (%s)" % (state,jlist[0]['return']))
                  job_finished = True
            else:
                  time.sleep(1)
      if (not job_failed):
            print("Job %s output:" % job_id)
            print(Jobs.read_output(job_id,"DSNTEP2","SYSPRINT"))
else:
      print('%s NOT FOUND in %s' % (mem,jclpds))

 

Here’s what it looks like when run:

IBMUSER:/u/ibmuser: >python3 prCurrent.py
Submitting the following JCL:
//IBMUSERJ JOB 'DB2 REST',CLASS=A,MSGCLASS=H,NOTIFY=&SYSUID
//*
//DSNTEP2 EXEC PGM=IKJEFT01,COND=(4,LT)
//STEPLIB  DD  DISP=SHR,DSN=SDB2T.SDSNEXIT
//         DD  DISP=SHR,DSN=SDB2T.SDSNLOAD
//SYSTSPRT DD  SYSOUT=*
//SYSPRINT DD  SYSOUT=*
//SYSTSIN  DD  *
  DSN S(DB2T)
    RUN PROGRAM(DSNTEP2) PLAN(DSNTEP12) +
        LIBRARY('SDB2T.RUNLIB.LOAD')
END
/*
//SYSIN    DD  *
  SELECT CURRENT SERVER,CATALOG_LEVEL,CURRENT TIMESTAMP
  FROM SYSIBM.SYSDUMMY1
  ;
/*
Job finished CC=0000
Job JOB00688 output:
1
1PAGE    1
 ***INPUT STATEMENT:
    SELECT CURRENT SERVER,CATALOG_LEVEL,CURRENT TIMESTAMP
    FROM SYSIBM.SYSDUMMY1
    ;
  +----------------------------------------------------------------------+
  |             |      CATALOG_LEVEL        |                            |
  +----------------------------------------------------------------------+
1_| DB2T        | V12R1M500                 | 2021-01-23-11.18.19.059917 |
  +----------------------------------------------------------------------+
0SUCCESSFUL RETRIEVAL OF          1 ROW(S)

 

Example 2 – Issuing Console Commands

This next example issues a dynamic APF display command and then reports the output:

import sys
from zoautil_py import Datasets,Jobs,OperatorCmd,ZSystem
resp = OperatorCmd.execute("D PROG,APF")
rlines = resp['message'].split("\n")
for line in rlines:
print(line)

 

Here’s what the output looks like when run:

IBMUSER:/u/ibmuser: >python3 prAPF.py
S0W1      2021024  12:30:37.23             ISF041I CONSOLE NAME IBMUSER MODIFIED
S0W1      2021024  12:30:37.23             ISF031I CONSOLE IBMUSER$ ACTIVATED
S0W1      2021024  12:30:37.23            -D PROG,APF
S0W1      2021024  12:30:37.27             CSV450I 12.30.37 PROG,APF DISPLAY 656
                                           FORMAT=DYNAMIC
                                           ENTRY VOLUME DSNAME
                                              1  C4RES1 SYS1.LINKLIB
                                              2  C4RES1 SYS1.SVCLIB
                                              3  C4RES1 SYS1.SHASLNKE
                                              :
                                             56  ZPDT01 ZPDT.VTAMLIB
                                             57  *SMS*  SDB2T.RUNLIB.LOAD

Note that we don’t have the same challenge with the OperatorCmd.execute() as with “opercmd” implementation – i.e. if we change the command issued in the above example to “-DB2T DIS DDF DETAIL” we get the following:

IBMUSER:/u/ibmuser: >python3 prDDF.py
S0W1      2021024  12:40:37.53             ISF041I CONSOLE NAME IBMUSER MODIFIED
S0W1      2021024  12:40:37.53             ISF031I CONSOLE IBMUSER$ ACTIVATED
S0W1      2021024  12:40:37.53            --DB2T DIS DDF DETAIL
S0W1      2021024  12:40:37.55  STC00464   DSNL080I  -DB2T DSNLTDDF DISPLAY DDF REPORT FOLLOWS:
                                           DSNL081I STATUS=STARTD
                                           DSNL082I LOCATION           LUNAME            GENERICLU
                                           DSNL083I DB2T               -NONE             -NONE
                                           DSNL084I TCPPORT=5000  SECPORT=0     RESPORT=5001  IPNAME=DB2T
                                           DSNL085I IPADDR=::192.168.2.25
                                           DSNL086I SQL    DOMAIN=s0w1.zpdt.local
                                           DSNL090I DT=I  CONDBAT=  10000 MDBAT=  200
                                           DSNL092I ADBAT=    0 QUEDBAT=      0 INADBAT=      0 CONQUED=      0
                                           DSNL093I DSCDBAT=      0 INACONN=      0 IUDBAT=      0
                                           DSNL105I CURRENT DDF OPTIONS ARE:
                                           DSNL106I PKGREL = COMMIT
                                           DSNL106I SESSIDLE = 001440
                                           DSNL099I DSNLTDDF DISPLAY DDF REPORT COMPLETE

 

Observations

This evolving set of utilities provides a whole set of additional capabilities into the USS environment that have not previously been available to scripting languages like Java and Python, let alone shell scripts. As an enabling technology, it provides a big uplift in capability for the environment and could make something like a Jenkins node running in USS significantly more capable.

We haven’t investigated driving these capabilities from Jenkins and Groovy, as our main focus has been on ansible. We will try and add this in to the end of the current blog run.

 

Next Time

In the next blog in this series, we’ll get Ansible installed and run some simple playbooks using the z/OS core and zOSMF collections.

 

View all blogs in James Gill’s Ansible series.

 

One thought on “Ansible on z/OS – 2 – z/OS Open Automation (ZOA) Utilities”

  1. Richard Groot says:

    Hi James,

    At our site we have python on z installed but it is not a regular tool yet. Think your example 1 is ‘sub’ and ‘sd:st;pre IBMUS*’ combined in a single screen output. Nice! Inspired me to rewrite it a bit and to practice these ZOAU modules. Hope you like it.

    Kind regards,

    Richard Groot

    from zoautil_py.jobs import submit, read_output, list_dds
    from zoautil_py.datasets import read
    from time import sleep

    def printjob(jobid):
    stepds = list_dds(jobid)
    for item in stepds:
    print(read_output(jobid, item[“stepname”], item[“dataset”]))

    def subsd(fname):

    print(120 * ‘*’)
    print(read(fname))
    print(120 * ‘*’)

    joppie = submit(fname, wait=True, timeout=20)

    print(“Job name: “, joppie.name, ” Job ID: “, joppie.id, ” Job owner: “, joppie.owner)
    print(120 * ‘*’)

    i=1
    while joppie.status == ‘AC’:
    sleep(0.1)
    print(‘\r Running job’, i * ‘.’, end =”, flush=True)
    i+=1
    joppie.refresh()

    printjob(joppie.id)
    print(120 * ‘*’)

    subsd(‘GROOTRR.TOOLS.JCL(DB2DISP)’)

Leave a Reply

Your email address will not be published. Required fields are marked *

« | »
Have a Question?

Get in touch with our expert team and see how we can help with your IT project, call us on +44(0) 870 2411 550 or use our contact form…