2023 Posted by James Gill

IBM Open Enterprise SDK for Python – ZIIPing Ahead

Introduction

A little while ago we looked at Ansible on z/OS, and as part of this, we also installed and had a quick look at Open Enterprise Python and the ZOA Utilities. Ansible is hugely useful for automating management and deployment tasks, so the importance of Python on the platform is already high. Python is also the driving language behind much of the machine learning (ML) software stack, raising its profile further.

An early criticism of the language release on z/OS was that whilst other interpretive languages – well, Java – were ZIIP offloaded (a circuitous route from ZAAP – remember them? – to ZIIP), Python was a hard burn on General Processors (GPs) instead, with a follow up cost impact on the four hour rolling average (4HRA) charging model.

With a huge step forwards, IBM has released Open Enterprise SDK for Python 3.11.3, which is now 70% ZIIP offloadable. This is a massive move, a huge statement about the importance of Python on the platform, and went by largely unnoticed by everyone. With this in mind, here’s a quick skim through, with a conclusion (spoiler alert) of Wow!

 

ZIIP Enabled?

The ZIIP processor was introduced with the z9 all the way back in 2005. This, along with the ZAAP and IFL processor types, was essentially a full blown and full speed General Purpose (GP) CPU, but with a couple of instructions disabled in the microcode. The goal was initially to support offloading some of the expensive activities in Db2 for z/OS onto a processor that was zero-rated for software licensing. Over time, the ZIIP was used by more IBM and Vendor products to help reduce the potential licensing impacts of products and tooling.

Some products which make use of ZIIP include:

• Db2 for z/OS
• TCPIP stack
• MQ Series
• Java (workload moved from ZAAP)
• z/OS Container Extensions (zCX – running Linux on Z container workloads in z/OS)

There are a couple of important things to know about ZIIPs:

1. ZIIP eligible workload is poured into a pipe, and will wait for (a default) 3.5ms for a ZIIP to be available to process the work. If this doesn’t happen, the task falls back to GP, where it adds to the 4HRA workload and therefore the license costs
2. Other users of ZIIP can be pretty important – e.g. Db2 for z/OS logging engines. ZIIPs should not be run 100% in-use, as this implies some work will be falling back to GPs. If this is Db2 log writes, it’s not just the 4HRA impact, but the wait of 3.5ms for each log write would be a massive drag on performance.
3. ZIIPs are always full capacity processors. If you are running a sub capacity configuration, where GPs are throttled, any ZIIPs that are enabled will be full capacity. As they have no impact on the software licensing charge, this can represent a fairly huge performance and capacity lift – for ZIIPable workloads.

 

What Do I Need To Use It?

As well as the Open Enterprise SDK for Python 3.11.3 release package, there is an APAR which needs to be applied to BCP (part of z/OS) – and this will require an IPL (with CLPA) to deliver. Details:

• APAR : OA63406
• PTF :
  • z/OS 2.4 : UJ92511
  • z/OS 2.5 : UK92512

 

Where Can I Get It?

IBM Open Enterprise SDK for Python has its own web page, here:

https://www.ibm.com/products/open-enterprise-python-zos

The pax version can be downloaded directly from this page. The SMP/E installable version will need to be ordered via ShopZ.

 

How Do I Install It?

We worked with the pax delivery (as opposed to SMP/E) version. To use this, download from the website (see above) – you’ll be asked to accept a license as part of the download.

Once downloaded, push to your (largish) maintenance / working OMVS / USS directory. In USS (OMVS), unpack the pax image:

• Unpack the pax image:
  • cd /u/ibmuser/maint
  • pax -r -ppAx -f HAMB3B0.nonsmpe.pax.Z
    • this unpacks into a subdirectory usr/lpp/IBM/cyp/v3r11
  • Create the target ZFS filesystem and aggregate, then
  • Create the mountpoint:
    • mkdir -p /usr/lpp/IBM/cyp/v3r11
  • Mount the new ZFS filesystem:
    • tso mount filesystem(‘IBMUSER.PYTHON.V3R11.ZFS’) mountpoint(‘/usr/lpp/IBM/cyp/v3r11’) mode(RDWR) type(ZFS)
  • Add this mount to the OMVS configuration to make it permanent:
    • PARMLIB(BPXPRMPY)
    • Update SYS1.PARMLIB(IEASYSxx) OMVS= to include PY
  • Populate the runtime library:
    • cd /u/ibmuser/maint
    • cp -r -p usr/lpp/IBM/cyp/v3r11/* /usr/lpp/IBM/cyp/v3r11/
    • NB the “-p” preserves permission bits, including APF authorisation

Verify the installation:

• Update your profile to include Python 3.11.3 in your PATH and LIBPATH:
  • In our case, update /u/ibmuser/.profile and add:
    • export PYTHON_HOME=/usr/lpp/IBM/cyp/v3r11
    • export PATH=$PYTHON_HOME/bin:$PATH
    • export LIBPATH=$PYTHON_HOME/lib:$LIBPATH
  • Quick check in USS (tso omvs):
    • python3 –version

IBMUSER:/u/ibmuser: >python3 --version
Python 3.11.3

 

But Is It Really 70% ZIIP Offloadable?

The following is a simple CPU burner, with an occasional message being pushed out. The idea is to churn some CPU and keep an eye on progress in SDSF, with a slightly rearranged display so that we can see how much of the burn is ZIIP and how much is GP:
/u/ibmuser/counter.py

#!python
count = 0
while count < 100000000:
if count % 9000 == 0:
print(count)
count += 1
print("Done")


 

And the batch job to drive it:

//IBMUSERA JOB 'PYTHON',CLASS=A,MSGCLASS=H,NOTIFY=&SYSUID
//*
//PYTHON EXEC PGM=IKJEFT01,DYNAMNBR=20,COND=(4,LT)
//SYSTSPRT DD SYSOUT=*
//STDOUT DD SYSOUT=*
//STDERR DD SYSOUT=*
//SYSTSIN DD *
BPXBATCH SH cd /u/ibmuser; +
python counter.py;
/*

Example SDSF output:

 

From this run, here are some numbers:

 

That’s looking pretty good!

 

Conclusion

If you’re running Python on z/OS, and it isn’t IBM Open Enterprise SDK for Python 3.11.3 (or higher), you should seriously be looking at this!

 

« Previous | Next »

• Share