Running Hercules and MVS3.8J in a Container

First why would you want to ?. There are many use cases, a few examples would be

Are there any benefits to using a container ?. For me yes, one container image can be customised in many ways at run time which suits my needs better than manually editing config files when I want to run it in a different configuration.

My workflow would be

  1. anything new or that might break something I test in the container; I keep the JCL and source code etc. as text files on a dev machine and submit (via the 3505 socket card reader) those to the container to create any needed datasets and run compiles etc. to test things
  2. Only when I am happy with them will I run the same JCL decks against my "master" on disk system, merge the shadows files into the main disks, and build a new container image
  3. push the new container to my local docker registry
  4. on any machines with docker, simply pull the new container image and it is on all my machines.

Containers will not be suitable for everyone, and if you are only interested in running MVS3.8J to play with rather than to make any major changes to then there is a supported CE container that would suit you better that may still be available at the MVS/CE github location https://github.com/MVS-sysgen/sysgen

Limitations, not really limitations with a container are before you mention them

What you need to create your own MVS3.8J container

At a minimum you will need

There are a few things to note, the major one being that when doing the docker build everything is expected to be under your working directory. For example you cannot copy your hercules dasd etc. from "/some/other/dir" it must be from "dir" under your work directory. That is a pain, but as my configs etc. for a container are different from my on-disk hercules system and static I just copy any changed dasd from my real system into the dasd directory under my build directory as needed without changing anything else.

A hands-off startup script is needed

The entire point of a container is that you just run it and everything is available; there must be no need to enter any commands. So you need a hands-off startup script that will start everything up, replying to any prompts needed by hercules or the consoles, and ending up with a running container you can simply start a TSO session to and start using it.

If you have been using hercules to run MVS3.8J for a while you probably have one. If you are using one of the 'packages' that expects a desktop and fires up lots of x3270 sessions that will not work, you do not want lots of GUI desktop packages in a container and my example Dockerfile is to build for a headless environment.

In the examples my example startup script is copied into place by my example Dockerfile as container_start_system.sh, and is designed for TK4-; however it was also used without issue on a normal TK3 system (less device 01F and one screen session) as that is what it was created for origionally.

A CMD entry point script that never ends

When the container starts up it will run a command as defined in the CMD entry, that is expected to start the 'critical service' the container provides and should it stop running (ie: when the a script completes and ends) that is considered a fatal error for the container. So whatever CMD startup script you use must after it has done all its work keep running, going into a endless loop to sleep for a few hours and loop back to sleeping for a few hours achieves that.

While not best practise, as that script must run, I also use that to perform all customisation of the hercules configuration file devices (based on environment variables passed to the container) prior to this script actually running the script above that will actually start hercules and the MVS3.8J system. This allows things like ip-addresses and ports on devices in the hercules config file to be easily customised allowing multiple copies of the container to run on a single host without port conflicts for things like CTC device ip connections.
I also use mine to based on environment variables passed to the container submit a JCL deck after hercules is started to perform further customisations, such as creating a new NJE38 config to match all the device editing done.
However the example I provide here just allows customisation of the tzoffset variable in the hercules config file to show it is possible (the huge script I have is unlikely to match you needs so just a small example). But customisation options are only limited by your imagination; and possibly an environment variable limit I have not hit yet.

Lets also have a health check script

While having a health check script is of course optional, for running MVS3.8J in a container with a hands-off startup it provides the following benefits

I have included an example healthcheck script based on my TK4- customised environment. It makes sure hercules and mvs38j are running OK, by checking the number of screen sessions running are correct, that the 3270 (tso) and 3505 (card reader) ports are listening, and uses curl to check port 3270 responds to connections to make sure that is not hung and TSO should be available.

The important thing to note in the hercules environment which you will see in my Dockerfile example is that there must be a huge delay before the first healthcheck poll. That is to give time for hercules to actually startup so the healthcheck script does not interfere with the startup, there are a lot of 'sleep' commands in my startup script to let things start in a staggered/clean order and we want to wait for the startup to copmplete fully.

The easy bit, the Dockerfile

First, do not use EXPOSE lines in your Dockerfile. If used they will start a docker-proxy process (if using docker) for every port listed and just waste machine resources. Using port mapping ensures only the ports you need are opened and have processes running for them.
And that is important ! as if you want two containers running on the same machine you do not want each container trying to use/expose the same ports; so always rely on mapping rather using expose within the container.

Ok, by now you should have a hands-off startup script, a CMD wrapper script, and a health check script in place. Now you can actually build a Dockerfile and create your image.

The Dockerfile I use myself is a little more verbose than the example I provide here as I copy locally compiled hercules binaries in and a build script to generate my Dockerfile depending on whether I am building for debian or fedora and whether I am using spinhawk or hyperion binaries etc.
You do not care about any of that; my example Dockerfile provided here just assumes you are going to use the distribution repo version of 'hercules' and debian:12-slim as the base for the cntainer to keep the example simple.

Also note that a temporary build environment target is used, and the near final step is to copy that as a single layer to the final container image; with the size of the dasd volumes that simple procedure can shave 500Mb off the size of the final container image.

My example Dockerfile is available here for a quick look at it, but it is also included in the example TGZ file (keep reading) along with all the scripts so don't cut/paste this one.

The startup script(s) needed

I have two of these. The history behind that is that for decades I have had a headless startup script for my installs of mvs3.8j and as it just works I chose to use that with mimimal changes to save a lot of debugging within the container.

The second script is a startup "wrapper" script required, and because it can do so much is a lot larger messier script. The main reasons it is required are

A headless startup script is needed

The actual startup script needs to be able to start hercules and the mvs3.8j system including replying to any hercules and console prompts needed.

My startup script I have tweaked slightly between TK3 and TK4-, little things like the IPL volume changing, config file and directory locations; but those are set as variables at the start of the script so easily customised for any site by just changing a few lines.
Although there were a few more tweaks to make it usable in the container so consider my example as a container script rather than a headless startup script.

It is important to note that in my examples I have the following consoles defined and the startup script is coded to expect these. These are a slightly tweaked setup from the TK4- distribution (TK3 from memory did not use 01F and had 009 as the telnet hardcopy interface and no console attached to the hercules interface itself) 

# 3270 consoles on 010 and 011
# 009 is used by hercules console interface
# 01F is a telnet hardcopy interface
0009 3215-C
0010 3270 CONS
0011 3270 CONS
001F 3215

It will run after the initial wrapper script has done any customistions to the hercules config file needed to

A working example of this script is in the TGZ file so read on.

A startup wrapper script is needed

OK you could stuff all the wrapper stuff into the hercules/mvs38j startup script but I have decided to keep all the customisations needed in a seperate script as most customisations need to be done before hercules is started.

My container wrapper start up script uses environment variables passed to the container to customise the hercules configuration file before hercules is started; things like enabling/configuring devices or commenting them out in the hercules configuration file.
It also uses the environment variables to build a JCL deck to submit to the running mvs38j system after it is started to update any parmlib configurations that may be needed in conjunction with the changes made.

You don't care about any of that so what I provide here is a simple wrapper script that only has as an example how to customize the tzoffset in the hercules config file as that is all most people would need (if you are somewhere that does daylight saving time).
It also has placeholders for waiting for startup to complete and submitting a JCL deck for after startup customisations.

This is the container CMD script which must never stop running so has the endless sleep a long time and loop back and sleep again to keep it running.

A working example of this script is in the TGZ file so read on.

OK so you just want to build a container

You probably do not care about all the above and just want to create a container for your existing MVSS3.8J environment. The impatient don't care about how or why and if read this far just want to create a container immediately, which is what the TGZ file here is for.

There are a few things you will have to edit, for example your mvs3.8j directory name is probably different to mine. The two places to customise are the Dockerfile itself, and an environment conf file that is used by all the bash scripts (so no need to edit any bash scripts themselves, just the variables they use).

What is needed to be changed is covered in a README.txt file and in comments at the start of the Dockerfile and in the envirnment config file.

Simply download the TGZ file, tar -zxvf it in the directory immediately above your mvs3.8j directory (will create a new directory called container_scripts), make the very few lines of editing needed in the Dockerfile and environmemt config file, copy the Dockerfile up into the higher level directory (so your mvs3.8j directory and the container_scripts directory are immediately below the Dockerfile), and run the docker build command. At which point you have your container and can simply run it under docker/podman etc. You should have a container running your setup in under 10mins.
This is the TGZ file you have been waiting for
I also test my containers under miniKube, because it is fun to make sure it works there as well.

Or if you really cannot be bothered details on my heavily customised pre-existing docker image may interest you; although as that has all the DLIBs and CBT volumes it is huge.