If any of you have been paying attention I have lately been looking into ansible.
First a disclaimer, I use ‘puppetserver’ and puppet agents as since they moved away from Ruby to their own scripting language which is pretty much english it is incredibly easy to configure. If / ifelse / else syntax means I can have a simple config for say ‘desktop’ that configures a desktop selecting appropiate packages for Ubuntu/Debian/Fedora/CentOS/Rocky and for specific versions (ie: CentOS8 is missing packages that were in centOS7, debian uses completely different package names etc.) And puppet has a fantastic templating feature that maybe one day in the future ansible will be able to match.
Ansible with the playbook parsing json responses from servers can have the playbook configured to run seperate tasks depending on the results returned in previous steps but yaml files are not plain english and it doesn’t really support targets such as ‘configure desktop reguardless of OS’ and at the moment you are better off having a seperate set of playbooks per-OS type… or more simply it is not as readable or manageble yet.
Ansible is also primarily targeted at (has builtin/sipplied modules for) Linux servers although it supports windoze as well; the main issue with ansible is that most of the modules are written in python(3). In the Linux world that is not really an issue as it is almost impossible to have a Linux machine without python. On RHEL based systems it is so tightly integrated it is impossible to remove (it’s needed by systemd, firewall, dnf etc.); fortunately even though Debian(11) installs it by default it is possible to remove it on that OS so I was able to test the examples here on a machine without python installed; although of course the tests were against Linux machines.
The advantage of ansible is that there are many operating systems that support some variant of ssh/scp/sftp even if they do not support python (yes people, there are operating systems that do not and probably never will have python ported to them; some I work on) and ansible allows modules to be written in any language so modules can in theory be written in any scripting language a target machine supports.
The basic function of an ansible playbook ‘task’ seems to be to use sftp to copy the module for a task from it’s local repository or filesystem to the target machine, also sftp a file containing all the data parameters in the playbook to the target machine, and run the script with the datafile of parameters as argument one for the module(script); at the end of the task it deletes the two files it copied to the target.
What I am not sure of is how it triggers the module to run, on *nix it probably uses ‘sh’, windoze powershell; but can it be configured to run other interpreters such as rexx/clist/tacl(or gtacl) etc. If it can then it can poke its gaping wide security hole into and manage every machine that exists in theory.
By security hole I of course just mean that it needs god-like access from ssh keys alone (you do not want 2FA for every step in a playbook) and despite decades of advice on keeping private ssh keys secure inadvertently published ones still keep popping up on forums that intend you no good; and ‘sudo’ does not care about where you connected from so anybody with that private key and access to you network is god on your infrastructure; of course you could be paranoid and have a seperate key pair per server but it you have hundreds of them a maintenance nightmare.
Anyway, my interest in ansible is primarily in if it can easily manage machines that will never have python installed on them. It is also fair to say that if I am to write anything extremely complicated and possibly destructive I would prefer to do it in bash rather than in a language like python I am not really familiar with yet.
As such I have no need of python modules and need to try to avoid letting any python modules be deployed on machines I am testing against. As noted above I managed to remove python from a Debian11 server so am able to test against that.
As all my lab servers are currently *nix servers I don’t really have the oportunity to test against all the different systems I would like to see if ansible works on them (although I might try and get netrexx or oorexx as targets on a few Linux servers).
This post is about how to write ansible modules using the bash shell rather than python.
It is also worth noting where to place your custom modules so they can be located by ansible. If on a Linux machine using ‘ansible-playbook’ from the command line for testing it is easiest to simply create a directory called ‘library’ under the directory your playbook is in, for testing that would simply be your working directory and they will be found in there. You can also use the environmanet variable ANSIBLE_LIBRARY to specify a list of locations if you want a module available to all your playbooks. Note: there are many other ansible environment variables that can be useful, refer to the ansible docs.
You should also note that the ‘ansible-doc’ command only works against python modules, and while it is recomended that you write the documentation in a Python file adjacent to the module file… don’t do that or the ansible-playbook command will try and run the .py file instead of the .sh file. Just ensure it is well documented in your site specific documentation.
While the example module shown in this post may seem a little complicated one thing you must note is that by default an ansible task will run a ‘gather_facts’ step on the target machine to populate variables such as OS family, OS version, hostname, and basically everything about the target machine. That is done by a python module so is not possible on target machines without python, so the example here sets ‘gather_facts: no’ and obtains whet it needs in the module itself as well as returning the information to the playbook for use.
It’s also a little more complicated than it needs to be in that I was curious as to how variables from the playbook would be passed if indented in the playbook under another value; they are passed as jsom if imbedded, for example
Task entries
dest: /some/dir
state: present
Results in the file passed as arg1 to the module on the target machine containing
dest="/some/dir"
state="present"
Task entries
parms:
dest: /some/dir
state: present
Results in the file passed as arg1 to the module on the target machine containing
parms="{'dest': '/tmp/hello', 'state': 'present'}"
Knowing in what format the data values are going to be provided to your script is a rather important thing to know :-). In a bash or sh script you can set the variables simply using “source $1” but you do need to know the values will be in different formats depending on indent level. My example script here will handle both of those above examples but not any further levels of indentation. There will be command line json parsers that could help on Linux but remembering I’m curious about non-linux servers I need the scripts to do everything themselves.
For my hosts file I included a ‘rocky8’, ‘debian11’, and ‘fedora32’ server. The bash example returned the expected results from all of them.
It should also be noted that the test against server name in the example playbook step always returns ‘changed’ as the ‘dummy_file.yml’ uses ‘local_action’ which seems to always return changed as true reguardless of what we stuff in stdout so the command function must return its own data fields. Where only the bash module is used we control that value.
But enough rambling. ‘cd’ to a working directory, ‘mkdir library’.
Create a file my_example_module.yml and paste in the below.
# Demo playbook to run my bash ansible module passing
# parms in 'dest' and 'state' in both ways permitted.
#
# Note: the last example also shows how to take action based on the returned result,
# in this case pull in a yml file containing a list of additional tasks for a specific
# hostname (in my test env the ansible 'hosts' file for the test had ip-addrs only
# so the hostname test is against the returned values).
# While ansible has builtins for hostname and OS info tests such as
# when: ansible_facts['os_family'] == "RedHat" and ansible_facts['lsb']['major_release'] | int >= 6
# that is only useful for target hosts that have python installed and you use the default 'gather_facts: yes',
# using a bash module implies the target does not have python so we use 'gather_facts: no' so we have
# to do our own tests; and it is a useful example anyway :-).
---
- hosts: all
gather_facts: no
tasks:
- name: test direct parms
become: false
my_example_module:
dest: /tmp/hello
state: present
register: result
- debug: var=result
- name: testing imbedded parms
become: false
my_example_module:
parms:
dest: /tmp/hello
state: present
register: result
- debug: var=result
- set_fact: target_host="{{ result.hostinfo[0].hostname}}"
- include: dummy_file.yml
when: target_host == 'nagios2'
ignore_errors: true
Change the hostname in the target_host test condition near the end of the above file to one of your hostnames, it is an example of running a tasks file for a single host and will be skipped if no hostnames match.
Create a file dummy_file.yml (used by the hostname test step) containing the below
- name: A dummy task to test it is triggered
local_action: "command echo '{\"changed\": false, \"msg\": \"Dummy task run\"'"
register: result
- debug: var=result
Create a file library/my_example_module.sh and paste in the below
#!/bin/bash
# =====================================================================================================
#
# my_example_module.sh - example bash ansible module 'my_example_module'
#
# Description:
# Demonstration of writing a ansible module using the bash shell
# (1) handles two parameters passed to it (dest, state) passed at either the
# top level or indented under a parms: field
# (2) returns useful host OS details (from os-release) as these are useful
# for logic branching (ie: centos7 and centos8 have different packages
# available so you need to know what targets modules run on).
# ---obviously, this method is only useful for linux target hosts
# Ansible functions such as
# when: ansible_facts['os_family'] == "RedHat" and ansible_facts['lsb']['major_release'] | int >= 6
# are obviously not useful for targets that do not run python where we must
# set gather_facts: no
#
# Items of note...
# * all values in the playbook are passed as var=value in a file sent to
# the remote host and passed as an argument to the script as $1 so script
# variables can be set withs a simple 'source' command from $1
# * data values placed immediately after the module name can be used 'as-is'
# (see example 1)
# dest=/tmp/hello
# state=present
# however if values are imbedded they will be passed as a
# JSON string which needs to be parsed
# (see example 2 where values are placed under a 'parms:' tag)
# parms='{'"'"'dest'"'"': '"'"'/tmp/hello'"'"', '"'"'state'"'"': '"'"'present'"'"'}'
# which after the 'source' command sets the parms variable to
# {'dest': '/tmp/hello', 'state': 'present'}
# which needs to be parsed to extract the variables
# * so, don't imbed more than needed for playbook readability or your script will be messy
#
# * the 'failed' variable indicates to the ansible caller if the script has failed
# or not so should be set by the script (failed value in ansible completion display)
# * the 'changed' variable indicates to the ansible caller if the script has changed
# anything on the host so should be set by the script (the changed value in ansible
# completion display). You should set that if your module changes anything, but
# this example has it hard coded as false in the response as the script changes nothing
# * oh yes, the output/response must be a JSON string, if you have trouble with your
# outputs try locating the error with https://jsonlint.com/
#
# Usage/Testing
# Under your currect working directory 'mkdir library' and place your script
# in there as (for this example) 'my_example_module.sh'.
# Then as long as the ansible-playbook command is run from your working
# directory the playbook will magically find and run module my_example_module
# Obviously for 'production' you would have a dedicated playbook directory
# in one of the normal locations or use an envronment variable to set the
# ansible library path, but for testing you do want to keep it isolated to
# your working directory path :-)
#
# Examples of use in a playbook,
# 1st example is vars at top level, 2nd is imbedded under parms:
# We use 'gather_facts: no' as using bash modules implies that the
# targets are servers without python installed so that would fail :-)
#
# ---
# - hosts: all
# gather_facts: no
# tasks:
# - name: example 1 test direct parms
# my_module_example:
# dest: /tmp/hello
# state: present
# register: result
# - debug: var=result
# - name: testing imbedded parms
# my_module_example:
# parms:
# dest: /tmp/hello
# state: present
# register: result
# - debug: var=result
#
#
# Example response produced
# {
# "changed": false,
# "failed": false,
# "msg": "test run, parms were dest=/somedir state=missing",
# "hostinfo": [{
# "hostname": "hawk",
# "osid": "rocky",
# "osname": "Rocky Linux",
# "osversion": [{
# "major": "8",
# "minor": "4"
# }]
# }]
# }
#
# =====================================================================================================
source $1 # load all the data values passed in the temporary file
failed="false" # default is that we have not failed
# If data was passed as a "parms:" subgroup it will be in JSON format such as the below
# {'dest': '/tmp/hello', 'state': 'present'}
# So we need to convert it to dest=xx and state=xx to set the variables
# Parsing for variable name as well as value allows them to be passed in any order
if [ "${parms}." != "." ];
then
isjson=${parms:0:1} # field name in playbook is parms:
if [ "${isjson}." == '{.' ] # If it is in json format will be {'dest': '/tmp/hello', 'state': 'present'}
then
f1=`echo "${parms}" | awk -F\' {'print $2'}`
d1=`echo "${parms}" | awk -F\' {'print $4'}`
f2=`echo "${parms}" | awk -F\' {'print $6'}`
d2=`echo "${parms}" | awk -F\' {'print $8'}`
export ${f1}="${d1}" # must use 'export' or the value of f1 is treated as a command
export ${f2}="${d2}"
else
failed="true"
printf '{ "changed": false, "failed": %s, "msg": "*** Invalid parameters ***" }' "${failed}"
exit 1
fi
fi
# Else data was passed as direct values so will have been set by the source command, no parsing needed
# You would of course always check all expected data was provided
if [ "${dest}." == "." -o "${state}." == "." ];
then
failed="true"
printf '{ "changed": false, "failed": %s, "msg": "*** Missing parameters ***" }' "${failed}"
exit 1
fi
OSHOST="$(uname -n)" # Get the node name (host name)
if [ -r /etc/os-release ];
then
# /etc/os-release is expected to have " around the values, we don't check in this
# example but assume correct and strip them out.
# In the real world test for all types of quotes or no quotes :-)
OSID=`grep '^ID=' /etc/os-release | awk -F\= {'print $2'} | sed -e 's/"//g'` # Get the OS ID (ie: "rocky")
OSNAME=`grep '^NAME=' /etc/os-release | awk -F\= {'print $2'} | sed -e 's/"//g'` # Get the OS Name (ie: "Rocky Linux")
osversion=`grep '^VERSION_ID=' /etc/os-release | awk -F\= {'print $2'} | sed -e 's/"//g'` # Get OS Version (ie: "8.4")
OSVER_MAJOR=`echo "${osversion}" | awk -F. {'print $1'}`
OSVER_MINOR=`echo "${osversion}" | awk -F. {'print $2'}`
if [ "${OSVER_MINOR}." == "." ]; # Debian 11 (at least what I run) does't have a minor version
then
OSVER_MINOR="0"
fi
hostinfo=`printf '{"hostname": "%s", "osid": "%s", "osname": "%s", "osversion": [{"major": "%s","minor": "%s"}]}' \
"${OSHOST}" "${OSID}" "${OSNAME}" "${OSVER_MAJOR}" "${OSVER_MINOR}"`
else
hostinfo=`printf '{"hostname": "%s", "osid": "missing", "osname": "missing", "osversion": [{"major": "0","minor": "0"}]}' "${OSHOST}"`
fi
# Return the JSON response string with a bunch of variables we want to pass back
printf '{ "changed": false, "failed": %s, "msg": "test run, parms were dest=%s state=%s", "hostinfo": [%s] }' \
"${failed}" "$dest" "${state}" "${hostinfo}"
exit 0
Create a file named hosts and enter a list of the hostnames or ip-addresses you want to test against as below (using you machines ids of course). Note that the python override is required for Debian11 servers, it does no harm using it on the others.
localhost 192.168.1.177 192.168.1.187 #192.168.1.9 #test_host ansible_port=5555 ansible_host=192.168.1.9 [all:vars] ansible_python_interpreter=/usr/bin/python3 [test_group] localhost 192.168.1.177
Create a file names TEST.sh, simply because it is easier to run that multiple times than type in the entire command. Place into that file
ansible-playbook -i hosts my_example_module.yml #ansible-playbook -i hosts my_example_nopython.yml
Yes the last line is commented, you have not created that file yet.
You are ready to go. Simply ‘bash TEST.sh’ and watch it run, you have your first working bash module.
Now, you are probably wondering about the commented example in the TEST.sh file above.
As mentioned I am curious as to how to use ansible to manage servers that do not have python installed, and have been thinking about how to do it.
This last example avoids the default ansible python modules, manually copies across the script/module to be executed, manually created the data input file to be used, and manually runs the ‘/bin/bash’ command to execute it, cleans up the files it copied.
While manually using the ‘/bin/bash’ command is overkill for Linux servers where you can just place at the start of the file what script execution program should run the script; it shows how you could in theory use the ‘raw’ function to invoke any script processor on the target machine.
I must point out it’s a badly written example, in that ansible is considered a configuration management tool so must have inbuilt functions to copy files from an ansible server to a managed server so having a manual ‘scp’ step is probably not necessary but I am trying to do it with as little inbuilt functions as possible for this example. Also in a managed environment you would probably not scp files from the local server but use curl/wget to pull them from a git repository; but not all operating systems support tools like wget/curl so knowing a manual scp is a way to get files there is useful.
Anyway this example copies exacly the same module as that used in the above example across to the target server, creates a data file of parms, runs the module explicitly specifying /bin/bash as the execution shell, and deletes the two copied files; just as ansible would in the background.
You could take it a lot further, for example not clean up the script file and have a prior step to see if it already existed on the target and skip the copy if it did, useful if you have a huge farm of servers and the files being moved about are large. But all that is beyong the scope of this post.
The playbook to do that is below. You can create the file my_example_nopython.yml, paste the contents below, and uncomment the line in the TEST.sh file to confirm it works. You must of course change the scriptsourcedir value to the working directory you are using, and it must be a full path; and of course change the host ip-addr used to one of your servers.
# Example of using a playbook to simulate what ansible does.
# MUST have 'gather_facts: no' if the target server does not have python3 installed as
# gathering facts is an ansible default module that is of course written in python.
#
# Obviously in the real world you would not copy scripts from a local filesystem but pull them
# from a source repository (but as examples go this you can copy and work on immediately...
# after updating the hosts (and hosts file) and script source location of course
#
# Uses the my demo bash module as the script to run so we must populate a file with the
# two data values it expects, done with a simple echo to a file in this example.
#
# NOTE: if the target server does not have python installed ansible will still happily
# (if you disabled facts gathering which of course would cause a failure)
# as part of it's inbuilt processing copy a python module to the target along with
# a file containing data values and try to run the module (which of course fails);
# that is the functionality we are duplicating here as an example, as you can
# easily build on this to make things a lot more complicated :-)
# (ansible probably uses sftp to copy the files, as the sftp subsystem needs to be enabled)
# And we of course copy my example module written in bash as we want the demo to work :-)
#
# Why is a playbook like this important ?.
# Many servers that are non-Linux (even non-*nix) support some form of ssh/scp/sftp.
# Using a playbook like this can let you handle the quirks of those systems where supplied
# ansible default modules cannot.
---
- hosts: 192.168.1.177
gather_facts: no
vars:
user: ansible
scriptsourcedir: /home/ansible/testing/library
scriptname: my_example_module.sh
tasks:
- name: copy script to remote host
local_action: "command scp {{scriptsourcedir}}/{{scriptname}} {{user}}@{{inventory_hostname}}:/var/tmp/{{scriptname}}"
register: result
- debug: var=result
- name: create remote parm file
raw: echo 'dest="/some/dir";state="present"' > /var/tmp/{{scriptname}}_data
become: false
register: result
- name: run remote script
raw: /bin/bash /var/tmp/{{scriptname}} /var/tmp/{{scriptname}}_data
become: false
register: result
- debug: var=result
- name: remove remote script
raw: /bin/rm /var/tmp/{{scriptname}} /var/tmp/{{scriptname}}_data
become: false
So, you have seen how to write and test a module in bash, not too complicated after all. There is one important thing you must always remember though. The output of your module must be valid JSON, get a bracket out of place and it will go splat; so two tips
- if you end up with bad JSON output I find https://jsonlint.com/ is a quick way of finding the problem
- if you are unsure of what data is being placed in the data value input file by ansible place a ‘sleep 30’ command in the script which gives you time on the target machine to look at the files under ~ansible/.ansible/tmp (replace ~ansible with the userid used on the target machine) and ‘cat’ the files under there to see what values are actually being set
Enjoy breaking things.
