Torque and Auks

Getting torque and auks to work together so that a batch job run by torque has access to a kerberos ticket of the user who submitted the batch job.

Overview

The idea behind this is to run an auks server to provide kerberos tickets to batch nodes when a job starts to run. For security the batch nodes do not access the auks server directly, if they did then a compromise on a batch node would compromise every kerberos ticket, which depending on what the kerberos tickets are being used for could well compromise the entire site and file system. Instead the batch headnode pulls the kerberos tickets from auks and then pushes the tickets to the batch jobs just before a job starts. For security the headnode should only run the torque and maui servers and should not run any batch jobs.

Install Torque and Maui

In order to push the kerberos ticket from the batch headnode to the batch worker node the torque server requires a patch. This adds the ability to run a script on the torque server before a job is sent to a batch node. Maui can be installed without any changes.

The server_prologue.patch for torque was written for torque 2.5.9 and while it works it is not production quality. It also has not been tested with other versions of torque.

I also use the a second patch, acl_fix.patch, which fixed some compilation issues. To make torque rpms I use this torque.spec file. Both the acl patch and spec file were not written by myself and come from searching the web.

It is probably best to install torque and maui without any patches first and get test batch jobs to run. After that install the patched version of torque and create a file /var/spool/torque/server_priv/prologue (this is hard coded in the patch, if you want to install torque somewhere else change the patch). The script is run just before a job is sent to a batch node with a single argument of the job id.

To test the server prologue script try the following for /var/spool/torque/server_priv/prologue:

#!/bin/bash
user_id=`id -u`
job_id=${1}
echo ${user_id} ${job_id} > /tmp/test_${user_id}_{job_id}

If everything is working this will create a file on the torque server for each batch job run which contains the submitting users uid and the batch job id.

Install AUKS

To follow ...

-- AndrewPickford - 2013-06-05