Grid Services

Jobs can be submitted to grid resources using the ARC tools, which are available in CVMFS. Our colleagues in Durham have written a good introductory tutorial; a summary of the steps required to submit and manage jobs, adapted for Glasgow users, is given below.

ARC tools

The tools required for grid job submission and management are available from CVMFS:

export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase
alias setupATLAS='source ${ATLAS_LOCAL_ROOT_BASE}/user/atlasLocalSetup.sh'
setupATLAS

lsetup emi

If you plan to submit jobs to the ScotGrid VO, at present you must also amend the X509_VOMSES environment variable as follows:

export X509_VOMSES=/etc/vomses

Certificates and proxies

To use grid resources, you will need a certificate, from which you can generate a proxy certificate. The proxy certificate has a relatively short lifetime, and is used to actually submit the job. A proxy is associated with a particular Virtual Organisation (VO), for example vo.scotgrid.ac.uk, which is selected when it is created. You can generate a proxy using the arcproxy command:

arcproxy -S <VO_ALIAS> -N

For example, to generate a proxy for the vo.scotgrid.ac.uk VO:

arcproxy -S vo.scotgrid.ac.uk -N

Job description (xRSL)

Before submitting a job, you need to create a file which describes the features of the job for ARC (its executable, the names of input and output files, what to do with logs, etc.). This file is written in the Extended Resource Specification Language (xRSL). A simple job description which runs a script called test.sh could look like this:

&
(executable = "test.sh")
(arguments = "")
(jobName = "TestJob")
(stdout = "stdout")
(stderr = "stderr")
(gmlog = "test.log")
(walltime="60")

A full description of xRSL can be found in the ARC reference manual:

http://www.nordugrid.org/documents/xrsl.pdf

Submitting a job

Jobs are submitted to a Compute Element (CE). The ScotGrid site at Glasgow has four CEs:

svr009.gla.scotgrid.ac.uk
svr010.gla.scotgrid.ac.uk
svr011.gla.scotgrid.ac.uk
svr019.gla.scotgrid.ac.uk

It does not matter which CE you choose to submit to. (If you've looked at the tutorial linked above, you'll see that Durham gave their CEs the sensible names ce1, ce2, etc. We thought that would be too easy.)

Jobs are submitted using the arcsub command:

arcsub -j <DATABASE_FILENAME> -c <CE_HOSTNAME> <XRSL_FILENAME>

For example, to submit test.xrsl to svr011 at Glasgow:

arcsub -j test.db -c svr011.gla.scotgrid.ac.uk test.xrsl