Difference: GridServices (15 vs. 16)

Revision 162018-04-16 - GordonStewart

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Grid Services

Line: 51 to 51
 (jobName = "TestJob") (stdout = "stdout") (stderr = "stderr")
Changed:
<
<
(gmlog = "test.log")
>
>
(gmlog = "gmlog")
 (walltime="60")
Line: 85 to 85
 Job submitted with jobid: gsiftp://svr011.gla.scotgrid.ac.uk:2811/jobs/NCKLDmEQkwrnZ4eC5pmRAbBiTBFKDmABFKDmpMFKDmABFKDmQffBxy
Changed:
<
<
When a job is submitted successfully, you will be presented with its job ID which can be used to refer to the job later. Information about submitted jobs is also recorded in a job list file; by default, this file is ~/.arc/jobs.dat (~/.arc/jobs.xml with some versions of ARC), but you can choose a different location by supplying the -j argument to arcsub:
>
>
When a job is submitted successfully, you will be presented with its job ID which can be used to refer to the job later. Information about submitted jobs is also recorded in a job list file; by default, this file is ~/.arc/jobs.dat (~/.arc/jobs.xml with some earlier versions of ARC), but you can choose a different location by supplying the -j argument to arcsub:
 
$ arcsub -j <JOBLIST_FILENAME> -c <CE_HOSTNAME> <XRSL_FILENAME>

Line: 142 to 142
 

You will only be able to retrieve job output once the job has finished.

Added:
>
>

Copying input and output files ("staging")

You can tell ARC to copy input and output files to and from the compute element by including additional attributes in your xRSL file:

&
(executable = "test.sh")
(arguments = "")
(jobName = "TestJob")
(inputFiles = ("input.dat" ""))
(outputFiles = ("output.txt" "")
               ("results.tgz" "")
)
(stdout = "stdout")
(stderr = "stderr")
(gmlog = "gmlog")
(walltime="60")

Files used in the exectuable, stdout and stderr attributes are transferred automatically, but other files should be listed in the inputFiles or outputFiles attribute as necessary. The inputFiles and outputFiles attributes each take one or more values like this:

("<FILENAME>" "<URL>")

Where <URL> is left blank, ARC transfers the file to or from the submission machine (this would be the case for input.dat, output.txt and results.tgz in the example xRSL above). Alternatively, a URL may be provided to copy the file to or from a remote resource:

("index.html" "http://www.example.org/index.html")
("rabbits.zip" "ftp://ftp.example.org/rabbits.zip")
("values.dat" "gsiftp://gridftp.example.org/data/values.dat")

Various protocols are supported, including Rucio and SRM, and details can be found in the ARC reference manual:

http://www.nordugrid.org/documents/xrsl.pdf

Due to the slightly convoluted path files follow to make their way from the submission machine through the CE to the compute node, it is easiest to avoid using paths when specifying outputFiles. Instead, if files are created in subdirectories, it may be simpler to copy these files back to $HOME at the end of the script (this is a working directory belonging to your job, and is not related to your home directory). You may also wish to add multiple output files or directories to an archive, in order to simplify the process of retrieving results further.

 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback