TorqueMarlinJobs < LinearCollider

LinearCollider Web>JetFlavourTagging>TorqueMarlinJobs (2013-07-26, DanProtopopescu) (raw view)

---++ Running Marlin Jobs on the PPE compute farm

It's very easy to run Marlin jobs on the PPE Torque/PBS compute farm: the AFS token is taken care of automatically, and access to the software packages, job input and output is via AFS. Attached to this page is a  [[%ATTACHURL%/22.tar][tarball]] containing the relevant scripts and template files mentioned in this wiki.

%TOC%

---+++ PBS Script

We found it easier to run processors 1 to 6 on batches of 10 input files , and save the outputs as /afs/phas.gla.ac.uk/data/ilc/datasets01/1.4tev/ZVRES_out/runX_out.slcio. The PBS script used is very simple:
<verbatim>
#PBS -N ZVRES_Job_Group_22
#PBS -m abe
#PBS -M my.email@example.com
#PBS -q long5

echo -n "Worker node: "
hostname

echo "Running ZVRES Job Group 22 ..." 

/afs/phas.gla.ac.uk/user/p/protopop/ILC/analysis/Marlin/ZZfusion/zvresRunPBS.sh 22

echo "Marlin done!"
echo
</verbatim>
where =ZVRES_Job_Group_22= is the name under which the job is listed when typing =qstat=, and =zvresRunPBS.sh= is our custom script (attached). We actually used another script to produce the individual PBS scripts from a template: sqbq.sh (attached).

---+++ Wrapper script
 
The wrapper script =zvresRunPBS.sh= (contained in the tarball attached) selects 10 input files, sets the *Marlin* environment, and runs [[JetFlavourTagging][processors 1 to 6]]. In addition, it prepares the steering files for the FTI and FT steps (which we ran on our desktops). This is not the only or the simplest way to do this, but it simplified our work: 
<verbatim>
#!/bin/bash
# Run the most time consuming LCFI processors on a subset of files
# by protopop@cern.ch - Jul 2013 (PBS version)

events="-1"

if [ "$1" -eq "$1" 2> /dev/null ]; then
  dstmp=`date | awk '{print $2$3}'`
  tag="$1"
  echo "Using run tag $tag"
else
  echo "Please provide a numeric tag!"
  exit 1
fi

cd /afs/phas.gla.ac.uk/user/p/protopop/ILC/analysis/Marlin/ZZfusion/batch

steer="zvres$tag.xml"
lcout="/afs/phas.gla.ac.uk/data/ilc/datasets01/1.4tev/ZVRES_out/run$tag-out.slcio"
gearx="../gear_ldc.xml"
flist="../steer/lcio_files.xml"

let "n1=$1*10"
let "n2=n1+9"
files=`sed -n "$n1,$n2"p < $flist`

cp ../steer/template_01.xml "$steer"
ls -lA

echo "Customising template for this batch run ..."
perl -p -i -e "s#MYNEVENTS#$events#g" $steer
perl -p -i -e "s#MYGEARFILE#$gearx#g" $steer
perl -p -i -e "s#MYLCIOFILES#$files#g" $steer
perl -p -i -e "s#MYVERBOSITY#SILENT#g" $steer
perl -p -i -e "s#MYOUTPUTLCIOFILE#$lcout#g" $steer

echo "Your steering file is $steer "

if [ "1" -eq "1" ]; then
 source /afs/phas.gla.ac.uk/data/ilc/software/ilcsoft/v01-14/init_ilcsoft.sh   
 Marlin $steer >/dev/null 2>&1  
else
 echo "Will not run Marlin"
fi

echo "Run $tag done."

nxtst="fti-$tag-steer.xml"
echo "Creating $nxtst steering file ..."
cp fti-3-pbs-steer.xml $nxtst
perl -p -i -e "s#run3#run$tag#g" $nxtst
perl -p -i -e "s#fti3#fti$tag#g" $nxtst

ftstr="ft$tag-gnets.xml"
echo "Creating $ftstr steering file ..."
cp ft1-pbs-gnets.xml $ftstr
perl -p -i -e "s#fti1#fti$tag#g" $ftstr
perl -p -i -e "s#flavourtag1#flavourtag$tag#g" $ftstr

echo "Bye!"
echo
</verbatim>

---+++ How to run it ?

For the bunch shown in the example above, we used the command =subq.sh 22=. This produced the PBS script, and submitted it. If other processors are needed, or other paths are preferred, then please feel free to download and modify our scripts as suitable.

---+++ Background set

For the background files, we used slightly modified version of the scripts, which can be found in [[%ATTACHURL%/bkg_scripts.tgz][this tarball]].

Attachments

Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
tar	22.tar	r1	manage	160.0 K	2013-07-26 - 11:35	DanProtopopescu	Scripts and templates mentioned in this wiki
tgz	bkg_scripts.tgz	r1	manage	8.6 K	2013-07-26 - 15:10	DanProtopopescu	Scripts and XML template for processing eeqq background files

Topic revision: r2 - 2013-07-26 - DanProtopopescu

LinearCollider