Using Ganga to submit jobs to the Panda backend on lxplus

References:

Full Ganga Atlas Tutorial

Data preparation reprocessing - using Ganga

NOTE: To set up for any release one must be familar with using CMT (bootstrap procedures and requirement files) - see here for more information. In this example (for reprocessing see the reference page above for the necessary release) once the requirements file is set up and a release directory in your test area is created then try:

For this example check out (and compile) the UserAnalysis package as in the "HelloWorld"example here or here.

4. Go to run directory and start ganga (once the code is compiled).NOTE ganga picks up your grid cert/key from your .globus directory on lxplus - if you have not created these files please follow the instructions here under the preparing the certificate section. Also you must create a file called 'vomses' in a new directory called .glite (again note the '.') and enter this line for the ATLAS VO - "atlas" "voms.cern.ch" "15001" "/C=CH/O=CERN/OU=GRID/CN=host/voms.cern.ch" "atlas" into this file.

cp ../share/AnalysisSkeleton_topOptions.py .

BUT to prepare your code for running on the Grid there are some changes needed for this Athena JO - please add these lines:

______________________________________________________________________________

include("RecExCommission/RecExCommissionFlags_jobOptions.py" )
ATLASCosmicFlags.useLocalCOOL  = True
# setup DBReplicaSvc to choose closest Oracle replica, configurables style
from AthenaCommon.AppMgr import ServiceMgr
from PoolSvc.PoolSvcConf import PoolSvc
ServiceMgr+=PoolSvc(SortReplicas=True)
from DBReplicaSvc.DBReplicaSvcConf import DBReplicaSvc
ServiceMgr+=DBReplicaSvc(UseCOOLSQLite=False) 

    ganga pandaBackend_test.py

6. You can monitor your job's progress by typing jobs inside Ganga or, if you submitted to the Panda backend by http://panda.cern.ch:25880/server/pandamon/query.

7. Once your job has finished you can copy the output data using the dq2 tools.

this

1026613035

The Ganga JO 'pandaBackend_test.py' could look like this (without line numbers):

Also for the LCG() backend, a site can be specified:
j.backend.requirements.other=['other.GlueSiteUniqueID=="UKI-SCOTGRID-GLASGOW"']

To submit to the UK Cloud add
j.backend.requirements.cloud='UK'

</pre>

NOTE: Line 3 is an example of overriding a database release to match the one needed to read ESD/DPD. In the case of the spring cosmic reprocessing,the DB release is 6.6.1.1. If the database releases don't match the jobs fail on the Grid ( remove this line if it is not necessary). Line 4 corresponds to your Athena jobOptions. Line 5 is set to False because we have already compiled the packages locally if you want your job to compile your checked out code before submitting then simply change this to True Line 6 tells Ganga to tar your user area and send it with the job. Line 10 specifies the backend to which you are sending your job. There are three options: LCG, Panda and NorduGrid. In the example above Panda was chosen because the data existed only in BNLPANDA, a site in the US cloud. Line 12 corresponds to the number of subjobs you want to split your job into. Finally in Line 13 you submit your job

The Ganga output looks something like this (Note the output is a dataset: Output dataset user09.chriscollins.ganga.2.20091210 ):

run% ganga pandaBackend_test.py

*** Welcome to Ganga ***
Version: Ganga-5-4-3
Documentation and support: http://cern.ch/ganga
Type help() or help('index') for online help.

This is free software (GPL), and you are welcome to redistribute it
under certain conditions; type license() for details.


For help visit the ATLAS Distributed Analysis Help eGroup:
  https://groups.cern.ch/group/hn-atlas-dist-analysis-help/

GangaAtlas                         : INFO     Found 0 tasks
Ganga.GPIDev.Lib.JobRegistry       : INFO     Found 2 jobs in "jobs", completed in 0 seconds
Ganga.GPIDev.Lib.JobRegistry       : INFO     Found 0 jobs in "templates", completed in 0 seconds

********************************************************************
New in 5.2.0: Change the configuration order w.r.t. Athena.prepare()
              New Panda backend schema - not backwards compatible
For details see the release notes or the wiki tutorials
********************************************************************

GangaAtlas.Lib.Athena              : WARNING  New prepare() method has been called. The old prepare method is c   alled now prepare_old()
GangaAtlas.Lib.Athena              : INFO     Found Working Directory /home/chrisc/atlas/GANGA-TEST-15.3.0.1/15   .3.0.1
GangaAtlas.Lib.Athena              : INFO     Found ATLAS Release 15.3.0
GangaAtlas.Lib.Athena              : INFO     Found ATLAS Production Release 15.3.0.1
GangaAtlas.Lib.Athena              : INFO     Found ATLAS Project AtlasProduction
GangaAtlas.Lib.Athena              : INFO     Found ATLAS CMTCONFIG i686-slc4-gcc34-opt
GangaAtlas.Lib.Athena              : INFO     Using run directory: PhysicsAnalysis/HiggsPhys/HiggsAssocTop/TtHH   bbDPDBasedAnalysis/run/
GangaAtlas.Lib.Athena              : INFO     Extracting athena run configuration ...
GangaAtlas.Lib.Athena              : INFO     Detected Athena run configuration: {'input': {'noInput': True}, '   other': {}, 'output': {'outNtuple': ['FILE1'], 'alloutputs': ['D3PD.root']}}
GangaAtlas.Lib.Athena              : INFO     Creating /tmp/chrisc/sources.f3f6d811-f7cd-42d3-8d50-47c47f58ae78   .tar ...
GangaAtlas.Lib.Athena              : INFO     Option athena_compile=False. Adding InstallArea to /tmp/chrisc/so   urces.f3f6d811-f7cd-42d3-8d50-47c47f58ae78.tar ...
Ganga.GPIDev.Lib.Job               : INFO     submitting job 2
Ganga.GPIDev.Lib.Job               : INFO     job 2 status changed to "submitting"
GangaPanda.Lib.Panda               : INFO     Panda brokerage results: cloud UK, site ANALY_GLASGOW
GangaAtlas.Lib.ATLASDataset        : WARNING  Dataset mc08.106314.Pythia_ttH120_2l2nu4b.merge.AOD.e364_s462_r63   5_t53_tid065132 has 8 locations
GangaAtlas.Lib.ATLASDataset        : WARNING  Please be patient - waiting for site-index update at site UKI-SCO   TGRID-GLASGOW_LOCALGROUPDISK ...
GangaAtlas.Lib.Athena              : WARNING  You are using DQ2JobSplitter.filesize or the backend used support   s only a maximum dataset size of 10000 MB per subjob - job splitting has been adjusted accordingly.
GangaPanda.Lib.Athena              : INFO     Input dataset(s) ['mc08.106314.Pythia_ttH120_2l2nu4b.merge.AOD.e3   64_s462_r635_t53_tid065132']
GangaPanda.Lib.Athena              : INFO     Output dataset user09.chriscollins.ganga.2.20091210
GangaPanda.Lib.Athena              : INFO     Running job options: TtAnalysis-ttHSignalSMALL-GRID.py
GangaPanda.Lib.Panda               : INFO     Uploading source tarball sources.f3f6d811-f7cd-42d3-8d50-47c47f58   ae78.tar.gz in /tmp/chrisc to Panda...
Ganga.GPIDev.Lib.Job               : INFO     job 2.0 status changed to "submitting"
Ganga.GPIDev.Lib.Job               : INFO     job 2.0 status changed to "submitted"
Ganga.GPIDev.Lib.Job               : INFO     job 2 status changed to "submitted"

Helpful commands inside Ganga:
jobs lists your jobs
jobs(1) lists the content of job 1
help() goes into help mode ( quit to leave help)
j=jobs(1) and j.kill() will kill job 1.

Your output will be in the dq2 registered dataset. For me this was user09.chriscollins.ganga.2.20091210 Again this is available from jobs(x)

Running Executable on panda Backend

This has been said to work on panda backend. It has not been tested. If found to be successful please UPDATE!

______________________________________________________________________________

j = Job()
j.application=Executable()
j.application.exe='myscript.sh'
j.backend=Panda()
j.backend.site='ANALY_FZK'
j.inputsandbox=['/path/to/myscript.sh']
j.submit()

_____________________________________________________________________________

Reusing the environment

First you need to find the name of the previous library. Say we want to reuse the library built in ganga job 323: given a job ID:

Either: in ganga:

In [19]:jobs(323).backend.libds

Out[19]: user.nedwards.20100610.323.lib

or use dq2:

$ dq2-ls %nedwards%323*

user.nedwards.20100610.323

user.nedwards.20100610.323.lib

user.nedwards.20100610.323_sub07797849

The library is user.nedwards.20100610.323.lib. In subsequent job options you can set:

j.backend.libds = 'user.nedwards.20100610.323.lib'