Difference: NA62MonteCarlo (1 vs. 57)

Revision 572016-03-11 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 14 to 14
 

Software

Changed:
<
<
  • Test tarball from Antonino Sergi: NA62MC.tgz or follow this howto
  • Obtain release number and read about modifications in NA62MC/HISTORY

Job submission and output management system code components are held on the na62.gla.ac.uk server. The files attached to this wiki are not the latest versions.

>
>
Source or precompiled production version provided by Antonino Sergi. Job submission and output management system code components are held on the na62.gla.ac.uk server.
 

Installation from scratch on a SL6 desktop

This was done for testing purposes and is documented in a separate wiki, NA62onSL6.

Deleted:
<
<

Code compilation

  • Versions:
    • v4 - Contains bug in the MC which Tonino spotted after giving us the tar ball
    • v5 - Installed by Janusz on May 10 (r170, see table below)
    • v6 - September 2012 version (r188)
    • v7 - r193: uses GEANT4 9.5p01, improves processing speed roughly by a factor of 5 wrt the previous version
    • v8 - r244: installed directly or via CVMFS (see table below)
    • v9 - r261: fatal bug in 4-lepton decay generator fixed, memory leak fixed, points to geant4 9.5.p02
  • Unpack tarball to NA62MC
  • Fix libs path problems in GNUMakefile-s (Generator/ and Beam/)
     cp ../NA62/backups/Beam.GNUmakefile Beam/GNUmakefile
     cp ../NA62/backups/Generator.GNUmakefile Generator/GNUmakefile 
  • Make changes to env.sh: source 'offline' environment, clear XAWLIBS variable, rearrange paths (move system paths after NA62-specific paths):
     export XAWLIBS=" "
  • Source the the environment file (source scripts/env.sh)
  • Type make and be patient
 

Code distribution

Deleted:
<
<
      tar -czvf NA62MC_v7.tgz NA62MC
      cd /afs/cern.ch
      tar -czvf ~/G4-libs.tgz sw/lcg/external/geant4/9.5.p01/x86_64-slc5-gcc43/lib
      tar -czvf ~/Qt-libs.tgz sw/lcg/external/qt/4.6.3p2/x86_64-slc5-gcc43-opt/lib
      tar -czvf ~/Boost_libs.tgz sw/lcg/external/Boost/1.48.0_python2.6/x86_64-slc5-gcc43-opt/lib
  • Add the G4EMLOW6.23.tgz tarball
      [protopop@ppepc102 gridsw]$ ls v8/
      Boost_libs_v8.tgz  G4EMLOW6.23.tgz  G4_libs_v8.tgz  NA62MC_v8.tgz  Qt_libs_v8.tgz  sqlite_v8.tgz
  • Run a special job (with an SGM Admin role) to wget and unpack these tarballs on sites (InstallSoftware.sh)
  • Run test job

CVMFS

 CVMFS is used for software distribution to sites. A guide is provided here: https://www.gridpp.ac.uk/wiki/RALnonLHCCVMFS

Scripts preparation

Line: 75 to 32
 
  • .mac file - check for new settings, new decay types etc.
  • input_files.tgz - check if any new files need to be included in this archive
Deleted:
<
<
Read the HISTORY file to find out what changes might have appeared.
 

Job submission

Changed:
<
<
  • Run parameters given in the StandardRun.mac file: run number, number of events, random seed, root output file name
  • Job environment parameters are set by sourcing the na62jobenv.sh script created by InstallSoftware.sh
  • The .mac is modified using sed on the worker node to modify paths to LAV matrix files, which are installed in the software area by InstallSoftware.sh
  • The actual job script is attached (needs editing and streamlining)

<--
Bulk job submission (first proposed option) 
  1. Create N .mac files using the template
  2. Create one Ganga script including the list of N .mac files
  3. Run this script via Ganga
-->
>
>
Job submission is fully automated via https://na62.gla.ac.uk/
 

Automated jobs scripts creator

Line: 97 to 42
 
Changed:
<
<
>
>
 
Changed:
<
<
>
>
  The script creator interface lets the production manager (PM) set up the job parameters (both MC parameters and output options) and choose the number of 'runs' to launch. The scripter prints out in the browser a list of commands that the PM has to run on his grid UI in order to launch the job or production with his credentials. For individual jobs, the PM updates the DB by hand, for multiple runs, it is done via a wrapper script. A cron job periodically updates the list of jobs and files in the DB.
Line: 117 to 62
 
This is a simple table to keep track of site resources (please update if changes occur):
Changed:
<
<
Site CPU WMS Support Storage CVMFS NA62 S/W
Glasgow DONE DONE DONE DONE r261
IC DONE DONE DONE DONE r261
RAL DONE DONE DONE DONE r261
Liverpool DONE DONE DONE DONE r261
Birmingham DONE DONE DONE DONE r261
UCL DONE DONE DONE - r261
>
>
Site CPU Dirac Support Storage CVMFS
Glasgow DONE DONE DONE DONE
IC DONE DONE DONE DONE
RAL DONE DONE DONE DONE
Liverpool DONE DONE DONE DONE
Birmingham DONE DONE DONE DONE
UCL DONE DONE DONE DONE
CNAF DONE DONE DONE DONE
 
Changed:
<
<
Note on s/w labelling: both a NA62 release number (e.g. r261) and a tarball version (e.g. v9) are used.
>
>
Note on s/w labelling: both a NA62 release/version number (e.g. r261/v0.9.0) and a tarball version (e.g. v14) are used.
 They are kept track of in a dedicated DB table. FTS channels might not be needed is FTS3 is used (latest wrapper scripts).

Site admins - persons to contact in case something goes wrong on one of the sites are:

Changed:
<
<
  • Glasgow - Dan Protopopescu (dan.protopopeascu#glasgow.ac.uk), Stuart Purdie (stuart.purdie#glasgow.ac.uk)
>
>
  • Glasgow - Dan Protopopescu (dan.protopopeascu#glasgow.ac.uk)
 
  • IC - Janusz Martyniak (janusz.martyniak#imperial.ac.uk)
  • RAL - Andrew Lahiff (andrew.lahiff#stfc.ac.uk)
  • Liverpool - Stephen Jones (sjones#hep.ph.liv.ac.uk)
  • Birmingham - Mark Slater (mslater#cern.ch)
  • UCL - Sergey Podolsky (spodolsky#mail.ru), Pavel Demin (pavel.demin#uclouvain.be)
Added:
>
>
  • CNAF - Daniele Cesini (daniele.cesini#cnaf.infn.it)
 Please replace the #-s.

Changed:
<
<
NA62-GridPP-Sites.png
>
>
NA62-GridPP-Sites.png
 
Line: 175 to 122
 
META FILEATTACHMENT attachment="CreateTarballs.sh.txt" attr="" comment="(not final!) This is run on the Software Manager's UI machine to create the software tarballs for distribution" date="1336659806" name="CreateTarballs.sh.txt" path="CreateTarballs.sh.txt" size="2734" stream="CreateTarballs.sh.txt" tmpFilename="/usr/tmp/CGItemp24496" user="DanProtopopescu" version="1"
META FILEATTACHMENT attachment="na62-button-apple-small.png" attr="h" comment="iPhone webapp button" date="1339769065" name="na62-button-apple-small.png" path="na62-button-apple-small.png" size="10818" stream="na62-button-apple-small.png" tmpFilename="/usr/tmp/CGItemp46125" user="DanProtopopescu" version="3"
META FILEATTACHMENT attachment="NA62MC-org-chart.png" attr="" comment="NA62 MC workflow" date="1340294677" name="NA62MC-org-chart.png" path="NA62MC-org-chart.png" size="57752" stream="NA62MC-org-chart.png" tmpFilename="/usr/tmp/CGItemp23581" user="DanProtopopescu" version="1"
Changed:
<
<
META FILEATTACHMENT attachment="NA62-GridPP-Sites.png" attr="" comment="Sites Map (September 2012)" date="1347961595" name="NA62-GridPP-Sites.png" path="NA62-GridPP-Sites.png" size="90381" stream="NA62-GridPP-Sites.png" tmpFilename="/usr/tmp/CGItemp21810" user="DanProtopopescu" version="1"
>
>
META FILEATTACHMENT attachment="NA62-GridPP-Sites.png" attr="" comment="Sites Map (March 2016)" date="1457693850" name="NA62-GridPP-Sites.png" path="NA62-GridPP-Sites.png" size="162689" user="DanProtopopescu" version="2"
 
META FILEATTACHMENT attachment="Cronjobs-small.png" attr="" comment="Cronjobs diagram" date="1360934943" name="Cronjobs-small.png" path="Cronjobs-robots-small.png" size="153549" stream="Cronjobs-robots-small.png" tmpFilename="/usr/tmp/CGItemp30372" user="DanProtopopescu" version="6"
META TOPICMOVED by="DanProtopopescu" date="1335878717" from="Main.NA62MonteCarlo" to="NA62.NA62MonteCarlo"

Revision 562015-02-23 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 21 to 21
 

Installation from scratch on a SL6 desktop

Changed:
<
<
This is documented in a separate wiki ... NA62onSL6
>
>
This was done for testing purposes and is documented in a separate wiki, NA62onSL6.
 

Code compilation

Revision 552015-02-20 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 19 to 19
  Job submission and output management system code components are held on the na62.gla.ac.uk server. The files attached to this wiki are not the latest versions.
Added:
>
>

Installation from scratch on a SL6 desktop

This is documented in a separate wiki ... NA62onSL6

 

Code compilation

  • Versions:

Revision 542015-01-28 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 14 to 14
 

Software

Changed:
<
<
>
>
 
  • Obtain release number and read about modifications in NA62MC/HISTORY

Job submission and output management system code components are held on the na62.gla.ac.uk server. The files attached to this wiki are not the latest versions.

Revision 532013-04-23 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 17 to 17
 
  • Test tarball from Antonino Sergi: NA62MC.tgz
  • Obtain release number and read about modifications in NA62MC/HISTORY
Changed:
<
<
Job submission and output management system code components are held on and SVN repo at IC (access list: Dan, Janusz, Stuart). The files attached to this wiki are not the latest versions.
>
>
Job submission and output management system code components are held on the na62.gla.ac.uk server. The files attached to this wiki are not the latest versions.
 

Code compilation

Revision 522013-03-18 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 119 to 119
 
RAL DONE DONE DONE DONE r261
Liverpool DONE DONE DONE DONE r261
Birmingham DONE DONE DONE DONE r261
Changed:
<
<
UCL DONE DONE DONE   r244
>
>
UCL DONE DONE DONE - r261
 
Changed:
<
<
Note on s/w labelling: both a NA62 release number (e.g. r170) and a tarball version (e.g. v5) are used.
>
>
Note on s/w labelling: both a NA62 release number (e.g. r261) and a tarball version (e.g. v9) are used.
 They are kept track of in a dedicated DB table. FTS channels might not be needed is FTS3 is used (latest wrapper scripts).

Site admins - persons to contact in case something goes wrong on one of the sites are:

Line: 144 to 144
 Cronjobs are employed to keep track of jobs and output files:
Cronjobs-small.png
Deleted:
<
<

To do before next production round

  • Check r261 outputs with Tonino
 

Troubleshooting

To log problems encountered (and solutions) use this wiki:

Revision 512013-03-12 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 117 to 117
 
Glasgow DONE DONE DONE DONE r261
IC DONE DONE DONE DONE r261
RAL DONE DONE DONE DONE r261
Changed:
<
<
Liverpool DONE DONE DONE DONE r193
>
>
Liverpool DONE DONE DONE DONE r261
 
Birmingham DONE DONE DONE DONE r261
UCL DONE DONE DONE   r244
Line: 147 to 147
 

To do before next production round

Changed:
<
<
  • Patch r244 and distribute via CVMFS (Dan & Janusz)
>
>
  • Check r261 outputs with Tonino
 

Troubleshooting

Revision 502013-03-11 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 114 to 114
 This is a simple table to keep track of site resources (please update if changes occur):

Site CPU WMS Support Storage CVMFS NA62 S/W
Changed:
<
<
Glasgow DONE DONE DONE DONE r244
IC DONE DONE DONE DONE r244
>
>
Glasgow DONE DONE DONE DONE r261
IC DONE DONE DONE DONE r261
 
RAL DONE DONE DONE DONE r261
Liverpool DONE DONE DONE DONE r193
Changed:
<
<
Birmingham DONE DONE DONE DONE r244
>
>
Birmingham DONE DONE DONE DONE r261
 
UCL DONE DONE DONE   r244

Note on s/w labelling: both a NA62 release number (e.g. r170) and a tarball version (e.g. v5) are used.

Revision 492013-03-11 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 114 to 114
 This is a simple table to keep track of site resources (please update if changes occur):

Site CPU WMS Support Storage CVMFS NA62 S/W
Changed:
<
<
Glasgow DONE DONE DONE   r244
>
>
Glasgow DONE DONE DONE DONE r244
 
IC DONE DONE DONE DONE r244
Changed:
<
<
RAL DONE DONE DONE DONE r244
Liverpool DONE DONE DONE   r193
Birmingham DONE DONE DONE   r244
>
>
RAL DONE DONE DONE DONE r261
Liverpool DONE DONE DONE DONE r193
Birmingham DONE DONE DONE DONE r244
 
UCL DONE DONE DONE   r244

Note on s/w labelling: both a NA62 release number (e.g. r170) and a tarball version (e.g. v5) are used.

Revision 482013-03-05 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 25 to 25
 
    • v4 - Contains bug in the MC which Tonino spotted after giving us the tar ball
    • v5 - Installed by Janusz on May 10 (r170, see table below)
    • v6 - September 2012 version (r188)
Changed:
<
<
    • v7 - r193 uses GEANT4 9.5p01, improves processing speed roughly by a factor of 5 wrt the previous version
    • v8 - r244 installed directly or via CVMFS (see table below)
>
>
    • v7 - r193: uses GEANT4 9.5p01, improves processing speed roughly by a factor of 5 wrt the previous version
    • v8 - r244: installed directly or via CVMFS (see table below)
    • v9 - r261: fatal bug in 4-lepton decay generator fixed, memory leak fixed, points to geant4 9.5.p02
 
  • Unpack tarball to NA62MC
  • Fix libs path problems in GNUMakefile-s (Generator/ and Beam/)
Line: 51 to 52
  tar -czvf ~/Boost_libs.tgz sw/lcg/external/Boost/1.48.0_python2.6/x86_64-slc5-gcc43-opt/lib
  • Add the G4EMLOW6.23.tgz tarball
Added:
>
>
      [protopop@ppepc102 gridsw]$ ls v8/
      Boost_libs_v8.tgz  G4EMLOW6.23.tgz  G4_libs_v8.tgz  NA62MC_v8.tgz  Qt_libs_v8.tgz  sqlite_v8.tgz
 
  • Run a special job (with an SGM Admin role) to wget and unpack these tarballs on sites (InstallSoftware.sh)
  • Run test job
Line: 58 to 63
  CVMFS is used for software distribution to sites. A guide is provided here: https://www.gridpp.ac.uk/wiki/RALnonLHCCVMFS
Added:
>
>

Scripts preparation

The following files need to be checked and updated for each new s/w release:

  • Job wrapper script - update version number
  • .mac file - check for new settings, new decay types etc.
  • input_files.tgz - check if any new files need to be included in this archive

Read the HISTORY file to find out what changes might have appeared.

 

Job submission

  • Run parameters given in the StandardRun.mac file: run number, number of events, random seed, root output file name

Revision 472013-02-28 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 54 to 54
 
  • Run a special job (with an SGM Admin role) to wget and unpack these tarballs on sites (InstallSoftware.sh)
  • Run test job
Added:
>
>

CVMFS

CVMFS is used for software distribution to sites. A guide is provided here: https://www.gridpp.ac.uk/wiki/RALnonLHCCVMFS

 

Job submission

  • Run parameters given in the StandardRun.mac file: run number, number of events, random seed, root output file name

Revision 462013-02-15 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 99 to 99
 
IC DONE DONE DONE DONE r244
RAL DONE DONE DONE DONE r244
Liverpool DONE DONE DONE   r193
Changed:
<
<
Birmingham DONE DONE DONE   r193
>
>
Birmingham DONE DONE DONE   r244
 
UCL DONE DONE DONE   r244

Note on s/w labelling: both a NA62 release number (e.g. r170) and a tarball version (e.g. v5) are used.

Revision 452013-02-15 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 81 to 81
  The script creator interface lets the production manager (PM) set up the job parameters (both MC parameters and output options) and choose the number of 'runs' to launch. The scripter prints out in the browser a list of commands that the PM has to run on his grid UI in order to launch the job or production with his credentials. For individual jobs, the PM updates the DB by hand, for multiple runs, it is done via a wrapper script. A cron job periodically updates the list of jobs and files in the DB.
Added:
>
>

Ganga

Mark Slater set up a Ganga service (http://ganga.web.cern.ch/ganga/) for managing entire production rounds. The Ganga request form is accessed via a link on the Production table. Once the request is submitted, a Ganga daemon from the Grid UI machine picks it up (within 10 minutes) and will submit and monitor jobs until the events target is reached.

How the Ganga service was set up and interacts with the other bot components is described in here: GangaService4NA62

 

Site resources

Line: 115 to 123
 

Cron jobs

Cronjobs are employed to keep track of jobs and output files:

Changed:
<
<
Cronjobs-small.png
>
>
Cronjobs-small.png
 

To do before next production round

Line: 150 to 158
 
META FILEATTACHMENT attachment="na62-button-apple-small.png" attr="h" comment="iPhone webapp button" date="1339769065" name="na62-button-apple-small.png" path="na62-button-apple-small.png" size="10818" stream="na62-button-apple-small.png" tmpFilename="/usr/tmp/CGItemp46125" user="DanProtopopescu" version="3"
META FILEATTACHMENT attachment="NA62MC-org-chart.png" attr="" comment="NA62 MC workflow" date="1340294677" name="NA62MC-org-chart.png" path="NA62MC-org-chart.png" size="57752" stream="NA62MC-org-chart.png" tmpFilename="/usr/tmp/CGItemp23581" user="DanProtopopescu" version="1"
META FILEATTACHMENT attachment="NA62-GridPP-Sites.png" attr="" comment="Sites Map (September 2012)" date="1347961595" name="NA62-GridPP-Sites.png" path="NA62-GridPP-Sites.png" size="90381" stream="NA62-GridPP-Sites.png" tmpFilename="/usr/tmp/CGItemp21810" user="DanProtopopescu" version="1"
Changed:
<
<
META FILEATTACHMENT attachment="Cronjobs-small.png" attr="" comment="Cronjobs diagram" date="1350557599" name="Cronjobs-small.png" path="Cronjobs-robots-small.png" size="130730" stream="Cronjobs-robots-small.png" tmpFilename="/usr/tmp/CGItemp27136" user="DanProtopopescu" version="5"
>
>
META FILEATTACHMENT attachment="Cronjobs-small.png" attr="" comment="Cronjobs diagram" date="1360934943" name="Cronjobs-small.png" path="Cronjobs-robots-small.png" size="153549" stream="Cronjobs-robots-small.png" tmpFilename="/usr/tmp/CGItemp30372" user="DanProtopopescu" version="6"
 
META TOPICMOVED by="DanProtopopescu" date="1335878717" from="Main.NA62MonteCarlo" to="NA62.NA62MonteCarlo"

Revision 442013-01-29 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 120 to 120
 

To do before next production round

Changed:
<
<
  • Install new tarball once it's available from Tonino, using CVMFS (Dan & Janusz)
>
>
  • Patch r244 and distribute via CVMFS (Dan & Janusz)
 

Troubleshooting

Revision 432013-01-28 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 26 to 26
 
    • v5 - Installed by Janusz on May 10 (r170, see table below)
    • v6 - September 2012 version (r188)
    • v7 - r193 uses GEANT4 9.5p01, improves processing speed roughly by a factor of 5 wrt the previous version
Changed:
<
<
    • v8 - to be installed
>
>
    • v8 - r244 installed directly or via CVMFS (see table below)
 
  • Unpack tarball to NA62MC
  • Fix libs path problems in GNUMakefile-s (Generator/ and Beam/)
Line: 86 to 86
 
This is a simple table to keep track of site resources (please update if changes occur):
Changed:
<
<
Site CPU WMS Support Storage FTS Channel to RAL FTS Channel to CERN NA62 S/W
Glasgow DONE DONE DONE DONE DONE r193
IC DONE DONE DONE DONE DONE r193
RAL DONE DONE DONE -- PEND r193
Liverpool DONE DONE DONE DONE PEND r193
Birmingham DONE DONE DONE DONE PEND r193
UCL DONE DONE DONE     PEND
>
>
Site CPU WMS Support Storage CVMFS NA62 S/W
Glasgow DONE DONE DONE   r244
IC DONE DONE DONE DONE r244
RAL DONE DONE DONE DONE r244
Liverpool DONE DONE DONE   r193
Birmingham DONE DONE DONE   r193
UCL DONE DONE DONE   r244
  Note on s/w labelling: both a NA62 release number (e.g. r170) and a tarball version (e.g. v5) are used. They are kept track of in a dedicated DB table. FTS channels might not be needed is FTS3 is used (latest wrapper scripts).

Revision 422013-01-08 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 48 to 48
  cd /afs/cern.ch tar -czvf ~/G4-libs.tgz sw/lcg/external/geant4/9.5.p01/x86_64-slc5-gcc43/lib tar -czvf ~/Qt-libs.tgz sw/lcg/external/qt/4.6.3p2/x86_64-slc5-gcc43-opt/lib
Added:
>
>
tar -czvf ~/Boost_libs.tgz sw/lcg/external/Boost/1.48.0_python2.6/x86_64-slc5-gcc43-opt/lib
 
Added:
>
>
  • Add the G4EMLOW6.23.tgz tarball
 
  • Run a special job (with an SGM Admin role) to wget and unpack these tarballs on sites (InstallSoftware.sh)
  • Run test job

Revision 412012-12-07 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 26 to 26
 
    • v5 - Installed by Janusz on May 10 (r170, see table below)
    • v6 - September 2012 version (r188)
    • v7 - r193 uses GEANT4 9.5p01, improves processing speed roughly by a factor of 5 wrt the previous version
Added:
>
>
    • v8 - to be installed
 
  • Unpack tarball to NA62MC
  • Fix libs path problems in GNUMakefile-s (Generator/ and Beam/)
Line: 117 to 118
 

To do before next production round

Changed:
<
<
  • Install new tarball once it's available from Tonino (Dan & Janusz)
  • Open remaining FTS channels (Janusz)
  • DONE Move web UI to a dedicated monitoring webserver (Dan)
  • Set up a controller-side cron job to double-check the FTS transfers of job outputs to CERN (Janusz)
>
>
  • Install new tarball once it's available from Tonino, using CVMFS (Dan & Janusz)
 

Troubleshooting

Revision 402012-12-05 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 15 to 15
 

Software

Changed:
<
<
  • Job submission and output management system code components are held on and SVN repo at IC (access list: Dan, Janusz, Stuart). The files attached to this wiki are not the latest versions.
>
>
  • Obtain release number and read about modifications in NA62MC/HISTORY

Job submission and output management system code components are held on and SVN repo at IC (access list: Dan, Janusz, Stuart). The files attached to this wiki are not the latest versions.

 

Code compilation

Revision 392012-10-25 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 85 to 85
 
Glasgow DONE DONE DONE DONE DONE r193
IC DONE DONE DONE DONE DONE r193
RAL DONE DONE DONE -- PEND r193
Changed:
<
<
Liverpool DONE DONE DONE DONE PEND PEND
>
>
Liverpool DONE DONE DONE DONE PEND r193
 
Birmingham DONE DONE DONE DONE PEND r193
UCL DONE DONE DONE     PEND

Revision 382012-10-24 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 82 to 82
 This is a simple table to keep track of site resources (please update if changes occur):

Site CPU WMS Support Storage FTS Channel to RAL FTS Channel to CERN NA62 S/W
Changed:
<
<
Glasgow DONE DONE DONE DONE DONE r188
IC DONE DONE DONE DONE DONE r188+r193
RAL DONE DONE DONE -- PEND r188+r193
Liverpool DONE DONE DONE DONE PEND  
Birmingham DONE DONE DONE DONE PEND r188+r193
UCL DONE DONE DONE      
>
>
Glasgow DONE DONE DONE DONE DONE r193
IC DONE DONE DONE DONE DONE r193
RAL DONE DONE DONE -- PEND r193
Liverpool DONE DONE DONE DONE PEND PEND
Birmingham DONE DONE DONE DONE PEND r193
UCL DONE DONE DONE     PEND
  Note on s/w labelling: both a NA62 release number (e.g. r170) and a tarball version (e.g. v5) are used. They are kept track of in a dedicated DB table. FTS channels might not be needed is FTS3 is used (latest wrapper scripts).

Revision 372012-10-22 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 84 to 84
 
Site CPU WMS Support Storage FTS Channel to RAL FTS Channel to CERN NA62 S/W
Glasgow DONE DONE DONE DONE DONE r188
IC DONE DONE DONE DONE DONE r188+r193
Changed:
<
<
RAL DONE DONE DONE -- PEND r188
>
>
RAL DONE DONE DONE -- PEND r188+r193
 
Liverpool DONE DONE DONE DONE PEND  
Birmingham DONE DONE DONE DONE PEND r188+r193
UCL DONE DONE DONE      

Revision 362012-10-22 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 86 to 86
 
IC DONE DONE DONE DONE DONE r188+r193
RAL DONE DONE DONE -- PEND r188
Liverpool DONE DONE DONE DONE PEND  
Changed:
<
<
Birmingham DONE DONE DONE DONE PEND r188
>
>
Birmingham DONE DONE DONE DONE PEND r188+r193
 
UCL DONE DONE DONE      

Note on s/w labelling: both a NA62 release number (e.g. r170) and a tarball version (e.g. v5) are used.

Revision 352012-10-19 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 83 to 83
 
Site CPU WMS Support Storage FTS Channel to RAL FTS Channel to CERN NA62 S/W
Glasgow DONE DONE DONE DONE DONE r188
Changed:
<
<
IC DONE DONE DONE DONE DONE r188
>
>
IC DONE DONE DONE DONE DONE r188+r193
 
RAL DONE DONE DONE -- PEND r188
Liverpool DONE DONE DONE DONE PEND  
Birmingham DONE DONE DONE DONE PEND r188

Revision 342012-10-19 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 86 to 86
 
IC DONE DONE DONE DONE DONE r188
RAL DONE DONE DONE -- PEND r188
Liverpool DONE DONE DONE DONE PEND  
Changed:
<
<
Birmingham DONE DONE DONE DONE PEND  
>
>
Birmingham DONE DONE DONE DONE PEND r188
 
UCL DONE DONE DONE      

Note on s/w labelling: both a NA62 release number (e.g. r170) and a tarball version (e.g. v5) are used.

Revision 332012-10-19 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 90 to 90
 
UCL DONE DONE DONE      

Note on s/w labelling: both a NA62 release number (e.g. r170) and a tarball version (e.g. v5) are used.

Changed:
<
<
They are kept track of in a dedicated DB table.
>
>
They are kept track of in a dedicated DB table. FTS channels might not be needed is FTS3 is used (latest wrapper scripts).
  Site admins - persons to contact in case something goes wrong on one of the sites are:
Changed:
<
<
  • Glasgow - Dan Protpopescu (dan.protopopeascu#glasgow.ac.uk), Stuart Purdie (stuart.purdie#glasgow.ac.uk)
>
>
  • Glasgow - Dan Protopopescu (dan.protopopeascu#glasgow.ac.uk), Stuart Purdie (stuart.purdie#glasgow.ac.uk)
 
  • IC - Janusz Martyniak (janusz.martyniak#imperial.ac.uk)
  • RAL - Andrew Lahiff (andrew.lahiff#stfc.ac.uk)
  • Liverpool - Stephen Jones (sjones#hep.ph.liv.ac.uk)

Revision 322012-10-18 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 84 to 84
 
Site CPU WMS Support Storage FTS Channel to RAL FTS Channel to CERN NA62 S/W
Glasgow DONE DONE DONE DONE DONE r188
IC DONE DONE DONE DONE DONE r188
Changed:
<
<
RAL DONE DONE DONE -- Processing, transparent r188
Liverpool DONE DONE DONE DONE Processing, transparent  
Birmingham DONE DONE DONE DONE Processing, transparent  
>
>
RAL DONE DONE DONE -- PEND r188
Liverpool DONE DONE DONE DONE PEND  
Birmingham DONE DONE DONE DONE PEND  
 
UCL DONE DONE DONE      

Note on s/w labelling: both a NA62 release number (e.g. r170) and a tarball version (e.g. v5) are used.

Line: 110 to 110
 

Cron jobs

Cronjobs are employed to keep track of jobs and output files:

Changed:
<
<
Cronjobs-small.png
>
>
Cronjobs-small.png
 

To do before next production round

  • Install new tarball once it's available from Tonino (Dan & Janusz)
  • Open remaining FTS channels (Janusz)
Changed:
<
<
  • Move web UI to a dedicated monitoring webserver (Dan)
>
>
  • DONE Move web UI to a dedicated monitoring webserver (Dan)
 
  • Set up a controller-side cron job to double-check the FTS transfers of job outputs to CERN (Janusz)

Troubleshooting

Line: 148 to 148
 
META FILEATTACHMENT attachment="na62-button-apple-small.png" attr="h" comment="iPhone webapp button" date="1339769065" name="na62-button-apple-small.png" path="na62-button-apple-small.png" size="10818" stream="na62-button-apple-small.png" tmpFilename="/usr/tmp/CGItemp46125" user="DanProtopopescu" version="3"
META FILEATTACHMENT attachment="NA62MC-org-chart.png" attr="" comment="NA62 MC workflow" date="1340294677" name="NA62MC-org-chart.png" path="NA62MC-org-chart.png" size="57752" stream="NA62MC-org-chart.png" tmpFilename="/usr/tmp/CGItemp23581" user="DanProtopopescu" version="1"
META FILEATTACHMENT attachment="NA62-GridPP-Sites.png" attr="" comment="Sites Map (September 2012)" date="1347961595" name="NA62-GridPP-Sites.png" path="NA62-GridPP-Sites.png" size="90381" stream="NA62-GridPP-Sites.png" tmpFilename="/usr/tmp/CGItemp21810" user="DanProtopopescu" version="1"
Changed:
<
<
META FILEATTACHMENT attachment="Cronjobs-small.png" attr="" comment="Cronjobs diagram" date="1349706502" name="Cronjobs-small.png" path="Cronjob-robots.png" size="120930" stream="Cronjob-robots.png" tmpFilename="/usr/tmp/CGItemp24356" user="DanProtopopescu" version="4"
>
>
META FILEATTACHMENT attachment="Cronjobs-small.png" attr="" comment="Cronjobs diagram" date="1350557599" name="Cronjobs-small.png" path="Cronjobs-robots-small.png" size="130730" stream="Cronjobs-robots-small.png" tmpFilename="/usr/tmp/CGItemp27136" user="DanProtopopescu" version="5"
 
META TOPICMOVED by="DanProtopopescu" date="1335878717" from="Main.NA62MonteCarlo" to="NA62.NA62MonteCarlo"

Revision 312012-10-15 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 30 to 30
  cp ../NA62/backups/Beam.GNUmakefile Beam/GNUmakefile cp ../NA62/backups/Generator.GNUmakefile Generator/GNUmakefile
Changed:
<
<
  • Make changes to env.sh: source 'offline' environment, clear XAWLIBS variable, rearrange paths (move system paths after NA62-specific paths), basically pay attention to these two lines:
>
>
  • Make changes to env.sh: source 'offline' environment, clear XAWLIBS variable, rearrange paths (move system paths after NA62-specific paths):
 
Deleted:
<
<
source /afs/cern.ch/na62/offline/geant4.9.5.p01/env.sh
  export XAWLIBS=" "
  • Source the the environment file (source scripts/env.sh)

Revision 302012-10-12 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 68 to 68
  There's now a suite of tools for submitting jobs either one-by-one or in bulk. Jobs submitted via this interface have the advantage that everything about the jobs (input and output files, scripts, run parameters and statistics) are archived for later reference.
Changed:
<
<
>
>
 
Changed:
<
<
>
>
  The script creator interface lets the production manager (PM) set up the job parameters (both MC parameters and output options) and choose the number of 'runs' to launch. The scripter prints out in the browser a list of commands that the PM has to run on his grid UI in order to launch the job or production with his credentials. For individual jobs, the PM updates the DB by hand, for multiple runs, it is done via a wrapper script. A cron job periodically updates the list of jobs and files in the DB.

Revision 292012-10-09 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 35 to 35
  source /afs/cern.ch/na62/offline/geant4.9.5.p01/env.sh export XAWLIBS=" "
Changed:
<
<
  • Source the the environment file (source scripts/env.sh), then type make and be patient
>
>
  • Source the the environment file (source scripts/env.sh)
  • Type make and be patient
 

Code distribution

Changed:
<
<
>
>
      tar -czvf NA62MC_v7.tgz NA62MC
      cd /afs/cern.ch
      tar -czvf ~/G4-libs.tgz sw/lcg/external/geant4/9.5.p01/x86_64-slc5-gcc43/lib
      tar -czvf ~/Qt-libs.tgz sw/lcg/external/qt/4.6.3p2/x86_64-slc5-gcc43-opt/lib
  • Run a special job (with an SGM Admin role) to wget and unpack these tarballs on sites (InstallSoftware.sh)
 
  • Run test job

Job submission

Line: 83 to 90
 
Birmingham DONE DONE DONE DONE Processing, transparent  
UCL DONE DONE DONE      
Changed:
<
<
Note on s/w labelling: both a NA62 release number (e.g. r170) and a tarball version (e.g. v5) are used. They are kept track of in a dedicated DB table.
>
>
Note on s/w labelling: both a NA62 release number (e.g. r170) and a tarball version (e.g. v5) are used. They are kept track of in a dedicated DB table.
 
Changed:
<
<
Cronjobs are employed to keep track of jobs and output files:
Cronjobs-small.png
>
>
Site admins - persons to contact in case something goes wrong on one of the sites are:
  • Glasgow - Dan Protpopescu (dan.protopopeascu#glasgow.ac.uk), Stuart Purdie (stuart.purdie#glasgow.ac.uk)
  • IC - Janusz Martyniak (janusz.martyniak#imperial.ac.uk)
  • RAL - Andrew Lahiff (andrew.lahiff#stfc.ac.uk)
  • Liverpool - Stephen Jones (sjones#hep.ph.liv.ac.uk)
  • Birmingham - Mark Slater (mslater#cern.ch)
  • UCL - Sergey Podolsky (spodolsky#mail.ru), Pavel Demin (pavel.demin#uclouvain.be)
Please replace the #-s.
 
Line: 94 to 108
 
Added:
>
>

Cron jobs

Cronjobs are employed to keep track of jobs and output files:
Cronjobs-small.png

 

To do before next production round

  • Install new tarball once it's available from Tonino (Dan & Janusz)
Line: 101 to 121
 
  • Move web UI to a dedicated monitoring webserver (Dan)
  • Set up a controller-side cron job to double-check the FTS transfers of job outputs to CERN (Janusz)
Added:
>
>

Troubleshooting

 To log problems encountered (and solutions) use this wiki:

Revision 282012-10-08 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 23 to 23
 
    • v4 - Contains bug in the MC which Tonino spotted after giving us the tar ball
    • v5 - Installed by Janusz on May 10 (r170, see table below)
    • v6 - September 2012 version (r188)
Changed:
<
<
    • v7 - r193 uses GEANT4 9.5p01, improves processing speed roughly by a facto 5 wrt the previous version
>
>
    • v7 - r193 uses GEANT4 9.5p01, improves processing speed roughly by a factor of 5 wrt the previous version
 
  • Unpack tarball to NA62MC
  • Fix libs path problems in GNUMakefile-s (Generator/ and Beam/)
Line: 85 to 85
  Note on s/w labelling: both a NA62 release number (e.g. r170) and a tarball version (e.g. v5) are used. They are kept track of in a dedicated DB table.
Changed:
<
<
Two cronjobs are employed to keep track of jobs and output files:
Cronjobs-small.png
>
>
Cronjobs are employed to keep track of jobs and output files:
Cronjobs-small.png
 
Line: 127 to 127
 
META FILEATTACHMENT attachment="na62-button-apple-small.png" attr="h" comment="iPhone webapp button" date="1339769065" name="na62-button-apple-small.png" path="na62-button-apple-small.png" size="10818" stream="na62-button-apple-small.png" tmpFilename="/usr/tmp/CGItemp46125" user="DanProtopopescu" version="3"
META FILEATTACHMENT attachment="NA62MC-org-chart.png" attr="" comment="NA62 MC workflow" date="1340294677" name="NA62MC-org-chart.png" path="NA62MC-org-chart.png" size="57752" stream="NA62MC-org-chart.png" tmpFilename="/usr/tmp/CGItemp23581" user="DanProtopopescu" version="1"
META FILEATTACHMENT attachment="NA62-GridPP-Sites.png" attr="" comment="Sites Map (September 2012)" date="1347961595" name="NA62-GridPP-Sites.png" path="NA62-GridPP-Sites.png" size="90381" stream="NA62-GridPP-Sites.png" tmpFilename="/usr/tmp/CGItemp21810" user="DanProtopopescu" version="1"
Changed:
<
<
META FILEATTACHMENT attachment="Cronjobs-small.png" attr="" comment="Cronjobs diagram" date="1349451072" name="Cronjobs-small.png" path="Cronjobs-robots-small.png" size="91574" stream="Cronjobs-robots-small.png" tmpFilename="/usr/tmp/CGItemp24078" user="DanProtopopescu" version="3"
>
>
META FILEATTACHMENT attachment="Cronjobs-small.png" attr="" comment="Cronjobs diagram" date="1349706502" name="Cronjobs-small.png" path="Cronjob-robots.png" size="120930" stream="Cronjob-robots.png" tmpFilename="/usr/tmp/CGItemp24356" user="DanProtopopescu" version="4"
 
META TOPICMOVED by="DanProtopopescu" date="1335878717" from="Main.NA62MonteCarlo" to="NA62.NA62MonteCarlo"

Revision 272012-10-05 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 85 to 85
  Note on s/w labelling: both a NA62 release number (e.g. r170) and a tarball version (e.g. v5) are used. They are kept track of in a dedicated DB table.
Added:
>
>
Two cronjobs are employed to keep track of jobs and output files:
Cronjobs-small.png
 
NA62-GridPP-Sites.png
Line: 124 to 127
 
META FILEATTACHMENT attachment="na62-button-apple-small.png" attr="h" comment="iPhone webapp button" date="1339769065" name="na62-button-apple-small.png" path="na62-button-apple-small.png" size="10818" stream="na62-button-apple-small.png" tmpFilename="/usr/tmp/CGItemp46125" user="DanProtopopescu" version="3"
META FILEATTACHMENT attachment="NA62MC-org-chart.png" attr="" comment="NA62 MC workflow" date="1340294677" name="NA62MC-org-chart.png" path="NA62MC-org-chart.png" size="57752" stream="NA62MC-org-chart.png" tmpFilename="/usr/tmp/CGItemp23581" user="DanProtopopescu" version="1"
META FILEATTACHMENT attachment="NA62-GridPP-Sites.png" attr="" comment="Sites Map (September 2012)" date="1347961595" name="NA62-GridPP-Sites.png" path="NA62-GridPP-Sites.png" size="90381" stream="NA62-GridPP-Sites.png" tmpFilename="/usr/tmp/CGItemp21810" user="DanProtopopescu" version="1"
Added:
>
>
META FILEATTACHMENT attachment="Cronjobs-small.png" attr="" comment="Cronjobs diagram" date="1349451072" name="Cronjobs-small.png" path="Cronjobs-robots-small.png" size="91574" stream="Cronjobs-robots-small.png" tmpFilename="/usr/tmp/CGItemp24078" user="DanProtopopescu" version="3"
 
META TOPICMOVED by="DanProtopopescu" date="1335878717" from="Main.NA62MonteCarlo" to="NA62.NA62MonteCarlo"

Revision 262012-10-04 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 23 to 23
 
    • v4 - Contains bug in the MC which Tonino spotted after giving us the tar ball
    • v5 - Installed by Janusz on May 10 (r170, see table below)
    • v6 - September 2012 version (r188)
Added:
>
>
    • v7 - r193 uses GEANT4 9.5p01, improves processing speed roughly by a facto 5 wrt the previous version
 
  • Unpack tarball to NA62MC
  • Fix libs path problems in GNUMakefile-s (Generator/ and Beam/)
     cp ../NA62/backups/Beam.GNUmakefile Beam/GNUmakefile
     cp ../NA62/backups/Generator.GNUmakefile Generator/GNUmakefile 
Changed:
<
<
  • Make changes to env.sh: source 'offline' environment, clear XAWLIBS variable, rearrange paths (move system paths after NA62-specific paths)
>
>
  • Make changes to env.sh: source 'offline' environment, clear XAWLIBS variable, rearrange paths (move system paths after NA62-specific paths), basically pay attention to these two lines:
 
Changed:
<
<
source /afs/cern.ch/na62/offline/geant4.9.4.p01/env.sh
>
>
source /afs/cern.ch/na62/offline/geant4.9.5.p01/env.sh
  export XAWLIBS=" "
Changed:
<
<
  • Type 'make' and be patient
>
>
  • Source the the environment file (source scripts/env.sh), then type make and be patient
 

Code distribution

Revision 252012-10-03 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

NA62 Monte Carlo Simulations

Line: 60 to 60
  There's now a suite of tools for submitting jobs either one-by-one or in bulk. Jobs submitted via this interface have the advantage that everything about the jobs (input and output files, scripts, run parameters and statistics) are archived for later reference.
Changed:
<
<
>
>
 
Line: 75 to 75
 This is a simple table to keep track of site resources (please update if changes occur):

Site CPU WMS Support Storage FTS Channel to RAL FTS Channel to CERN NA62 S/W
Changed:
<
<
Glasgow DONE DONE   DONE DONE r170
IC DONE DONE DONE DONE   r170
RAL DONE DONE   -   r170
Liverpool DONE DONE DONE      
Birmingham DONE DONE DONE      
UCL DONE          
>
>
Glasgow DONE DONE DONE DONE DONE r188
IC DONE DONE DONE DONE DONE r188
RAL DONE DONE DONE -- Processing, transparent r188
Liverpool DONE DONE DONE DONE Processing, transparent  
Birmingham DONE DONE DONE DONE Processing, transparent  
UCL DONE DONE DONE      
  Note on s/w labelling: both a NA62 release number (e.g. r170) and a tarball version (e.g. v5) are used. They are kept track of in a dedicated DB table.
Added:
>
>
 
NA62-GridPP-Sites.png
Line: 91 to 92
 

To do before next production round

Changed:
<
<
  • Install new tarball once it's available from Tonino
  • DONE Resolve CERN FTS channel problem GGUS ticket #83133
  • DONE Scale scripts for largers submissions
>
>
  • Install new tarball once it's available from Tonino (Dan & Janusz)
  • Open remaining FTS channels (Janusz)
  • Move web UI to a dedicated monitoring webserver (Dan)
  • Set up a controller-side cron job to double-check the FTS transfers of job outputs to CERN (Janusz)
  To log problems encountered (and solutions) use this wiki:

Revision 242012-09-21 - DanProtopopescu

Line: 1 to 1
Changed:
<
<
META TOPICPARENT name="NA62"
>
>
META TOPICPARENT name="WebHome"
 

NA62 Monte Carlo Simulations

Revision 232012-09-18 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="NA62"

NA62 Monte Carlo Simulations

Line: 29 to 29
  cp ../NA62/backups/Beam.GNUmakefile Beam/GNUmakefile cp ../NA62/backups/Generator.GNUmakefile Generator/GNUmakefile
Changed:
<
<
  • Make changes to env.sh: source 'offline' environment; clear XAWLIBS variable
  • Rearrange paths (move system paths after NA62-specific paths)
>
>
  • Make changes to env.sh: source 'offline' environment, clear XAWLIBS variable, rearrange paths (move system paths after NA62-specific paths)
     source /afs/cern.ch/na62/offline/geant4.9.4.p01/env.sh
     export XAWLIBS=" "
 
  • Type 'make' and be patient

Code distribution

Revision 222012-09-18 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="NA62"

NA62 Monte Carlo Simulations

Line: 68 to 68
 

Site resources

Added:
>
>
 This is a simple table to keep track of site resources (please update if changes occur):

Site CPU WMS Support Storage FTS Channel to RAL FTS Channel to CERN NA62 S/W
Changed:
<
<
Glasgow DONE DONE   DONE   r170
>
>
Glasgow DONE DONE   DONE DONE r170
 
IC DONE DONE DONE DONE   r170
RAL DONE DONE   -   r170
Liverpool DONE DONE DONE      
Changed:
<
<
UCL            
>
>
Birmingham DONE DONE DONE      
UCL DONE          
  Note on s/w labelling: both a NA62 release number (e.g. r170) and a tarball version (e.g. v5) are used. They are kept track of in a dedicated DB table.
Added:
>
>
NA62-GridPP-Sites.png
 

To do before next production round

Line: 109 to 116
 
META FILEATTACHMENT attachment="CreateTarballs.sh.txt" attr="" comment="(not final!) This is run on the Software Manager's UI machine to create the software tarballs for distribution" date="1336659806" name="CreateTarballs.sh.txt" path="CreateTarballs.sh.txt" size="2734" stream="CreateTarballs.sh.txt" tmpFilename="/usr/tmp/CGItemp24496" user="DanProtopopescu" version="1"
META FILEATTACHMENT attachment="na62-button-apple-small.png" attr="h" comment="iPhone webapp button" date="1339769065" name="na62-button-apple-small.png" path="na62-button-apple-small.png" size="10818" stream="na62-button-apple-small.png" tmpFilename="/usr/tmp/CGItemp46125" user="DanProtopopescu" version="3"
META FILEATTACHMENT attachment="NA62MC-org-chart.png" attr="" comment="NA62 MC workflow" date="1340294677" name="NA62MC-org-chart.png" path="NA62MC-org-chart.png" size="57752" stream="NA62MC-org-chart.png" tmpFilename="/usr/tmp/CGItemp23581" user="DanProtopopescu" version="1"
Added:
>
>
META FILEATTACHMENT attachment="NA62-GridPP-Sites.png" attr="" comment="Sites Map (September 2012)" date="1347961595" name="NA62-GridPP-Sites.png" path="NA62-GridPP-Sites.png" size="90381" stream="NA62-GridPP-Sites.png" tmpFilename="/usr/tmp/CGItemp21810" user="DanProtopopescu" version="1"
 
META TOPICMOVED by="DanProtopopescu" date="1335878717" from="Main.NA62MonteCarlo" to="NA62.NA62MonteCarlo"

Revision 212012-09-17 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="NA62"

NA62 Monte Carlo Simulations

Line: 21 to 21
 
  • Versions:
    • v4 - Contains bug in the MC which Tonino spotted after giving us the tar ball
Changed:
<
<
    • v5 - Installed by Janusz on May 10 (see table below)
>
>
    • v5 - Installed by Janusz on May 10 (r170, see table below)
    • v6 - September 2012 version (r188)
 
  • Unpack tarball to NA62MC
  • Fix libs path problems in GNUMakefile-s (Generator/ and Beam/)
Added:
>
>
     cp ../NA62/backups/Beam.GNUmakefile Beam/GNUmakefile
     cp ../NA62/backups/Generator.GNUmakefile Generator/GNUmakefile 
 
  • Make changes to env.sh: source 'offline' environment; clear XAWLIBS variable
  • Rearrange paths (move system paths after NA62-specific paths)
  • Type 'make' and be patient

Revision 202012-09-17 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="NA62"

NA62 Monte Carlo Simulations

Line: 14 to 14
 

Software

Changed:
<
<
>
>
 
  • Job submission and output management system code components are held on and SVN repo at IC (access list: Dan, Janusz, Stuart). The files attached to this wiki are not the latest versions.

Code compilation

Line: 96 to 96
  VOMS_CA_DN="'/C=UK/O=eScienceCA/OU=Authority/CN=UK e-Science CA 2B'" verbatim -->
Deleted:
<
<
META FILEATTACHMENT attachment="NA62MC.tar.zip" attr="" comment="Test tarball from Antonino Sergi" date="1333620431" moveby="DanProtopopescu" movedto="Main.NA62MonteCarlo.NA62MC.tar.zip" movedwhen="1335804947" movefrom="Main.NA62.NA62MC.tar.zip" name="NA62MC.tar.zip" path="NA62MC.tar.zip" size="16919575" stream="NA62MC.tar.zip" tmpFilename="/usr/tmp/CGItemp30898" user="Main.DanProtopopescu" version="1"
 
META FILEATTACHMENT attachment="StandardRun_template.mac.txt" attr="" comment="StandardRun.mac template" date="1335868566" name="StandardRun_template.mac.txt" path="StandardRun_template.mac.txt" size="15837" stream="StandardRun_template.mac.txt" tmpFilename="/usr/tmp/CGItemp21850" user="Main.DanProtopopescu" version="1"
META FILEATTACHMENT attachment="InstallSoftware.sh" attr="" comment="Script to install NA62 MC software on a given site" date="1335870411" name="InstallSoftware.sh" path="InstallSoftware.sh" size="1267" stream="InstallSoftware.sh" tmpFilename="/usr/tmp/CGItemp21516" user="Main.DanProtopopescu" version="1"
META FILEATTACHMENT attachment="jobscript.sh.txt" attr="" comment="Job script" date="1335871339" name="jobscript.sh.txt" path="jobscript.sh.txt" size="4799" stream="jobscript.sh.txt" tmpFilename="/usr/tmp/CGItemp21726" user="Main.DanProtopopescu" version="1"

Revision 192012-08-22 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="NA62"

NA62 Monte Carlo Simulations

Line: 77 to 77
 

To do before next production round

  • Install new tarball once it's available from Tonino
Changed:
<
<
  • Resolve CERN FTS channel problem GGUS ticket #83133
  • Scale scripts for largers submissions
>
>
  • DONE Resolve CERN FTS channel problem GGUS ticket #83133
  • DONE Scale scripts for largers submissions
  To log problems encountered (and solutions) use this wiki:

Revision 182012-08-17 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="NA62"

NA62 Monte Carlo Simulations

Line: 65 to 65
  This is a simple table to keep track of site resources (please update if changes occur):
Changed:
<
<
Site CPU WMS Support Storage FTS Channel NA62 S/W
Glasgow DONE DONE     r170
IC DONE DONE DONE DONE r170
RAL DONE DONE     r170
Liverpool DONE DONE DONE    
UCL          
>
>
Site CPU WMS Support Storage FTS Channel to RAL FTS Channel to CERN NA62 S/W
Glasgow DONE DONE   DONE   r170
IC DONE DONE DONE DONE   r170
RAL DONE DONE   -   r170
Liverpool DONE DONE DONE      
UCL            
  Note on s/w labelling: both a NA62 release number (e.g. r170) and a tarball version (e.g. v5) are used. They are kept track of in a dedicated DB table.

Revision 172012-07-04 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="NA62"

NA62 Monte Carlo Simulations

Line: 55 to 55
 

Changed:
<
<
  • Submitted jobs list (with a MySQL DB backend) is located here
>
>
  • Submitted jobs list (with a MySQL DB backend) is located here
 

Revision 162012-07-02 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="NA62"

NA62 Monte Carlo Simulations

Line: 69 to 69
 
Glasgow DONE DONE     r170
IC DONE DONE DONE DONE r170
RAL DONE DONE     r170
Changed:
<
<
UCL (Belgium)          
Liverpool          
>
>
Liverpool DONE DONE DONE    
UCL          
  Note on s/w labelling: both a NA62 release number (e.g. r170) and a tarball version (e.g. v5) are used. They are kept track of in a dedicated DB table.

Revision 152012-06-21 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="NA62"

NA62 Monte Carlo Simulations

Line: 61 to 61
  The script creator interface lets the production manager (PM) set up the job parameters (both MC parameters and output options) and choose the number of 'runs' to launch. The scripter prints out in the browser a list of commands that the PM has to run on his grid UI in order to launch the job or production with his credentials. For individual jobs, the PM updates the DB by hand, for multiple runs, it is done via a wrapper script. A cron job periodically updates the list of jobs and files in the DB.
Deleted:
<
<

Workflow

NA62MC-org-chart.png

 

Site resources

This is a simple table to keep track of site resources (please update if changes occur):

Line: 84 to 80
 
  • Resolve CERN FTS channel problem GGUS ticket #83133
  • Scale scripts for largers submissions
Changed:
<
<
To log problems encountered and solutions use this wiki:
>
>
To log problems encountered (and solutions) use this wiki:
 

Links

Changed:
<
<
>
>
 

Revision 142012-06-21 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="NA62"

NA62 Monte Carlo Simulations

Line: 57 to 57
 
Added:
>
>
  The script creator interface lets the production manager (PM) set up the job parameters (both MC parameters and output options) and choose the number of 'runs' to launch. The scripter prints out in the browser a list of commands that the PM has to run on his grid UI in order to launch the job or production with his credentials. For individual jobs, the PM updates the DB by hand, for multiple runs, it is done via a wrapper script. A cron job periodically updates the list of jobs and files in the DB.
Line: 77 to 78
  Note on s/w labelling: both a NA62 release number (e.g. r170) and a tarball version (e.g. v5) are used. They are kept track of in a dedicated DB table.
Changed:
<
<

Problems encountered

>
>

To do before next production round

  • Install new tarball once it's available from Tonino
  • Resolve CERN FTS channel problem GGUS ticket #83133
  • Scale scripts for largers submissions

To log problems encountered and solutions use this wiki:

 
Deleted:
<
<
<-- DPM does not work properly  -->
 

Links

Revision 132012-06-15 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="NA62"

NA62 Monte Carlo Simulations

Line: 52 to 52
  There's now a suite of tools for submitting jobs either one-by-one or in bulk. Jobs submitted via this interface have the advantage that everything about the jobs (input and output files, scripts, run parameters and statistics) are archived for later reference.
Changed:
<
<
>
>
 

Revision 122012-06-15 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="NA62"

NA62 Monte Carlo Simulations

Line: 52 to 52
  There's now a suite of tools for submitting jobs either one-by-one or in bulk. Jobs submitted via this interface have the advantage that everything about the jobs (input and output files, scripts, run parameters and statistics) are archived for later reference.
Added:
>
>
 
Added:
>
>
  The script creator interface lets the production manager (PM) set up the job parameters (both MC parameters and output options) and choose the number of 'runs' to launch. The scripter prints out in the browser a list of commands that the PM has to run on his grid UI in order to launch the job or production with his credentials. For individual jobs, the PM updates the DB by hand, for multiple runs, it is done via a wrapper script. A cron job periodically updates the list of jobs and files in the DB.
Line: 98 to 101
 
META FILEATTACHMENT attachment="ganga-script.txt" attr="" comment="Ganga script for multiple job submission: na62job.gpy" date="1335878174" name="ganga-script.txt" path="na62job.gpy" size="621" stream="na62job.gpy" tmpFilename="/usr/tmp/CGItemp25104" user="Main.DanProtopopescu" version="2"
META FILEATTACHMENT attachment="JobSubmissionComponents.png" attr="" comment="Job submission components diagram" date="1335879460" name="JobSubmissionComponents.png" path="JobSubmissionComponents.png" size="85200" stream="JobSubmissionComponents.png" tmpFilename="/usr/tmp/CGItemp20928" user="DanProtopopescu" version="1"
META FILEATTACHMENT attachment="CreateTarballs.sh.txt" attr="" comment="(not final!) This is run on the Software Manager's UI machine to create the software tarballs for distribution" date="1336659806" name="CreateTarballs.sh.txt" path="CreateTarballs.sh.txt" size="2734" stream="CreateTarballs.sh.txt" tmpFilename="/usr/tmp/CGItemp24496" user="DanProtopopescu" version="1"
Changed:
<
<
META FILEATTACHMENT attachment="NA62MC-org-chart.png" attr="" comment="NA62 MC organisational chart" date="1337183954" moveby="DanProtopopescu" movedto="NA62.NA62MonteCarlo.NA62MC-org-chart.png" movedwhen="1339761567" movefrom="NA62.NA62.NA62MC-org-chart.png" name="NA62MC-org-chart.png" path="NA62MC-org-chart.png" size="57317" stream="NA62MC-org-chart.png" tmpFilename="/usr/tmp/CGItemp18961" user="DanProtopopescu" version="1"
>
>
META FILEATTACHMENT attachment="NA62MC-org-chart.png" attr="" comment="NA62 MC organisational chart" date="1339769035" name="NA62MC-org-chart.png" path="NA62MC-org-chart.png" size="57317" stream="NA62MC-org-chart.png" tmpFilename="/usr/tmp/CGItemp45267" user="DanProtopopescu" version="3"
META FILEATTACHMENT attachment="na62-button-apple-small.png" attr="h" comment="iPhone webapp button" date="1339769065" name="na62-button-apple-small.png" path="na62-button-apple-small.png" size="10818" stream="na62-button-apple-small.png" tmpFilename="/usr/tmp/CGItemp46125" user="DanProtopopescu" version="3"
 
META TOPICMOVED by="DanProtopopescu" date="1335878717" from="Main.NA62MonteCarlo" to="NA62.NA62MonteCarlo"

Revision 112012-06-15 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="NA62"

NA62 Monte Carlo Simulations

Line: 41 to 41
 
  • The .mac is modified using sed on the worker node to modify paths to LAV matrix files, which are installed in the software area by InstallSoftware.sh
  • The actual job script is attached (needs editing and streamlining)
Added:
>
>
<--
Bulk job submission (first proposed option) 
  1. Create N .mac files using the template
  2. Create one Ganga script including the list of N .mac files
  3. Run this script via Ganga
-->
 

Automated jobs scripts creator

Changed:
<
<
>
>
There's now a suite of tools for submitting jobs either one-by-one or in bulk. Jobs submitted via this interface have the advantage that everything about the jobs (input and output files, scripts, run parameters and statistics) are archived for later reference.
 
Changed:
<
<

Bulk job submission (job splitting)

>
>
 
Changed:
<
<
  1. Create N .mac files using the template
  2. Create one Ganga script including the list of N .mac files
  3. Run this script via Ganga
>
>
The script creator interface lets the production manager (PM) set up the job parameters (both MC parameters and output options) and choose the number of 'runs' to launch. The scripter prints out in the browser a list of commands that the PM has to run on his grid UI in order to launch the job or production with his credentials. For individual jobs, the PM updates the DB by hand, for multiple runs, it is done via a wrapper script. A cron job periodically updates the list of jobs and files in the DB.

Workflow

NA62MC-org-chart.png

 

Site resources

This is a simple table to keep track of site resources (please update if changes occur):

Site CPU WMS Support Storage FTS Channel NA62 S/W
Changed:
<
<
Glasgow DONE DONE     v5
IC DONE DONE   DONE v5
RAL          
>
>
Glasgow DONE DONE     r170
IC DONE DONE DONE DONE r170
RAL DONE DONE     r170
 
UCL (Belgium)          
Added:
>
>
Liverpool          

Note on s/w labelling: both a NA62 release number (e.g. r170) and a tarball version (e.g. v5) are used. They are kept track of in a dedicated DB table.

 

Problems encountered

Changed:
<
<
  • DPM does not work properly
>
>
<-- DPM does not work properly  -->
 
Added:
>
>

Links

<-- verbatim
  SW_DIR=$VO_SW_DIR/na62
  DEFAULT_SE="$DPM_HOST"
  VOMS_SERVERS="vomss://voms.gridpp.ac.uk:8443/voms/na62.vo.gridpp.ac.uk"
  VOMSES='"na62.vo.gridpp.ac.uk" "voms.gridpp.ac.uk" "15501" "/C=UK/O=eScience/OU=Manchester/L=HEP/CN=voms.gridpp.ac.uk" "na62.vo.gridpp.ac.uk"'
  STORAGE_DIR=$DPM_BASE_PATH/na62.vo.gridpp.ac.uk
  VOMS_CA_DN="'/C=UK/O=eScienceCA/OU=Authority/CN=UK e-Science CA 2B'"
verbatim -->
 
META FILEATTACHMENT attachment="NA62MC.tar.zip" attr="" comment="Test tarball from Antonino Sergi" date="1333620431" moveby="DanProtopopescu" movedto="Main.NA62MonteCarlo.NA62MC.tar.zip" movedwhen="1335804947" movefrom="Main.NA62.NA62MC.tar.zip" name="NA62MC.tar.zip" path="NA62MC.tar.zip" size="16919575" stream="NA62MC.tar.zip" tmpFilename="/usr/tmp/CGItemp30898" user="Main.DanProtopopescu" version="1"
META FILEATTACHMENT attachment="StandardRun_template.mac.txt" attr="" comment="StandardRun.mac template" date="1335868566" name="StandardRun_template.mac.txt" path="StandardRun_template.mac.txt" size="15837" stream="StandardRun_template.mac.txt" tmpFilename="/usr/tmp/CGItemp21850" user="Main.DanProtopopescu" version="1"
META FILEATTACHMENT attachment="InstallSoftware.sh" attr="" comment="Script to install NA62 MC software on a given site" date="1335870411" name="InstallSoftware.sh" path="InstallSoftware.sh" size="1267" stream="InstallSoftware.sh" tmpFilename="/usr/tmp/CGItemp21516" user="Main.DanProtopopescu" version="1"
Line: 74 to 98
 
META FILEATTACHMENT attachment="ganga-script.txt" attr="" comment="Ganga script for multiple job submission: na62job.gpy" date="1335878174" name="ganga-script.txt" path="na62job.gpy" size="621" stream="na62job.gpy" tmpFilename="/usr/tmp/CGItemp25104" user="Main.DanProtopopescu" version="2"
META FILEATTACHMENT attachment="JobSubmissionComponents.png" attr="" comment="Job submission components diagram" date="1335879460" name="JobSubmissionComponents.png" path="JobSubmissionComponents.png" size="85200" stream="JobSubmissionComponents.png" tmpFilename="/usr/tmp/CGItemp20928" user="DanProtopopescu" version="1"
META FILEATTACHMENT attachment="CreateTarballs.sh.txt" attr="" comment="(not final!) This is run on the Software Manager's UI machine to create the software tarballs for distribution" date="1336659806" name="CreateTarballs.sh.txt" path="CreateTarballs.sh.txt" size="2734" stream="CreateTarballs.sh.txt" tmpFilename="/usr/tmp/CGItemp24496" user="DanProtopopescu" version="1"
Added:
>
>
META FILEATTACHMENT attachment="NA62MC-org-chart.png" attr="" comment="NA62 MC organisational chart" date="1337183954" moveby="DanProtopopescu" movedto="NA62.NA62MonteCarlo.NA62MC-org-chart.png" movedwhen="1339761567" movefrom="NA62.NA62.NA62MC-org-chart.png" name="NA62MC-org-chart.png" path="NA62MC-org-chart.png" size="57317" stream="NA62MC-org-chart.png" tmpFilename="/usr/tmp/CGItemp18961" user="DanProtopopescu" version="1"
 
META TOPICMOVED by="DanProtopopescu" date="1335878717" from="Main.NA62MonteCarlo" to="NA62.NA62MonteCarlo"

Revision 102012-05-31 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="NA62"

NA62 Monte Carlo Simulations

Line: 41 to 41
 
  • The .mac is modified using sed on the worker node to modify paths to LAV matrix files, which are installed in the software area by InstallSoftware.sh
  • The actual job script is attached (needs editing and streamlining)
Added:
>
>

Automated jobs scripts creator

 

Bulk job submission (job splitting)

  1. Create N .mac files using the template

Revision 92012-05-16 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="NA62"

NA62 Monte Carlo Simulations

Line: 52 to 52
 This is a simple table to keep track of site resources (please update if changes occur):

Site CPU WMS Support Storage FTS Channel NA62 S/W
Changed:
<
<
Glasgow DONE DONE      
>
>
Glasgow DONE DONE     v5
 
IC DONE DONE   DONE v5
RAL          
UCL (Belgium)          

Revision 82012-05-10 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="NA62"

NA62 Monte Carlo Simulations

Line: 19 to 19
 

Code compilation

Added:
>
>
  • Versions:
    • v4 - Contains bug in the MC which Tonino spotted after giving us the tar ball
    • v5 - Installed by Janusz on May 10 (see table below)
 
  • Unpack tarball to NA62MC
  • Fix libs path problems in GNUMakefile-s (Generator/ and Beam/)
  • Make changes to env.sh: source 'offline' environment; clear XAWLIBS variable
Line: 44 to 47
 
  1. Create one Ganga script including the list of N .mac files
  2. Run this script via Ganga
Added:
>
>

Site resources

This is a simple table to keep track of site resources (please update if changes occur):

Site CPU WMS Support Storage FTS Channel NA62 S/W
Glasgow DONE DONE      
IC DONE DONE   DONE v5
RAL          
UCL (Belgium)          
 

Problems encountered

  • DPM does not work properly

Revision 72012-05-10 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="NA62"

NA62 Monte Carlo Simulations

Line: 27 to 27
 

Code distribution

Changed:
<
<
>
>
 
  • Run special job (with an SGM Admin role) to wget and unpack these tarballs on sites (InstallSoftware.sh)
  • Run test job
Line: 52 to 52
 
META FILEATTACHMENT attachment="NA62MC.tar.zip" attr="" comment="Test tarball from Antonino Sergi" date="1333620431" moveby="DanProtopopescu" movedto="Main.NA62MonteCarlo.NA62MC.tar.zip" movedwhen="1335804947" movefrom="Main.NA62.NA62MC.tar.zip" name="NA62MC.tar.zip" path="NA62MC.tar.zip" size="16919575" stream="NA62MC.tar.zip" tmpFilename="/usr/tmp/CGItemp30898" user="Main.DanProtopopescu" version="1"
META FILEATTACHMENT attachment="StandardRun_template.mac.txt" attr="" comment="StandardRun.mac template" date="1335868566" name="StandardRun_template.mac.txt" path="StandardRun_template.mac.txt" size="15837" stream="StandardRun_template.mac.txt" tmpFilename="/usr/tmp/CGItemp21850" user="Main.DanProtopopescu" version="1"
META FILEATTACHMENT attachment="InstallSoftware.sh" attr="" comment="Script to install NA62 MC software on a given site" date="1335870411" name="InstallSoftware.sh" path="InstallSoftware.sh" size="1267" stream="InstallSoftware.sh" tmpFilename="/usr/tmp/CGItemp21516" user="Main.DanProtopopescu" version="1"
Deleted:
<
<
META FILEATTACHMENT attachment="CreateTarballs.sh" attr="" comment="This is run on the Software Manager's UI machine to create the software tarballs for distribution" date="1335870957" name="CreateTarballs.sh" path="CreateTarballs.sh" size="85" stream="CreateTarballs.sh" tmpFilename="/usr/tmp/CGItemp21627" user="Main.DanProtopopescu" version="1"
 
META FILEATTACHMENT attachment="jobscript.sh.txt" attr="" comment="Job script" date="1335871339" name="jobscript.sh.txt" path="jobscript.sh.txt" size="4799" stream="jobscript.sh.txt" tmpFilename="/usr/tmp/CGItemp21726" user="Main.DanProtopopescu" version="1"
META FILEATTACHMENT attachment="ganga-script.txt" attr="" comment="Ganga script for multiple job submission: na62job.gpy" date="1335878174" name="ganga-script.txt" path="na62job.gpy" size="621" stream="na62job.gpy" tmpFilename="/usr/tmp/CGItemp25104" user="Main.DanProtopopescu" version="2"
META FILEATTACHMENT attachment="JobSubmissionComponents.png" attr="" comment="Job submission components diagram" date="1335879460" name="JobSubmissionComponents.png" path="JobSubmissionComponents.png" size="85200" stream="JobSubmissionComponents.png" tmpFilename="/usr/tmp/CGItemp20928" user="DanProtopopescu" version="1"
Added:
>
>
META FILEATTACHMENT attachment="CreateTarballs.sh.txt" attr="" comment="(not final!) This is run on the Software Manager's UI machine to create the software tarballs for distribution" date="1336659806" name="CreateTarballs.sh.txt" path="CreateTarballs.sh.txt" size="2734" stream="CreateTarballs.sh.txt" tmpFilename="/usr/tmp/CGItemp24496" user="DanProtopopescu" version="1"
 
META TOPICMOVED by="DanProtopopescu" date="1335878717" from="Main.NA62MonteCarlo" to="NA62.NA62MonteCarlo"

Revision 62012-05-08 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="NA62"

NA62 Monte Carlo Simulations

Line: 47 to 47
 

Problems encountered

  • DPM does not work properly
Added:
>
>
 
META FILEATTACHMENT attachment="NA62MC.tar.zip" attr="" comment="Test tarball from Antonino Sergi" date="1333620431" moveby="DanProtopopescu" movedto="Main.NA62MonteCarlo.NA62MC.tar.zip" movedwhen="1335804947" movefrom="Main.NA62.NA62MC.tar.zip" name="NA62MC.tar.zip" path="NA62MC.tar.zip" size="16919575" stream="NA62MC.tar.zip" tmpFilename="/usr/tmp/CGItemp30898" user="Main.DanProtopopescu" version="1"
META FILEATTACHMENT attachment="StandardRun_template.mac.txt" attr="" comment="StandardRun.mac template" date="1335868566" name="StandardRun_template.mac.txt" path="StandardRun_template.mac.txt" size="15837" stream="StandardRun_template.mac.txt" tmpFilename="/usr/tmp/CGItemp21850" user="Main.DanProtopopescu" version="1"

Revision 52012-05-01 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="NA62"

NA62 Monte Carlo Simulations

Line: 15 to 15
 

Software

Changed:
<
<
  • Job submission and output management system code components are held on and SVN repo at IC (access list: Dan, Janusz, Stuart)
>
>
  • Job submission and output management system code components are held on and SVN repo at IC (access list: Dan, Janusz, Stuart). The files attached to this wiki are not the latest versions.
 

Code compilation

Revision 42012-05-01 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="NA62"

NA62 Monte Carlo Simulations

Line: 15 to 15
 

Software

Added:
>
>
  • Job submission and output management system code components are held on and SVN repo at IC (access list: Dan, Janusz, Stuart)
 

Code compilation

Revision 32012-05-01 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="NA62"

NA62 Monte Carlo Simulations

This wiki describes how to set up and distribute the NA62 Monte Carlo simulation code.

Line: 43 to 43
 
  1. Create one Ganga script including the list of N .mac files
  2. Run this script via Ganga
Changed:
<
<
META FILEATTACHMENT attachment="NA62MC.tar.zip" attr="" comment="Test tarball from Antonino Sergi" date="1333620431" moveby="DanProtopopescu" movedto="Main.NA62MonteCarlo.NA62MC.tar.zip" movedwhen="1335804947" movefrom="Main.NA62.NA62MC.tar.zip" name="NA62MC.tar.zip" path="NA62MC.tar.zip" size="16919575" stream="NA62MC.tar.zip" tmpFilename="/usr/tmp/CGItemp30898" user="DanProtopopescu" version="1"
META FILEATTACHMENT attachment="StandardRun_template.mac.txt" attr="" comment="StandardRun.mac template" date="1335868566" name="StandardRun_template.mac.txt" path="StandardRun_template.mac.txt" size="15837" stream="StandardRun_template.mac.txt" tmpFilename="/usr/tmp/CGItemp21850" user="DanProtopopescu" version="1"
META FILEATTACHMENT attachment="InstallSoftware.sh" attr="" comment="Script to install NA62 MC software on a given site" date="1335870411" name="InstallSoftware.sh" path="InstallSoftware.sh" size="1267" stream="InstallSoftware.sh" tmpFilename="/usr/tmp/CGItemp21516" user="DanProtopopescu" version="1"
META FILEATTACHMENT attachment="CreateTarballs.sh" attr="" comment="This is run on the Software Manager's UI machine to create the software tarballs for distribution" date="1335870957" name="CreateTarballs.sh" path="CreateTarballs.sh" size="85" stream="CreateTarballs.sh" tmpFilename="/usr/tmp/CGItemp21627" user="DanProtopopescu" version="1"
META FILEATTACHMENT attachment="jobscript.sh.txt" attr="" comment="Job script" date="1335871339" name="jobscript.sh.txt" path="jobscript.sh.txt" size="4799" stream="jobscript.sh.txt" tmpFilename="/usr/tmp/CGItemp21726" user="DanProtopopescu" version="1"
META FILEATTACHMENT attachment="ganga-script.txt" attr="" comment="Ganga script for multiple job submission" date="1335871749" name="ganga-script.txt" path="ganga-script.txt" size="505" stream="ganga-script.txt" tmpFilename="/usr/tmp/CGItemp21726" user="DanProtopopescu" version="1"
>
>

Problems encountered

  • DPM does not work properly

META FILEATTACHMENT attachment="NA62MC.tar.zip" attr="" comment="Test tarball from Antonino Sergi" date="1333620431" moveby="DanProtopopescu" movedto="Main.NA62MonteCarlo.NA62MC.tar.zip" movedwhen="1335804947" movefrom="Main.NA62.NA62MC.tar.zip" name="NA62MC.tar.zip" path="NA62MC.tar.zip" size="16919575" stream="NA62MC.tar.zip" tmpFilename="/usr/tmp/CGItemp30898" user="Main.DanProtopopescu" version="1"
META FILEATTACHMENT attachment="StandardRun_template.mac.txt" attr="" comment="StandardRun.mac template" date="1335868566" name="StandardRun_template.mac.txt" path="StandardRun_template.mac.txt" size="15837" stream="StandardRun_template.mac.txt" tmpFilename="/usr/tmp/CGItemp21850" user="Main.DanProtopopescu" version="1"
META FILEATTACHMENT attachment="InstallSoftware.sh" attr="" comment="Script to install NA62 MC software on a given site" date="1335870411" name="InstallSoftware.sh" path="InstallSoftware.sh" size="1267" stream="InstallSoftware.sh" tmpFilename="/usr/tmp/CGItemp21516" user="Main.DanProtopopescu" version="1"
META FILEATTACHMENT attachment="CreateTarballs.sh" attr="" comment="This is run on the Software Manager's UI machine to create the software tarballs for distribution" date="1335870957" name="CreateTarballs.sh" path="CreateTarballs.sh" size="85" stream="CreateTarballs.sh" tmpFilename="/usr/tmp/CGItemp21627" user="Main.DanProtopopescu" version="1"
META FILEATTACHMENT attachment="jobscript.sh.txt" attr="" comment="Job script" date="1335871339" name="jobscript.sh.txt" path="jobscript.sh.txt" size="4799" stream="jobscript.sh.txt" tmpFilename="/usr/tmp/CGItemp21726" user="Main.DanProtopopescu" version="1"
META FILEATTACHMENT attachment="ganga-script.txt" attr="" comment="Ganga script for multiple job submission: na62job.gpy" date="1335878174" name="ganga-script.txt" path="na62job.gpy" size="621" stream="na62job.gpy" tmpFilename="/usr/tmp/CGItemp25104" user="Main.DanProtopopescu" version="2"
META FILEATTACHMENT attachment="JobSubmissionComponents.png" attr="" comment="Job submission components diagram" date="1335879460" name="JobSubmissionComponents.png" path="JobSubmissionComponents.png" size="85200" stream="JobSubmissionComponents.png" tmpFilename="/usr/tmp/CGItemp20928" user="DanProtopopescu" version="1"
META TOPICMOVED by="DanProtopopescu" date="1335878717" from="Main.NA62MonteCarlo" to="NA62.NA62MonteCarlo"

Revision 22012-05-01 - DanProtopopescu

Line: 1 to 1
 
META TOPICPARENT name="NA62"

NA62 Monte Carlo Simulations

Line: 26 to 26
 

Code distribution

Changed:
<
<
  • Create tarballs for geant4, root, X11 and sw (see attached scripts)
  • Run special job to wget and unpack these tarballs on sites
>
>
 
  • Run test job
Added:
>
>

Job submission

 
Changed:
<
<
-- DanProtopopescu - 2012-04-30
>
>
  • Run parameters given in the StandardRun.mac file: run number, number of events, random seed, root output file name
  • Job environment parameters are set by sourcing the na62jobenv.sh script created by InstallSoftware.sh
  • The .mac is modified using sed on the worker node to modify paths to LAV matrix files, which are installed in the software area by InstallSoftware.sh
  • The actual job script is attached (needs editing and streamlining)

Bulk job submission (job splitting)

  1. Create N .mac files using the template
  2. Create one Ganga script including the list of N .mac files
  3. Run this script via Ganga
 
META FILEATTACHMENT attachment="NA62MC.tar.zip" attr="" comment="Test tarball from Antonino Sergi" date="1333620431" moveby="DanProtopopescu" movedto="Main.NA62MonteCarlo.NA62MC.tar.zip" movedwhen="1335804947" movefrom="Main.NA62.NA62MC.tar.zip" name="NA62MC.tar.zip" path="NA62MC.tar.zip" size="16919575" stream="NA62MC.tar.zip" tmpFilename="/usr/tmp/CGItemp30898" user="DanProtopopescu" version="1"
Added:
>
>
META FILEATTACHMENT attachment="StandardRun_template.mac.txt" attr="" comment="StandardRun.mac template" date="1335868566" name="StandardRun_template.mac.txt" path="StandardRun_template.mac.txt" size="15837" stream="StandardRun_template.mac.txt" tmpFilename="/usr/tmp/CGItemp21850" user="DanProtopopescu" version="1"
META FILEATTACHMENT attachment="InstallSoftware.sh" attr="" comment="Script to install NA62 MC software on a given site" date="1335870411" name="InstallSoftware.sh" path="InstallSoftware.sh" size="1267" stream="InstallSoftware.sh" tmpFilename="/usr/tmp/CGItemp21516" user="DanProtopopescu" version="1"
META FILEATTACHMENT attachment="CreateTarballs.sh" attr="" comment="This is run on the Software Manager's UI machine to create the software tarballs for distribution" date="1335870957" name="CreateTarballs.sh" path="CreateTarballs.sh" size="85" stream="CreateTarballs.sh" tmpFilename="/usr/tmp/CGItemp21627" user="DanProtopopescu" version="1"
META FILEATTACHMENT attachment="jobscript.sh.txt" attr="" comment="Job script" date="1335871339" name="jobscript.sh.txt" path="jobscript.sh.txt" size="4799" stream="jobscript.sh.txt" tmpFilename="/usr/tmp/CGItemp21726" user="DanProtopopescu" version="1"
META FILEATTACHMENT attachment="ganga-script.txt" attr="" comment="Ganga script for multiple job submission" date="1335871749" name="ganga-script.txt" path="ganga-script.txt" size="505" stream="ganga-script.txt" tmpFilename="/usr/tmp/CGItemp21726" user="DanProtopopescu" version="1"

Revision 12012-04-30 - DanProtopopescu

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="NA62"

NA62 Monte Carlo Simulations

<--
Access control:  
-->

This wiki describes how to set up and distribute the NA62 Monte Carlo simulation code.

Software

Code compilation

  • Unpack tarball to NA62MC
  • Fix libs path problems in GNUMakefile-s (Generator/ and Beam/)
  • Make changes to env.sh: source 'offline' environment; clear XAWLIBS variable
  • Rearrange paths (move system paths after NA62-specific paths)
  • Type 'make' and be patient

Code distribution

  • Create tarballs for geant4, root, X11 and sw (see attached scripts)
  • Run special job to wget and unpack these tarballs on sites
  • Run test job

-- DanProtopopescu - 2012-04-30

META FILEATTACHMENT attachment="NA62MC.tar.zip" attr="" comment="Test tarball from Antonino Sergi" date="1333620431" moveby="DanProtopopescu" movedto="Main.NA62MonteCarlo.NA62MC.tar.zip" movedwhen="1335804947" movefrom="Main.NA62.NA62MC.tar.zip" name="NA62MC.tar.zip" path="NA62MC.tar.zip" size="16919575" stream="NA62MC.tar.zip" tmpFilename="/usr/tmp/CGItemp30898" user="DanProtopopescu" version="1"
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback