Difference: NA62MonteCarlo (1 vs. 57)

NA62 Monte Carlo Simulations

Software

Installation from scratch on a SL6 desktop

This was done for testing purposes and is documented in a separate wiki, NA62onSL6.

Code distribution

https://www.gridpp.ac.uk/wiki/RALnonLHCCVMFS

Scripts preparation

• .mac file - check for new settings, new decay types etc.
• input_files.tgz - check if any new files need to be included in this archive

Job submission

Automated jobs scripts creator

• Jobs status iPhone webapp.

Site admins - persons to contact in case something goes wrong on one of the sites are:

• IC - Janusz Martyniak (janusz.martyniak#imperial.ac.uk)
• RAL - Andrew Lahiff (andrew.lahiff#stfc.ac.uk)
• Liverpool - Stephen Jones (sjones#hep.ph.liv.ac.uk)
• Birmingham - Mark Slater (mslater#cern.ch)
• UCL - Sergey Podolsky (spodolsky#mail.ru), Pavel Demin (pavel.demin#uclouvain.be)

NA62 Monte Carlo Simulations

Installation from scratch on a SL6 desktop

Code compilation

NA62 Monte Carlo Simulations

Code compilation

• Versions:

NA62 Monte Carlo Simulations

Software

• Obtain release number and read about modifications in NA62MC/HISTORY

Job submission and output management system code components are held on the na62.gla.ac.uk server. The files attached to this wiki are not the latest versions.

NA62 Monte Carlo Simulations

• Test tarball from Antonino Sergi: NA62MC.tgz
• Obtain release number and read about modifications in NA62MC/HISTORY

Code compilation

NA62 Monte Carlo Simulations

Site admins - persons to contact in case something goes wrong on one of the sites are:

Troubleshooting

To log problems encountered (and solutions) use this wiki:

NA62 Monte Carlo Simulations

To do before next production round

Troubleshooting

NA62 Monte Carlo Simulations

UCL

r244

Note on s/w labelling: both a NA62 release number (e.g. r170) and a tarball version (e.g. v5) are used.

NA62 Monte Carlo Simulations

UCL

r244

Note on s/w labelling: both a NA62 release number (e.g. r170) and a tarball version (e.g. v5) are used.

NA62 Monte Carlo Simulations

• • v4 - Contains bug in the MC which Tonino spotted after giving us the tar ball
  • v5 - Installed by Janusz on May 10 (r170, see table below)
  • v6 - September 2012 version (r188)

• Unpack tarball to NA62MC
• Fix libs path problems in GNUMakefile-s (Generator/ and Beam/)

• Add the G4EMLOW6.23.tgz tarball

• Run a special job (with an SGM Admin role) to wget and unpack these tarballs on sites (InstallSoftware.sh)
• Run test job

Job submission

• Run parameters given in the StandardRun.mac file: run number, number of events, random seed, root output file name

NA62 Monte Carlo Simulations

• Run a special job (with an SGM Admin role) to wget and unpack these tarballs on sites (InstallSoftware.sh)
• Run test job

Job submission

• Run parameters given in the StandardRun.mac file: run number, number of events, random seed, root output file name

NA62 Monte Carlo Simulations

UCL

r244

Note on s/w labelling: both a NA62 release number (e.g. r170) and a tarball version (e.g. v5) are used.

NA62 Monte Carlo Simulations

Site resources

Cron jobs

Cronjobs are employed to keep track of jobs and output files:

To do before next production round

NA62 Monte Carlo Simulations

To do before next production round

Troubleshooting

NA62 Monte Carlo Simulations

• • v5 - Installed by Janusz on May 10 (r170, see table below)
  • v6 - September 2012 version (r188)
  • v7 - r193 uses GEANT4 9.5p01, improves processing speed roughly by a factor of 5 wrt the previous version

• Unpack tarball to NA62MC
• Fix libs path problems in GNUMakefile-s (Generator/ and Beam/)

NA62 Monte Carlo Simulations

• Run a special job (with an SGM Admin role) to wget and unpack these tarballs on sites (InstallSoftware.sh)
• Run test job

NA62 Monte Carlo Simulations

• • v5 - Installed by Janusz on May 10 (r170, see table below)
  • v6 - September 2012 version (r188)
  • v7 - r193 uses GEANT4 9.5p01, improves processing speed roughly by a factor of 5 wrt the previous version

• Unpack tarball to NA62MC
• Fix libs path problems in GNUMakefile-s (Generator/ and Beam/)

To do before next production round

Troubleshooting

NA62 Monte Carlo Simulations

Software

• Test tarball from Antonino Sergi: NA62MC.tgz

Code compilation

NA62 Monte Carlo Simulations

NA62 Monte Carlo Simulations

NA62 Monte Carlo Simulations

NA62 Monte Carlo Simulations

UCL

Note on s/w labelling: both a NA62 release number (e.g. r170) and a tarball version (e.g. v5) are used.

NA62 Monte Carlo Simulations

NA62 Monte Carlo Simulations

UCL

Note on s/w labelling: both a NA62 release number (e.g. r170) and a tarball version (e.g. v5) are used.

NA62 Monte Carlo Simulations

UCL

Note on s/w labelling: both a NA62 release number (e.g. r170) and a tarball version (e.g. v5) are used.

• IC - Janusz Martyniak (janusz.martyniak#imperial.ac.uk)
• RAL - Andrew Lahiff (andrew.lahiff#stfc.ac.uk)
• Liverpool - Stephen Jones (sjones#hep.ph.liv.ac.uk)

NA62 Monte Carlo Simulations

UCL

Note on s/w labelling: both a NA62 release number (e.g. r170) and a tarball version (e.g. v5) are used.

Cron jobs

Cronjobs are employed to keep track of jobs and output files:

To do before next production round

• Install new tarball once it's available from Tonino (Dan & Janusz)
• Open remaining FTS channels (Janusz)

• Set up a controller-side cron job to double-check the FTS transfers of job outputs to CERN (Janusz)

Troubleshooting

NA62 Monte Carlo Simulations

• Source the the environment file (source scripts/env.sh)

NA62 Monte Carlo Simulations

NA62 Monte Carlo Simulations

Code distribution

• Run test job

Job submission

To do before next production round

• Install new tarball once it's available from Tonino (Dan & Janusz)

• Move web UI to a dedicated monitoring webserver (Dan)
• Set up a controller-side cron job to double-check the FTS transfers of job outputs to CERN (Janusz)

• Troubleshooting page

NA62 Monte Carlo Simulations

• • v4 - Contains bug in the MC which Tonino spotted after giving us the tar ball
  • v5 - Installed by Janusz on May 10 (r170, see table below)
  • v6 - September 2012 version (r188)

• Unpack tarball to NA62MC
• Fix libs path problems in GNUMakefile-s (Generator/ and Beam/)

NA62 Monte Carlo Simulations

NA62 Monte Carlo Simulations

• • v4 - Contains bug in the MC which Tonino spotted after giving us the tar ball
  • v5 - Installed by Janusz on May 10 (r170, see table below)
  • v6 - September 2012 version (r188)

• Unpack tarball to NA62MC
• Fix libs path problems in GNUMakefile-s (Generator/ and Beam/)

     cp ../NA62/backups/Beam.GNUmakefile Beam/GNUmakefile
     cp ../NA62/backups/Generator.GNUmakefile Generator/GNUmakefile 

Code distribution

NA62 Monte Carlo Simulations

• Toolbox located at: http://ppewww.physics.gla.ac.uk/~protopop/na62/mc/tools/scripter.php
• Submitted jobs list (with a MySQL DB backend) is located here

To do before next production round

NA62 Monte Carlo Simulations

     source /afs/cern.ch/na62/offline/geant4.9.4.p01/env.sh
     export XAWLIBS=" "

Revision 222012-09-18 - DanProtopopescu

Line: 1 to 1
	META TOPICPARENT name="NA62" NA62 Monte Carlo Simulations
Line: 68 to 68
	Site resources
Added:
> >
	This is a simple table to keep track of site resources (please update if changes occur): Site CPU WMS Support Storage FTS Channel to RAL FTS Channel to CERN NA62 S/W
Changed:
< <	Glasgow     r170
> >	Glasgow   r170
	IC   r170 RAL   -   r170 Liverpool
Changed:
< <	UCL
> >	Birmingham       UCL
	Note on s/w labelling: both a NA62 release number (e.g. r170) and a tarball version (e.g. v5) are used. They are kept track of in a dedicated DB table.
Added:
> >
	To do before next production round
Line: 109 to 116
	META FILEATTACHMENT attachment="CreateTarballs.sh.txt" attr="" comment="(not final!) This is run on the Software Manager's UI machine to create the software tarballs for distribution" date="1336659806" name="CreateTarballs.sh.txt" path="CreateTarballs.sh.txt" size="2734" stream="CreateTarballs.sh.txt" tmpFilename="/usr/tmp/CGItemp24496" user="DanProtopopescu" version="1" META FILEATTACHMENT attachment="na62-button-apple-small.png" attr="h" comment="iPhone webapp button" date="1339769065" name="na62-button-apple-small.png" path="na62-button-apple-small.png" size="10818" stream="na62-button-apple-small.png" tmpFilename="/usr/tmp/CGItemp46125" user="DanProtopopescu" version="3" META FILEATTACHMENT attachment="NA62MC-org-chart.png" attr="" comment="NA62 MC workflow" date="1340294677" name="NA62MC-org-chart.png" path="NA62MC-org-chart.png" size="57752" stream="NA62MC-org-chart.png" tmpFilename="/usr/tmp/CGItemp23581" user="DanProtopopescu" version="1"
Added:
> >	META FILEATTACHMENT attachment="NA62-GridPP-Sites.png" attr="" comment="Sites Map (September 2012)" date="1347961595" name="NA62-GridPP-Sites.png" path="NA62-GridPP-Sites.png" size="90381" stream="NA62-GridPP-Sites.png" tmpFilename="/usr/tmp/CGItemp21810" user="DanProtopopescu" version="1"
	META TOPICMOVED by="DanProtopopescu" date="1335878717" from="Main.NA62MonteCarlo" to="NA62.NA62MonteCarlo"

     cp ../NA62/backups/Beam.GNUmakefile Beam/GNUmakefile
     cp ../NA62/backups/Generator.GNUmakefile Generator/GNUmakefile 

Revision 182012-08-17 - DanProtopopescu

Line: 1 to 1
	META TOPICPARENT name="NA62" NA62 Monte Carlo Simulations
Line: 65 to 65
	This is a simple table to keep track of site resources (please update if changes occur):
Changed:
< <	Site CPU WMS Support Storage FTS Channel NA62 S/W Glasgow     r170 IC r170 RAL     r170 Liverpool     UCL
> >	Site CPU WMS Support Storage FTS Channel to RAL FTS Channel to CERN NA62 S/W Glasgow     r170 IC   r170 RAL   -   r170 Liverpool       UCL
	Note on s/w labelling: both a NA62 release number (e.g. r170) and a tarball version (e.g. v5) are used. They are kept track of in a dedicated DB table.

Revision 172012-07-04 - DanProtopopescu

Line: 1 to 1
	META TOPICPARENT name="NA62" NA62 Monte Carlo Simulations
Line: 55 to 55
	Toolbox located at: http://ppewww.physics.gla.ac.uk/~protopop/na62/mc/tools/scripter.php
Changed:
< <	Submitted jobs list (with a MySQL DB backend) is located here
> >	Submitted jobs list (with a MySQL DB backend) is located here
	Jobs status iPhone webapp. Production rounds: listed here.

Revision 162012-07-02 - DanProtopopescu

Line: 1 to 1
	META TOPICPARENT name="NA62" NA62 Monte Carlo Simulations
Line: 69 to 69
	Glasgow     r170 IC r170 RAL     r170
Changed:
< <	UCL (Belgium)           Liverpool
> >	Liverpool     UCL
	Note on s/w labelling: both a NA62 release number (e.g. r170) and a tarball version (e.g. v5) are used. They are kept track of in a dedicated DB table.

Revision 152012-06-21 - DanProtopopescu

Line: 1 to 1
	META TOPICPARENT name="NA62" NA62 Monte Carlo Simulations
Line: 61 to 61
	The script creator interface lets the production manager (PM) set up the job parameters (both MC parameters and output options) and choose the number of 'runs' to launch. The scripter prints out in the browser a list of commands that the PM has to run on his grid UI in order to launch the job or production with his credentials. For individual jobs, the PM updates the DB by hand, for multiple runs, it is done via a wrapper script. A cron job periodically updates the list of jobs and files in the DB.
Deleted:
< <	Workflow
	Site resources This is a simple table to keep track of site resources (please update if changes occur):
Line: 84 to 80
	Resolve CERN FTS channel problem GGUS ticket #83133 Scale scripts for largers submissions
Changed:
< <	To log problems encountered and solutions use this wiki:
> >	To log problems encountered (and solutions) use this wiki:
	Troubleshooting page Links NA62 voms server: https://voms.gridpp.ac.uk:8443/voms/na62.vo.gridpp.ac.uk/
Changed:
< <	Info from https://www.gridpp.ac.uk/wiki/GridPP_approved_VOs:
> >	VO Info https://www.gridpp.ac.uk/wiki/GridPP_approved_VOs
	Revision 142012-06-21 - DanProtopopescu Line: 1 to 1   META TOPICPARENT name="NA62" NA62 Monte Carlo Simulations Line: 57 to 57   Toolbox located at: http://ppewww.physics.gla.ac.uk/~protopop/na62/mc/tools/scripter.php Submitted jobs list (with a MySQL DB backend) is located here Jobs status iPhone webapp. Added: > > Production rounds: listed here.   The script creator interface lets the production manager (PM) set up the job parameters (both MC parameters and output options) and choose the number of 'runs' to launch. The scripter prints out in the browser a list of commands that the PM has to run on his grid UI in order to launch the job or production with his credentials. For individual jobs, the PM updates the DB by hand, for multiple runs, it is done via a wrapper script. A cron job periodically updates the list of jobs and files in the DB. Line: 77 to 78   Note on s/w labelling: both a NA62 release number (e.g. r170) and a tarball version (e.g. v5) are used. They are kept track of in a dedicated DB table. Changed: < < Problems encountered > > To do before next production round Install new tarball once it's available from Tonino Resolve CERN FTS channel problem GGUS ticket #83133 Scale scripts for largers submissions To log problems encountered and solutions use this wiki:   Deleted: < < <-- DPM does not work properly -->   Troubleshooting page Links Revision 132012-06-15 - DanProtopopescu Line: 1 to 1   META TOPICPARENT name="NA62" NA62 Monte Carlo Simulations Line: 52 to 52   There's now a suite of tools for submitting jobs either one-by-one or in bulk. Jobs submitted via this interface have the advantage that everything about the jobs (input and output files, scripts, run parameters and statistics) are archived for later reference. Changed: < < > >   Toolbox located at: http://ppewww.physics.gla.ac.uk/~protopop/na62/mc/tools/scripter.php Submitted jobs list (with a MySQL DB backend) is located here Revision 122012-06-15 - DanProtopopescu Line: 1 to 1   META TOPICPARENT name="NA62" NA62 Monte Carlo Simulations Line: 52 to 52   There's now a suite of tools for submitting jobs either one-by-one or in bulk. Jobs submitted via this interface have the advantage that everything about the jobs (input and output files, scripts, run parameters and statistics) are archived for later reference. Added: > >   Toolbox located at: http://ppewww.physics.gla.ac.uk/~protopop/na62/mc/tools/scripter.php Submitted jobs list (with a MySQL DB backend) is located here Added: > > Jobs status iPhone webapp.   The script creator interface lets the production manager (PM) set up the job parameters (both MC parameters and output options) and choose the number of 'runs' to launch. The scripter prints out in the browser a list of commands that the PM has to run on his grid UI in order to launch the job or production with his credentials. For individual jobs, the PM updates the DB by hand, for multiple runs, it is done via a wrapper script. A cron job periodically updates the list of jobs and files in the DB. Line: 98 to 101   META FILEATTACHMENT attachment="ganga-script.txt" attr="" comment="Ganga script for multiple job submission: na62job.gpy" date="1335878174" name="ganga-script.txt" path="na62job.gpy" size="621" stream="na62job.gpy" tmpFilename="/usr/tmp/CGItemp25104" user="Main.DanProtopopescu" version="2" META FILEATTACHMENT attachment="JobSubmissionComponents.png" attr="" comment="Job submission components diagram" date="1335879460" name="JobSubmissionComponents.png" path="JobSubmissionComponents.png" size="85200" stream="JobSubmissionComponents.png" tmpFilename="/usr/tmp/CGItemp20928" user="DanProtopopescu" version="1" META FILEATTACHMENT attachment="CreateTarballs.sh.txt" attr="" comment="(not final!) This is run on the Software Manager's UI machine to create the software tarballs for distribution" date="1336659806" name="CreateTarballs.sh.txt" path="CreateTarballs.sh.txt" size="2734" stream="CreateTarballs.sh.txt" tmpFilename="/usr/tmp/CGItemp24496" user="DanProtopopescu" version="1" Changed: < < META FILEATTACHMENT attachment="NA62MC-org-chart.png" attr="" comment="NA62 MC organisational chart" date="1337183954" moveby="DanProtopopescu" movedto="NA62.NA62MonteCarlo.NA62MC-org-chart.png" movedwhen="1339761567" movefrom="NA62.NA62.NA62MC-org-chart.png" name="NA62MC-org-chart.png" path="NA62MC-org-chart.png" size="57317" stream="NA62MC-org-chart.png" tmpFilename="/usr/tmp/CGItemp18961" user="DanProtopopescu" version="1" > > META FILEATTACHMENT attachment="NA62MC-org-chart.png" attr="" comment="NA62 MC organisational chart" date="1339769035" name="NA62MC-org-chart.png" path="NA62MC-org-chart.png" size="57317" stream="NA62MC-org-chart.png" tmpFilename="/usr/tmp/CGItemp45267" user="DanProtopopescu" version="3" META FILEATTACHMENT attachment="na62-button-apple-small.png" attr="h" comment="iPhone webapp button" date="1339769065" name="na62-button-apple-small.png" path="na62-button-apple-small.png" size="10818" stream="na62-button-apple-small.png" tmpFilename="/usr/tmp/CGItemp46125" user="DanProtopopescu" version="3"   META TOPICMOVED by="DanProtopopescu" date="1335878717" from="Main.NA62MonteCarlo" to="NA62.NA62MonteCarlo" Revision 112012-06-15 - DanProtopopescu Line: 1 to 1   META TOPICPARENT name="NA62" NA62 Monte Carlo Simulations Line: 41 to 41   The .mac is modified using sed on the worker node to modify paths to LAV matrix files, which are installed in the software area by InstallSoftware.sh The actual job script is attached (needs editing and streamlining) Added: > > <-- Bulk job submission (first proposed option) Create N .mac files using the template Create one Ganga script including the list of N .mac files Run this script via Ganga -->   Automated jobs scripts creator Changed: < < Toolbox located at: http://ppewww.physics.gla.ac.uk/~protopop/na62/mc/tools/ Submitted jobs list (with a MySQL DB backend) is located here > > There's now a suite of tools for submitting jobs either one-by-one or in bulk. Jobs submitted via this interface have the advantage that everything about the jobs (input and output files, scripts, run parameters and statistics) are archived for later reference.   Changed: < < Bulk job submission (job splitting) > > Toolbox located at: http://ppewww.physics.gla.ac.uk/~protopop/na62/mc/tools/scripter.php Submitted jobs list (with a MySQL DB backend) is located here   Changed: < < Create N .mac files using the template Create one Ganga script including the list of N .mac files Run this script via Ganga > > The script creator interface lets the production manager (PM) set up the job parameters (both MC parameters and output options) and choose the number of 'runs' to launch. The scripter prints out in the browser a list of commands that the PM has to run on his grid UI in order to launch the job or production with his credentials. For individual jobs, the PM updates the DB by hand, for multiple runs, it is done via a wrapper script. A cron job periodically updates the list of jobs and files in the DB. Workflow   Site resources This is a simple table to keep track of site resources (please update if changes occur): Site CPU WMS Support Storage FTS Channel NA62 S/W Changed: < < Glasgow     v5 IC   v5 RAL           > > Glasgow     r170 IC r170 RAL     r170   UCL (Belgium)           Added: > > Liverpool           Note on s/w labelling: both a NA62 release number (e.g. r170) and a tarball version (e.g. v5) are used. They are kept track of in a dedicated DB table.   Problems encountered Changed: < < DPM does not work properly > > <-- DPM does not work properly -->   Troubleshooting page Added: > > Links NA62 voms server: https://voms.gridpp.ac.uk:8443/voms/na62.vo.gridpp.ac.uk/ Info from https://www.gridpp.ac.uk/wiki/GridPP_approved_VOs: <-- verbatim SW_DIR=$VO_SW_DIR/na62 DEFAULT_SE="$DPM_HOST" VOMS_SERVERS="vomss://voms.gridpp.ac.uk:8443/voms/na62.vo.gridpp.ac.uk" VOMSES='"na62.vo.gridpp.ac.uk" "voms.gridpp.ac.uk" "15501" "/C=UK/O=eScience/OU=Manchester/L=HEP/CN=voms.gridpp.ac.uk" "na62.vo.gridpp.ac.uk"' STORAGE_DIR=$DPM_BASE_PATH/na62.vo.gridpp.ac.uk VOMS_CA_DN="'/C=UK/O=eScienceCA/OU=Authority/CN=UK e-Science CA 2B'" verbatim -->   META FILEATTACHMENT attachment="NA62MC.tar.zip" attr="" comment="Test tarball from Antonino Sergi" date="1333620431" moveby="DanProtopopescu" movedto="Main.NA62MonteCarlo.NA62MC.tar.zip" movedwhen="1335804947" movefrom="Main.NA62.NA62MC.tar.zip" name="NA62MC.tar.zip" path="NA62MC.tar.zip" size="16919575" stream="NA62MC.tar.zip" tmpFilename="/usr/tmp/CGItemp30898" user="Main.DanProtopopescu" version="1" META FILEATTACHMENT attachment="StandardRun_template.mac.txt" attr="" comment="StandardRun.mac template" date="1335868566" name="StandardRun_template.mac.txt" path="StandardRun_template.mac.txt" size="15837" stream="StandardRun_template.mac.txt" tmpFilename="/usr/tmp/CGItemp21850" user="Main.DanProtopopescu" version="1" META FILEATTACHMENT attachment="InstallSoftware.sh" attr="" comment="Script to install NA62 MC software on a given site" date="1335870411" name="InstallSoftware.sh" path="InstallSoftware.sh" size="1267" stream="InstallSoftware.sh" tmpFilename="/usr/tmp/CGItemp21516" user="Main.DanProtopopescu" version="1" Line: 74 to 98   META FILEATTACHMENT attachment="ganga-script.txt" attr="" comment="Ganga script for multiple job submission: na62job.gpy" date="1335878174" name="ganga-script.txt" path="na62job.gpy" size="621" stream="na62job.gpy" tmpFilename="/usr/tmp/CGItemp25104" user="Main.DanProtopopescu" version="2" META FILEATTACHMENT attachment="JobSubmissionComponents.png" attr="" comment="Job submission components diagram" date="1335879460" name="JobSubmissionComponents.png" path="JobSubmissionComponents.png" size="85200" stream="JobSubmissionComponents.png" tmpFilename="/usr/tmp/CGItemp20928" user="DanProtopopescu" version="1" META FILEATTACHMENT attachment="CreateTarballs.sh.txt" attr="" comment="(not final!) This is run on the Software Manager's UI machine to create the software tarballs for distribution" date="1336659806" name="CreateTarballs.sh.txt" path="CreateTarballs.sh.txt" size="2734" stream="CreateTarballs.sh.txt" tmpFilename="/usr/tmp/CGItemp24496" user="DanProtopopescu" version="1" Added: > > META FILEATTACHMENT attachment="NA62MC-org-chart.png" attr="" comment="NA62 MC organisational chart" date="1337183954" moveby="DanProtopopescu" movedto="NA62.NA62MonteCarlo.NA62MC-org-chart.png" movedwhen="1339761567" movefrom="NA62.NA62.NA62MC-org-chart.png" name="NA62MC-org-chart.png" path="NA62MC-org-chart.png" size="57317" stream="NA62MC-org-chart.png" tmpFilename="/usr/tmp/CGItemp18961" user="DanProtopopescu" version="1"   META TOPICMOVED by="DanProtopopescu" date="1335878717" from="Main.NA62MonteCarlo" to="NA62.NA62MonteCarlo" Revision 102012-05-31 - DanProtopopescu Line: 1 to 1   META TOPICPARENT name="NA62" NA62 Monte Carlo Simulations Line: 41 to 41   The .mac is modified using sed on the worker node to modify paths to LAV matrix files, which are installed in the software area by InstallSoftware.sh The actual job script is attached (needs editing and streamlining) Added: > > Automated jobs scripts creator Toolbox located at: http://ppewww.physics.gla.ac.uk/~protopop/na62/mc/tools/ Submitted jobs list (with a MySQL DB backend) is located here   Bulk job submission (job splitting) Create N .mac files using the template Revision 92012-05-16 - DanProtopopescu Line: 1 to 1   META TOPICPARENT name="NA62" NA62 Monte Carlo Simulations Line: 52 to 52   This is a simple table to keep track of site resources (please update if changes occur): Site CPU WMS Support Storage FTS Channel NA62 S/W Changed: < < Glasgow       > > Glasgow     v5   IC   v5 RAL           UCL (Belgium)           Revision 82012-05-10 - DanProtopopescu Line: 1 to 1   META TOPICPARENT name="NA62" NA62 Monte Carlo Simulations Line: 19 to 19   Code compilation Added: > > Versions: v4 - Contains bug in the MC which Tonino spotted after giving us the tar ball v5 - Installed by Janusz on May 10 (see table below)   Unpack tarball to NA62MC Fix libs path problems in GNUMakefile-s (Generator/ and Beam/) Make changes to env.sh: source 'offline' environment; clear XAWLIBS variable Line: 44 to 47   Create one Ganga script including the list of N .mac files Run this script via Ganga Added: > > Site resources This is a simple table to keep track of site resources (please update if changes occur): Site CPU WMS Support Storage FTS Channel NA62 S/W Glasgow       IC   v5 RAL           UCL (Belgium)             Problems encountered DPM does not work properly Revision 72012-05-10 - DanProtopopescu Line: 1 to 1   META TOPICPARENT name="NA62" NA62 Monte Carlo Simulations Line: 27 to 27   Code distribution Changed: < < Create tarballs for geant4, root, X11 and sw (CreateTarballs.sh) > > Create tarballs for geant4, root, X11 and sw (CreateTarballs.sh)   Run special job (with an SGM Admin role) to wget and unpack these tarballs on sites (InstallSoftware.sh) Run test job Line: 52 to 52   META FILEATTACHMENT attachment="NA62MC.tar.zip" attr="" comment="Test tarball from Antonino Sergi" date="1333620431" moveby="DanProtopopescu" movedto="Main.NA62MonteCarlo.NA62MC.tar.zip" movedwhen="1335804947" movefrom="Main.NA62.NA62MC.tar.zip" name="NA62MC.tar.zip" path="NA62MC.tar.zip" size="16919575" stream="NA62MC.tar.zip" tmpFilename="/usr/tmp/CGItemp30898" user="Main.DanProtopopescu" version="1" META FILEATTACHMENT attachment="StandardRun_template.mac.txt" attr="" comment="StandardRun.mac template" date="1335868566" name="StandardRun_template.mac.txt" path="StandardRun_template.mac.txt" size="15837" stream="StandardRun_template.mac.txt" tmpFilename="/usr/tmp/CGItemp21850" user="Main.DanProtopopescu" version="1" META FILEATTACHMENT attachment="InstallSoftware.sh" attr="" comment="Script to install NA62 MC software on a given site" date="1335870411" name="InstallSoftware.sh" path="InstallSoftware.sh" size="1267" stream="InstallSoftware.sh" tmpFilename="/usr/tmp/CGItemp21516" user="Main.DanProtopopescu" version="1" Deleted: < < META FILEATTACHMENT attachment="CreateTarballs.sh" attr="" comment="This is run on the Software Manager's UI machine to create the software tarballs for distribution" date="1335870957" name="CreateTarballs.sh" path="CreateTarballs.sh" size="85" stream="CreateTarballs.sh" tmpFilename="/usr/tmp/CGItemp21627" user="Main.DanProtopopescu" version="1"   META FILEATTACHMENT attachment="jobscript.sh.txt" attr="" comment="Job script" date="1335871339" name="jobscript.sh.txt" path="jobscript.sh.txt" size="4799" stream="jobscript.sh.txt" tmpFilename="/usr/tmp/CGItemp21726" user="Main.DanProtopopescu" version="1" META FILEATTACHMENT attachment="ganga-script.txt" attr="" comment="Ganga script for multiple job submission: na62job.gpy" date="1335878174" name="ganga-script.txt" path="na62job.gpy" size="621" stream="na62job.gpy" tmpFilename="/usr/tmp/CGItemp25104" user="Main.DanProtopopescu" version="2" META FILEATTACHMENT attachment="JobSubmissionComponents.png" attr="" comment="Job submission components diagram" date="1335879460" name="JobSubmissionComponents.png" path="JobSubmissionComponents.png" size="85200" stream="JobSubmissionComponents.png" tmpFilename="/usr/tmp/CGItemp20928" user="DanProtopopescu" version="1" Added: > > META FILEATTACHMENT attachment="CreateTarballs.sh.txt" attr="" comment="(not final!) This is run on the Software Manager's UI machine to create the software tarballs for distribution" date="1336659806" name="CreateTarballs.sh.txt" path="CreateTarballs.sh.txt" size="2734" stream="CreateTarballs.sh.txt" tmpFilename="/usr/tmp/CGItemp24496" user="DanProtopopescu" version="1"   META TOPICMOVED by="DanProtopopescu" date="1335878717" from="Main.NA62MonteCarlo" to="NA62.NA62MonteCarlo" Revision 62012-05-08 - DanProtopopescu Line: 1 to 1   META TOPICPARENT name="NA62" NA62 Monte Carlo Simulations Line: 47 to 47   Problems encountered DPM does not work properly Added: > > Troubleshooting page   META FILEATTACHMENT attachment="NA62MC.tar.zip" attr="" comment="Test tarball from Antonino Sergi" date="1333620431" moveby="DanProtopopescu" movedto="Main.NA62MonteCarlo.NA62MC.tar.zip" movedwhen="1335804947" movefrom="Main.NA62.NA62MC.tar.zip" name="NA62MC.tar.zip" path="NA62MC.tar.zip" size="16919575" stream="NA62MC.tar.zip" tmpFilename="/usr/tmp/CGItemp30898" user="Main.DanProtopopescu" version="1" META FILEATTACHMENT attachment="StandardRun_template.mac.txt" attr="" comment="StandardRun.mac template" date="1335868566" name="StandardRun_template.mac.txt" path="StandardRun_template.mac.txt" size="15837" stream="StandardRun_template.mac.txt" tmpFilename="/usr/tmp/CGItemp21850" user="Main.DanProtopopescu" version="1" Revision 52012-05-01 - DanProtopopescu Line: 1 to 1   META TOPICPARENT name="NA62" NA62 Monte Carlo Simulations Line: 15 to 15   Software Test tarball from Antonino Sergi: NA62MC.tar.zip Changed: < < Job submission and output management system code components are held on and SVN repo at IC (access list: Dan, Janusz, Stuart) > > Job submission and output management system code components are held on and SVN repo at IC (access list: Dan, Janusz, Stuart). The files attached to this wiki are not the latest versions.   Code compilation Revision 42012-05-01 - DanProtopopescu Line: 1 to 1   META TOPICPARENT name="NA62" NA62 Monte Carlo Simulations Line: 15 to 15   Software Test tarball from Antonino Sergi: NA62MC.tar.zip Added: > > Job submission and output management system code components are held on and SVN repo at IC (access list: Dan, Janusz, Stuart)   Code compilation Revision 32012-05-01 - DanProtopopescu Line: 1 to 1   META TOPICPARENT name="NA62" NA62 Monte Carlo Simulations This wiki describes how to set up and distribute the NA62 Monte Carlo simulation code. Line: 43 to 43   Create one Ganga script including the list of N .mac files Run this script via Ganga Changed: < < META FILEATTACHMENT attachment="NA62MC.tar.zip" attr="" comment="Test tarball from Antonino Sergi" date="1333620431" moveby="DanProtopopescu" movedto="Main.NA62MonteCarlo.NA62MC.tar.zip" movedwhen="1335804947" movefrom="Main.NA62.NA62MC.tar.zip" name="NA62MC.tar.zip" path="NA62MC.tar.zip" size="16919575" stream="NA62MC.tar.zip" tmpFilename="/usr/tmp/CGItemp30898" user="DanProtopopescu" version="1" META FILEATTACHMENT attachment="StandardRun_template.mac.txt" attr="" comment="StandardRun.mac template" date="1335868566" name="StandardRun_template.mac.txt" path="StandardRun_template.mac.txt" size="15837" stream="StandardRun_template.mac.txt" tmpFilename="/usr/tmp/CGItemp21850" user="DanProtopopescu" version="1" META FILEATTACHMENT attachment="InstallSoftware.sh" attr="" comment="Script to install NA62 MC software on a given site" date="1335870411" name="InstallSoftware.sh" path="InstallSoftware.sh" size="1267" stream="InstallSoftware.sh" tmpFilename="/usr/tmp/CGItemp21516" user="DanProtopopescu" version="1" META FILEATTACHMENT attachment="CreateTarballs.sh" attr="" comment="This is run on the Software Manager's UI machine to create the software tarballs for distribution" date="1335870957" name="CreateTarballs.sh" path="CreateTarballs.sh" size="85" stream="CreateTarballs.sh" tmpFilename="/usr/tmp/CGItemp21627" user="DanProtopopescu" version="1" META FILEATTACHMENT attachment="jobscript.sh.txt" attr="" comment="Job script" date="1335871339" name="jobscript.sh.txt" path="jobscript.sh.txt" size="4799" stream="jobscript.sh.txt" tmpFilename="/usr/tmp/CGItemp21726" user="DanProtopopescu" version="1" META FILEATTACHMENT attachment="ganga-script.txt" attr="" comment="Ganga script for multiple job submission" date="1335871749" name="ganga-script.txt" path="ganga-script.txt" size="505" stream="ganga-script.txt" tmpFilename="/usr/tmp/CGItemp21726" user="DanProtopopescu" version="1" > > Problems encountered DPM does not work properly META FILEATTACHMENT attachment="NA62MC.tar.zip" attr="" comment="Test tarball from Antonino Sergi" date="1333620431" moveby="DanProtopopescu" movedto="Main.NA62MonteCarlo.NA62MC.tar.zip" movedwhen="1335804947" movefrom="Main.NA62.NA62MC.tar.zip" name="NA62MC.tar.zip" path="NA62MC.tar.zip" size="16919575" stream="NA62MC.tar.zip" tmpFilename="/usr/tmp/CGItemp30898" user="Main.DanProtopopescu" version="1" META FILEATTACHMENT attachment="StandardRun_template.mac.txt" attr="" comment="StandardRun.mac template" date="1335868566" name="StandardRun_template.mac.txt" path="StandardRun_template.mac.txt" size="15837" stream="StandardRun_template.mac.txt" tmpFilename="/usr/tmp/CGItemp21850" user="Main.DanProtopopescu" version="1" META FILEATTACHMENT attachment="InstallSoftware.sh" attr="" comment="Script to install NA62 MC software on a given site" date="1335870411" name="InstallSoftware.sh" path="InstallSoftware.sh" size="1267" stream="InstallSoftware.sh" tmpFilename="/usr/tmp/CGItemp21516" user="Main.DanProtopopescu" version="1" META FILEATTACHMENT attachment="CreateTarballs.sh" attr="" comment="This is run on the Software Manager's UI machine to create the software tarballs for distribution" date="1335870957" name="CreateTarballs.sh" path="CreateTarballs.sh" size="85" stream="CreateTarballs.sh" tmpFilename="/usr/tmp/CGItemp21627" user="Main.DanProtopopescu" version="1" META FILEATTACHMENT attachment="jobscript.sh.txt" attr="" comment="Job script" date="1335871339" name="jobscript.sh.txt" path="jobscript.sh.txt" size="4799" stream="jobscript.sh.txt" tmpFilename="/usr/tmp/CGItemp21726" user="Main.DanProtopopescu" version="1" META FILEATTACHMENT attachment="ganga-script.txt" attr="" comment="Ganga script for multiple job submission: na62job.gpy" date="1335878174" name="ganga-script.txt" path="na62job.gpy" size="621" stream="na62job.gpy" tmpFilename="/usr/tmp/CGItemp25104" user="Main.DanProtopopescu" version="2" META FILEATTACHMENT attachment="JobSubmissionComponents.png" attr="" comment="Job submission components diagram" date="1335879460" name="JobSubmissionComponents.png" path="JobSubmissionComponents.png" size="85200" stream="JobSubmissionComponents.png" tmpFilename="/usr/tmp/CGItemp20928" user="DanProtopopescu" version="1" META TOPICMOVED by="DanProtopopescu" date="1335878717" from="Main.NA62MonteCarlo" to="NA62.NA62MonteCarlo" Revision 22012-05-01 - DanProtopopescu Line: 1 to 1   META TOPICPARENT name="NA62" NA62 Monte Carlo Simulations Line: 26 to 26   Code distribution Changed: < < Create tarballs for geant4, root, X11 and sw (see attached scripts) Run special job to wget and unpack these tarballs on sites > > Create tarballs for geant4, root, X11 and sw (CreateTarballs.sh) Run special job (with an SGM Admin role) to wget and unpack these tarballs on sites (InstallSoftware.sh)   Run test job Added: > > Job submission   Changed: < < -- DanProtopopescu - 2012-04-30 > > Run parameters given in the StandardRun.mac file: run number, number of events, random seed, root output file name Job environment parameters are set by sourcing the na62jobenv.sh script created by InstallSoftware.sh The .mac is modified using sed on the worker node to modify paths to LAV matrix files, which are installed in the software area by InstallSoftware.sh The actual job script is attached (needs editing and streamlining) Bulk job submission (job splitting) Create N .mac files using the template Create one Ganga script including the list of N .mac files Run this script via Ganga   META FILEATTACHMENT attachment="NA62MC.tar.zip" attr="" comment="Test tarball from Antonino Sergi" date="1333620431" moveby="DanProtopopescu" movedto="Main.NA62MonteCarlo.NA62MC.tar.zip" movedwhen="1335804947" movefrom="Main.NA62.NA62MC.tar.zip" name="NA62MC.tar.zip" path="NA62MC.tar.zip" size="16919575" stream="NA62MC.tar.zip" tmpFilename="/usr/tmp/CGItemp30898" user="DanProtopopescu" version="1" Added: > > META FILEATTACHMENT attachment="StandardRun_template.mac.txt" attr="" comment="StandardRun.mac template" date="1335868566" name="StandardRun_template.mac.txt" path="StandardRun_template.mac.txt" size="15837" stream="StandardRun_template.mac.txt" tmpFilename="/usr/tmp/CGItemp21850" user="DanProtopopescu" version="1" META FILEATTACHMENT attachment="InstallSoftware.sh" attr="" comment="Script to install NA62 MC software on a given site" date="1335870411" name="InstallSoftware.sh" path="InstallSoftware.sh" size="1267" stream="InstallSoftware.sh" tmpFilename="/usr/tmp/CGItemp21516" user="DanProtopopescu" version="1" META FILEATTACHMENT attachment="CreateTarballs.sh" attr="" comment="This is run on the Software Manager's UI machine to create the software tarballs for distribution" date="1335870957" name="CreateTarballs.sh" path="CreateTarballs.sh" size="85" stream="CreateTarballs.sh" tmpFilename="/usr/tmp/CGItemp21627" user="DanProtopopescu" version="1" META FILEATTACHMENT attachment="jobscript.sh.txt" attr="" comment="Job script" date="1335871339" name="jobscript.sh.txt" path="jobscript.sh.txt" size="4799" stream="jobscript.sh.txt" tmpFilename="/usr/tmp/CGItemp21726" user="DanProtopopescu" version="1" META FILEATTACHMENT attachment="ganga-script.txt" attr="" comment="Ganga script for multiple job submission" date="1335871749" name="ganga-script.txt" path="ganga-script.txt" size="505" stream="ganga-script.txt" tmpFilename="/usr/tmp/CGItemp21726" user="DanProtopopescu" version="1" Revision 12012-04-30 - DanProtopopescu Line: 1 to 1 Added: > > META TOPICPARENT name="NA62" NA62 Monte Carlo Simulations <-- Access control: Set ALLOWTOPICCHANGE = DanProtopopescu,JanuszMartyniak,StuartPurdie Set ALLOWTOPICRENAME = DanProtopopescu,JanuszMartyniak --> This wiki describes how to set up and distribute the NA62 Monte Carlo simulation code. NA62 Monte Carlo Simulations Software Installation from scratch on a SL6 desktop Code distribution Scripts preparation Job submission Automated jobs scripts creator Ganga Site resources Cron jobs Troubleshooting Links Software Test tarball from Antonino Sergi: NA62MC.tar.zip Code compilation Unpack tarball to NA62MC Fix libs path problems in GNUMakefile-s (Generator/ and Beam/) Make changes to env.sh: source 'offline' environment; clear XAWLIBS variable Rearrange paths (move system paths after NA62-specific paths) Type 'make' and be patient Code distribution Create tarballs for geant4, root, X11 and sw (see attached scripts) Run special job to wget and unpack these tarballs on sites Run test job -- DanProtopopescu - 2012-04-30 META FILEATTACHMENT attachment="NA62MC.tar.zip" attr="" comment="Test tarball from Antonino Sergi" date="1333620431" moveby="DanProtopopescu" movedto="Main.NA62MonteCarlo.NA62MC.tar.zip" movedwhen="1335804947" movefrom="Main.NA62.NA62MC.tar.zip" name="NA62MC.tar.zip" path="NA62MC.tar.zip" size="16919575" stream="NA62MC.tar.zip" tmpFilename="/usr/tmp/CGItemp30898" user="DanProtopopescu" version="1" View topic | History: r57 < r56 < r55 < r54 | More topic actions...   Copyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors. Ideas, requests, problems regarding TWiki? Send feedback