NA62MonteCarlo < NA62

NA62 Web>NA62MonteCarlo (2016-03-11, DanProtopopescu) (raw view)

---++ NA62 Monte Carlo Simulations

<!--
__Access control:__
   * Set ALLOWTOPICCHANGE = Main.DanProtopopescu,Main.JanuszMartyniak,StuartPurdie
   * Set ALLOWTOPICRENAME = Main.DanProtopopescu,Main.JanuszMartyniak
-->

This wiki describes how to set up and distribute the NA62 Monte Carlo simulation code.

%TOC%

---+++ Software

Source or precompiled production version provided by Antonino Sergi. Job submission and output management system code components are held on the na62.gla.ac.uk server. 

---+++ Installation from scratch on a SL6 desktop

This was done for testing purposes and is documented in a separate wiki, NA62onSL6.

---+++ Code distribution

CVMFS is used for software distribution to sites. A guide is provided here: https://www.gridpp.ac.uk/wiki/RALnonLHCCVMFS

---+++ Scripts preparation

The following files need to be checked and updated for each new s/w release:

   * Job wrapper script - update version number
   * =.mac= file - check for new settings, new decay types etc.
   * =input_files.tgz= - check if any new files need to be included in this archive

---+++ Job submission

Job submission is fully automated via https://na62.gla.ac.uk/

---+++ Automated jobs scripts creator

There's now a suite of tools for submitting jobs either one-by-one or in bulk. Jobs submitted via this interface have the advantage that everything about the jobs (input and output files, scripts, run parameters and statistics) are archived for later reference.

<a href="http://na62.gla.ac.uk/app" alt='iPhone webapp'><img alt="" src="%ATTACHURLPATH%/na62-button-apple-small.png" style="float: right; width: 60px; height: 60px; padding-right: 50px;" /></a>

   * Toolbox located at: http://na62.gla.ac.uk/index.php?task=scripter
   * Submitted jobs list is located [[https://na62.gla.ac.uk/index.php?task=jobs][here]]
   * Jobs status [[http://na62.gla.ac.uk/app][iPhone webapp]].
   * Production rounds: [[https://na62.gla.ac.uk/index.php?task=production][listed here]].

The *script creator* interface lets the production manager (PM) set up the job parameters (both MC parameters and output options) and choose the number of 'runs' to launch. The *scripter* prints out in the browser a list of commands that the PM has to run on his grid UI in order to launch the job or production with _his credentials_. For individual jobs, the PM updates the DB by hand, for multiple runs, it is done via a wrapper script. A cron job periodically updates the list of jobs and files in the DB.

---+++ Ganga 

Mark Slater set up a Ganga service (http://ganga.web.cern.ch/ganga/) for managing entire production rounds. The Ganga request form is accessed via a link on the 
[[http://na62.gla.ac.uk/index.php?task=production][Production table]]. Once the request is submitted, a Ganga daemon from the Grid UI machine picks it up (within 10 minutes) and will submit and monitor jobs until the events target  is reached. 

How the Ganga service was set up and interacts with the other *bot* components is described in here: GangaService4NA62


---+++ Site resources

<table border="0" cellspacing="5"><tr style='vertical-align: top'><td>
This is a simple table to keep track of site resources (please update if changes occur):

 |   *Site*                  | *CPU* | *Dirac Support* | *Storage* |  *CVMFS*  | 
 |  Glasgow              |  %Y%  |        %Y%          |    %Y%     |       %Y%   |  
 |  IC                        |  %Y%  |        %Y%         |     %Y%     |      %Y%    |    
 |  RAL                     |  %Y%  |        %Y%         |     %Y%     |      %Y%    |   
 |  Liverpool             |  %Y%  |       %Y%          |    %Y%      |      %Y%    |      
 |  Birmingham        |  %Y%  |       %Y%          |     %Y%      |     %Y%     |      
 |  UCL                    |  %Y%  |        %Y%         |     %Y%      |      %Y%    |      
 |  CNAF                 |  %Y%  |        %Y%         |     %Y%      |       %Y%    |      

Note on s/w labelling: both a NA62 release/version number (e.g. r261/v0.9.0) and a tarball version (e.g. v14) are used. 
They are kept track of in a dedicated DB table. FTS channels might not be needed is FTS3 is used (latest wrapper scripts). 

*Site admins* - persons to contact in case something goes wrong on one of the sites are:
   * Glasgow - Dan Protopopescu (dan.protopopeascu#glasgow.ac.uk)
   * IC - Janusz Martyniak (janusz.martyniak#imperial.ac.uk)
   * RAL - Andrew Lahiff (andrew.lahiff#stfc.ac.uk)
   * Liverpool - Stephen Jones (sjones#hep.ph.liv.ac.uk)
   * Birmingham - Mark Slater (mslater#cern.ch)
   * UCL - Sergey Podolsky (spodolsky#mail.ru), Pavel Demin (pavel.demin#uclouvain.be)
   * CNAF - Daniele Cesini (daniele.cesini#cnaf.infn.it)
Please replace the #-s.

</td>
<td>
  <img src="%ATTACHURLPATH%/NA62-GridPP-Sites.png" alt="NA62-GridPP-Sites.png" width='384' height='500' />
</td>
</tr></table>

---+++ Cron jobs

Cronjobs are employed to keep track of jobs and output files:<br/>
<img src="%ATTACHURLPATH%/Cronjobs-small.png" alt="Cronjobs-small.png" width='600' height='318' />

---+++ Troubleshooting

To log problems encountered (and solutions) use this wiki:

   * [[TroubleshootingMC][Troubleshooting page]]

---+++ Links
   * NA62 voms server: https://voms.gridpp.ac.uk:8443/voms/na62.vo.gridpp.ac.uk/
   * VO Info https://www.gridpp.ac.uk/wiki/GridPP_approved_VOs

<!-- verbatim
  SW_DIR=$VO_SW_DIR/na62
  DEFAULT_SE="$DPM_HOST"
  VOMS_SERVERS="vomss://voms.gridpp.ac.uk:8443/voms/na62.vo.gridpp.ac.uk"
  VOMSES='"na62.vo.gridpp.ac.uk" "voms.gridpp.ac.uk" "15501" "/C=UK/O=eScience/OU=Manchester/L=HEP/CN=voms.gridpp.ac.uk" "na62.vo.gridpp.ac.uk"'
  STORAGE_DIR=$DPM_BASE_PATH/na62.vo.gridpp.ac.uk
  VOMS_CA_DN="'/C=UK/O=eScienceCA/OU=Authority/CN=UK e-Science CA 2B'"
verbatim -->

Attachments

Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
txt	CreateTarballs.sh.txt	r1	manage	2.7 K	2012-05-10 - 14:23	DanProtopopescu	(not final!) This is run on the Software Manager's UI machine to create the software tarballs for distribution
png	Cronjobs-small.png	r6 r5 r4 r3 r2	manage	150.0 K	2013-02-15 - 13:29	DanProtopopescu	Cronjobs diagram
sh	InstallSoftware.sh	r1	manage	1.2 K	2012-05-01 - 11:06	DanProtopopescu	Script to install NA62 MC software on a given site
png	JobSubmissionComponents.png	r1	manage	83.2 K	2012-05-01 - 13:37	DanProtopopescu	Job submission components diagram
png	NA62-GridPP-Sites.png	r2 r1	manage	158.9 K	2016-03-11 - 10:57	DanProtopopescu	Sites Map (March 2016)
png	NA62MC-org-chart.png	r1	manage	56.4 K	2012-06-21 - 16:04	DanProtopopescu	NA62 MC workflow
txt	StandardRun_template.mac.txt	r1	manage	15.5 K	2012-05-01 - 10:36	DanProtopopescu	StandardRun.mac template
txt	ganga-script.txt	r2 r1	manage	0.6 K	2012-05-01 - 13:16	DanProtopopescu	Ganga script for multiple job submission: na62job.gpy
txt	jobscript.sh.txt	r1	manage	4.7 K	2012-05-01 - 11:22	DanProtopopescu	Job script

Topic revision: r57 - 2016-03-11 - DanProtopopescu

NA62