Automating job submission

There's a script to simplify running Mokka simulations on the PPE compute farm. You can find it here:

/afs/phas.gla.ac.uk/data/ilc/datasets01/350gev/hzee/mokka_dp/run-ecal-run.sh

    ppepc102>  ./run-ecal-run.sh <#events> <geom_version> <gen_set>

One will have to customise a few settings before using it, and download the stdhep files for input. Can also define a few more Ecal geometries (see templates/) to play with.

The script will create the relevant mac, steer and PBS script files, and submit the job straight away. If you modify the script, it’s best to test your settings before actually submitting the jobs.

How to find the optimal number of events

One can use the stdhepjob utility from LCIO to count the number of events in the input .stdhep file, e.g.

     ppepc102> stdhepjob hzee_gen_1335_16.stdhep /tmp/out.slcio -1 

We found out that on our farm a 500 events jobs runs approximately 15h, so we could split the input in 2-3 runs by making use of /generator/skipEvents.

/Mokka/init/rangeCut 0.1 mm # Geant4 production range cut [default is 0.005 mm]

/Mokka/init/TPCCut 10 MeV # Default is 10MeV

/Mokka/init/lcioDetailedTRKHitMode VXDCollection /Mokka/init/lcioDetailedTRKHitMode FTDCollection /Mokka/init/lcioDetailedTRKHitMode ETDCollection /Mokka/init/lcioDetailedTRKHitMode SETCollection /Mokka/init/lcioDetailedTRKHitMode SITCollection /Mokka/init/lcioDetailedTRKHitMode TPCCollection

# To run with modified detector parameters use command line option '-U'

Re-running a set of events

Say we want to run a set of simulations with modified detector configurations, while using the same generated set, for example:

/ilc/prod/clic/350gev/hzee/gen/00001335/000/hzee_gen_1335_16.stdhep

/Mokka/init/lcioFilename hzee_10evt_CLIC_ILD_CDR500.slcio
/Mokka/init/lcioWriteMode WRITE_NEW
/Mokka/init/MokkaGearFileName GearCLIC_ILD_CDR500.xml
/Mokka/init/detectorModel CLIC_ILD_CDR500
/Mokka/init/user consult
/Mokka/init/dbPasswd consult

# To modify detector parameters use command line option '-U' 
# For a list of specs visit the Mokka Detector Model DB Browser at 
# http://www-flc.desy.de/ldcoptimization/tools/mokkamodels.php?model=CLIC_ILD_CDR500
# /Mokka/init/globalModelParameter Hcal_nlayers 36
# /Mokka/init/globalModelParameter Ecal_Si_thickness 0.45

/Mokka/init/randomSeed 16
/Mokka/init/mcRunNumber 1335
/Mokka/init/initialMacroFile hzee_1335_16.g4macro

/Mokka/init/visumode false
/Mokka/init/BatchMode true

/run/verbose 0
/event/verbose 0
/tracking/verbose 0
/generator/generator /afs/phas.gla.ac.uk/data/ilc/datasets01/350gev/hzee/gen/00001335/000/hzee_gen_1335_16.stdhep
/run/beamOn 10
exit

/vis/viewer/set/viewpointThetaPhi 30 40
/vis/viewer/panTo -5 -1
/vis/viewer/zoom 4.

/gun/info                 
Particle:       e- (m = 0.000510999 GeV)
Kinetic Energy: 100 GeV (momentum = 100.001 GeV)
Position:       0 0 0 cm
Direction:      0.707107 0 0.707107 (theta = 45 deg, phi = 0 deg)
(Steps and smearing are not displayed.)

/gun/direction 0 0 1
/gun/directionSmearingMode uniform
/gun/phiSmearing 360 deg
/gun/thetaSmearing 180 deg

release notes 05-02

Outlined in the Mokka release_notes_mokka-01-02. Note that this has been replaced in newer versions of Mokka such that the new commands are detailed below.

The new method (which works) is outlined in section 'VI. New Commands for the Particle Gun' in the Mokka release_notes_mokka-05-02. Relevant subsections of IV. are "/gun/thetaSmearing", "/gun/phiSmearing" and "/gun/directionSmearingMode".

Works if replace "/gun/direction 1 0 1" with "/gun/directionSmearingMode gaussian" in the e-gun.g4macro

/vis/scene/add/trajectories 
/vis/scene/add/hits 
/vis/viewer/flush

Running from your own work directory

To be able to customise your scripts and better keep track of the outputs, you should run Mokka from your own directory, e.g.

cd ~
mkdir ILC_Simulations
cd ILC_Simulations
cp /afs/phas.gla.ac.uk/data/ilc/software/validation/StandardConfig/current/bbudsc_3evt* .

Mokka -M ILD_o1_v05 -U -e ./particle.tbl my_sim.steer

Using a particle gun

The example above needs around 3 minutes per event. For certain purposes, it is enough to use a simple particle gun instead, and this will run much faster. To do this, replace in the example steering file

/Mokka/init/initialMacroFile bbudsc_3evt.g4macro

/Mokka/init/initialMacroFile e-gun.g4macro

/gun/particle e-
/gun/energy 100 GeV
/gun/momentumSmearing 5 GeV
/gun/direction 0 0 1
/run/beamOn 3000
exit

Running Mokka on the PPE batch farm

Here's a simple script (mokka_test.pbs) that runs the Mokka example on the PPE batch farm:

#PBS -N Mokka_test_1
#PBS -m abe
#PBS -M gary.smythe@glasgow.ac.uk
#PBS -q long5

echo "Running Mokka test 1 ..." 

. /afs/phas.gla.ac.uk/data/ilc/software/ilcsoft/v01-14/init_ilcsoft.sh
cd /afs/phas.gla.ac.uk/data/ilc/software/validation/StandardConfig/current
Mokka -M ILD_o1_v05 -e ./particle.tbl bbudsc_3evt.steer

echo "Mokka test 1 done!"

qsub mokka_test.pbs

Visualisation

To visualise the detector geometry set in the steering file for example:

/Mokka/init/BatchMode false

/vis/open OGL
/vis/scene/create 
/vis/drawVolume
/vis/viewer/set/viewpointThetaPhi 60 45

Running Mokka simulations locally

Extensive sets of simulations are already available or can

Testing Mokka

This is described in the ILCSoft wiki. One can try quickly:

. /afs/phas.gla.ac.uk/data/ilc/software/ilcsoft/v01-14/init_ilcsoft.sh
cd /afs/phas.gla.ac.uk/data/ilc/software/validation/StandardConfig/current
Mokka -M ILD_o1_v05 -e ./particle.tbl bbudsc_3evt.steer

Changing certain detector parameters

Detector parameters are taken from (a default) database. There are many versions of the ILC detectors. One can browse this database via the Mokka Detector Model Database Browser.

To change for example the number of layers in the Hcal, and the sensitive Si layer thickness in Ecal, one would add

/Mokka/init/globalModelParameter Hcal_nlayers 36
/Mokka/init/globalModelParameter Ecal_Si_thickness 0.45

Mokka -M ILD_o1_v05 -U -e ./particle.tbl my_bbudsc_3evt.steer