Difference: ILCSoft (1 vs. 16)

Problems to expect and fix

Some of the packages had to be recompiled for various reasons, as described below.

MarlinReco

MarlinReco

cd /afs/phas.gla.ac.uk/data/ilc/software/ilcsoft/v01-14/MarlinReco/v01-03

cd build
cmake -DILCUTIL_DIR=../../../ilcutil/ -DROOT_DIR=$ROOTSYS -DFORTRAN_LIBRARY=/usr/lib/gcc/x86_64-redhat-linux/4.1.2/libgfortran.so ..
make install

Building Root Dict manually for certain packages

simjob simjob.slcio

(5) To run one of the processors in the steering file, you must name it amongst the processor names at the beginning of the steering file. This will inform Marlin you want to execute the steering file with your processor.

For the ExampleDSTAnalysis processor:

<execute>
 <processor name="MyAIDAProcessor" />
 <processor name="MyTestProcessor" />
 <processor name="MyLCIOOutputProcessor"/> 
 <processor name="MyExampleDSTAnalysis" />
 </execute>

This will create ExampleDSTAnalysis.root which contains the list of histograms.

Software Installation

Installation of ILC software at Glasgow is described below:

Getting the latest version

Two choices for downloading the software is through svn or via a tarball. For the first attempt I opted for the latter. To do this I

went to:

https://svnsrv.desy.de/viewvc/ilctools/ilcinstall/tags/

and downloaded the tar file accessible here:

https://svnsrv.desy.de/viewvc/ilctools/ilcinstall/tags/?view=tar

into the afs area

/afs/phas.gla.ac.uk/data/ilc/software

I then untarred the file:

tar xvfz ilctools-tags.tar.gz

I then went into the directory:

/afs/phas.gla.ac.uk/data/ilc/software/tags

and removed all other tags except for the latest v01-14.

(Using svn to check out this tag would have been a more sensible way - will do next time!)

Preparing for build - separate packages to install

First we must prepare four packages that don't come automatically with the ilcsoft installation.

They are MySQL, Java, xerces-c and cernlib. MySQL was installed by sysadmin (Andy) as root

privileges are necessary. Java is already installed on our network. So it was necessary to install

xerces-c and Cernlib. I installed them in the:

/afs/phas.gla.ac.uk/data/ilc/software

directory. For xerces-c I did the following:

mkdir xerces-c

And in this directory I:

wget http://apache.mirror.rbftpnetworks.com//xerces/c/3/binaries/xerces-c-3.1.1-x86_64-linux-gcc-3.4.tar.gz

for 64 bit. And untarred.

For Cernlib I realised that the installation here ( /usr/lib64/cernlib) did not have the inlcude directories needed so I also went to /afs/phas.gla.ac.uk/data/ilc/software

and downloaded:

wget http://wwwasd.web.cern.ch/wwwasd/cernlib/download/2006b_x86_64-slc5-gcc41-opt/tar/cernlib.tar.gz
wget http://wwwasd.web.cern.ch/wwwasd/cernlib/download/2006b_x86_64-slc5-gcc41-opt/tar/include.tar.gz

and untarred them.

Preparing release version file

First of all I created a directory where the ilcsoft software packages will actually be installed into. This is called /data/atlas01/ILC/ilcsoft

It is now necessary to point to the above installations (MySQL etc.) in the release file that comes with the ilcsoft installer. To do this I went to file:

/afs/phas.gla.ac.uk/data/ilc/software/tags/v01-14/releases/v01-14/release-versions.py

But I first tell this file where the ilcsoft install directory is by setting:

ilcsoft_install_prefix = "/afs/phas.gla.ac.uk/data/ilc/software/ilcsoft/"

I then added:

MySQL_path = "/usr/"
Java_path = "/usr/java/latest/"
CERNLIB_path = "/afs/phas.gla.ac.uk/data/ilc/software/2006b"
XERCESC_ROOT_DIR = "/afs/phas.gla.ac.uk/data/ilc/software/xcerces-c/xerces-c-3.1.1-x86_64-linux-gcc-3.4"

to this file

Starting a full build for all packages

The file at the centre of this build is called 'release-scratch.cfg' which can be found in the same directory as the release-versions.py: /afs/phas.gla.ac.uk/data/ilc/software/tags/v01-14/releases/v01-14/

This file has the list and order in which all packages must be installed. First of all it is necessary to point to the release-version.py file from within this file:

path_where_this_file_lives = "/afs/phas.gla.ac.uk/data/ilc/software/tags/v01-14/releases/v01-14"

Then do:

./ilcsoft-install -p releases/v01-14/release-scratch.cfg -p (to check all is ok)
./ilcsoft-install -p releases/v01-14/release-scratch.cfg -I (to install)

Problems to expect and fix:

LCIO compilation problems

(1) Failed to find tools required to build lcio user manual - to fix this just turn off documentation installation. I did this for all pacakages by changing:

ilcsoft.envcmake["INSTALL_DOC"]="ON" to ilcsoft.envcmake["INSTALL_DOC"]="OFF"

in /afs/phas.gla.ac.uk/data/ilc/software/tags/v01-14/releases/v01-14/release-scratch.cfg, but it could be done at just a package level here:

/afs/phas.gla.ac.uk/data/ilc/software/ilcsoft/lcio/v02-01-02/CMakeLists.txt

(2) We don't want Fortran examples - and they cause problems in compilation anyway - so I commented out:

#MESSAGE( STATUS "BUILD_F77_TESTJOBS = ${BUILD_F77_TESTJOBS}" )

And:

# fortran examples
#OPTION( BUILD_F77_TESTJOBS "Set to ON to build LCIO F77 testjobs" OFF )
#IF( BUILD_F77_TESTJOBS )
# ADD_SUBDIRECTORY( ./src/f77 f77 )
#ENDIF()

again in: /afs/phas.gla.ac.uk/data/ilc/software/ilcsoft/lcio/v02-01-02/CMakeLists.txt

(3) I also turned off the optional root dict build for LCIO

ilcsoft.module("LCIO").envcmake['BUILD_ROOTDICT']='OFF'

/afs/phas.gla.ac.uk/data/ilc/software/tags/v01-14/releases/v01-14/release-scratch.py

Building Root Dict manually for certain packages

The Root Dict compilation does not work for some packages when building using the main compilation script so it is necessary to do the following:

KALTEST:

Comment out of /afs/phas.gla.ac.uk/data/ilc/software/tags/v01-14/releases/v01-14/release-scratch.py

cd /afs/phas.gla.ac.uk/data/ilc/software/ilcsoft/v01-14/KalTest/v01-05/build
# set up root (installed with ilcsoft)
source ../../../root/5.28.00f/bin/thisroot.sh
cmake -DILCUTIL_DIR=../../../ilcutil/ -DROOT_DIR=$ROOTSYS ..
make install

When installed – can uncomment in release-scratch.cfg again.

KALDET:

Comment out of /afs/phas.gla.ac.uk/data/ilc/software/tags/v01-14/releases/v01-14/release-scratch.py

cd /afs/phas.gla.ac.uk/data/ilc/software/ilcsoft/KalDet/v01-09/build
source ../../../root/5.28.00f/bin/thisroot.sh
cmake -DILCUTIL_DIR=../../../ilcutil/ -DROOT_DIR=$ROOTSYS ..
make install

When installed – can uncomment in release-scratch.cfg

LCFIPlus:

Comment out of /afs/phas.gla.ac.uk/data/ilc/software/tags/v01-14/releases/v01-14/release-scratch.py

cd /afs/phas.gla.ac.uk/data/ilc/software/ilcsoft/LCFIPlus/v00-05/build
source ../../../root/5.28.00f/bin/thisroot.sh
cmake -DILCUTIL_DIR=../../../ilcutil/ -DROOT_DIR=$ROOTSYS ..
make install

When installed – can uncomment in release-scratch.cfg but make sure not to reinstall.

BBQ:

Comment out of /afs/phas.gla.ac.uk/data/ilc/software/tags/v01-14/releases/v01-14/release-scratch.py

cd /afs/phas.gla.ac.uk/data/ilc/software/ilcsoft/bbq/v00-01-02/build
source ../../../root/5.28.00f/bin/thisroot.sh
cmake -DILCUTIL_DIR=../../../ilcutil/ -DROOT_DIR=$ROOTSYS ..

When installed – can uncomment in release-scratch.cfg but make sure not to reinstall.

Installing a new tag for one package within ilcsoft suite

Mokka needed to be reinstalled because of problem:

http://forum.linearcollider.org/index.php?t=tree&th=780&goto=2237&rid=151&S=18debae024982841a2da6aff71fca849#msg_2237

This ended out to be a lot simpler than I had expected. From the thread above the

new tag for mokka was mokka-08-00-03. All that was necessary to install it was

to change the version in:

/afs/phas.gla.ac.uk/data/ilc/software/tags/v01-14/releases/v01-14/release-versions.py

from Mokka_version = "mokka-08-00-02" to Mokka_version = "mokka-08-00-03"

and when you run:

./ilcsoft-install -p releases/v01-14/release-scratch.cfg -i

it automatically picks up the new version. Downloads it and builds it.

Running a validation test for Mokka and Marlin installations

Follow the instructions in StandardConfig/current/README, but using the default central database (do not use the script that creates a local DB).

. /afs/phas.gla.ac.uk/data/ilc/software/ilcsoft/v01-14/init_ilcsoft.sh
cd StandardConfig/current
Mokka -M ILD_o1_v05 -e ./particle.tbl bbudsc_3evt.steer
anajob bbudsc_3evt.slcio
Marlin bbudsc_3evt_stdreco.xml
dumpevent bbudsc_3evt_DST.slcio 2 

The visualisation bit did not work though.

See https://svnsrv.desy.de/viewvc/marlinreco/ILDConfig/trunk/StandardConfig/current/README?view=markup

Running a Marlin processor example in your home area

Preparing the home area

In this example we are building and running a Marlin processor called ExampleDSTAnalysis.

(1) Make the directory named as you wish - for example: /home/tdoherty/ILCWork/Marlin/ExampleDSTAnalysis

(2) In the ExampleDSTAnalysis directory create build, src, include and lib directories - like:

[tdoherty@ppepc144 ExampleDSTAnalysis]$ ls -lart
total 908
drwxr-xr-x 3 tdoherty grid 77 Aug 3 15:12 include
drwxr-xr-x 2 tdoherty grid 66 Aug 3 15:14 src
drwxr-xr-x 3 tdoherty grid 121 Aug 3 15:17 lib
drwxr-xr-x 5 tdoherty grid 4096 Aug 3 15:26 build

(3) Place the source ( ExampleDSTAnalysis.cc) code for your processor in the src directory and the header ( ExampleDSTAnalysis.h) file in the include/ directory

(4) Copy the CMakeLists.txt file from the mymarlin example given with Marlin into the ExampleDSTAnalysis folder:

cp /data/atlas01/ILC/ilcsoft/Marlin/v01-03/examples/mymarlin/CMakeLists.txt .

(5) Open up this file and just change the name of the project from mymarlin to ExampleDSTAnalysis: from PROJECT(mymarlin) to PROJECT(ExampleDSTAnalysis).

(6) If your processor code requires ROOT then add:

FIND_PACKAGE( ROOT REQUIRED )

and

FOREACH( pkg ROOT ) 
 IF( ${pkg}_FOUND )
  INCLUDE_DIRECTORIES( ${${pkg}_INCLUDE_DIRS} )
  ADD_DEFINITIONS( ${${pkg}_DEFINITIONS} )
 ENDIF()
ENDFOREACH()

to CMakeLists.txt so that your C++ code can pull in ROOT header files.

Building the processor

(1) First set up the ilcsoft software by running this script

. /afs/phas.gla.ac.uk/data/ilc/software/ilcsoft/v01-14/init_ilcsoft.sh

(2) cd into the build directory of this processor in your home area

cd build

(3) Then:

cmake -C $ILCSOFT/ILCSoft.cmake ..

(4) And:

make install

This should create library: lib/libExampleDSTAnalysis.so.

Loading the processor into Marlin

(1) Point to the .so file

export MARLIN_DLL=$PWD/lib/libExampleDSTAnalysis.so

(2) Create the steering file. This steering file is used to tell Marlin in what order to run your processor with other standard processors. It is used to set some global variables aswell as the input stdio file and the output file name. You also need to point to the gear (geometry) file that was used to create your input stdio file. Marlin automatically creates this file for you but you must edit the input/output files at least. The global variables will be taken from your C++ code. To create the file just do:

Marlin -x > mysteer.xml

(3) As mentioned you must edit the steering file to point to the input .stdio and gear.xml file you are using for your analysis. A very basic gear file can be used in this example.

[protopop@ppepc102 ExampleDSTAnalysis]$ cat mygear.xml 
<gear>
    <!--minimal Gear XML file that can be used to read simjob.slcio with Marlin (for testing) ...-->
    <global detectorName="CLIC_ILD_CDR500" />
    <BField type="ConstantBField" x="0.000000000e+00" y="0.000000000e+00" z="3.500000000e+00" />
    <detectors>
   </detectors>
</gear>

Running Marlin processor

Simply do:

Marlin mysteer.xml