ILCSoft < LinearCollider

---++ Software Installation

Installation of ILC software at Glasgow is described below:

%TOC%

---++++ Getting the latest version

Two choices for downloading the software is through =svn= or via a tarball. For the first attempt I opted for the latter. To do this I went to:

https://svnsrv.desy.de/viewvc/ilctools/ilcinstall/tags/

and downloaded the tar file accessible here:

[[https://svnsrv.desy.de/viewvc/ilctools/ilcinstall/tags/?view=tar]]

into the afs area
<verbatim>
/afs/phas.gla.ac.uk/data/ilc/software
</verbatim>

I then untarred the file:
<verbatim>
tar xvfz ilctools-tags.tar.gz
</verbatim>

I then went into the directory:
<verbatim>
/afs/phas.gla.ac.uk/data/ilc/software/tags
</verbatim>

and removed all other tags except for the latest *v01-14*.

(Using svn to check out this tag would have been a more sensible way - will do next time!)

---++++ Preparing for build - separate packages to install

First we must prepare four packages that don't come automatically with the ilcsoft installation.

<ins> </ins> They are *MySQL*, *Java*, *xerces-c* and *cernlib*. MySQL was installed by sysadmin (Andy) as root

privileges are necessary. Java is already installed on our network. So it was necessary to install

xerces-c and Cernlib. I installed them in the:
<verbatim>/afs/phas.gla.ac.uk/data/ilc/software</verbatim>

directory. For *xerces-c* I did the following:
<verbatim>
mkdir xerces-c
</verbatim>

And in this directory I:
<verbatim>
wget http://apache.mirror.rbftpnetworks.com//xerces/c/3/binaries/xerces-c-3.1.1-x86_64-linux-gcc-3.4.tar.gz
</verbatim>

for 64 bit. And untarred.

For *Cernlib* I realised that the installation here ( =/usr/lib64/cernlib=) did not have the inlcude directories needed so I also went to /afs/phas.gla.ac.uk/data/ilc/software

and downloaded:
<verbatim>
wget http://wwwasd.web.cern.ch/wwwasd/cernlib/download/2006b_x86_64-slc5-gcc41-opt/tar/cernlib.tar.gz
wget http://wwwasd.web.cern.ch/wwwasd/cernlib/download/2006b_x86_64-slc5-gcc41-opt/tar/include.tar.gz
</verbatim>

and untarred them.

---++++ Preparing release version file

First of all I created a directory where the _ilcsoft_ software packages will actually be installed into. This is called =/afs/phas.gla.ac.uk/data/ilc/software/ilcsoft=

It is now necessary to point to the above installations (MySQL etc.) in the release file that comes with the ilcsoft installer. To do this I went to file:
<verbatim>
/afs/phas.gla.ac.uk/data/ilc/software/tags/v01-14/releases/v01-14/release-versions.py
</verbatim>

But I first tell this file where the ilcsoft install directory is by setting:
<verbatim>
ilcsoft_install_prefix = "/afs/phas.gla.ac.uk/data/ilc/software/ilcsoft/"
</verbatim>

I then added:
<verbatim>
MySQL_path = "/usr/"
Java_path = "/usr/java/latest/"
CERNLIB_path = "/afs/phas.gla.ac.uk/data/ilc/software/2006b"
XERCESC_ROOT_DIR = "/afs/phas.gla.ac.uk/data/ilc/software/xcerces-c/xerces-c-3.1.1-x86_64-linux-gcc-3.4"
</verbatim>

to this file

---++++ Starting a full build for all packages

The file at the centre of this build is called 'release-scratch.cfg' which can be found in the same directory as the *release-versions.py*: /afs/phas.gla.ac.uk/data/ilc/software/tags/v01-14/releases/v01-14/

This file has the list and order in which all packages must be installed. First of all it is necessary to point to the =release-version.py= file from within this file:
<verbatim>
path_where_this_file_lives = "/afs/phas.gla.ac.uk/data/ilc/software/tags/v01-14/releases/v01-14"
</verbatim>

Then do:
<verbatim>
./ilcsoft-install -p releases/v01-14/release-scratch.cfg -p (to check all is ok)
./ilcsoft-install -p releases/v01-14/release-scratch.cfg -i (to install)
</verbatim>

---++++ Problems to expect and fix

Some of the packages had to be recompiled for various reasons, as described below.

---+++++ LCIO compilation problems

(1) Failed to find tools required to build lcio user manual - to fix this just turn off documentation installation. I did this for all pacakages by changing:
<verbatim>
ilcsoft.envcmake["INSTALL_DOC"]="ON" to ilcsoft.envcmake["INSTALL_DOC"]="OFF"
</verbatim>

in /afs/phas.gla.ac.uk/data/ilc/software/tags/v01-14/releases/v01-14/release-scratch.cfg, but it could be done at just a package level here:
<verbatim>
/afs/phas.gla.ac.uk/data/ilc/software/ilcsoft/v01-14/lcio/v02-01-02/CMakeLists.txt
</verbatim>

(2) We don't want _Fortran_ examples - and they cause problems in compilation anyway - so I commented out:

<verbatim>
#MESSAGE( STATUS "BUILD_F77_TESTJOBS = ${BUILD_F77_TESTJOBS}" )
</verbatim>

_And:_
<verbatim>
# fortran examples
#OPTION( BUILD_F77_TESTJOBS "Set to ON to build LCIO F77 testjobs" OFF )
#IF( BUILD_F77_TESTJOBS )
# ADD_SUBDIRECTORY( ./src/f77 f77 )
#ENDIF()
</verbatim>

again in: /afs/phas.gla.ac.uk/data/ilc/software/ilcsoft/v01-14/lcio/v02-01-02/CMakeLists.txt

(3) I also turned off the optional root dict build for *LCIO*
<verbatim>
ilcsoft.module("LCIO").envcmake['BUILD_ROOTDICT']='OFF'
</verbatim>

in =/afs/phas.gla.ac.uk/data/ilc/software/tags/v01-14/releases/v01-14/release-scratch.cfg=.

---+++++ !MarlinReco

This needs to be recompiled to include two items that were commented out in the list of processors (=Wolf= and =TrackwiseClustering=):
<verbatim>
cd /afs/phas.gla.ac.uk/data/ilc/software/ilcsoft/v01-14/MarlinReco/v01-03
</verbatim>
Edit file =CMakeLists.txt= and uncomment the lines corresponding to the two processors, then
<verbatim>
cd build
cmake -DILCUTIL_DIR=../../../ilcutil/  -DFORTRAN_LIBRARY=/usr/lib/gcc/x86_64-redhat-linux/4.1.2/libgfortran.so ..
make install
</verbatim>

---+++++ Building Root Dict manually for certain packages

The _Root Dict_ compilation does not work for some packages when building using the main compilation script so it is necessary to do the following:

*KALTEST*:

Comment out of =/afs/phas.gla.ac.uk/data/ilc/software/tags/v01-14/releases/v01-14/release-scratch.py=, and do by hand
<verbatim>
cd /afs/phas.gla.ac.uk/data/ilc/software/ilcsoft/v01-14/KalTest/v01-05/build
# set up root (installed with ilcsoft)
source ../../../root/5.28.00f/bin/thisroot.sh
cmake -DILCUTIL_DIR=../../../ilcutil/ -DROOT_DIR=$ROOTSYS ..
make install
</verbatim>

When installed, you can uncomment in =release-scratch.cfg= again.

*KALDET*:

Comment out in =/afs/phas.gla.ac.uk/data/ilc/software/tags/v01-14/releases/v01-14/release-scratch.py=, and do by hand
<verbatim>
cd /afs/phas.gla.ac.uk/data/ilc/software/ilcsoft/v01-14/KalDet/v01-09/build
source ../../../root/5.28.00f/bin/thisroot.sh
cmake -DILCUTIL_DIR=../../../ilcutil/ -DROOT_DIR=$ROOTSYS ..
make install
</verbatim>

When installed, you can uncomment in =release-scratch.cfg=

*LCFIPlus*:

Comment out of /afs/phas.gla.ac.uk/data/ilc/software/tags/v01-14/releases/v01-14/release-scratch.py
<verbatim>
cd /afs/phas.gla.ac.uk/data/ilc/software/ilcsoft/v01-14/LCFIPlus/v00-05/build
source ../../../root/5.28.00f/bin/thisroot.sh
cmake -DILCUTIL_DIR=../../../ilcutil/ -DROOT_DIR=$ROOTSYS ..
make install
</verbatim>

When installed &ndash; can uncomment in =release-scratch.cfg= but make sure not to reinstall.

*BBQ:*

Comment out of /afs/phas.gla.ac.uk/data/ilc/software/tags/v01-14/releases/v01-14/release-scratch.py
<verbatim>
cd /afs/phas.gla.ac.uk/data/ilc/software/ilcsoftv01-14//bbq/v00-01-02/build
source ../../../root/5.28.00f/bin/thisroot.sh
cmake -DILCUTIL_DIR=../../../ilcutil/ -DROOT_DIR=$ROOTSYS ..
make install
</verbatim>

When installed &ndash; can uncomment in *release-scratch.cfg* but make sure not to reinstall.
---+++ Installing a new tag for one package within ilcsoft suite

Mokka needed to be reinstalled because of problem:

http://forum.linearcollider.org/index.php?t=tree&th=780&goto=2237&rid=151&S=18debae024982841a2da6aff71fca849#msg_2237

This ended out to be a lot simpler than I had expected. From the thread above the

new tag for mokka was mokka-08-00-03. All that was necessary to install it was

to change the version in:
<verbatim>
/afs/phas.gla.ac.uk/data/ilc/software/tags/v01-14/releases/v01-14/release-versions.py
</verbatim>
from Mokka_version = "mokka-08-00-02" to Mokka_version = "mokka-08-00-03"

and when you run:
<verbatim>
./ilcsoft-install -p releases/v01-14/release-scratch.cfg -i
</verbatim>
it automatically picks up the new version. Downloads it and builds it.

---+++ Running a validation test for Mokka and Marlin installations

Follow the instructions in =validation/StandardConfig/current/README=, but *using the default central database* (do not use the script that creates a local DB).

<verbatim>
. /afs/phas.gla.ac.uk/data/ilc/software/ilcsoft/v01-14/init_ilcsoft.sh
cd /afs/phas.gla.ac.uk/data/ilc/software/validation/StandardConfig/current
Mokka -M ILD_o1_v05 -e ./particle.tbl bbudsc_3evt.steer
anajob bbudsc_3evt.slcio
Marlin bbudsc_3evt_stdreco.xml
dumpevent bbudsc_3evt_DST.slcio 2 
</verbatim>

%X% The visualisation bit did not work though.

See https://svnsrv.desy.de/viewvc/marlinreco/ILDConfig/trunk/StandardConfig/current/README?view=markup

---+++ Running a Marlin processor example in your home area

---++++ Preparing the home area

In this example we are building and running a Marlin processor called ExampleDSTAnalysis.

(1) Make the directory named as you wish - for example: =/home/tdoherty/ILCWork/Marlin/ExampleDSTAnalysis=

(2) In the =ExampleDSTAnalysis= directory create =build=, =src=, =include= and =lib= directories - like:
<verbatim>
[tdoherty@ppepc144 ExampleDSTAnalysis]$ ls -lart
total 908
drwxr-xr-x 3 tdoherty grid 77 Aug 3 15:12 include
drwxr-xr-x 2 tdoherty grid 66 Aug 3 15:14 src
drwxr-xr-x 3 tdoherty grid 121 Aug 3 15:17 lib
drwxr-xr-x 5 tdoherty grid 4096 Aug 3 15:26 build
</verbatim>

(3) Place the source ( =ExampleDSTAnalysis.cc=) code for your processor in the src directory and the header ( =ExampleDSTAnalysis.h=) file in the =include/= directory

(4) Copy the =CMakeLists.txt= file from the =mymarlin= example given with Marlin into the =ExampleDSTAnalysis= folder:
<verbatim>
cp /afs/phas.gla.ac.uk/data/ilc/software/ilcsoft/v01-14/Marlin/v01-03/examples/mymarlin/CMakeLists.txt .
</verbatim>

(5) Open up this file and just change the name of the project from _mymarlin_ to _ExampleDSTAnalysis_: from PROJECT(mymarlin) to PROJECT(ExampleDSTAnalysis).

(6) If your processor code requires ROOT then add:
<verbatim>
FIND_PACKAGE( ROOT REQUIRED )
</verbatim>

and
<verbatim>
FOREACH( pkg ROOT ) 
 IF( ${pkg}_FOUND )
  INCLUDE_DIRECTORIES( ${${pkg}_INCLUDE_DIRS} )
  ADD_DEFINITIONS( ${${pkg}_DEFINITIONS} )
 ENDIF()
ENDFOREACH()
</verbatim>

to =CMakeLists.txt= so that your C++ code can pull in ROOT header files.

(7) Edit =/afs/phas.gla.ac.uk/data/ilc/software/ilcsoft/v01-14/ILCSoft.cmake= and move ROOT before !MySQL, else the wrong =root-config= is picked up

---++++ Building the processor

(1) First set up the _ilcsoft_ software by running this script

=. /afs/phas.gla.ac.uk/data/ilc/software/ilcsoft/v01-14/init_ilcsoft.sh=

(2) cd into the =build= directory of this processor in your home area
<verbatim>
cd build
</verbatim>

(3) Then:
<verbatim>
cmake -C $ILCSOFT/ILCSoft.cmake ..
</verbatim>

(4) And:
<verbatim>
make install
</verbatim>

This should create library: *lib/libExampleDSTAnalysis.so*.

---++++ Loading the processor into Marlin

(1) Point to the =.so= file
<verbatim>
export MARLIN_DLL=$MARLIN_DLL:$PWD/lib/libExampleDSTAnalysis.so
</verbatim>

(2) Create the steering file. This steering file is used to tell *Marlin* in what order to run your processor with other standard processors. It is used to set some global variables aswell as the input stdio file and the output file name. You also need to point to the _gear_ (geometry) file that was used to create your input stdio file. Marlin automatically creates this file for you but you must edit the input/output files at least. The global variables will be taken from your C++ code. To create the file just do:

=Marlin -x &gt; mysteer.xml=

(This is a one-time step -- thereafter you will probably have edited mysteer.xml)

(3) As mentioned you must edit the steering file to point to the input =.stdio= and =gear.xml= file you are using for your analysis. A very basic _gear_ file can be used in this example.
<verbatim>
[protopop@ppepc102 ExampleDSTAnalysis]$ cat mygear.xml 
<gear>
    <!--minimal Gear XML file that can be used to read simjob.slcio with Marlin (for testing) ...-->
    <global detectorName="CLIC_ILD_CDR500" />
    <BField type="ConstantBField" x="0.000000000e+00" y="0.000000000e+00" z="3.500000000e+00" />
    <detectors>
   </detectors>
</gear>
</verbatim>

(4) To run one of the processors in the steering file, you must name it amongst the processor names at the beginning of the steering file. This will inform Marlin you want to execute the steering file with your processor.

For the !ExampleDSTAnalysis processor:
<pre>&lt;execute&gt;
 &lt;processor name="MyAIDAProcessor" /&gt;
 &lt;processor name="MyTestProcessor" /&gt;
 &lt;processor name="MyExampleDSTAnalysis"/&gt; 
 &lt;processor name="MyLCIOOutputProcessor" /&gt;
 &lt;/execute&gt;</pre>

This will create !ExampleDSTAnalysis.root which contains the list of histograms.
---++++ Running Marlin processor

Simply do:

=Marlin mysteer.xml=
This topic: LinearCollider > WebHome > ILCSoft
Topic revision: r16 - 2013-12-05 - DanProtopopescu