(r18) WebHome < LinearCollider

LinearCollider Web>WebHome (revision 18) (raw view)~~EditAttach~~
---+ Linear Collider activities in Glasgow

<!-- img style="width: 222; float: right" alt="ild.gif" src="https://ppes8.physics.gla.ac.uk/twiki/pub/LinearCollider/WebHome/ild.gif" title="ild.gif" height="169" --> <img alt="clic_event.gif" src="https://ppes8.physics.gla.ac.uk/twiki/pub/LinearCollider/WebHome/clic_event.gif" style="width: 200; float: right;" title="clic_event.gif" height="199" />

%TOC%

---++ Software

   * VO membership via ilc link at http://grid.desy.de/certs/vos.html
   * ilcsoft local installation temporarily at /data/atlas01/ILC/ilcsoft %X%

---+++ Installation of ILC software at Glasgow

---++++ Getting the latest version

Two choices for downloading the software is through svn or via a tarball. For the first attempt I opted for the latter. To do this I

went to:

https://svnsrv.desy.de/viewvc/ilctools/ilcinstall/tags/

and downloaded the tar file accessible here:

[[https://svnsrv.desy.de/viewvc/ilctools/ilcinstall/tags/?view=tar]]

into the afs area
<verbatim>
/afs/phas.gla.ac.uk/data/ilc/software
</verbatim>

I then untarred the file:
<verbatim>
tar xvfz ilctools-tags.tar.gz
</verbatim>

I then went into the directory:
<verbatim>
/afs/phas.gla.ac.uk/data/ilc/software/tags
</verbatim>

and removed all other tags except for the latest *v01-14*.

(Using svn to check out this tag would have been a more sensible way - will do next time!)

---++++ Preparing for build - separate packages to install

First we must prepare four packages that don't come automatically with the ilcsoft installation.

<ins> </ins> They are *MySQL*, *Java*, *xerces-c* and *cernlib*. MySQL was installed by sysadmin (Andy) as root

privileges are necessary. Java is already installed on our network. So it was necessary to install

xerces-c and Cernlib. I installed them in the:
<verbatim>/afs/phas.gla.ac.uk/data/ilc/software</verbatim>

directory. For *xerces-c* I did the following:
<verbatim>
mkdir xerces-c
</verbatim>

And in this directory I:
<verbatim>
wget http://apache.mirror.rbftpnetworks.com//xerces/c/3/binaries/xerces-c-3.1.1-x86_64-linux-gcc-3.4.tar.gz
</verbatim>

for 64 bit. And untarred.

For *Cernlib* I realised that the installation here ( =/usr/lib64/cernlib=) did not have the inlcude directories needed so I also went to /afs/phas.gla.ac.uk/data/ilc/software

and downloaded:
<verbatim>
wget http://wwwasd.web.cern.ch/wwwasd/cernlib/download/2006b_x86_64-slc5-gcc41-opt/tar/cernlib.tar.gz
wget http://wwwasd.web.cern.ch/wwwasd/cernlib/download/2006b_x86_64-slc5-gcc41-opt/tar/include.tar.gz
</verbatim>

and untarred them.

---++++ Preparing release version file

First of all I created a directory where the _ilcsoft_ software packages will actually be installed into. This is called =/data/atlas01/ILC/ilcsoft=

It is now necessary to point to the above installations (MySQL etc.) in the release file that comes with the ilcsoft installer. To do this I went to file:
<verbatim>
/afs/phas.gla.ac.uk/data/ilc/software/tags/v01-14/releases/v01-14/release-versions.py
</verbatim>

But I first tell this file where the ilcsoft install directory is by setting:
<verbatim>
ilcsoft_install_prefix = "/afs/phas.gla.ac.uk/data/ilc/software/ilcsoft/"
</verbatim>

I then added:
<verbatim>
MySQL_path = "/usr/"
Java_path = "/usr/java/latest/"
CERNLIB_path = "/afs/phas.gla.ac.uk/data/ilc/software/2006b"
XERCESC_ROOT_DIR = "/afs/phas.gla.ac.uk/data/ilc/software/xcerces-c/xerces-c-3.1.1-x86_64-linux-gcc-3.4"
</verbatim>

to this file

---++++ Starting a full build for all packages

The file at the centre of this build is called 'release-scratch.cfg' which can be found in the same directory as the *release-versions.py*: /afs/phas.gla.ac.uk/data/ilc/software/tags/v01-14/releases/v01-14/

This file has the list and order in which all packages must be installed. First of all it is necessary to point to the =release-version.py= file from within this file:
<verbatim>
path_where_this_file_lives = "/afs/phas.gla.ac.uk/data/ilc/software/tags/v01-14/releases/v01-14"
</verbatim>

Then do:
<verbatim>
./ilcsoft-install -p releases/v01-14/release-scratch.cfg -p (to check all is ok)
./ilcsoft-install -p releases/v01-14/release-scratch.cfg -I (to install)
</verbatim>

---++++ Problems to expect and fix:

---+++++ LCIO compilation problems

(1) Failed to find tools required to build lcio user manual - to fix this just turn off documentation installation. I did this for all pacakages by changing:
<verbatim>
ilcsoft.envcmake["INSTALL_DOC"]="ON" to ilcsoft.envcmake["INSTALL_DOC"]="OFF"
</verbatim>

in /afs/phas.gla.ac.uk/data/ilc/software/tags/v01-14/releases/v01-14/release-scratch.cfg, but it could be done at just a package level here:
<verbatim>
/afs/phas.gla.ac.uk/data/ilc/software/ilcsoft/lcio/v02-01-02/CMakeLists.txt
</verbatim>

(2) We don't want _Fortran_ examples - and they cause problems in compilation anyway - so I commented out:

<verbatim>
#MESSAGE( STATUS "BUILD_F77_TESTJOBS = ${BUILD_F77_TESTJOBS}" )
</verbatim>

_And:_
<verbatim>
# fortran examples
#OPTION( BUILD_F77_TESTJOBS "Set to ON to build LCIO F77 testjobs" OFF )
#IF( BUILD_F77_TESTJOBS )
# ADD_SUBDIRECTORY( ./src/f77 f77 )
#ENDIF()
</verbatim>

again in: /afs/phas.gla.ac.uk/data/ilc/software/ilcsoft/lcio/v02-01-02/CMakeLists.txt

(3) I also turned off the optional root dict build for *LCIO*
<verbatim>
ilcsoft.module("LCIO").envcmake['BUILD_ROOTDICT']='OFF'
</verbatim>

/afs/phas.gla.ac.uk/data/ilc/software/tags/v01-14/releases/v01-14/release-scratch.py
---+++++ _Building Root Dict manually for certain packages_

The _Root Dict_ compilation does not work for some packages when building using the main compilation script so it is necessary to do the following:

*KALTEST*:

Comment out of =/data/atlas01/ILC/tags/v01-14/releases/v01-14/release-scratch.cfg=
<verbatim>
cd /afs/phas.gla.ac.uk/data/ilc/software/ilcsoft/v01-14/KalTest/v01-05/build
# set up root (installed with ilcsoft)
source ../../../root/5.28.00f/bin/thisroot.sh
cmake -DILCUTIL_DIR=../../../ilcutil/ -DROOT_DIR=$ROOTSYS ..
make install
</verbatim>

When installed &ndash; can uncomment in =release-scratch.cfg= again.

*KALDET*:

Comment out of =/data/atlas01/ILC/tags/v01-14/releases/v01-14/release-scratch.cfg=
<verbatim>
cd /data/atlas01/ILC/ilcsoft/KalDet/v01-09/build
source ../../../root/5.28.00f/bin/thisroot.sh
cmake -DILCUTIL_DIR=../../../ilcutil/ -DROOT_DIR=$ROOTSYS ..
make install
</verbatim>

When installed &ndash; can uncomment in =release-scratch.cfg=

*LCFIPlus*:

Comment out of =/data/atlas01/ILC/tags/v01-14/releases/v01-14/release-scratch.cfg=
<verbatim>
cd /data/atlas01/ILC/ilcsoft/LCFIPlus/v00-05/build
source ../../../root/5.28.00f/bin/thisroot.sh
cmake -DILCUTIL_DIR=../../../ilcutil/ -DROOT_DIR=$ROOTSYS ..
make install
</verbatim>

When installed &ndash; can uncomment in =release-scratch.cfg= but make sure not to reinstall.

*BBQ:*

Comment out of =/data/atlas01/ILC/tags/v01-14/releases/v01-14/release-scratch.cfg=
<verbatim>
cd /data/atlas01/ILC/ilcsoft/bbq/v00-01-02/build
source ../../../root/5.28.00f/bin/thisroot.sh
cmake -DILCUTIL_DIR=../../../ilcutil/ -DROOT_DIR=$ROOTSYS ..
</verbatim>

When installed &ndash; can uncomment in *release-scratch.cfg* but make sure not to reinstall.

---+++ Installing a new tag for one package within ilcsoft suite

%X% STILL TO BE COMPLETED: Mokka needed to be reinstalled because of problem:

http://forum.linearcollider.org/index.php?t=tree&th=780&goto=2237&rid=151&S=18debae024982841a2da6aff71fca849#msg_2237

This ended out to be a lot simpler than I had expected. From the thread above the

new tag for mokka was mokka-08-00-03. All that was necessary to install it was

to change the version in:

/afs/phas.gla.ac.uk/data/ilc/software/tags/v01-14/releases/v01-14/release-versions.py

from Mokka_version = "mokka-08-00-02" to Mokka_version = "mokka-08-00-03"

and when you run:

./ilcsoft-install -p releases/v01-14/release-scratch.cfg -i

it automatically picks up the new version. Downloads it and builds it.
---+++ Running a validation test for Mokka and Marlin installations

%X% STILL TO BE COMPLETED: See https://svnsrv.desy.de/viewvc/marlinreco/ILDConfig/trunk/StandardConfig/current/README?view=markup

---+++ Running a Marlin processor example in your home area

---++++ Preparing the home area

In this example we are building and running a Marlin processor called ExampleDSTAnalysis.

(1) Make the directory named as you wish - for example: =/home/tdoherty/ILCWork/Marlin/ExampleDSTAnalysis=

(2) In the =ExampleDSTAnalysis= directory create =build=, =src=, =include= and =lib= directories - like:
<verbatim>
[tdoherty@ppepc144 ExampleDSTAnalysis]$ ls -lart
total 908
drwxr-xr-x 3 tdoherty grid 77 Aug 3 15:12 include
drwxr-xr-x 2 tdoherty grid 66 Aug 3 15:14 src
drwxr-xr-x 3 tdoherty grid 121 Aug 3 15:17 lib
drwxr-xr-x 5 tdoherty grid 4096 Aug 3 15:26 build
</verbatim>

(3) Place the source ( =ExampleDSTAnalysis.cc=) code for your processor in the src directory and the header ( =ExampleDSTAnalysis.h=) file in the =include/= directory

(4) Copy the =CMakeLists.txt= file from the =mymarlin= example given with Marlin into the =ExampleDSTAnalysis= folder:
<verbatim>
cp /data/atlas01/ILC/ilcsoft/Marlin/v01-03/examples/mymarlin/CMakeLists.txt .
</verbatim>

(5) Open up this file and just change the name of the project from _mymarlin_ to _ExampleDSTAnalysis_: from PROJECT(mymarlin) to PROJECT(ExampleDSTAnalysis).

(6) If your processor code requires ROOT then add:
<verbatim>
FIND_PACKAGE( ROOT REQUIRED )
</verbatim>

and
<verbatim>
FOREACH( pkg ROOT ) 
 IF( ${pkg}_FOUND )
  INCLUDE_DIRECTORIES( ${${pkg}_INCLUDE_DIRS} )
  ADD_DEFINITIONS( ${${pkg}_DEFINITIONS} )
 ENDIF()
ENDFOREACH()
</verbatim>

to =CMakeLists.txt= so that your C++ code can pull in ROOT header files.

---++++ Building the processor

(1) First set up the _ilcsoft_ software by running this script

=./data/atlas01/ILC/ilcsoft/init_ilcsoft.sh=

(2) cd into the =build= directory of this processor in your home area
<verbatim>
cd build
</verbatim>

(3) Then:
<verbatim>
cmake -C $ILCSOFT/ILCSoft.cmake ..
</verbatim>

(4) And:
<verbatim>
make install
</verbatim>

This should create library: *lib/libExampleDSTAnalysis.so*.

---++++ Loading the processor into Marlin

(1) Point to the =.so= file
<verbatim>
export MARLIN_DLL=$PWD/lib/libExampleDSTAnalysis.so
</verbatim>

(2) Create the steering file. This steering file is used to tell *Marlin* in what order to run your processor with other standard processors. It is used to set some global variables aswell as the input stdio file and the output file name. You also need to point to the _gear_ (geometry) file that was used to create your input stdio file. Marlin automatically creates this file for you but you must edit the input/output files at least. The global variables will be taken from your C++ code. To create the file just do:

=Marlin -x &gt; mysteer.xml=

(3) As mentioned you must edit the steering file to point to the input =.stdio= and =gear.xml= file you are using for your analysis.

---++++ Running Marlin processor

Simply do:

=Marlin mysteer.xml=

---++ Datasets

   * <p>[[DatasetAccess]]</p>

---++ Useful Links

   * [[http://ilcsoft.desy.de/portal/][ILCSoft]]
   * [[http://lcd.web.cern.ch/LCD/Documents/Documents.html][CERN LC Physics & Detectors Group]]
   * [[http://indico.cern.ch/categoryDisplay.py?categId=3061][LCD-WG Indico]]
   * [[https://twiki.cern.ch/twiki/bin/view/CLIC/Detector][CLIC Detector Twiki]]
   * Documents: 
      * [[http://arxiv.org/abs/1202.5940][CLIC Conceptual Design Report: Physics and Detectors]] (Feb 2012)
      * [[http://arxiv.org/abs/1006.3396/][ILD Letter of Intent]] (Feb 2010)
      * [[%ATTACHURL%/gaede_ildsoft_chep09.pdf][ILC Software Framework - Frank Gaede's CHEP 2009 talk]]

<!--
--++ <nop>%WEB% Web Utilities

<form action='%SCRIPTURLPATH{"search"}%/%WEB%/'>
   * <input type="text" name="search" size="22" /> <input type="submit" class="twikiSubmit" value="%MAKETEXT{"Search"}%" /> - [[WebSearchAdvanced][%MAKETEXT{"advanced search"}%]]
   * WebTopicList - all topics in alphabetical order
   * WebChanges - recent topic changes in this web
   * WebNotify - subscribe to an e-mail alert sent when topics change
   * WebRss, WebAtom - RSS and ATOM news feeds of topic changes
   * WebStatistics - listing popular topics and top contributors
   * WebPreferences - preferences of this web

</form>
-->
Attachments
Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
png	ILD-LDC-SWF.png	r1	manage	144.5 K	2012-08-08 - 15:28	DanProtopopescu	SW framework overview
pdf	gaede_ildsoft_chep09.pdf	r1	manage	2139.7 K	2012-08-08 - 15:23	DanProtopopescu	ILC Software Framework - Frank Gaede's CHEP 2009 talk
Topic revision: r18 - 2012-08-14 - ThomasDoherty
LinearCollider