Difference: CustomReconstruction (1 vs. 14)

Revision 142014-07-09 - KennyWraight

Line: 1 to 1
 
META TOPICPARENT name="EUTelescopeAndIBLTestbeam"

Customising IBL Reconstruction

Line: 237 to 237
 * edit appropriate branch * add changes to staging area (git add) * commit changes to branch on local repository (git commit ... -m "COMMENT")
Added:
>
>
*set ssh keys (use this link for easy recipe) https://help.github.com/articles/generating-ssh-keys * check urls are git not https then...
 * push to remote repository (git push )

upload for later...

Revision 132014-06-25 - KennyWraight

Line: 1 to 1
 
META TOPICPARENT name="EUTelescopeAndIBLTestbeam"

Customising IBL Reconstruction

Line: 6 to 6
 

Overview

Changed:
<
<
To make a custom reconstruction you will need appropriate gear, config, and steering files. If possible, ask the author or someone who might know where to get these files as it is much easier to edit working code than to invent your own.
>
>
To make a custom reconstruction you will need appropriate gear, config, and steering files. If possible, ask the author or someone who might know where to get these files as it is much easier to edit working code than to invent your own. PS git is a useful cool repository tool.
  The gear file defines the positions and orientation of the devices present: Mimosas and devices under test (DUTs). The config file sets the essential variable values which the reconstruction code will need to find and write files and run the code successfully. The main part of the of the reconstruction comes in calling the appropriate sub-routines via the steering files.
Line: 224 to 224
 When these are edited, the code must be recompiled in the ../external/eudaq/tags/v01-00-00/main directory (using make ) and in the ../vXX-YY-ZZ/Eutelescope/vxx-yy-zz/build/ directory (using make install , NB the extra.potion this time).This way all the relevant executables are updated.
Added:
>
>

Git repository (very) basics (just added sketch, needs proper home)

To update a git repository with your customised code follow these instructions:

assumed cloned version... * set git directory (git init) * clone remote repository (git clone )

make changes... * edit appropriate branch * add changes to staging area (git add) * commit changes to branch on local repository (git commit ... -m "COMMENT") * push to remote repository (git push )

upload for later... * fetch from remote repository (git fetch ) * merge new changes (git merge )

 -- KennyWraight - 2013-06-11

META FILEATTACHMENT attachment="anemone_clu_DUT20p0_YcorShiftProj_good.gif" attr="" comment="Form example with good gear file" date="1403000699" name="anemone_clu_DUT20p0_YcorShiftProj_good.gif" path="anemone_clu_DUT20p0_YcorShiftProj_good.gif" size="7963" user="KennyWraight" version="1"

Revision 122014-06-18 - KennyWraight

Line: 1 to 1
 
META TOPICPARENT name="EUTelescopeAndIBLTestbeam"

Customising IBL Reconstruction

Line: 174 to 173
 anemone_hit_p0DUT20_XhitCor.gif
Added:
>
>
Align fixing relations between planes

DafFitter output histograms can be used to diagnose problems using the resolution plots, which should be single peaked usually with some offset. Any offset in resolution is automatically used by the code for fitting purposes and should not be manually applied. After pre-alignment residuals should not shift more than the pitch size. Residuals should be centred very well at 0, with +-few tens of um for mimosas and up to +- couple 100 um for DUTs.

 

steering files comments

(Integrate this section later.)
Line: 209 to 212
  Notes (integrate in text fully later)
Deleted:
<
<
Check

DafFitter output histograms can be used to diagnose problems using the resolution plots, which should be single peaked usually with some offset. Any offset in resolution is automatically used by the code for fitting purposes and should not be manually applied. After pre-alignment residuals should not shift more than the pitch size.

  Filter data in later version of the reconstruction software (i.e. >= v01-17-05) cleans up hot pixels/clusters before hitmaker steps which should aid pre-alignment.

Revision 112014-06-18 - KennyWraight

Line: 1 to 1
 
META TOPICPARENT name="EUTelescopeAndIBLTestbeam"

Customising IBL Reconstruction

Line: 6 to 6
 

Overview

Changed:
<
<
To make a custom reconstruction you will need appropriate =gear=, config, and steering files. If possible, ask the author or someone who might know where to get these files as it is much easier to edit working code than to invent your own.
>
>
To make a custom reconstruction you will need appropriate gear, config, and steering files. If possible, ask the author or someone who might know where to get these files as it is much easier to edit working code than to invent your own.
  The gear file defines the positions and orientation of the devices present: Mimosas and devices under test (DUTs). The config file sets the essential variable values which the reconstruction code will need to find and write files and run the code successfully. The main part of the of the reconstruction comes in calling the appropriate sub-routines via the steering files.
Line: 92 to 92
 

steering files

Deleted:
<
<
If you editing the steering files you must be careful that the correct referencing is done between the values set in the config file and the values used in the steering files. Problems are often generated when directories or values defined in the config file are not properly referenced or simply hard-coded in the steering file. The code will fail if files, directories or values mentioned in the steering files do not exist. Always check the error messages if running jobsub fails.

Be sure to replace hard-coded references with values set in config file, e.g.:

someplacewithgeomfile

change to

@GeometryPath@/@GearFile@"

where both variables GeometryPath and GearFile are set in config.cfg:

# Geometry file GearFile = my_gear_file.xml

# Path to the geometry file GeometryPath = %(BasePath)s

 Common Features

Steering files for each run or set of runs are produced on execution from the template files (found in the steering-templates directory). The general templates become specific when the variables in specified in the config files are substituted into the templates. To check the of specific executions are correct use the jobsub -dry-run option on execution. This will copy the steering files used for each run to the current directory.

Line: 122 to 100
  Each steering file has several sections in common. To begin there is an execute section where the list of all processes to be run is defined. There is also a global section where the global (e.g. gear file location) from the config file are passed. The rest of the file is taken up with setting the variables of each processor defined in the execute section. The task specific parameters (e.g. FiringFrequency) are passed to the processors in these sections.
Changed:
<
<
Converter read DUT to into EUTelescope framework
>
>
Converter Read DUT to into EUTelescope framework
  EUDaq processor is used to record testbeam data in organised fashion, however this organisation is not standard, rather it is specific to categories of device (e.g. Atlas Pixel, "APIX"). Converters can be found in $EUTELESCOPE/external/eudaq/tags/v01-00-00/main/src/ (NB slight differences in directory structure between versions). For current (!) testbeam data APIX-CT-ConverterPlugin.cc is the converter used. Any manipulation of data as it enters the EUTelescope software framework should take place here. As EUDAq is separate from EUTelescope, i.e. the converter is not an EUTelescope processor, you cannot read in variables through the variables through the steering file.
Changed:
<
<
Clustering Manipulating DUT pixels
>
>
Clustering Manipulating DUT pixels
 
Changed:
<
<
Clustering collects pixels into clusters according to some constraints (such as minimum and maximum cluster size). It also contains a correlation processor which can be used to check the orientation of the DUTs with respect to one another. NB Correlations at the clustering stage are done on a pixel-to-pixel basis.
>
>
Clustering collects pixels into clusters according to some constraints (such as minimum and maximum cluster size). It also contains a correlation processor which can be used to calculate the orientation of the DUTs with respect to one another. NB Correlations at the clustering stage are done on a pixel-to-pixel basis.
  Using the ROOT file
Line: 150 to 126
  The correlation plots compare pixels based on their X or Y position. If pixels are correctly orientated with respect to one another then the correlations should be positive, i.e. greatest intensity from bottom left to top right: in the case of comparing devices with the same number of pixels, correctly orientated devices will produce a correlation running up the central diagonal; any offset in the devices will displace the diagonal. Devices which are flipped with respect to one another will produce an anti-correlation, i.e. greatest intensity from bottom left to top right: similar comments apply for case when devices have the same number of pixels. Devices which are rotated will lose correlation, such that for 90 rotation case no correlation will be seen. Changes to orientation are made in the gear file, either by the XY-rotation matrix or individual rotation parameters.
Changed:
<
<
This is an example of an X-correlation map between planes 0 and 1 in the telescope. The correlation can be seen in the increasing in the positive direction as the diagonal line between corresponding pixels. Here it runs for x=y as planes are well aligned and orientated and devices have same number of pixels.
>
>
This is an example of an X-correlation map between planes 0 and 1 in the telescope. The correlation can be seen in the increasing in the positive direction as the diagonal line between corresponding pixels. Here it runs for x=y as planes are well aligned and orientated and devices have same number of pixels. NB changes in orientation will not be seen in clustering tep as this only deals with pixels not global orientation of devices. Changes in orientation will have affect in hitmaker.
 anemone_clu_p0p1_Xcor_good.gif

This is an example of an X-correlation map between planes the above DUT and plane 0 in the telescope. The correlation here is positive but offset (as is typical), there is also a mismatch in axes bins due to the difference in detector geometries.

Line: 166 to 146
 anemone_clu_DUT20p0_XcorShift_good.gif anemone_clu_DUT20p0_XcorShiftProj_good.gif
Added:
>
>
The correlation shift plots are used to identify the offset peaks. Cutting around these peaks is used by the prealign processor in hitmaker to improve track fitting. The peak parameters are used in the PreAligner and Correlator processors in the ResidualsXMax, ResidualsXMin, ResidualsYMax and ResidualsYMin vectors.

Hitmaker Changing to global telescope geometry

Hitmaker moves the detector planes and DUTs into the global geometry of the telescope so that alignment can be preformed. It also contains a correlation processor which can be used to check the orientation of the DUTs with respect to one another. NB Correlations at the hitmaker stage are done on a pitch-to-pitch basis i.e. the pixels are translated into the global coordinate system of the telescope.

Using the ROOT file

The hitmaker root file has similar hitmaps for each mimosa plane and DUT as the clustering output but with axes translated into the global telescope geometry. Changes to correlations due to gear file gemometry parameters will be seen in the correlations and hitmaps.

These are the local an global hitmaps corresponding to the above DUT.

anemone_hit_DUT20_local.gif anemone_hit_DUT20_global.gif

These are correlation and shift plots corresponding to the above DUT. Note the cuts around the peaks.

anemone_hit_DUT20p0_Xcor.gif anemone_hit_DUT20p0_XcorShift.gif anemone_hit_DUT20p0_XcorShiftProj.gif

There are also plots from the pre-alignment which show the same results as the shift plots. Here are plots for the plane 1 and DUT20 offset with respect to plane 0.

anemone_hit_p0p1_XhitCor.gif anemone_hit_p0DUT20_XhitCor.gif

 

steering files comments

(Integrate this section later.)
Line: 173 to 181
  Tips
Changed:
<
<
In general, make sure any definition of the number of planes is correct: #plane = #mimosas + #DUTs.
>
>
If you editing the steering files you must be careful that the correct referencing is done between the values set in the config file and the values used in the steering files. Problems are often generated when directories or values defined in the config file are not properly referenced or simply hard-coded in the steering file. The code will fail if files, directories or values mentioned in the steering files do not exist. Always check the error messages if running jobsub fails.

Be sure to replace hard-coded references with values set in config file, e.g.:

 
Added:
>
>
someplacewithgeomfile

change to

@GeometryPath@/@GearFile@"

where both variables GeometryPath and GearFile are set in config.cfg:

# Geometry file GearFile = my_gear_file.xml

# Path to the geometry file GeometryPath = %(BasePath)s

In general, make sure any definition of the number of planes is correct: #plane = #mimosas + #DUTs.

  In the steering files where number of planes is defined (e.g. in DafFitter processor in align-tmp, fitter-tmp) by TelescopePlanes, the set-up can be defined in two ways. Firstly all planes (mimosas & DUTs) can be defined in one ordered vector (gear file is not used for z-ordering here). This uses all planes for the fitting. Alternatively, only mimosa planes can be defined and DUTs can be referenced in the DutPlanes. If this is done the FitDuts parameter mus be set true. Either way the RequireNTelPlanes parameter should be appropriately set (e.g. to 6 if the mimosa planes alone are used).
Line: 208 to 237
 
META FILEATTACHMENT attachment="anemone_clu_DUT20_bad.gif" attr="" comment="Form example with bad gear file" date="1403003407" name="anemone_clu_DUT20_bad.gif" path="anemone_clu_DUT20_bad.gif" size="11000" user="KennyWraight" version="1"
META FILEATTACHMENT attachment="anemone_clu_p0p1_XcorShiftProj_good.gif" attr="" comment="Form example with bad gear file" date="1403003750" name="anemone_clu_p0p1_XcorShiftProj_good.gif" path="anemone_clu_p0p1_XcorShiftProj_good.gif" size="4313" user="KennyWraight" version="1"
META FILEATTACHMENT attachment="anemone_clu_p0p1_XcorShift_good.gif" attr="" comment="Form example with bad gear file" date="1403003750" name="anemone_clu_p0p1_XcorShift_good.gif" path="anemone_clu_p0p1_XcorShift_good.gif" size="13881" user="KennyWraight" version="1"
Added:
>
>
META FILEATTACHMENT attachment="anemone_hit_p0DUT20_XhitCor.gif" attr="" comment="Form example with good gear file" date="1403086771" name="anemone_hit_p0DUT20_XhitCor.gif" path="anemone_hit_p0DUT20_XhitCor.gif" size="3804" user="KennyWraight" version="1"
META FILEATTACHMENT attachment="anemone_hit_p0p1_XhitCor.gif" attr="" comment="Form example with good gear file" date="1403086771" name="anemone_hit_p0p1_XhitCor.gif" path="anemone_hit_p0p1_XhitCor.gif" size="3577" user="KennyWraight" version="1"
META FILEATTACHMENT attachment="anemone_hit_DUT20p0_XcorShiftProj.gif" attr="" comment="Form example with good gear file" date="1403086771" name="anemone_hit_DUT20p0_XcorShiftProj.gif" path="anemone_hit_DUT20p0_XcorShiftProj.gif" size="4193" user="KennyWraight" version="1"
META FILEATTACHMENT attachment="anemone_hit_DUT20p0_XcorShift.gif" attr="" comment="Form example with good gear file" date="1403086771" name="anemone_hit_DUT20p0_XcorShift.gif" path="anemone_hit_DUT20p0_XcorShift.gif" size="5767" user="KennyWraight" version="1"
META FILEATTACHMENT attachment="anemone_hit_DUT20p0_Xcor.gif" attr="" comment="Form example with good gear file" date="1403086771" name="anemone_hit_DUT20p0_Xcor.gif" path="anemone_hit_DUT20p0_Xcor.gif" size="8501" user="KennyWraight" version="1"
META FILEATTACHMENT attachment="anemone_hit_DUT20_local.gif" attr="" comment="Form example with good gear file" date="1403086771" name="anemone_hit_DUT20_local.gif" path="anemone_hit_DUT20_local.gif" size="20021" user="KennyWraight" version="1"
META FILEATTACHMENT attachment="anemone_hit_DUT20_global.gif" attr="" comment="Form example with good gear file" date="1403086771" name="anemone_hit_DUT20_global.gif" path="anemone_hit_DUT20_global.gif" size="10886" user="KennyWraight" version="1"

Revision 102014-06-17 - KennyWraight

Line: 1 to 1
 
META TOPICPARENT name="EUTelescopeAndIBLTestbeam"

Customising IBL Reconstruction

Changed:
<
<
Notes on customising reconstruction process for IBL testbeam data using EUTelescope software. NB this is a perilous process and, so far, not perfected.
>
>
Notes on customising reconstruction process for ITk (formerly IBL) testbeam data using EUTelescope software. NB this is a perilous process and, so far, not perfected.
 
Changed:
<
<

Required files

>
>

Overview

 
Changed:
<
<
To make a custom reconstruction you will need appropriate gear, config, and steering files. If possible, ask the author or someone who might know where to get these files as it is much easier to edit working code than to invent your own. The gear file defines the positions and orientation of the devices present: Mimosas and devices under test (DUTs). The config file sets the essential variable values which the reconstruction code will need to find and write files and run the code successfully. The main part of the of the reconstruction comes in calling the appropriate sub-routines via the steering files.

If you editing the steering files you must be careful that the correct referencing is done between the values set in the config file and the values used in the steering files. Problems are often generated when directories or values defined in the config file are not properly referenced or simply hard-coded in the steering file. The code will fail if files, directories or values mentioned in the steering files do not exist. Always check the error messages if running jobsub fails.

Be sure to replace hard-coded references with values set in config file, e.g.:

someplacewithgeomfile

change to

@GeometryPath@/@GearFile@"

where both variables GeometryPath and GearFile are set in config.cfg:

# Geometry file GearFile = my_gear_file.xml

# Path to the geometry file GeometryPath = %(BasePath)s

>
>
To make a custom reconstruction you will need appropriate =gear=, config, and steering files. If possible, ask the author or someone who might know where to get these files as it is much easier to edit working code than to invent your own.
 
Added:
>
>
The gear file defines the positions and orientation of the devices present: Mimosas and devices under test (DUTs). The config file sets the essential variable values which the reconstruction code will need to find and write files and run the code successfully. The main part of the of the reconstruction comes in calling the appropriate sub-routines via the steering files.
 

config file layout

Added:
>
>
  The config file is written in python. It lists all the variables set by the user for the reconstruction. Essential parameters are things like the file structure the reconstruction will run in, e.g. the directories to find input files and write output files to, and the location of the gear and steering files. These files must exist (i.e. be set correctly) or the program will not run.
Line: 111 to 90
 

steering files

Added:
>
>

If you editing the steering files you must be careful that the correct referencing is done between the values set in the config file and the values used in the steering files. Problems are often generated when directories or values defined in the config file are not properly referenced or simply hard-coded in the steering file. The code will fail if files, directories or values mentioned in the steering files do not exist. Always check the error messages if running jobsub fails.

Be sure to replace hard-coded references with values set in config file, e.g.:

someplacewithgeomfile

change to

@GeometryPath@/@GearFile@"

where both variables GeometryPath and GearFile are set in config.cfg:

# Geometry file GearFile = my_gear_file.xml

# Path to the geometry file GeometryPath = %(BasePath)s

  Common Features
Line: 186 to 188
 

EUTelescope analysis

Added:
>
>
  The source code of the EUTelescope analysis can be found in the ../vXX-YY-ZZ/Eutelescope/vxx-yy-zz/ directory. For example the converters are found in ../external/eudaq/tags/v01-00-00/main/src/ .

Revision 92014-06-17 - KennyWraight

Line: 1 to 1
 
META TOPICPARENT name="EUTelescopeAndIBLTestbeam"

Customising IBL Reconstruction

Line: 46 to 46
 FiringFrequency = 0.1
Changed:
<
<
Here, the FiringFrequency variable used by the conveter steering file is set. Other parameters tasks are similarly defined.
>
>
Here, the FiringFrequency variable used by the converter steering file is set. Other parameters tasks are similarly defined.
 

gear file layout

Added:
>
>
  The gear file is written in xml. It lists the geometry and orientation of the Mimosas use for track formation and the DUTs. The essential parameters of each include the position, rotation, size and pitch of each detector. gear files should be obtained with raw data as they must correspond to the devices used to record data in the testbeam.
Changed:
<
<
To edit or amend a file be sure to set the parameters according to the following conventions:
>
>
To edit or amend a file be sure to set the parameters according to the following conventions (regrettably, these may change between testbeams):
 
Line: 61 to 62
 
Changed:
<
<
The rotataion matrix has the form
>
>
Rotations are specified by the individual rotation parameters: rotationZY, rotationZX and rotationXY; or, in the XY case the rotation matrix. The XY-rotataion matrix has the form
 \begin{bmatrix} Cos\theta & -Sin\theta \ Sin\theta & Cos\theta \
Line: 69 to 70
  so the entries are usually 1,0,-1 for 90 degree rotations.
Added:
>
>
Here is some lined of code for a standard (250x50) DUT from the anemone example:

<--APIX-Plane 1 - FE-I4 DBM -->

<ladder ID="20"

positionX="0.00" positionY="0.00" positionZ="334.0"

rotationZY="180.0" rotationZX="0.0" rotationXY="0.0"

sizeX="20.0" sizeY="16.8" thickness="0.700"

radLength="65.000000"

/>

<sensitive ID="20"

positionX="0.00" positionY="0.00" positionZ="334.0"

sizeX="20.0" sizeY="16.8" thickness="0.500"

npixelX="80" npixelY="336"

pitchX="0.250" pitchY="0.05" resolution="0.1000"

rotation1="0.0" rotation2="-1.0"

rotation3="1.0" rotation4="0.0"

radLength="65.000000"

/>

 

steering files

Line: 81 to 121
 Each steering file has several sections in common. To begin there is an execute section where the list of all processes to be run is defined. There is also a global section where the global (e.g. gear file location) from the config file are passed. The rest of the file is taken up with setting the variables of each processor defined in the execute section. The task specific parameters (e.g. FiringFrequency) are passed to the processors in these sections.

Converter

Changed:
<
<
Uses EUDAq software to read output of DUT to into EUTelescope framework.
>
>
read DUT to into EUTelescope framework
 
Changed:
<
<
EUDaq processor is used to recorde testbeam data in organised fashion, however this organisation is not standard, rather it is specific to categories of device (e.g. Atlas Pixel, "APIX"). Converters can be found in $EUTELESCOPE/external/eudaq/tags/v01-00-00/main/src/ (NB slight differences in directory structure between versions). For current (!) testbeam data APIX-CT-ConverterPlugin.cc is the converter used. Any manipulation of data as it enters the EUTelescope software framework should take place here. As EUDAq is separate from EUTelescope, i.e. the converter is not an EUTelescope processor, you cannot read in variables through the variables through the steering file.
>
>
EUDaq processor is used to record testbeam data in organised fashion, however this organisation is not standard, rather it is specific to categories of device (e.g. Atlas Pixel, "APIX"). Converters can be found in $EUTELESCOPE/external/eudaq/tags/v01-00-00/main/src/ (NB slight differences in directory structure between versions). For current (!) testbeam data APIX-CT-ConverterPlugin.cc is the converter used. Any manipulation of data as it enters the EUTelescope software framework should take place here. As EUDAq is separate from EUTelescope, i.e. the converter is not an EUTelescope processor, you cannot read in variables through the variables through the steering file.
  Clustering
Changed:
<
<
Manipulating DUT pixels.
>
>
Manipulating DUT pixels

Clustering collects pixels into clusters according to some constraints (such as minimum and maximum cluster size). It also contains a correlation processor which can be used to check the orientation of the DUTs with respect to one another. NB Correlations at the clustering stage are done on a pixel-to-pixel basis.

Using the ROOT file

Consistency checks of detector geometry and orientation can be performed by inspecting the histograms produced by the clustering root file. Inside (assuming relevant processors have been used) are hit maps for the mimosas and DUTs (from clustering processors), as well a correlation maps (from correlation processor).

The hitmap axes should correspond to the number of pixels and rows defined in the gear file. If there is a cosmetic problem (i.e. the histogram is wrongly binned) then change the gear file as this is referenced to set the histogram axes. If the gear file is correct but the histogram is not correctly filled then there has been a problem in the converter stage. To remedy the converter step must be rerun with the converter file appropriately changed (and recompiled) to read the correct number of row and column pixels.

This is an example of a hitmap with correct number of pixels for a mimosa: 1152x576 pixels. anemone_clu_m26_good.gif

 
Changed:
<
<
Clustering collects pixels into clusters according to some constraints (such as minimum and maximum cluster size). It also contains a correlation processor which can be used to check the orientation of the DUTs with respect to one another.
>
>
This is an example of a hitmap with correct number of pixels for a standard DUT (note, the X-edge pixels are masked). Standard devices have 80 columns (20cm/250um) and 336 rows (16.8cm/50um). anemone_clu_DUT20_good.gif

This is an example of a hitmap with incorrect number of pixels for the same standard (250x50um**2) DUT. The gear file specifies too many rows and too few columns. anemone_clu_DUT20_bad.gif

The correlation plots compare pixels based on their X or Y position. If pixels are correctly orientated with respect to one another then the correlations should be positive, i.e. greatest intensity from bottom left to top right: in the case of comparing devices with the same number of pixels, correctly orientated devices will produce a correlation running up the central diagonal; any offset in the devices will displace the diagonal. Devices which are flipped with respect to one another will produce an anti-correlation, i.e. greatest intensity from bottom left to top right: similar comments apply for case when devices have the same number of pixels. Devices which are rotated will lose correlation, such that for 90 rotation case no correlation will be seen. Changes to orientation are made in the gear file, either by the XY-rotation matrix or individual rotation parameters.

This is an example of an X-correlation map between planes 0 and 1 in the telescope. The correlation can be seen in the increasing in the positive direction as the diagonal line between corresponding pixels. Here it runs for x=y as planes are well aligned and orientated and devices have same number of pixels. anemone_clu_p0p1_Xcor_good.gif

This is an example of an X-correlation map between planes the above DUT and plane 0 in the telescope. The correlation here is positive but offset (as is typical), there is also a mismatch in axes bins due to the difference in detector geometries. anemone_clu_DUT20p0_Xcor_good.gif

The correlation ROOT file also contains plots pertaining to the offset of devices. These are used to quantify the misalignment of devices in the code (in normal circumstances no manual input of offset should be necessary). There are also projection plots of the maps which peak at offset values. The projections come from offset plots w.r.t. plane 0.

This is an example of an X-correlation shift plot (and corresponding projection) between planes the above planes 0 and 1 in the telescope. anemone_clu_p0p1_XcorShift_good.gif anemone_clu_p0p1_XcorShiftProj_good.gif

This is the corresponding X-correlation shift plot (and corresponding projection) for the above DUT and plane 0 in the telescope. anemone_clu_DUT20p0_XcorShift_good.gif anemone_clu_DUT20p0_XcorShiftProj_good.gif

 

steering files comments

(Integrate this section later.)
Line: 119 to 193
 

-- KennyWraight - 2013-06-11

Added:
>
>
META FILEATTACHMENT attachment="anemone_clu_DUT20p0_YcorShiftProj_good.gif" attr="" comment="Form example with good gear file" date="1403000699" name="anemone_clu_DUT20p0_YcorShiftProj_good.gif" path="anemone_clu_DUT20p0_YcorShiftProj_good.gif" size="7963" user="KennyWraight" version="1"
META FILEATTACHMENT attachment="anemone_clu_DUT20p0_YcorShift_good.gif" attr="" comment="Form example with good gear file" date="1403000722" name="anemone_clu_DUT20p0_YcorShift_good.gif" path="anemone_clu_DUT20p0_YcorShift_good.gif" size="8852" user="KennyWraight" version="1"
META FILEATTACHMENT attachment="anemone_clu_DUT20p0_XcorShiftProj_good.gif" attr="" comment="Form example with good gear file" date="1403000722" name="anemone_clu_DUT20p0_XcorShiftProj_good.gif" path="anemone_clu_DUT20p0_XcorShiftProj_good.gif" size="5398" user="KennyWraight" version="1"
META FILEATTACHMENT attachment="anemone_clu_DUT20p0_XcorShift_good.gif" attr="" comment="Form example with good gear file" date="1403000722" name="anemone_clu_DUT20p0_XcorShift_good.gif" path="anemone_clu_DUT20p0_XcorShift_good.gif" size="8343" user="KennyWraight" version="1"
META FILEATTACHMENT attachment="anemone_clu_m26p0_good.gif" attr="" comment="Form example with good gear file" date="1403000747" name="anemone_clu_m26p0_good.gif" path="anemone_clu_m26p0_good.gif" size="29356" user="KennyWraight" version="1"
META FILEATTACHMENT attachment="anemone_clu_DUT20_good.gif" attr="" comment="Form example with good gear file" date="1403000747" name="anemone_clu_DUT20_good.gif" path="anemone_clu_DUT20_good.gif" size="19213" user="KennyWraight" version="1"
META FILEATTACHMENT attachment="anemone_clu_p0p1_Xcor_good.gif" attr="" comment="Form example with good gear file" date="1403003373" name="anemone_clu_p0p1_Xcor_good.gif" path="anemone_clu_p0p1_Xcor_good.gif" size="20742" user="KennyWraight" version="1"
META FILEATTACHMENT attachment="anemone_clu_DUT20p0_Xcor_good.gif" attr="" comment="Form example with good gear file" date="1403003373" name="anemone_clu_DUT20p0_Xcor_good.gif" path="anemone_clu_DUT20p0_Xcor_good.gif" size="13237" user="KennyWraight" version="1"
META FILEATTACHMENT attachment="anemone_clu_DUT20_bad.gif" attr="" comment="Form example with bad gear file" date="1403003407" name="anemone_clu_DUT20_bad.gif" path="anemone_clu_DUT20_bad.gif" size="11000" user="KennyWraight" version="1"
META FILEATTACHMENT attachment="anemone_clu_p0p1_XcorShiftProj_good.gif" attr="" comment="Form example with bad gear file" date="1403003750" name="anemone_clu_p0p1_XcorShiftProj_good.gif" path="anemone_clu_p0p1_XcorShiftProj_good.gif" size="4313" user="KennyWraight" version="1"
META FILEATTACHMENT attachment="anemone_clu_p0p1_XcorShift_good.gif" attr="" comment="Form example with bad gear file" date="1403003750" name="anemone_clu_p0p1_XcorShift_good.gif" path="anemone_clu_p0p1_XcorShift_good.gif" size="13881" user="KennyWraight" version="1"

Revision 82014-06-16 - KennyWraight

Line: 1 to 1
 
META TOPICPARENT name="EUTelescopeAndIBLTestbeam"

Customising IBL Reconstruction

Line: 70 to 70
 so the entries are usually 1,0,-1 for 90 degree rotations.
Changed:
<
<

steering file layout

>
>

steering files

Common Features

Steering files for each run or set of runs are produced on execution from the template files (found in the steering-templates directory). The general templates become specific when the variables in specified in the config files are substituted into the templates. To check the of specific executions are correct use the jobsub -dry-run option on execution. This will copy the steering files used for each run to the current directory.

  The steering files are also written in xml. There is a file for each stage of the reconstruction. The program which is being steered is called Marlin and it is executed by jobsub. Information from the config file, i.e. variable values, can be referred to in the steering file using @VARIABLENAME@ syntax.

Each steering file has several sections in common. To begin there is an execute section where the list of all processes to be run is defined. There is also a global section where the global (e.g. gear file location) from the config file are passed. The rest of the file is taken up with setting the variables of each processor defined in the execute section. The task specific parameters (e.g. FiringFrequency) are passed to the processors in these sections.

Added:
>
>
Converter Uses EUDAq software to read output of DUT to into EUTelescope framework.

EUDaq processor is used to recorde testbeam data in organised fashion, however this organisation is not standard, rather it is specific to categories of device (e.g. Atlas Pixel, "APIX"). Converters can be found in $EUTELESCOPE/external/eudaq/tags/v01-00-00/main/src/ (NB slight differences in directory structure between versions). For current (!) testbeam data APIX-CT-ConverterPlugin.cc is the converter used. Any manipulation of data as it enters the EUTelescope software framework should take place here. As EUDAq is separate from EUTelescope, i.e. the converter is not an EUTelescope processor, you cannot read in variables through the variables through the steering file.

Clustering Manipulating DUT pixels.

Clustering collects pixels into clusters according to some constraints (such as minimum and maximum cluster size). It also contains a correlation processor which can be used to check the orientation of the DUTs with respect to one another.

steering files comments

(Integrate this section later.)
 A few variables are used several times throughout steering files and so can be defined for ease in the config file e.g. residuals.

Tips

Line: 86 to 104
  Notes (integrate in text fully later)
Added:
>
>
Check
 DafFitter output histograms can be used to diagnose problems using the resolution plots, which should be single peaked usually with some offset. Any offset in resolution is automatically used by the code for fitting purposes and should not be manually applied. After pre-alignment residuals should not shift more than the pitch size.

Filter data in later version of the reconstruction software (i.e. >= v01-17-05) cleans up hot pixels/clusters before hitmaker steps which should aid pre-alignment.

Revision 72014-06-13 - KennyWraight

Line: 1 to 1
 
META TOPICPARENT name="EUTelescopeAndIBLTestbeam"

Customising IBL Reconstruction

Line: 76 to 76
  Each steering file has several sections in common. To begin there is an execute section where the list of all processes to be run is defined. There is also a global section where the global (e.g. gear file location) from the config file are passed. The rest of the file is taken up with setting the variables of each processor defined in the execute section. The task specific parameters (e.g. FiringFrequency) are passed to the processors in these sections.
Added:
>
>
A few variables are used several times throughout steering files and so can be defined for ease in the config file e.g. residuals.

Tips

In general, make sure any definition of the number of planes is correct: #plane = #mimosas + #DUTs.

In the steering files where number of planes is defined (e.g. in DafFitter processor in align-tmp, fitter-tmp) by TelescopePlanes, the set-up can be defined in two ways. Firstly all planes (mimosas & DUTs) can be defined in one ordered vector (gear file is not used for z-ordering here). This uses all planes for the fitting. Alternatively, only mimosa planes can be defined and DUTs can be referenced in the DutPlanes. If this is done the FitDuts parameter mus be set true. Either way the RequireNTelPlanes parameter should be appropriately set (e.g. to 6 if the mimosa planes alone are used).

Notes (integrate in text fully later)

DafFitter output histograms can be used to diagnose problems using the resolution plots, which should be single peaked usually with some offset. Any offset in resolution is automatically used by the code for fitting purposes and should not be manually applied. After pre-alignment residuals should not shift more than the pitch size.

Filter data in later version of the reconstruction software (i.e. >= v01-17-05) cleans up hot pixels/clusters before hitmaker steps which should aid pre-alignment.

 

EUTelescope analysis

Revision 62014-01-09 - KennyWraight

Line: 1 to 1
 
META TOPICPARENT name="EUTelescopeAndIBLTestbeam"

Customising IBL Reconstruction

Line: 81 to 81
  The source code of the EUTelescope analysis can be found in the ../vXX-YY-ZZ/Eutelescope/vxx-yy-zz/ directory. For example the converters are found in ../external/eudaq/tags/v01-00-00/main/src/ .
Changed:
<
<
When these are edited, the code must be recompiled in the ../external/eudaq/tags/v01-00-00/main directory (using make ) and in the ../vXX-YY-ZZ/Eutelescope/vxx-yy-zz/build/ directory (using make again). This way all the relevant executables are updated.
>
>
When these are edited, the code must be recompiled in the ../external/eudaq/tags/v01-00-00/main directory (using make ) and in the ../vXX-YY-ZZ/Eutelescope/vxx-yy-zz/build/ directory (using make install , NB the extra.potion this time).This way all the relevant executables are updated.
 

-- KennyWraight - 2013-06-11 \ No newline at end of file

Revision 52014-01-06 - KennyWraight

Line: 1 to 1
 
META TOPICPARENT name="EUTelescopeAndIBLTestbeam"

Customising IBL Reconstruction

Line: 76 to 76
  Each steering file has several sections in common. To begin there is an execute section where the list of all processes to be run is defined. There is also a global section where the global (e.g. gear file location) from the config file are passed. The rest of the file is taken up with setting the variables of each processor defined in the execute section. The task specific parameters (e.g. FiringFrequency) are passed to the processors in these sections.
Added:
>
>

EUTelescope analysis

The source code of the EUTelescope analysis can be found in the ../vXX-YY-ZZ/Eutelescope/vxx-yy-zz/ directory. For example the converters are found in ../external/eudaq/tags/v01-00-00/main/src/ .

When these are edited, the code must be recompiled in the ../external/eudaq/tags/v01-00-00/main directory (using make ) and in the ../vXX-YY-ZZ/Eutelescope/vxx-yy-zz/build/ directory (using make again). This way all the relevant executables are updated.

 -- KennyWraight - 2013-06-11

Revision 42013-09-26 - KennyWraight

Line: 1 to 1
 
META TOPICPARENT name="EUTelescopeAndIBLTestbeam"

Customising IBL Reconstruction

Line: 53 to 53
 The gear file is written in xml. It lists the geometry and orientation of the Mimosas use for track formation and the DUTs. The essential parameters of each include the position, rotation, size and pitch of each detector. gear files should be obtained with raw data as they must correspond to the devices used to record data in the testbeam.
Changed:
<
<
To edit or amend a file be sure to set the parameters according to the following conventsions:
>
>
To edit or amend a file be sure to set the parameters according to the following conventions:
 
Line: 61 to 61
 
Added:
>
>
The rotataion matrix has the form so the entries are usually 1,0,-1 for 90 degree rotations.
 

steering file layout

Revision 32013-09-26 - KennyWraight

Line: 1 to 1
 
META TOPICPARENT name="EUTelescopeAndIBLTestbeam"

Customising IBL Reconstruction

Line: 21 to 21
  @GeometryPath@/@GearFile@"
Changed:
<
<
where both variables GeometryPath and GearFile are set in config.h:
>
>
where both variables GeometryPath and GearFile are set in config.cfg:
  # Geometry file
Line: 53 to 53
 The gear file is written in xml. It lists the geometry and orientation of the Mimosas use for track formation and the DUTs. The essential parameters of each include the position, rotation, size and pitch of each detector. gear files should be obtained with raw data as they must correspond to the devices used to record data in the testbeam.
Added:
>
>
To edit or amend a file be sure to set the parameters according to the following conventsions:

 

steering file layout

The steering files are also written in xml. There is a file for each stage of the reconstruction. The program which is being steered is called Marlin and it is executed by jobsub. Information from the config file, i.e. variable values, can be referred to in the steering file using @VARIABLENAME@ syntax.

Revision 22013-06-12 - KennyWraight

Line: 1 to 1
 
META TOPICPARENT name="EUTelescopeAndIBLTestbeam"

Customising IBL Reconstruction

Notes on customising reconstruction process for IBL testbeam data using EUTelescope software. NB this is a perilous process and, so far, not perfected.

Added:
>
>

Required files

 To make a custom reconstruction you will need appropriate gear, config, and steering files. If possible, ask the author or someone who might know where to get these files as it is much easier to edit working code than to invent your own.
Changed:
<
<
The gear file defines the positions and orientation of the devices present: Mimosas and DUTs. The config file sets the essential variable values which the reconstruction code will need to find and write files and run the code successfully. The main part of the of the reconstruction comes in calling the appropriate sub-routines via the steering files.
>
>
The gear file defines the positions and orientation of the devices present: Mimosas and devices under test (DUTs). The config file sets the essential variable values which the reconstruction code will need to find and write files and run the code successfully. The main part of the of the reconstruction comes in calling the appropriate sub-routines via the steering files.
 
Changed:
<
<
If you editiing the steering fiels you must be careful that the correct referencing is done between the values set in the config file and the values used in the steering files. Problems are often generated when directories or values defined in the config file are not properly referenced or simply hard-coded in the steering file. The code will fail if files, directories or values mentioned in the steering files do not exist. Always check the error messages if running jobsub fails.
>
>
If you editing the steering files you must be careful that the correct referencing is done between the values set in the config file and the values used in the steering files. Problems are often generated when directories or values defined in the config file are not properly referenced or simply hard-coded in the steering file. The code will fail if files, directories or values mentioned in the steering files do not exist. Always check the error messages if running jobsub fails.
  Be sure to replace hard-coded references with values set in config file, e.g.:
Line: 30 to 32
 
Added:
>
>

config file layout

The config file is written in python. It lists all the variables set by the user for the reconstruction. Essential parameters are things like the file structure the reconstruction will run in, e.g. the directories to find input files and write output files to, and the location of the gear and steering files. These files must exist (i.e. be set correctly) or the program will not run.

In addition to essential parameters other variables can be set which will overwrite the defaults in the code. Alternately these could be set on the command line using the -o VARIABLE=VALUE option, but this is cumbersome for several variables or longwinded values. The variables for each steering file are listed in the config file after the global values have been set, each set is introduced in by the task name in square brackets. Here is an example of the converter section:

# Section for the converter step [converter]

# Hot Pixel detection FiringFrequency = 0.1

Here, the FiringFrequency variable used by the conveter steering file is set. Other parameters tasks are similarly defined.

gear file layout

The gear file is written in xml. It lists the geometry and orientation of the Mimosas use for track formation and the DUTs. The essential parameters of each include the position, rotation, size and pitch of each detector. gear files should be obtained with raw data as they must correspond to the devices used to record data in the testbeam.

steering file layout

The steering files are also written in xml. There is a file for each stage of the reconstruction. The program which is being steered is called Marlin and it is executed by jobsub. Information from the config file, i.e. variable values, can be referred to in the steering file using @VARIABLENAME@ syntax.

Each steering file has several sections in common. To begin there is an execute section where the list of all processes to be run is defined. There is also a global section where the global (e.g. gear file location) from the config file are passed. The rest of the file is taken up with setting the variables of each processor defined in the execute section. The task specific parameters (e.g. FiringFrequency) are passed to the processors in these sections.

  -- KennyWraight - 2013-06-11

Revision 12013-06-11 - KennyWraight

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="EUTelescopeAndIBLTestbeam"

Customising IBL Reconstruction

Notes on customising reconstruction process for IBL testbeam data using EUTelescope software. NB this is a perilous process and, so far, not perfected.

To make a custom reconstruction you will need appropriate gear, config, and steering files. If possible, ask the author or someone who might know where to get these files as it is much easier to edit working code than to invent your own. The gear file defines the positions and orientation of the devices present: Mimosas and DUTs. The config file sets the essential variable values which the reconstruction code will need to find and write files and run the code successfully. The main part of the of the reconstruction comes in calling the appropriate sub-routines via the steering files.

If you editiing the steering fiels you must be careful that the correct referencing is done between the values set in the config file and the values used in the steering files. Problems are often generated when directories or values defined in the config file are not properly referenced or simply hard-coded in the steering file. The code will fail if files, directories or values mentioned in the steering files do not exist. Always check the error messages if running jobsub fails.

Be sure to replace hard-coded references with values set in config file, e.g.:

someplacewithgeomfile

change to

@GeometryPath@/@GearFile@"

where both variables GeometryPath and GearFile are set in config.h:

# Geometry file GearFile = my_gear_file.xml

# Path to the geometry file GeometryPath = %(BasePath)s

-- KennyWraight - 2013-06-11

 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback