JetFlavourTagging < LinearCollider

LinearCollider Web>JetFlavourTagging (2017-03-22, DanProtopopescu) (raw view)

---++ Jet Flavour Tagging Howto with LCFIVertex

This is a detailed record on how the Marlin framework and included LCFIVertex packages are used for jet flavour tagging. b-jet flavour tagging is part of our analysis of the feasibility of the ZZ fusion channel with CLIC ILD at 1.4 !TeV.

Note: this wiki refers to the LCFIVertex package, that can be found now on !GitHub [[https://github.com/iLCSoft/LCFIVertex][here]].

%TOC%

---+++ Jet Finder and Truth Tagging 

The LCFI package consists of a topological vertex finder ZVTOP, which reconstructs secondary interactions, and a multivariate classifier which combines several jet-related variables to tag bottom, charm, and light quark jets (see [[%ATTACHURLPATH%/LCFI_Flow_Diagram.pdf][diagram]]).

Our steering file will contain the jet finder, flavour tagging and LCFI processors, and we will write new =slcio= files containing the added collections: 
<verbatim>
  <group name="JetFinders"/>
  <group name="MyTrueAngularJetFlavourProcessorCollection"/>
  <processor name="IPRPCutProcessor"/>
  <processor name="MyPerEventIPFitterProcessor"/>
  <processor name="ZVRESRPCutProcessor"/>
  <processor name="MyZVTOP_ZVRES"/>
  <processor name="FTRPCutProcessor"/>
  <processor name="MyFlavourTagInputsProcessor"/>
  <processor name="MyLCIOOutputProcessor"/>  
</verbatim>

The *JetFinder* processor reconstructs 2 and 4 jets events from the input collection (!LooseSelectedPandoraPFANewPFOs was used). For the reconstructed 4 jets, *MyTrueAngularJetFlavourProcessor* determines MC Jet Flavour by angular matching of heavy quarks to jets, and also determines hadronic and partonic charge of the jet. The *LCIOOutput* processor creates new =slcio= files containing the new collections added by the above processors. 

The LCFI processors have the following functions:

   * !IPRPCut		        - selects Reconstructed Particles based on track parameters, number of hits etc.
   * !MyPerEventIPFitter	- determines IP position and error from the tracks in an event by simple fitting 
   * !ZVRESRPCut            - applies cuts on the d0 and z0 values of the track
   * !MyZVTOP_ZVRES 	- topological vertex finder 
   * !FTRPCut                   - flavour tagging reconstructed particle cuts (on d0, z0 and P<sub>T</sub>)
   * !MyFlavourTagInputs - from vertices and tracks calculates discriminating variables for the neural net

Table of [[%ATTACHURLPATH%/LCFI_Flow_Diagram.pdf][input and output collections]] for our setup (one can choose other names, of course):

   |  #  |          *Processor*           |  *Type*    |              *Input Collection name*            |         *Output Collection name*       |
   | 1 |  !JetFinder                       |   !SatoruJetFinder  | !LooseSelectedPandoraPFANewPFOs   | Durham_4Jets   |
   | 2 |  !MyTrueAngularJetFlavour  |   !TrueAngularJetFlavour  | !MCParticlesSkimmed, Durham_4Jets | !TrueJetFlavour_4Jets    |
   | 3 |       !IPRPCut              |  !RPCut  | !LooseSelectedPandoraPFANewPFOs  | !IPFitSelectedParticles  |
   | 4 |    !MyPerEventIPFitter      |   !PerEventIPFitter    | !IPFitSelectedParticles      | !IPVertex  |
   | 5 |   !ZVRESRPCut            |  !RPCut    | !RecoMCTruthLink, Durham_4Jets  | !ZVRESSelectedJets   |
   | 6 |     !MyZVTOP_ZVRES      |  !ZVTOP_ZVRES    | !IPVertex, !ZVRESSelectedJets | !ZVRESDecayChains,  !ZVRESDecayChainRPTracks, !ZVRESSelectedJets  |
   | 7 |      !FTRPCut               |  !RPCut   |  !RecoMCTruthLink,  !ZVRESDecayChains  | !FTSelectedJets  |
   | 8 |   !MyFlavourTagInputs     |  !FlavourTagInputs  | !ZVRESDecayChains,  !FTSelectedJets | !FlavourTagInputs   |

Our input =slcio= files contain the collections: !LooseSelectedPandoraPFANewPFOs, !MCParticlesSkimmed, !PandoraPFANewClusters, !PandoraPFANewPFOs, !PandoraPFANewReclusterMonitoring, !ProngVertices, !RecoMCTruthLink, !SelectedLDCTracks, !SelectedPandoraPFANewPFOs, !TightSelectedPandoraPFANewPFOs  and !V0Vertices.
  
The processors listed above could be run in sequence, or split in several steps, invoking a *LCIOOutput* processor to write intermediate =slcio= outputs at every step. Here's [[%ATTACHURLPATH%/runLCFI.txt][a script]] for that, where the intermediate xml files are slight modifications of the files provided in =LCFIVertex/steering= examples. We found that the most [[%ATTACHURLPATH%/Timing-ScreenShot.png][time-consuming]] processor is *ZVTOP_ZVRES* with more than 10 s/event. 

We found it easier to [[TorqueMarlinJobs][run processors 1 to 6 on batches of 10 input files]], and save the outputs as =/afs/phas.gla.ac.uk/data/ilc/datasets01/1.4tev/ZVRES_out/runX_out.slcio=. Then we ran processors 7 and 8 on these files to produce a new set =ftiX_out.slcio=. We used half of these files to train the neural net, but then again was easier to run flavour tagging (see below) on the individual =ftiX_out.slcio= files. We used the other half of the files files for the purity vs. efficiency plots (see below). These =slcio= output files were stored in =/afs/phas.gla.ac.uk/data/ilc/datasets01/1.4tev/ZVRES_out/= as well.

*Troubleshooting:* The =b3_D0CutValue= parameter of the !IPRPCutProcessor was set to =5O= instead of =50=, and was causing a crash. For the !ZVRESRPCut processor, =h1_MCPIDEnable= had to be set to =false=. See also [[http://forum.linearcollider.org/index.php?t=tree&goto=1547&rid=0&S=6e268f6996033e2e63e8595ffdcf73e9&srch=FTSelected#msg_1547][this post]]. 

---+++ Neural Network Training

In the previous step, we have extracted the discriminating parameters and truth-tagged the jets. The =slcio= files created are stored in:
<verbatim>
/afs/phas.gla.ac.uk/data/ilc/datasets01/1.4tev/ZVRES_out/
</verbatim>
These files contain the collections *Durham_4Jets*, *FlavourTagInputs* and *TrueJetFlavour_4Jets*, which we will use now to train our neural nets. We use a customised version of the  =NeuralNetTrainer= code included in the LCFI package. Separate nets were trained for 1, 2, or 3+ vertices to identify b-jets, c-jets, and c-jets with b background. Our steering file contains only:
<verbatim>
  <processor name="MyNeuralNetTrainer" type="GlasgowNeuralNetTrainer"/>
</verbatim>
This processor is a slightly modified version of the NeuralNetTrainer included in the LCFI package, where the polar angle cut was introduced as a steering parameter:
<verbatim>
// Theta cut parameters: _jetThetaAngleCut < theta < (180 - _jetThetaAngleCut)
        registerProcessorParameter( "JetThetaAngleCut" ,
                                    "Cut on the jets theta angle"  ,
                                    _jetThetaAngleCut,
                                    (float)15.);
</verbatim>
such that we can pass a cut angle different of the  30º default:
<verbatim>
<!-- Jet theta angle cut (only the Glasgow NN trainer has this option) -->
                <parameter name="JetThetaAngleCut" type="float"> 24. </parameter>
</verbatim>
The neural nets are saved as XML files in =gnets/= and will be used for flavour tagging (next step). No =slcio= output is written at this time. These neural networks can be downloaded from here:  [[%ATTACHURLPATH%/gnets.tgz][gnets.tgz]]


---+++ Flavour Tagging

Now we are ready to employ the *FlavourTag* processor, which will do flavour tagging using the neural nets trained in the previous step. The input =slcio= files contain the !FlavourTagInputs and !FTSelectedJets (or !Durham_4Jets, not sure if there's a difference at this level) collections.
<verbatim>
  <processor name="MyFlavourTag"/>
  <processor name="MyLCIOOutputProcessor"/>  
</verbatim>
The output =slcio= will contain a new collection !FlavourTag (or !FlavourTagGla in our customised configuration).


---+++ Purity and Efficiency Studies

To determine the optimal cut for our b-tagging, a purity vs. efficiency study needs to be done. We use the =MakePurityVsEfficiencyRootPlot.C= macro provided by the !LCFIVertex package.  
First, we have to run:
<verbatim>
  <processor name="MyAIDAProcessor"/>
  <processor name="MyPlot"/>
  <processor name="MyLCFIAIDAPlotProcessor"/>
</verbatim>
We had to provide !MyPlot with the actual names of our collections:
<verbatim>
<parameter name="TrueJetFlavourCollection" type="string">TrueJetFlavour_4Jets </parameter>
<!--In fti-steer.xml this parameter is called "FlavourTagCollection", without the 's' -->  
<parameter name="FlavourTagCollections" type="string">FlavourTagGla </parameter>
</verbatim> 
Note that LCFI must be compiled with ROOT if one wants *.root* output from !PlotProcessor (instead of .txt). For this, we added  
<verbatim>
FIND_PACKAGE( ROOT REQUIRED )
FOREACH( pkg ROOT ) 
 IF( ${pkg}_FOUND )
  INCLUDE_DIRECTORIES( ${${pkg}_INCLUDE_DIRS} )
  ADD_DEFINITIONS( ${${pkg}_DEFINITIONS} )
 ENDIF()
ENDFOREACH()
</verbatim>
to the =LCFIVertex/CMakeLists.txt= file, sourced the root environment, then ran =cmake= and =make install=. 

Once the AIDA Plots processors are run via =Marlin=, a text file =PurityEfficiencyOutput.txt= and a RAIDA root file are produced. We customised the =MakePurityVsEfficiencyRootPlot.C= macro and ran it to use the RAIDA file as input to produce the purity vs. efficiency plots:
<verbatim>
root -l MakePurityVsEfficiencyRootPlotGla.C
</verbatim>
Here's a plot of our flavour tagging purity vs. efficiencies (using cca. 25k events): <br />
<img src="%ATTACHURLPATH%/Screen_Shot_2013-07-16_at_13.40.40.png" alt="Screen_Shot_2013-07-16_at_13.40.40.png" width='633' height='452' />

The cut values and corresponding purity and efficiencies are tabulated in the file =PurityEfficiencyOutput.txt=. We used another Root macro to plot these values and determine the optimum _b-cut_. 

For b-tagging, we've compared the plot produced by =MakePurityVsEfficiencyRootPlotGla.C= and graphs drawn with the data tabulated in =PurityEfficiencyOutput.txt=, for 1, 2, 3 (corresponding to distinct neural networks) or any number of vertices (which we don't know yet how to interpret):<br />
<img src="%ATTACHURLPATH%/PEcomparison.png" alt="PEcomparison.png" width='696' height='472' />

Here's [[%ATTACHURLPATH%/xtractPEv.txt][the script]] used to extract the numbers from  =PurityEfficiencyOutput.txt=


---+++ Adding background

Up to now all that input slcio files contained was signal, so the 'background' in the plots above is not really background. To add background, we've [[DatasetAccess][downloaded]] 1.4!TeV =eeqq= files to:
<verbatim>
/afs/phas.gla.ac.uk/data/ilc/datasets01/1.4tev/qq/ILD/DST/00002091/000/
</verbatim>
We had to run processors 1-7 on these files to identify jets and calculate the discriminating parameters. For this, we used slightly modified versions of the scripts, which can be found [[%ATTACHURLPATH%/bkg_scripts.tgz][here]]. The resulting outputs, containing approximately 25k events, are stored in =/afs/phas.gla.ac.uk/data/ilc/datasets01/1.4tev/ZVRES_out/= as well. 

<img src="%ATTACHURLPATH%/PEcomparisonBkg.png" alt="PEcomparisonBkg.png" width='696' height='472' />


---+++ Training the neural nets with eeqq background added

We have trained another set of neural nets but adding this time _eeqq_ background to the input. The resulting neural nets (25k signal + 25k background) are here: [[%ATTACHURLPATH%/bnets.tgz][bnets.tgz]].

<img src="%ATTACHURLPATH%/PEcomparisonBNN.png" alt="PEcomparisonBNN.png" width='696' height='472' />

---+++ Using an alternate Jet Finder

The whole sequence can be redone with and alternate jet finder. We used '!FastJet' and we replaced the Durham algorithm with KT. For this, we changed =Durham_4Jets= to =Kt_4Jets= and used
<verbatim>
<processor name="MyFastJetProcessor4" type="FastJetProcessor">
       <parameter name="algorithm" type="StringVec"> kt_algorithm 1.0 </parameter>
       <parameter name="clusteringMode" type="StringVec"> ExclusiveNJets 4 </parameter>
       <parameter name="jetOut" type="string" lcioOutType="ReconstructedParticle"> Kt_4Jets </parameter>
       <parameter name="recParticleIn" type="string" lcioInType="ReconstructedParticle"> LooseSelectedPandoraPFANewPFOs </parameter>
       <parameter name="recombinationScheme" type="string">E_scheme </parameter>
       <parameter name="recParticleOut" type="string" lcioOutType="ReconstructedParticle"> LooseSelectedPandoraPFOsInKt_4Jets </parameter>
</processor>
</verbatim>
The relevant scripts and XML templates are [[%ATTACHURL%/KTscripts.tgz][here]]. The outputs are stored in =/afs/phas.gla.ac.uk/data/ilc/datasets01/1.4tev/ZVRES_out/= and have the prefix =kt= added the file name.

---+++ Using 4-Jet background

This is done following the same procedure, but using =qqll= 1.4TeV, CLIC_ILD DST-s (2645) and the script =subqqll.sh= and its dependencies, =zvresQQllPBS.sh=, and =train_qnets.xml=. The relevant files are:

   * [[%ATTACHURL%/qqll_scripts.tgz][qqll_scripts.tgz]]: Scripts for signal + qqll background, KT algorithm
   * [[%ATTACHURL%/qnets.tgz][qnets.tgz]]: Neural nets for signal + qqll background, KT jet finder algorithm

The outputs are stored in the usual directory: =/afs/phas.gla.ac.uk/data/ilc/datasets01/1.4tev/ZVRES_out/*qnets*=. Purity vs. efficiency plot using qnets (1 vertex NN is gibberish): <br/>
     <img src="%ATTACHURLPATH%/PEcomparisonQQll.png" alt="PEcomparisonQQll.png" width='696' height='472' />


---++ Useful Links

   * !LCFIVertex Tutorial: http://www-pnp.physics.ox.ac.uk/~hillert/VP/LCFIVertex-v00-01/Tutorials.html
   * Trained Neural Nets: https://svnsrv.desy.de/viewvc/tagnet/LCFI_MokkaBasedNets/

Attachments

Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
png	BRTotalUncertBands_lm.png	r2 r1	manage	111.7 K	2013-07-01 - 14:41	DanProtopopescu	Higgs branching ratios (from A. Denner et al., EPJ C71, p.1753)
tgz	KTscripts.tgz	r2 r1	manage	13.5 K	2013-08-12 - 09:09	DanProtopopescu	Scripts and templates modified to use FastJet/KT algorithm
pdf	LCFI_Flow_Diagram.pdf	r1	manage	87.7 K	2013-07-01 - 16:10	DanProtopopescu	LCFI processors - flow diagram
pdf	LCFI_Hillert.pdf	r1	manage	787.8 K	2013-07-24 - 10:21	DanProtopopescu	LCFI Vertex package talk by S. Hillert
pdf	LCFI_ecfa_krakow.pdf	r1	manage	296.4 K	2013-07-24 - 10:22	DanProtopopescu	Flavour tagging at the linear collider - talk by M. Wing
c	MakePurityVsEfficiencyRootPlotGla.C	r1	manage	2.5 K	2013-07-16 - 14:06	DanProtopopescu	Customised macro to draw purity vs. efficiency plots
png	PEcomparison.png	r2 r1	manage	12.0 K	2013-07-24 - 11:09	DanProtopopescu	B-tagging Purity vs. Efficiency comparison
png	PEcomparisonBNN.png	r1	manage	14.5 K	2013-08-06 - 15:58	DanProtopopescu	B-tagging Purity vs. Efficiency comparison (signal + background, see text)
png	PEcomparisonBkg.png	r2 r1	manage	17.1 K	2013-07-31 - 13:57	DanProtopopescu	B-tagging Purity vs. Efficiency comparison (25k eeqq background)
png	PEcomparisonQQll.png	r1	manage	12.6 K	2013-09-13 - 15:04	DanProtopopescu	Purity vs. efficiency plot using qnets (1 vertex NN is gibberish)
png	Screen_Shot_2013-07-16_at_13.40.40.png	r1	manage	21.5 K	2013-07-16 - 13:02	DanProtopopescu	Flavour tagging purity vs. efficiency plots (using cca. 22k events)
png	Timing-ScreenShot.png	r1	manage	70.9 K	2013-07-01 - 16:03	DanProtopopescu	Screen shot: time used by Marlin processors
pdf	Vertexing_Howto.pdf	r1	manage	743.5 K	2013-07-01 - 16:52	DanProtopopescu	Vertexing HowTo talk by B. Jeffery
tgz	bkg_scripts.tgz	r1	manage	8.5 K	2013-08-06 - 15:46	DanProtopopescu	Scripts for running processors 1-7 on eeqq background files
tgz	bnets.tgz	r3 r2 r1	manage	18.2 K	2013-08-06 - 16:04	DanProtopopescu	Neural nets (Satoru Jet Finder, trained with signal + eeqq background)
tgz	gnets.tgz	r1	manage	11.7 K	2013-08-06 - 14:02	DanProtopopescu	Neural nets (Satoru Jet Finder, trained with signal only)
xml	jet_truth_tag-steer.xml	r1	manage	82.1 K	2013-07-01 - 17:20	DanProtopopescu	Steering file 1: jet finder and truth flavour tagging
tgz	qnets.tgz	r1	manage	9.4 K	2013-09-13 - 15:00	DanProtopopescu	Neural nets for signal + qqll background, KT jet finder algorithm
tgz	qqll_scripts.tgz	r1	manage	14.4 K	2013-09-13 - 15:00	DanProtopopescu	Scripts for signal + qqll background, KT algorithm
txt	runLCFI.txt	r1	manage	1.1 K	2013-07-04 - 16:15	DanProtopopescu	Script to run LCFI sequence of processors
txt	xtractPEv.txt	r1	manage	1.6 K	2013-07-29 - 15:46	DanProtopopescu	Script used to extract the numbers from PurityEfficiencyOutput.txt
sh	zvresRun.sh	r1	manage	1.7 K	2013-07-16 - 14:05	DanProtopopescu	Script to run processors 1-7, see text

Topic revision: r30 - 2017-03-22 - DanProtopopescu

LinearCollider