Difference: AtlasDataAnalysis (143 vs. 144)

Revision 1442012-03-07 - AdrianBuzatu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"
Changed:
<
<
<-- p { margin-bottom: 0.21cm; }h1 { margin-bottom: 0.21cm; }h1.western { font-family: "Liberation Serif",serif; }h1.cjk { font-family: "DejaVu Sans"; }h1.ctl { font-family: "DejaVu Sans"; }h2 { margin-bottom: 0.21cm; }h4 { margin-bottom: 0.21cm; }h5 { margin-bottom: 0.21cm; }h3 { margin-bottom: 0.21cm; }h3.western { font-family: "Liberation Serif",serif; }h3.cjk { font-family: "DejaVu Sans"; }h3.ctl { font-family: "DejaVu Sans"; }pre.cjk { font-family: "DejaVu Sans",monospace; }a:link { } -->
>
>		Est_12_120.eps
<-- p { margin-bottom: 0.21cm; }h1 { margin-bottom: 0.21cm; }h1.western { font-family: "Liberation Serif",serif; }h1.cjk { font-family: "DejaVu Sans"; }h1.ctl { font-family: "DejaVu Sans"; }h2 { margin-bottom: 0.21cm; }h4 { margin-bottom: 0.21cm; }h5 { margin-bottom: 0.21cm; }h3 { margin-bottom: 0.21cm; }h3.western { font-family: "Liberation Serif",serif; }h3.cjk { font-family: "DejaVu Sans"; }h3.ctl { font-family: "DejaVu Sans"; }pre.cjk { font-family: "DejaVu Sans",monospace; }a:link { } -->
 
Line: 122 to 122
 

Overview of the process

Changed:
<
<
>
>
 
Line: 209 to 209
 

  • You then need to set up your environment ready for the validation. This is done with the setup_glantp.sh script, which is available within the NNFitter package. (Yes, I know - another case of getting the code before getting the code...) You run the script (which is also used for debugging the code) with 

    source setup_glantp.sh

  • Make a directory to hold the code itself:

    mkdir GlaNtpPackage

Changed:
<
<

  • GlaNtpScript.sh not only checks out and compiles the code, it also then goes and validates it. setup_glantp.sh sets up the environment variables so the validation data can be found.

>
>

  • GlaNtpScript.sh not only checks out and compiles the code, it also then goes and validates it. setup_glantp.sh sets up the environment variables so the validation data can be found.

 

  • You now run the script in the parent directory of GlaNtpPackage, specifying whether you want a specific tag (e.g. 00-00-10), or just from the head of the trunk (h) so you're more free to play around with it. It's always a good idea to check out a specific tag, so that whatever you do to the head, you can still run over a valid release.

    ./GlaNtpScript.sh SVN 00-00-10
Changed:
<
<

  • This will check out everything, and run a few simple validations - the final output should look like this (i.e. don't be worried that not everything seems to have passed validation!):

>
>

  • This will check out everything, and run a few simple validations - the final output should look like this (i.e. don't be worried that not everything seems to have passed validation!):

 
HwwFlatFitATLAS Validation succeeded
Done with core tests
Line: 247 to 247
 Result of FlatAscii validation: OK Result of FlatAscii_global validation: OK Result of FlatTRntp validation: OK
Changed:
<
<

>
>
 

Variables used by the GlaNtp package

Line: 268 to 268
 
ListParameter   EvInfoTree:1  1 NN_BJetWeight_Jet1:NN_BJetWeight_Jet1/NN_BJetWeight_Jet1
Deleted:
<
<		If you want a parameter to be found in the output, best to list it here....
 
Added:
>
>		If you want a parameter to be found in the output, best to list it here....
 

Calculating my_integral (The Magic Formula)

Changed:
<
<		1. Check the critical formula: The most important forumla is the first thing to check:

weight*= GetSF()*GetXsect()*GetBrFrac()*GetFilterEff()*GetLumiForType()/GetNGenForType(); my_brFrac my_filterEff my_xSect my_lumiForType my_nGenForType

>
>		1. Check the critical formula: The most important forumla is the first thing to check:
 
Added:
>
>		weight*= GetSF()*GetXsect()*GetBrFrac()*GetFilterEff()*GetLumiForType()/GetNGenForType(); my_brFrac my_filterEff my_xSect my_lumiForType my_nGenForType
  You need a scale factor: my_sf?
Added:
>
>		2. Check the stuff that FlatReader uses. This is documented in GlaNtp/NtpAna/test/VariableTreeToNtp.txt. Variable Tree to Ntp is the one that maps logical values to their physical branch/leaf. Anything prefaced with FlatTupleVar needs to be specified or is useful to specify.
 
Changed:
<
<		 2. Check the stuff that FlatReader uses. This is documented in GlaNtp/NtpAna/test/VariableTreeToNtp.txt. Variable Tree to Ntp is the one that maps logical values to their physical branch/leaf. Anything prefaced with FlatTupleVar needs to be specified or is useful to specify.

Values are divided in to those that can change on each event (kept in the "ev" tree) and those that are the same for a file (kept in the "global" tree). As you know you can set the tree names. You really should create a global tree for the global file values now. We have procrastinated on this a long time.

>
>		Values are divided in to those that can change on each event (kept in the "ev" tree) and those that are the same for a file (kept in the "global" tree). As you know you can set the tree names. You really should create a global tree for the global file values now. We have procrastinated on this a long time.
 
Changed:
<
<		 There you see # # Values that are required from global # GeneralParameter string 1 FlatTupleVar/BrFrac=globalInfo/BrFrac GeneralParameter string 1 FlatTupleVar/FilterEff=globalInfo/FilterEff GeneralParameter string 1 FlatTupleVar/Fraction=Fraction/Fraction GeneralParameter string 1 FlatTupleVar/Integral=Integral/Integral GeneralParameter string 1 FlatTupleVar/XSect=globalInfo/Xsect
>
>		There you see # # Values that are required from global # GeneralParameter string 1 FlatTupleVar/BrFrac=globalInfo/BrFrac GeneralParameter string 1 FlatTupleVar/FilterEff=globalInfo/FilterEff GeneralParameter string 1 FlatTupleVar/Fraction=Fraction/Fraction GeneralParameter string 1 FlatTupleVar/Integral=Integral/Integral GeneralParameter string 1 FlatTupleVar/XSect=globalInfo/Xsect
 
Changed:
<
<		 # This specifies the name of the leaf for the cutmask and invert word. #Again, these are global values for a file. GeneralParameter string 1 CutMaskString=cutMask GeneralParameter string 1 InvertWordString=invertWord
>
>		# This specifies the name of the leaf for the cutmask and invert word. #Again, these are global values for a file. GeneralParameter string 1 CutMaskString=cutMask GeneralParameter string 1 InvertWordString=invertWord
 
Changed:
<
<		 This confirms that Fraction and Integral are needed. my_fraction my_integral
>
>		This confirms that Fraction and Integral are needed. my_fraction my_integral
 
Changed:
<
<		 Here are the ones that are required for ev: # # Values that are required from ev # GeneralParameter string 1 FlatTupleVar/Channel=evInfo/Channel GeneralParameter string 1 FlatTupleVar/DilMass=evInfo/Mll GeneralParameter string 1 FlatTupleVar/Entry=evInfo/ientry GeneralParameter string 1 FlatTupleVar/Event=evInfo/eventNumber GeneralParameter string 1 FlatTupleVar/Lep1En=evInfo/lep1_E GeneralParameter string 1 FlatTupleVar/Lep2En=evInfo/lep2_E GeneralParameter string 1 FlatTupleVar/LumiForType=evInfo/lumiForType GeneralParameter string 1 FlatTupleVar/MEVal=LRInfo/LRHWW GeneralParameter string 1 FlatTupleVar/NGenForType=evInfo/nGenForType GeneralParameter string 1 FlatTupleVar/Njets=evInfo/Njets GeneralParameter string 1 FlatTupleVar/Rand=evInfo/Rand GeneralParameter string 1 FlatTupleVar/Run=evInfo/runNumber GeneralParameter string 1 FlatTupleVar/Weight=LRInfo/weight GeneralParameter string 1 FlatTupleVar/cutWord=evInfo/cutWord GeneralParameter string 1 FlatTupleVar/lep1_Type=evInfo/lep1_Type GeneralParameter string 1 FlatTupleVar/lep2_Type=evInfo/lep2_Type GeneralParameter string 1 FlatTupleVar/sf=evInfo/sf

Some can be used with the default values that FlatTuple gives: GeneralParameter string 1 FlatTupleVar/Channel=evInfo/Channel GeneralParameter string 1 FlatTupleVar/DilMass=evInfo/Mll GeneralParameter string 1 FlatTupleVar/MEVal=LRInfo/LRHWW GeneralParameter string 1 FlatTupleVar/Rand=evInfo/Rand GeneralParameter string 1 FlatTupleVar/lep1_Type=evInfo/lep1_Type GeneralParameter string 1 FlatTupleVar/lep2_Type=evInfo/lep2_Type

Some are ok to leave if you dont want to use it: there are switches that turn on the use of these GeneralParameter string 1 FlatTupleVar/Lep1En=evInfo/lep1_E GeneralParameter string 1 FlatTupleVar/Lep2En=evInfo/lep2_E GeneralParameter string 1 FlatTupleVar/Weight=LRInfo/weight

>
>		Here are the ones that are required for ev: # # Values that are required from ev # GeneralParameter string 1 FlatTupleVar/Channel=evInfo/Channel GeneralParameter string 1 FlatTupleVar/DilMass=evInfo/Mll GeneralParameter string 1 FlatTupleVar/Entry=evInfo/ientry GeneralParameter string 1 FlatTupleVar/Event=evInfo/eventNumber GeneralParameter string 1 FlatTupleVar/Lep1En=evInfo/lep1_E GeneralParameter string 1 FlatTupleVar/Lep2En=evInfo/lep2_E GeneralParameter string 1 FlatTupleVar/LumiForType=evInfo/lumiForType GeneralParameter string 1 FlatTupleVar/MEVal=LRInfo/LRHWW GeneralParameter string 1 FlatTupleVar/NGenForType=evInfo/nGenForType GeneralParameter string 1 FlatTupleVar/Njets=evInfo/Njets GeneralParameter string 1 FlatTupleVar/Rand=evInfo/Rand GeneralParameter string 1 FlatTupleVar/Run=evInfo/runNumber GeneralParameter string 1 FlatTupleVar/Weight=LRInfo/weight GeneralParameter string 1 FlatTupleVar/cutWord=evInfo/cutWord GeneralParameter string 1 FlatTupleVar/lep1_Type=evInfo/lep1_Type GeneralParameter string 1 FlatTupleVar/lep2_Type=evInfo/lep2_Type GeneralParameter string 1 FlatTupleVar/sf=evInfo/sf
 
Changed:
<
<		 Some are useful for plotting: GeneralParameter string 1 FlatTupleVar/Njets=evInfo/Njets
>
>		Some can be used with the default values that FlatTuple gives: GeneralParameter string 1 FlatTupleVar/Channel=evInfo/Channel GeneralParameter string 1 FlatTupleVar/DilMass=evInfo/Mll GeneralParameter string 1 FlatTupleVar/MEVal=LRInfo/LRHWW GeneralParameter string 1 FlatTupleVar/Rand=evInfo/Rand GeneralParameter string 1 FlatTupleVar/lep1_Type=evInfo/lep1_Type GeneralParameter string 1 FlatTupleVar/lep2_Type=evInfo/lep2_Type
 
Added:
>
>		Some are ok to leave if you dont want to use it: there are switches that turn on the use of these GeneralParameter string 1 FlatTupleVar/Lep1En=evInfo/lep1_E GeneralParameter string 1 FlatTupleVar/Lep2En=evInfo/lep2_E GeneralParameter string 1 FlatTupleVar/Weight=LRInfo/weight
 
Changed:
<
<		 I think you have my_Eventtype as channel my_failEvent as cutword
>
>		Some are useful for plotting: GeneralParameter string 1 FlatTupleVar/Njets=evInfo/Njets
 
Added:
>
>		I think you have my_Eventtype as channel my_failEvent as cutword
 

Variables that must be listed in the event (not the global) tree

Line: 477 to 410
 ColumnParameter SignalList 1 ttH=1 ColumnParameter DataList 1 Data=11
Changed:
<
<		Here you specify once again the numbers assigned to the processes by my_Eventtype (for tt0j it equals zero), and list things as BackgroundList, SignalList or DataList. The number after 'BackgroundList' or 'SignalList' is unique for each process (to preserve the uniqueness of <tag>:<sequence>), but it must be sequential, running from 0 to n-1 (where you have n samples) - apart from for DataList entries (as shown above). It also does not need to correspond to my_Eventtype, however, for completeness' sake within this file I have set it as such. The number at the end of this declaration (tt0j=0 in this case) needs to be sequential - it instructs the net of the order in which to process the samples, so it must go from 0 to n-1 (when you have n samples). It must match up with the numbers provided in atlastth_histlist_flat-v16.txt and AtlasttHRealTitles.txt so that processes and data can be matched to the various individual files.
>
>		Here you specify once again the numbers assigned to the processes by my_Eventtype (for tt0j it equals zero), and list things as BackgroundList, SignalList or DataList. The number after 'BackgroundList' or 'SignalList' is unique for each process (to preserve the uniqueness of <tag>:<sequence>), but it must be sequential, running from 0 to n-1 (where you have n samples) - apart from for DataList entries (as shown above). It also does not need to correspond to my_Eventtype, however, for completeness' sake within this file I have set it as such. The number at the end of this declaration (tt0j=0 in this case) needs to be sequential - it instructs the net of the order in which to process the samples, so it must go from 0 to n-1 (when you have n samples). It must match up with the numbers provided in atlastth_histlist_flat-v16.txt and AtlasttHRealTitles.txt so that processes and data can be matched to the various individual files.
  
ColumnParameter PseudoDataList 0 tt0j=0
Line: 729 to 663
 

teststeerFlatPlotterATLAStthSemileptonic-v16.txt and teststeerFlatReaderATLAStthSemileptonic-v16.txt 

GeneralParameter bool   1 LoadGlobalOnEachEvent=0
Added:
>
>
 Determines if you have a separate global tree or not. If you do not, set this equal to one, and the relevant global values will be read out anew for each event from the event tree.

Important notes about running parts of the code (not a complete run - for debugging, replotting etc)

Changed:
<
<
  1. When you want to only make stacked input plots (e.g. when you've decided you want to change the scale of one plot), you must also redo the templating. Just redoing the stacked inputs on their own does nothing new. The histograms for the stacked inputs are booked during the templating.
>
>
  1. When you want to only make stacked input plots (e.g. when you've decided you want to change the scale of one plot), you must also redo the templating. Just redoing the stacked inputs on their own does nothing new. The histograms for the stacked inputs are booked during the templating.
 

Where the output is stored

Line: 805 to 738
  OnOff : 1 Process : 2.19001e-314 SorB : 0
Changed:
<
<		 These reflect the parameters as set in TMVAsteer.txt (created via genemflat_batch_Complete2_SL5_sh). Note that the number following 'Process' is nonsense (and in later releases of GlaNtp is not present) - that parameter is there in the steering file simply to make it more human-readable. However, the code still tries to read it in, but can only handle doubles - the net result varies from run to run, but can always be safely ignored.
  • >
    >    		 These reflect the parameters as set in TMVAsteer.txt (created via genemflat_batch_Complete2_SL5_sh). Note that the number following 'Process' is nonsense (and in later releases of GlaNtp is not present) - that parameter is there in the steering file simply to make it more human-readable. However, the code still tries to read it in, but can only handle doubles - the net result varies from run to run, but can always be safely ignored.
  •   templates/out/FlatPlotter${prefix}.out
    Line: 896 to 827
     
    Channel: SemiLeptonic(0) Process: eFake(10)
ib= 1 0.88551 1.50073 wgt=      0.88551 wgtE=     1.50073 wgtEsum2= 2.2522
    Changed:
    <
    <    		 The first number is the bin number being considered. wgt is the weighted integral of that bin, and all preceding bins (i.e. the total integral up to that point)
    >
    >    		 The first number is the bin number being considered. wgt is the weighted integral of that bin, and all preceding bins (i.e. the total integral up to that point)
      Immediately following this is the record of generating the first pseudoexperiment. It lists the weighted contents of each of the bins of a neural net histogram, assuming background only, with poisson fluctuations. It then gives the integral of this pseudoexperiment:
    Changed:
    <
    <
    Pseudodata Integral: 11506
    For obvious reasons this should be similar to the projected background yield.
    >
    >
    Pseudodata Integral: 11506
    For obvious reasons this should be similar to the projected background yield.
      Later on, at the start of the fitting we also have the following:
    Line: 916 to 845
     NLOAccep : -0.0627014 0.727165 pdf : -0.0106981 0.99073 xsec : 0.00620052 0.923289
    Changed:
    <
    <    		 These values come from a Minuit fit, so should be taken with a pinch of salt. The 'Value' compares the results of the pseudoexperiment for all the various errors, and compares it to what you told it. E.g. if you said you had 1fb-1 for luminosity, but the pseudodata suggested a luminosity of 1.01, then Value would be 0.01 - you are 'out' by 1%. 'Error' says how much of your proposed error you have 'used' - if you say you have a 10% error on your luminosity, but the fit suggests at 1% error, then 'Error' would be 0.10 - you are using 10% of your 'allowed' error.
  • >
    >    		 These values come from a Minuit fit, so should be taken with a pinch of salt. The 'Value' compares the results of the pseudoexperiment for all the various errors, and compares it to what you told it. E.g. if you said you had 1fb-1 for luminosity, but the pseudodata suggested a luminosity of 1.01, then Value would be 0.01 - you are 'out' by 1%. 'Error' says how much of your proposed error you have 'used' - if you say you have a 10% error on your luminosity, but the fit suggests at 1% error, then 'Error' would be 0.10 - you are using 10% of your 'allowed' error.
  •   drivetestFlatFitAtlastth.rootUnscaledTemplates.root.
    Line: 949 to 876
      ttH /data/atlas09/ahgemmell/NNInputFiles_v16/mergedfilesProcessed/ttH-v16.root 120 1 556 556 1 1
    Changed:
    <
    <    		Some of the values are established through steerComputentp.txt in the line 
    ListParameter  Process:ttH       1 Filename:/data/atlas09/ahgemmell/NNInputFiles_v16/mergedfilesProcessed/ttH-v16.root:File:120:IntLumi:1.0
    >
    >    		Some of the values are established through steerComputentp.txt in the line 
    ListParameter  Process:ttH       1 Filename:/data/atlas09/ahgemmell/NNInputFiles_v16/mergedfilesProcessed/ttH-v16.root:File:120:IntLumi:1.0
     
    Changed:
    <
    <
    1. Process Name / File Name : The mapping between these two is established in steerComputentp.txt
    2. File : The number which Computentp uses to differentiate between the various files it's processing - established in steerComputentp.txt
    3. IntLumi : The luminosity which Computentp is aiming to simulate by applying 'weight' to your samples - established in steerComputentp.txt
    4. Integral : The number of events you would expect that sample to have within your desired luminosity
    5. Alpha : This should equal TrainWeight
    >
    >
    1. Process Name / File Name : The mapping between these two is established in steerComputentp.txt
    2. File : The number which Computentp uses to differentiate between the various files it's processing - established in steerComputentp.txt
    3. IntLumi : The luminosity which Computentp is aiming to simulate by applying 'weight' to your samples - established in steerComputentp.txt
    4. Integral : The number of events you would expect that sample to have within your desired luminosity
    5. Alpha : This should equal TrainWeight
     

    Limitations

    Line: 1108 to 1024
     WtEvents:01 Passing Mask Selection for Higgs : 2.59829
    Changed:
    <
    <    		These entries correspond to the yields - the numbers of events expected in our specified luminosity.
    >
    >    		These entries correspond to the yields - the numbers of events expected in our specified luminosity.
      If you want to get more debugging from Computentp, then run it with another argument (doesn't matter what the argument is - in the example below it's simply 1):
    Line: 1146 to 1062
     
    META FILEATTACHMENT attachment="Est_12_120.eps" attr="" comment="" date="1248437193" name="Est_12_120.eps" path="Est_12_120.eps" size="16358" stream="Est_12_120.eps" tmpFilename="/usr/tmp/CGItemp41869" user="GavinKirby" version="1"
    META FILEATTACHMENT attachment="score_12_120.eps" attr="" comment="" date="1248437206" name="score_12_120.eps" path="score_12_120.eps" size="29195" stream="score_12_120.eps" tmpFilename="/usr/tmp/CGItemp41862" user="GavinKirby" version="1"
    META FILEATTACHMENT attachment="Report-FINAL.pdf" attr="" comment="" date="1254221251" name="Report-FINAL.pdf" path="Report-FINAL.pdf" size="207168" stream="Report-FINAL.pdf" tmpFilename="/usr/tmp/CGItemp39289" user="ChrisCollins" version="1"
    Added:
    >
    >
    META FILEATTACHMENT attachment="FlatStackParams.txt" attr="" comment="Description of the parameters that can be set in the plotting control." date="1331136419" name="FlatStackParams.txt" path="FlatStackParams.txt" size="4980" stream="FlatStackParams.txt" tmpFilename="/usr/tmp/CGItemp30631" user="AdrianBuzatu" version="1"
    View topic | History: r161 < r160 < r159 < r158 | More topic actions...
     
    This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2025 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
    Ideas, requests, problems regarding TWiki? Send feedback