MCLimits Fitting Code: Tool for Measuring Sensitivity

The MC Limits fitting code, developed by Tom Junk for the CDF Collaboration, is a hypothesis testing tool. Making use of the Likelihood function to discriminate between two hypothesised theories, it is possible to utilise the output of the test to measure expected discovery and exclusion limits for new Physics.

This tool is currently utilised at both CDF and D0 at Fermilab for the combined exclusion measurements. This page will detail the use of the code at Glasgow as a tool for determining the sensitivity of the ATLAS Experiment to the SM Higgs Boson. Thus far, this tool has been utilised both as a tool to assess the sensitivity of a Neural Network analysis on the Higgs plus associated top channel and to assess the combined sensitivity of four low-mass Higgs channels in the mass range 110 - 190 Gev.

The code can be downloaded in a tarball from the PhyStat website, by following this link. The file you download (as of 15th Feb 2010) is mclimit_feb17_2009.tgz.

MCLimits Tarball

The MC Limits tarball should be downloaded following the link above. Several files are included in the tarball in a directory, mclimit/. These are detailed below:

Documentation

There is significant documentation which comes along with the tarball of the code. This contains details of the methods and functions available in mclimits as well as giving an outline of the statistical framework

• mclimit_csm.pdf - Details general statistical framework and also has details of each of the methods available in the mclimits code.
• chisquare.pdf - Describes the relationship between a chi^2 and a likelihood, detailing the specific chi^2 function minimised. Discusses how systematic uncertainties are handled.
• genlimit.pdf - Describes the Bayesian Limit Calculator, developed by J. Heinrich and utilised by CDF/D0 to produce the combined exclusion limits for the two experiments as a function of the SM cross-section.
• mclimit.html - Contains details of the updates to the code along with brief explanations of what each file is
• README - Warning about the example code for running. See below.

The Code

• mclimits_csm.C - This is the code, containing all the methods and functions necessary to test two hypotheses and produce discovery sensitivity and exclusion limits.
• mclimits_csm.h

Test Files - Beware

These files DO NOT work (see the README file for details):

• tchanlc.C
• tchan_cls.C
• preparetchan.h
• Makefile
• Makefile.arch

This file might work, but hasn't been tested. It takes the output of the mclimits method, csm_model::print() (which is detailed in the documentation, mclimits_csm.pdf) and produces a webpage of template names and systematics sources for each template.

• ptohtml.pl

Running the Code

To run the mclimits package, you need to produce a file which prepares the inputs to mclimits. This is what tchan* is an example of, however these examples require some specific root files that don't come along with the tarball. As such, and a simple example is provided here.

To run the code:

> root

> .x ttH_massDistribution_Example.C

ttH_massDistribution_Example.C - this is the 'driver' code effectively for MCLIMITS. It defines the input to mclimits and which methods from mclimits should be run.

This file is available for testing here: /home/cwright/roofit/Rick/tthiggs_CSC_updateForThesis/ttH_massDistribution_Example.C

You will also need the following rootfiles in order to run: /home/cwright/roofit/Rick/tthiggs_CSC_updateForThesis/rootfiles/

(You need them all!)

Useful Output

The outcomes from the MCLimits code is heavily dependent on the methods which are selected and run. However there are a set of standard outputs that are of use when doing a hypothesis test, and assessing sensitivity. These are detailed here.

- PDFs

- Lumi95

- Lumi3sigma

- Lumi5sigma

- CLs +/- 1,2sigma

- 1-CLb +/- 1,2sigma

- J. Heinrich's Bayesian Limit results (s95med, s95p1,s95p2,s95m1,s95m2)

The details of these tools are all included in the MCLIMITS documentation (that comes along with the tarball), and rather than repeat the details of each tool here, I recommend you read the paper!

Useful Statistics Documentation

There is a myriad of statistics books and papers available. A simple list of papers that are useful for understanding the statistical concepts adopted in the mclimits code are listed here:

• "Confidence Level Computation for Combining Searches with Small Statistics", Thomas Junk, arXiv:hep-ex/9902006v1
• "Presentation of search results: the CLs technique", A.L. Read, J.Phys. G: Nucl.Part. Phys. 28 (2002) 2693-2704
• "Signal Significance in Particle Physics", Pekka K. Sinervo, arXiv:hep-ex/0208005v1 (CDF/PUB/STATISTICS/PUBLIC/6031)
• "How to Claim a Discovery", W.A.Rolke and A.M. Lopez, PHYSTAT-2003-MOBT002
• "Sensitivity of Searches for New Signals and Its Optimization", Giovanni Punzi, PHYSTAT-2003-MODT002
• "Evaluation of three methods for calculating statistical significance when incorporating a systematic uncertainty into a test of the background-only hypothesis for a Poisson process", Robert D. Cousins, James T. Linnemann, Jordan Tucker, arXiv:physics/0702156v3
• "Combined CDF and D0 Upper Limits on Standard Model Higgs-Boson Production with 2.1 - 5.4 fb-1 of Data", The TEVNPH Working Group, arXiv:0911.3930v1

-- CatherineWright - 2010-02-12