Following HF meeting (20150416) :

1. same as HFmeeting (reference)
2. Classification using like sign only for background
3. Classification w/o variables to recreate a fake peak (momentum of daughters and daughters opening angle)
4. Classification using # signal sample ~ # background sample (to avoid overtraining)

Classifiers used : Cuts(Rectangular cuts), Fisher(Linear discriminant) , Likelihood, BDT(Boosted Tree Decision ), MLP(Neural Network, Multi Layer Perceptron)

plots

Some printing :

test 1 :

--- Factory                  : An event is signal-like, if its MVA output exceeds the following value:
--- Factory                  : -----------------------
--- Factory                  :     Method:  Cut value:
--- Factory                  : -----------------------
--- Factory                  : Likelihood:     -0.021
--- Factory                  :     Fisher:     +0.003
--- Factory                  :        MLP:     +0.027
--- Factory                  :        BDT:     -0.118
--- Factory                  : -----------------------

--- Factory                  : Testing efficiency compared to training efficiency (overtraining check)
--- Factory                  : --------------------------------------------------------------------------------
--- Factory                  : MVA              Signal efficiency: from test sample (from training sample) 
--- Factory                  : Method:          @B=0.01             @B=0.10            @B=0.30   
--- Factory                  : --------------------------------------------------------------------------------
--- Factory                  : BDT            : 0.415 (0.428)       0.667 (0.672)      0.862 (0.863)
--- Factory                  : MLP            : 0.360 (0.371)       0.628 (0.627)      0.815 (0.812)
--- Factory                  : Fisher         : 0.160 (0.152)       0.574 (0.569)      0.809 (0.805)
--- Factory                  : Likelihood     : 0.211 (0.201)       0.535 (0.529)      0.753 (0.751)
--- Factory                  : Cuts           : 0.010 (0.128)       0.100 (0.384)      0.300 (1.000)
--- Factory                  : --------------------------------------------------------------------------------

 

test 2 :

--- Factory                  : An event is signal-like, if its MVA output exceeds the following value:
--- Factory                  : -----------------------
--- Factory                  :     Method:  Cut value:
--- Factory                  : -----------------------
--- Factory                  : Likelihood:     -0.017
--- Factory                  :     Fisher:     +0.004
--- Factory                  :        MLP:     +0.043
--- Factory                  :        BDT:     -0.111
--- Factory                  : -----------------------

--- Factory                  : Testing efficiency compared to training efficiency (overtraining check)
--- Factory                  : --------------------------------------------------------------------------------
--- Factory                  : MVA              Signal efficiency: from test sample (from training sample) 
--- Factory                  : Method:          @B=0.01             @B=0.10            @B=0.30   
--- Factory                  : --------------------------------------------------------------------------------
--- Factory                  : BDT            : 0.404 (0.416)       0.664 (0.673)      0.863 (0.867)
--- Factory                  : MLP            : 0.374 (0.378)       0.620 (0.616)      0.820 (0.819)
--- Factory                  : Fisher         : 0.147 (0.153)       0.563 (0.571)      0.805 (0.808)
--- Factory                  : Likelihood     : 0.226 (0.230)       0.545 (0.538)      0.763 (0.765)
--- Factory                  : Cuts           : 0.010 (0.152)       0.100 (0.356)      0.300 (1.000)
--- Factory                  : --------------------------------------------------------------------------------

test 3 :

--- Factory                  : -----------------------
--- Factory                  :     Method:  Cut value:
--- Factory                  : -----------------------
--- Factory                  : Likelihood:     -0.010
--- Factory                  :     Fisher:     +0.012
--- Factory                  :        MLP:     +0.037
--- Factory                  :        BDT:     -0.107
--- Factory                  : -----------------------

--- Factory                  : Testing efficiency compared to training efficiency (overtraining check)
--- Factory                  : --------------------------------------------------------------------------------
--- Factory                  : MVA              Signal efficiency: from test sample (from training sample) 
--- Factory                  : Method:          @B=0.01             @B=0.10            @B=0.30   
--- Factory                  : --------------------------------------------------------------------------------
--- Factory                  : BDT            : 0.368 (0.377)       0.618 (0.624)      0.787 (0.788)
--- Factory                  : MLP            : 0.339 (0.342)       0.580 (0.578)      0.754 (0.752)
--- Factory                  : Fisher         : 0.226 (0.248)       0.566 (0.566)      0.735 (0.735)
--- Factory                  : Likelihood     : 0.200 (0.214)       0.504 (0.502)      0.723 (0.722)
--- Factory                  : Cuts           : 0.013 (0.218)       0.128 (0.335)      0.384 (0.630)
--- Factory                  : --------------------------------------------------------------------------------

test 4 :

--- Factory                  : -----------------------
--- Factory                  :     Method:  Cut value:
--- Factory                  : -----------------------
--- Factory                  : Likelihood:     -0.002
--- Factory                  :     Fisher:     +0.040
--- Factory                  :        MLP:     +0.455
--- Factory                  :        BDT:     -0.017
--- Factory                  : -----------------------

--- Factory                  : Testing efficiency compared to training efficiency (overtraining check)
--- Factory                  : --------------------------------------------------------------------------------
--- Factory                  : MVA              Signal efficiency: from test sample (from training sample) 
--- Factory                  : Method:          @B=0.01             @B=0.10            @B=0.30   
--- Factory                  : --------------------------------------------------------------------------------
--- Factory                  : BDT            : 0.322 (0.448)       0.628 (0.664)      0.786 (0.816)
--- Factory                  : MLP            : 0.298 (0.317)       0.600 (0.591)      0.758 (0.769)
--- Factory                  : Fisher         : 0.189 (0.197)       0.581 (0.571)      0.734 (0.740)
--- Factory                  : Likelihood     : 0.231 (0.203)       0.548 (0.529)      0.743 (0.748)
--- Factory                  : Cuts           : 0.013 (0.174)       0.130 (0.387)      0.390 (0.530)
--- Factory                  : --------------------------------------------------------------------------------

comments :

• the main difference is a decrease in the rejection power for all classifiers, but Fisher,  when going from (2) --> (3)

blah blah