Here I discuss the problem relating to how statistical errors arising from weighted simulation samples are treated in the RooUnfold package ...

The RooUnfold packages implements several unfolding methods and interfaces with ROOT. I chose to use the SVD method from RooUnfold because it supposedly treated the statistical errors arising from finite MC statistics correctly. While I was investigating the use of the package, I discovered that the size of the error bars on the unfolded yields from SVD depended on the normalization I used on my simulation sample.

The simulation was created in 10 partonic pt bins and each bin is assigned a weight such that the pt spectrum will be smooth. The ratio of the weights of any two partonic pt bins is constant, but I should be able to scale all weights up or down by an arbitrary factor without changing any physics quantity. The fact that the size of the statistical error on my unfolded yield depends on the normalization means that the statistical significance of the various partonic pt bins is not being accounted for correctly.

To explore/confirm this, I have created two simu samples in addition to the full sample which I have been using: a sample which contains 25% of the events of the full sample and a sample contains the same events as the 25% sample but has a normalization which is a factor of 4 times larger than the 25% sample. Using these samples, I have created a spread sheet showing the unfolded yields and errors for the various samples.

The spread sheet can be found here.

The spread sheet contains info on two different error methods provided by RooUnfold: Method 1 and Method 3. Method 1 takes into account the errors from the Simu statistics whereas Method 3 only takes into account the errors arising from the data sample itself. The bottom line of the spread sheet is that the errors from the 25% sample with a normalization of 4 are the same size as the full sample. The size of the errors from the 25% sample are rougly 25% larger than the full sample. This shows that I could take a simu sample with a small number of events and just scale up the weights to get arbitrarily small error bars.