Run 11 pp500 Transverse bXing QA

Quick summarization: Out of the 1378 runs available to calculate the bXing for in the pp500 transverse run from 2011, 867 had files on disk. Of these 867 runs, there were 40 marked bad overall. These runs were QA'ed and it was identified that only 10 were really bad (plus one extra that was found during ivestigation that wasn't on the investigate list), and othe other 29 needed to be uploaded with a zero offset in the bX7 counter. The bad and good runs will be outlined below. 

**Note: It's unclear why 511 runs had no data on disk. A list of these runs is attached to this post below, any resolution to this would be very much welcome.
 
The run-by-run beam crossing analysis is complete and ready for upload, this blog post will serve as documentation and results from the QA process. I will soon create a good "How-To" PDF that outlines common things to look for and how to operate the code that creates all of the histgrams/upload content necessary to complete the task.
 
This process begins upon completion of the spin QA, which was completed by Brian Page for run11. Upon receiving the outline of fills and the runs therein, there are a total of 1378 runs passed on. Of these runs there are 867 of them with files on disk for running over, and 511 that have zero files archived. The process begins by running TestChain.C over all of the files on disk. This will create the bX7 and bX48 spectra for the run up to a maximum of 10,000 events. It also outputs an "ideal" histogram that is created from the spin database beam crossing. For instance the v124 files that are uploaded include a header that is either "0 240" or "0 120" depending on the first bunch collision point. For 2011, this header says "0 240" which means bunch zero of Yellow Beam (YB) and bunch zero of Blue Beam (BB) collide at 2 O'clock and 8 O'clock on the RHIC beam. Thus the ideal histogram is created so that the abort gaps for BB and YB are in the appropriate places and all other bunches have a count of 1.
 
Now we can use the findBXoffDynamic.C macro to take each ideal histogram and "slide" it over the bX7 and bX48 counters to determine if the beam crossing is as it should be, or if there is an offset. Of the 867 good runs, the following 40 were determined to be offset, and were in need of QA to determine if they should be thrown out, or if they were good after all.
 
 Bad bxoff7 for Run# =12096020
 Bad bxoff7 for Run# =12095055
 Bad bxoff7 for Run# =12091048
 Bad bxoff7 for Run# =12087037
 Bad bxoff7 for Run# =12087034
 Bad bxoff7 for Run# =12083058
 Bad bxoff7 for Run# =12083057
 Bad bxoff7 for Run# =12083054
 Bad bxoff7 for Run# =12083053
 Bad bxoff7 for Run# =12081015
 Bad bxoff7 for Run# =12076034
 Bad bxoff7 for Run# =12064068
 Bad bxoff7 for Run# =12062012
 Bad bxoff7 for Run# =12059035
 Bad bxoff7 for Run# =12059021
 Bad bxoff7 for Run# =12059019
 Bad bxoff7 for Run# =12059018
 Bad bxoff7 for Run# =12059017
 Bad bxoff7 for Run# =12059012
 Bad bxoff7 for Run# =12058058
 Bad bxoff7 for Run# =12058055
 Bad bxoff7 for Run# =12058049
 Bad bxoff7 for Run# =12056020
 Bad bxoff7 for Run# =12056014
 Bad bxoff7 for Run# =12056010
 Bad bxoff7 for Run# =12056007
 Bad bxoff7 for Run# =12054030
 Bad bxoff7 for Run# =12054028
 Bad bxoff7 for Run# =12054026
 Bad bxoff7 for Run# =12054025
 Bad bxoff7 for Run# =12054024
 Bad bxoff7 for Run# =12054021
 Bad bxoff7 for Run# =12054019
 Bad bxoff7 for Run# =12047018
 Bad bxoff7 for Run# =12047016
 Bad bxoff7 for Run# =12047012
 Bad bxoff7 for Run# =12042033
 Bad bxoff7 for Run# =12042027
 Bad bxoff7 for Run# =12038107
 Bad bxoff7 for Run# =12038106
 
 
This macro also creates a PDF for each run that documents the bX7 spectrum and the chi2 distribution that calculates the offset (a minimum at zero means there is no offset). So to QA these runs we used these as well as the same bX7 plots that are created with PlotBadBXing.C, which show larger plots and are grouped in a way that makes the process a bit faster. These are attached to this post. Now I will go through these on a fill by fill basis, using the cdev record as the primary investigative tool.
 
The following runs are those that were completely bad. The abort gaps were not shown at all, and the counter was pure noise. This is a bit strange since runs previous and runs after show great bX7 spectra. These were thrown out, and not uploaded.

UPDATE: Carl mentioned that we should reason to throw these out. All of these runs seem to have problems in the ESL. They will still be thrown out. R12087037 had no entries, but it looks really bad. I don't believe it worth it to try and save this one run.

R12096019 -- Found to be bad during the QA process. Events higher than normal, and run finished in half the time as normal.
R12096020 -- Event rates really high. Asked to remove FMSHT and restart.
R12095055 -- EEMC errors in previous run. Put EEMC back in during this run but was still running into some problems
R12091048 -- FMS trigger rates skyrocketed during the run.
R12087037 -- 
R12087034 -- High FMS trigger rates
R12076034 -- The beam was lost unexpectedly causing shutdown of run and detectors.
R12064068 -- DAQ rates dropped to zero, run stopped
R12059035 -- DAQ rates dropped to zero, run stopped.
R12038106 -- High rates in FPD
R12038107 -- High rates in FPD
 
The following runs remaining are the good ones. In all cases missing bunches in one or both beams are the cause for being marked bad. Using cdev it was possible to locate the missing bunches and proper abort gap locations, and thus make the final call that they were in fact good runs. 
 
These are from Fill 15337. Blue Beam (BB) bunch 1 is missing in the records, as well as Yellow Beam (YB) bunch 80. The overall shift of 40 bunches at STAR means that this yellow bunch will show up as bunch zero in the bX7 spectrum. Also YB bunch 1 is missing, which shows up just to the right of the YB abort gap, and YB bunch 70 which shows up just to the left of the BB abort gap. But the first two bunches in the bX7 spectrum is causing the code to assume that this is the abort gap being shifted by 2 bunches, and a 118 offset it calculated. This is not true, these bunches are simply missing and the abort gaps are in the proper places. These runs will be uploaded with a zero offset.
R12083058
R12083057
R12083054
R12083053
 
The following run has YB bunch 80 missing, which shows up as bunch zero on the counter. This is causing the calculated offset to be 119, but really there is no offset. It will be uploaded with an offset of zero.
R12081015
 
The following run had a very noisy BB abort gap and YB bunch 81 is missing (bunch 1 in the counter). So the shift of 118 is not to be believed, the combination of the missing bunch and noisy abort gap gives ample cause to believe that the code shifted the ideal histogram just enough to have an offset. But it will be uploaded with a zero offset.
R12062012
 
 
The following several runs are from Fill 15251 which has YB bunch 80 missing (bunch zero on the counter). This means that the code could slide the ideal histogram in such a way to believe that this missing bunch is part of the blue abort gap, but it isnt, we know it's missing from the fill. So these will be uploaded with a zero offset.
R12059021
R12059019
R12059018
R12059017
R12059012
 
The following runs are from Fill 15249, which has BB bunch 40 missing, which shows up just to the right of the YB abort gap. This is causing the calculated offset of 119, but this again isnt true. The abort gaps are in the proper location and with the proper number of bunches. These will be uploaded with a zero offset.
R12058058
R12058055
R12058049
 
The next few runs were from Fill 15233 which has several confirmed missing bunches in the neighbor hood of bX7 = 70-80. The calculated shift of 40 is consistent with the ideal histogram being slid in such a way that the BB abort gap lines up with these missing bunches, but it's strange since there are some filled bunches here also. But the end result is that the abort gaps are properly placed with the right number of bunches in them, thus indicating a zero offset. 
R12056020
R12056014
R12056010
R12056007
 
These runs came from Fill 15217, which has BB bunch zero missing (bunch zero on the counter), which is consistent with the code calculating an offset of 119. These will be uploaded with a zero offset.
R12054030
R12054028
R12054026
R12054025
R12054024
R12054021
R12054019
 
These runs are from Fill 15173 which has bunch 40 missing again. The abort gaps are correct, so the offset calculated is a miscalculation. These will be uploaded with a zero offset
R12047018
R12047016
R12047012
 
These are from Fill 15150, which has missing bunches in the neighborhood of bX7 = 80, so the program slides the ideal histogram 39 bunches assuming that these missing bunches are an abort gap. This isn't the case, the abort gaps are correct and the missing bunches are causing a miscalculation. They have a zero offset.
R12042033
R12042027
 
 To fix these bad offsets we must first upload all of the bX7 spectra with an offset of zero. Now from this we can calculate the proper bX48 offset. The given bX48 offset tells us we must shift the bX48 spectrum to the right by X units. For instance, an offset of 9 means we take our bX48 spectrum and shift it 9 units to the right and we will have the bX48 aligned with the bX7 spectrum. But the offset in the bX7 spectrum is messing up the overall shift in bX48. To get proper alignment, we can use the following simple formula

bXoff48_true = bXoff_calculated + (120 - bXoff7_calculated)

This accounts for the false shift in the bX7 spectra due to missing bunches, and gives the final proper resulting bX48 spectrum. This has been applied to the 30 runs outlined above and they have been uploaded which finalizes all of the spin QA/bXing uploads for the run11 transverse period.