This post is to document how I determined which runs from Run 18 AuAu27 were affected by the operation of the Booster Main Magnet (BMM), which was determined to be the fundamental cause of The 6.6 second anomaly.

I was able to obtain access to the logs of BMM operation from C-AD. There are 8 "users" of the BMM during Run 18, and each user has their own logs of operation, each of which comes from a different source which may not be running a time-synchronizing daemon, so the timestamps may not be perfectly accurate nor in agreement with each other. These 8 users pass off their control of the BMM to each other on sub-second time scales. Discussions with various C-AD personnel have convinced me that only 1 user (#3) of the BMM has been historically responsible for impacting RHIC operation. That user rapidly pulls and pushes energy from the power grid, which (according to the working hypothesis) causes power fluctuations for the RHIC magnets. In turn, those power fluctuations cause the RHIC beam position to fluctuate, and any beam halo may then scrape constrictions in the RHIC rings. That scraping leads to backgrounds in STAR, which can ionize the TPC gas and cause space charge distortions as seen in my previous blog post.

Here I show a screen capture of using the BMM log viewer on the C-AD machines:



Using the C-AD log viewer has a couple important quirks: large blocks of time may have no logged data (during which that user is presumably inactive, not turning the BMM on), and the time granularity of the presented data is dependent on the time range viewed. I found that viewing the entire AuAu27 period led to too coarse granularity for my purposes (i.e. some occurrences of switching on or off of the BMM were missed), while viewing individual hours of time led to both an excessive amount of information and a rather large number of views to digest. I settled on day-by-day views as a reasonable amount of data. I exported the data from the log views, concatenated that data, and transferred it to RCF.

Next, I spun through that dataset and tried to determine periods of operation, ascertaining start and stop times. To do so, I ignored records of low current in the BMM which were in windows of 30-seconds or less between high current readings as these may simply have been sampled mid-fluctuation. The result was 571 periods of operation between 2018-05-10 and 2018-06-18. I then looped over these to see if the "on" period coincided with any produced AuAu27 physics production runs, finding 286 of 804 such runs (~36%) to have at least some overlap. I also determined the affected time for each of the 804 runs, finding 223,647 affected seconds out of 1,158,547 seconds of total data acquisition time during these runs (~19%). The running of these integrals vs. chronological run index is shown here:



If I sort the runs by the percentage of their DAQ time affected, and plot the percentage of DAQ time affected, one can see 106 runs that are 100% affected, while 180 runs are fractionally affected (and most of those are in the 0%-60% range) and really should not be completely discarded in the end:



The portions of the above plots which are red is the affected ~19% of data.

I cannot rule out that some fraction of time during the BMM operations may be recoverable in the sense that it is undistorted, as the BMM was only on for a portion of each 6.6-second cycle. However, there is some evidence I saw that the unaffected part of each cycle may only be ~1 second (i.e. maybe only ~15% of the red area in the plot above), so the effort to keep these sub-cycle portions may not be worth the small gain in physics. With even more work, one could imagine attempting to correct for the distortions in the data as we have with the Abort Gap Cleaning distortions, but this would be a longer and challenging project.

Another caveat about this affected data is that I have not shown that every time the BMM is operated in this mode ("user 3"), the TPC becomes distorted. Angelika Drees states that she sometimes managed to collimate away much of the beam halo, and subsequently the beam oscillations may not cause scraping that results in additional backgrounds at STAR. This kind of spot-checking will require a more detailed look at the production data. My hope is that it may be determinable from the RICH scalers, which are easier to extract from the PicoDsts than are the DCAs vs. sub-second time in the MuDsts necessary to see the 6.6 second anomaly there. To be seen...

-Gene