Just as we saw in BES-I, it appears that are events with large hit counts in BES-II.

When processing 8000 events per file from Run 19 AuAu 14.5 GeV collision data, we found that somewhere around 33% of the jobs crashed. This was traced to the large number of hits in the TPC causing memory issues. Note that the rarity of these events means that processing fewer events per file means fewer job crashes (e.g. only ~1/8th, or 4% of jobs would crash if we processed 1000 events per file; and ~50% of jobs would crash if we process 12000 events per file, and most files have somewhere in the 10k-14k events in the 14.5 GeV data).

The hits seem notably more enhanced in the outer sectors (old TPX electronics, just as in BES-I), though the inner sectors can look a bit odd too. Here are the row vs. sector distributions per event for a small selection of good events from the 14.5 GeV data, followed by the same for two bad events from separate runs:



I expanded the plot for the second bad event to show the pads within each sector (i.e. the x axis is sector*150 + pad):



Here are the per-event time-bin distributions for those same two bad events, along with the good events near them in time, showing distributions for..

• good event inner sectors (green)
• good event outer sectors (blue)
• bad event inner sectors (red)
• bad event outer sectors (black)

In general, the bad events have hits everywhere, not just one part of the TPC.

The way we dealt with this before was to place a maximum on the number of hits allowed in a given sector. Here is a plot of that hits-per-sector count for the two bad events, along with the events on either side of them:



So from these plots, we can probably imagine a cut placed somewhere in the 10000 to 30000 range. But it is also helpful to see a distribution of the number of hits we have in the data in general. Here is a plot similar to what I did back in BES-I, showing N_hits^1/4 for Run 19 AuAu 19.6 GeV and 14.5 GeV data, as well as the Run 21 AuAu 7.7 GeV data (being taken right now):

(note: with the iTPC, we'll have more hits than we did in BES-I in general)

• Run 19 19.6 GeV (red)
• Run 19 14.5 GeV (black)
• Run 21 7.7 GeV (blue)

There isn't a very clear cut-off for the 14.5 GeV data, but it's clear there is a high side tail that we can cut somewhere. The 7.7 GeV data also shows a clear second hump that is likely worth trimming off as well. But even if we just use a cut at N_hits^1/4 < 25, which corresponds to on average N_hits < ~16000 in a sector, all of these datasets' good events will likely make it through reconstruction and the production jobs won't die (which is really my goal here, not to remove all of these crazy events). So I'll set the cut just a little above this for some margin for now, at 18000 hits maximum per sector. However, I will not use such a cut for the Run 19 AuAu 200 GeV data as we saw N_hits^1/4 reach 25 even in the days of the old inner TPC.

I also took a look at some Run 19 FXT data (production_4p59GeV_fixedTarget_2019 and production_31GeV_fixedTarget_2019) and saw that events can have up to ~6000 hits in a sector, but only for the case of a low p_T looper within the sector. So again, that should be fine with 16000.

I'll place the cut at 18000 for the Run 21 7.7 GeV data as well, but this should probably be opened up when we return to high energy running later during Run 21. I also need to look at the Run 20 data and decide on reasonable maxima there.

-Gene