Spikes in the TPC IFC current have been seen with beam on March 30^th, 2018, and without beam on April 4^th, 2018.

I looked at the data from March 30^th to look for whether the data is actually distorted. The answer appears to be yes, with signed DCA shifts of up to half a millimeter for the largest current spikes.

Here are plots of the IFCW current during runs 19089047-19089052 (a period of about 10,000 seconds, a little under 3 hours) and then a profile of the signed DCA versus time [seconds] (arbitrary 0), as plotted from the FastOffline MuDsts using this formula (which is unfortunately not optimal as the mDCAGlobal vector is from a 3D DCA, not a 2D DCA):

MuDst.Draw("sqrt(sq(PrimaryTracks.mDCAGlobal.mX1)+sq(PrimaryTracks.mDCAGlobal.mX2))
*sign(PrimaryTracks.mHelix.mP.mX2*(PrimaryTracks.mDCAGlobal.mX1)-
(PrimaryTracks.mDCAGlobal.mX2)*PrimaryTracks.mHelix.mP.mX1):
MuEvent.mEventInfo.mTime-1522e6>>hh(200,435000,445000,100,-1.,1.)","PrimaryTracks.mEta>0");

Red arrows show when spikes occurred. Solid arrows show what appear to be a good correlation between a change in the signed DCA and the excess current, while dashed arrows show weaker or no notable correlation. It seems probably that there is indeed a correlation.



Presently there is no understanding of what can be the cause of the FC current spikes. The following observations may be relevant in understanding the issue:

• While the primary spikes of increased current are seen in IFCW, corresponding spikes of increased current are seen at about the same time in IFCE, with magnitudes about 1/5^th of those in the IFCW. No corresponding spikes are noticeable in the OFCs (e.g. they are below the noise if they are there).
• The IFCE spikes appear to have a slower rise and fall than the IFCE spikes (as if they occur with a slightly longer time constant).
• All FCs appear to have a similar relaxation time constant of ~5 minutes as seen by a slow settling from slightly high currents at the time the cathode is ramped up until steady state is reached. The OFCs start with an excess of ~4 nA, while the IFCs start with an excess of ~6-7 nA (this is out of ~76.4 μA).
• The effective resistance as measured by R_182,eff = V₁₈₂/I on IFCW increases during the spikes from 309.55 kΩ to 309.75 kΩ (i.e. 1 part per ~1500) [reference]
• The effective resistance as measured by R_181,eff = V₁₈₁/I on IFCW decreases during the spikes from 2.323 MΩ to 2.319 MΩ (i.e. 1 part per ~500).
• The effective resistances on IFCE show no effects visible above the noise.

Additional thoughts:

• Excess current represents either a drop in the effective resistance of the resistor chain (e.g. shorting occurring in the chain itself, or between FC rings, or through the FC structure) or electrons collecting on the rings from elsewhere.
• The resistor box has resistors which may fail, but there is no clear mechanism for such a failure to increase one of the effective resistances, but decrease the other.
• Occurrence on both IFCE and IFCW hints at something beam related, but the observation of spikes when there was no beam conflicts with this.
  • Jim Thomas also reported coincidence with C-AD "LISA" and "orbit correction" events, but no clear correlations are seen in the BPM (Beam Position Monitor) data.

-Gene

_____________

UPDATE: 2018-04-06

New hypothesis from Blair (& Alexei?):
Blower is supposed to bring dehumidified (air conditioned) air from one end of the TPC to keep temperature regulated. If it is bringing in moist air, then the field cages begin to lose some of their electrical resistance and provide an alternate channel for current down from the central membrane. This would likely affect one end more than the other as the blower is only on one end (west end?).

Jim and I checked historical humidity records from the past few weeks, and they seem to support this hypothesis. However, we've seen humidities like this on the platform in previous years and had no problems, so it still implies something like a malfunctioning dehumidifier (compressor).

Here are some supporting plots of dew point and humidity on the platform during Run 18 so far (Feb. 20^th - Apr. 5^th). I've tried to put the IFCW currents on the same time scale to elucidate the correlation in time with the peaks in the water content:

        



Here's a zoom in of the IFCW current and humidity from the 48 hour period of March 29^th & 30^th, 2018, again demonstrating good correlation:




Here were the records from Run 17, showing that as summer arrived, these values exceeded what we've seen in Run 18, implying that the dehumidification may not be working: