2022 Shutdown - FST Maintenance Log

Note: This note is not in the reversed time order

 May 10 2022

  1. Issue: abnormal current in u207 on MPOD01 and u0 on MPOD03
    • Turn on the MPOD crates; check the current on those 2 channels, compared to other channels. u207 on MOD01 is higher than the others; u0 on MPOD03 has significant fluctuation.
    • Turn off the MPOD crates; switch the purple cables of the problematic channel with the adjacent ones
      • 1-12 (B-6) <-> 1-11 (B-5)
      • 3-1 (C-10) <-> 3-2  (C-11)
    • Turn on the MPOD crates; check the current again; the issue is now on u205 on MPOD01 and u2 on MPOD03
    • Turn off the MPOD crates; switch back the purple cables
  2. Issue: Temperature of PPB_A-11 is comparatively higher then the others when running
    • Turn on the MPOD crates; run DAQ for 10 mins in order to warm up the PPB. Check the temperature of PPB_A-11 and compare to other PPBs every 5 mins. The temperature of PPB_A-11 is about 0.7 °C higher than the others once DAQ is on
    • Turn off the MPOD crates; switch the purple cables of the problematic PPB with the adjacent one
      • 3-11 (PPB_A-11, 5_2_0_3 ) <-> 3-12 (PPB_A-10, 5_2_1_3)
    • Turn on the MPOD crates; check the temperature of PPB_A-11 and PPB_A-10; PPB_A-11 (5_2_0_3) is still higher than PPB_A-10 (5_2_1_3)


====================================================================================================
May 11 2022

  1. Turn off the MPOD crates; disconnect the purple and gray cables on PPB_A
  2. Organized the spare/broken/empty module boxes in the physics building


====================================================================================================
May 16 2022
Disconnect the purple cables on PPB_B and PPB_C
====================================================================================================
May 17 2022
Platform built for up coming work

====================================================================================================
May 18 2022 - Check Leakage 
Some photos of unknown structures on the sensors(3-2,6,7,8):

In comparison to the similar spots on IST:

Some photos of unknown residue (3-1,3):

The FST was already slide out prior to the following operation

  1. Refill the coolant to  ~ 70% 
  2. Turn on the cooling system and let it run for few minutes
  3. Check as many modules as possible (limited by space)
    • Leak
      • 3-1 "straight"
      • 3-2 "straight"
      • 3-3 both
      • 3-4 "straight"
      • 3-5 both; the straight tube had droplets formed (used towel to absorb and collect; residue visible on the towel)
      • 3-11 "bent" 
    • No Leak
      • 1-1 
      • 1-3
      • 1-11 
      • 2-3 
      • 2-4 
      • 2-8 
      • 2-10 
      • 2-12 
      • 3-7 
      • 3-8
      • 3-9 
      • 3-10
  4. Slightly tighten the screw on 3-5 - "straight"; expect less leakage the next day

====================================================================================================
May 19 2022

  1. Dismount PPB_A-11 from the PPB_A tray
  2. Double checked the leakage on 3-5 - "straight"; no noticeable leakage on this pipe any more

====================================================================================================
May 20 2022

  1. Test the current on the spare modules (#35 and #77). They perform the same as before (normal noise readout)
  2. Use the sniffer to double check accessible pipes/connectors


#35

#77
====================================================================================================
May 23 2022

  1.  Install the backup PPB to the uninstalled PPB_A tray
  2. Use the sniffer to double check accessible pipes/connectors


====================================================================================================
May 24 2022

  1. Test the current on the modules with leaking/missing pipes but working sensors (#53, 56, 58 66).
  2. Use the sniffer to double check accessible pipes/connectors


#53

#56

#58

#66

Inner Outer Comment
#35 0.1 muA 1.2 muA -> 1.7 muA (+20 min) -> 1.9 muA (+10 min) Spare module
#53 0.1 muA 0.2 muA Major leak
#56 0.1 muA 0.2 muA Disconnected connector
#58 0.1 muA 0.1 muA Missing connector
#66 0.1 muA 0.2 muA Disconnected connector
#77 0.6 muA 0.3 muA Spare module
====================================================================================================
May 25 2022

  1. Flemming pointed out that 3-11 had similar grease flowing out although not as significant
  2. Use the sniffer to double check accessible pipes/connectors


====================================================================================================
May 26 2022
Use the sniffer to double check accessible pipes/connectors
====================================================================================================
May 27 2022

  1. Use the sniffer to double check accessible pipes/connectors
  2. Touch the grease: softened, but not flowing

====================================================================================================
May 31 2022

  1. The FST is pushed back into TPC.
  2. All cables were reconnected.
  3. In the epoxy testing, one of the soft tube connections in the testing setup had a leak (not the soft tube connected to the FST module cooling tubes). Greg fixed this leak 

====================================================================================================
June 1 2022

  1. Refilled the coolant to 95.1%, leak rate = -0.68% per day
  2. Ramp up the voltage; 1-12 (outer) tripped; need to manually ramp up slowly and only to 140V instead of 160V


Coolant level vs time (updated on July 5)

====================================================================================================
June 2 2022

  1. Re-organize the cable which was a quick fix during the run
  2. Re-organize the purple cables and cooling tube so that they are not interfering with the active area and not floating
  3. Pedestal test demonstration by Xu

====================================================================================================
June 3 2022
Pedestal tests
https://drupal.star.bnl.gov/STAR/system/files/fst_pedestals_23154003.pdf
https://drupal.star.bnl.gov/STAR/system/files/fst_totStdDev_23154003.pdf
https://drupal.star.bnl.gov/STAR/system/files/fst_ranStdDev_23154003.pdf 
====================================================================================================
June 6 2022
Online Monitoring Plots
https://drupal.star.bnl.gov/STAR/system/files/OnlineMonitor.pdf
====================================================================================================
June 9 2022
The table is already removed. Cooling tubes at the bottom are re-organized

=
===================================================================================================
Jun 13 2022
1. Another pedestal test before shutting off FST
https://drupal.star.bnl.gov/STAR/system/files/fst_pedestals_23164002.pdf
https://drupal.star.bnl.gov/STAR/system/files/fst_totStdDev_23164002.pdf
https://drupal.star.bnl.gov/STAR/system/files/fst_ranStdDev_23164002.pdf
2. Shut down FST and the cooling system from the online software
3. Shut down the hardware: shut off EMO on the cooling rack, and turn off power supply on all 3 MPOD crates
====================================================================================================
Dec 27
1. Refill the coolant to ~50%
2. Turn on the cooling system and MPOD crates; conduct the pedestal tests
        The cooling system required a power cycle in order to be turned on
        u101 and u207 on MPOD1 tripped with default ramp-up configuration
        Rampped up u101 with smaller step
        Rampped up u207 with smaller target voltage (100V)
3. Turn off the FST; switch off MPOD crates; turn off the cooling system
4. Turn off the valve in the front face of FST; disconnect cooling tubes; collect remaining NOVEC in the cooling tube
5. Disconnect the purple cables
6. Tape the valve opening, end of cooling tubes to prevent dust; tape the connector of purple cables to prevent damage
==========================================================================================
Dec 28
Drain the residual NOVEC in the cooling tube
==========================================================================================
Dec 29
The FST is pulled out; re-organize the cables so that they are in a steady and safe place

Winter 2022 - Part 1