Offline QA: Run 15 stories

These anecdotes document some of the findings performed ONLY by Offline QA during Run 15:

 

  • Anode voltages for a large fraction of TPC sectors were reduced (unexplained) midway through run 16069060 (6.65M events) without any alarm. The reduction was noticed by the online shift crew a few minutes into the next run, but only Offline QA observed that the problem had started much earlier.
     
  • Although issues with the experiment's recordings of the RHIC clock had already been found elsewhere, the automatic calibration of TPC drift velocities went forward for a couple laser runs with incorrect RHIC clocks. The result was invalid drift velocities entered into the database whose impacts only Offline QA noticed, and may not otherwise have been found before physics production.
     
  • As described in RT ticket #3048, Offline QA was the first to determine that the offline software library became broken (in this case, on a weekend). It would have eventually been found by others, but not for a couple days.
     
  • As described in RT ticket #3078, Offline QA found issues due to halted migration of databases from online to offline, which were in turn caused by a hiccup in the online LDAP system. Who knows how long it would have taken before someone else found this; for now, it is Offline QA which determines if the database migration process is truly fully functioning.

-Gene