Attendees: Jeff L., Jérôme L., Wayne B., Gene V.B., Leve H., Dmitry A.

Minutes:

jevp : the new online QA package

• Deployment intended before run starts
  • Will run concurrently with OnlinePlots package on the evp.starp machine
    • That machine is believed, at this point, to be sufficiently up to the task
• Infrastructure ready at 90% (old adage: last 10% takes 90% of the time)
  • Using QtRoot
• New architecture has 3 components
  • server: repository for plots
  • builder: stand-alone process for reading event pool, filling hists, and ships plots of them to the server in a ROOT container class (StJEvpPlotSet)
    • StJevpPlotSet contains the histograms, plus lines and texts, and axis scales
  • display: it asks the server for the plots it wants to present at any given time (and tabs); not the content of other plots
• All components communicate over ethernet and can be on separate machines
  • Multiple builder instances will all read the same files simultaneously
  • Suggestion to possibly leverage other available online nodes
    • Would need to mount evp.starp: /a
    • Won't pursue until a need is shown
• Deployed all existing plots to server (999 exactly!)
  • Divided into 17 builders: generic, daq, hlt, l3, each subsystem
• Main features:
  • Easily configured plotting
    • There is an xml configuration file, plus an editor for managing the xml file
    • Hierarchy of tabs and plots easily changed (directory-like arrangement)
    • Top level is shift, HLT, other expert sets
    • Same plot can be in more than one place
    • Control over rows and columns, log scaling (part of plot definition, but over-ridable in the configuration), plot maxima and minima (setting can span multiple plots)
    • Display comes up with shift histograms by default
  • Reference histogram capability
    • Clicking on a hist brings up the data and reference histograms (and possible old references)
    • This page allows one to set the data as the reference along with a comment
    • Discussion of expanding this capability for automated analysis and alerts
      • Analytic comparisons be done in the builder (that's where the analysis happens)
      • Suggestions to keep this in mind as programming continues
• Future Planned Features:
  • Conditional suppression of unwanted plots
    • Distinction between not-in-run, and broken
    • Some plots shown only if some state exists
  • Fast trigger stream: possible send all trigger data (not DAQ data) to the event pool
  • Monitors from within the package (e.g. how long does each builder take)
    • Needs to be coordinated in the server
• Additional To-Do tasks:
  • Run stop/start isn't well-coordinated yet (when to clear histograms for start of a new run; codes need more understanding of state)
  • Basic shift histogram group needs to be set-up
    • Plan: present a set of plots to the trigger board for feedback & input
  • Need testing with real data for each subsystem for possible code-copying mistakes
  • Need extended operation testing (e.g. does it have memory leaks? eventually crashes?)
    • Already some concerns about ownership of objects in the plotting code (an issue we had with the OnlinePlots package)
  • Plan to run display component locally on rts01
    • Needs STAR environment to do so
    • Interest in a second display screen
    • Interest in upgrading rts01 hardware
      • Wayne agreed to spec out new hardware
• Plan to learn as we go (from problems during the Run) of what are the optimal ways to present data which make the problem(s) evident
• Plan for now is to demo the jevp package in 3 weeks time

Other discussion topics:

• Rotations of DBs done for new year, but not all hyperlinks have been rotated
• Online disk space usage needs: users need to be polled (e.g. spin QA programs and data), and who the users are needs to be determined
• Solaris phase out: systems (slow controls and trigger) are not in S&C control; admins need to be convinced that support is gone and catastrophes loom (e.g. ITD can disconnect machines from the network)
• Switch to using dbbak for daily DB backups not yet done
• DB flush planned for Monday, and that should be it until online daemons start up (awaits EPICS ramp up)
• Jérôme mentioned workflow tests are in order for QA-RunLog-ShiftLog interconnections
• Discussion of RTS/ShiftLeader status editing/updating (continued post-meeting):
  • Conclusion seemed to be that the current set-up is what is needed, but that edits/updates should be in the hands of shift leaders. This policy should be made very clear and known.