Yearly Tasks

This page will serve as documentation for several yearly tasks. Not all of these are spin-pwg related tasks, but should be documented somewhere public, and then can be linked elsewhere if necessary. It's a work in progress currently, but hopefully soon all bullets below will be filled out with information.

1.) CDEV Monitoring:
If the current run is a polarized (proton) run, then CDEV should be running - and is generally associated with the spin PWG. This code grabs spin information output by C-AD, and stores it into a MySQL database for use later after the run. This code is documented year-to-year on the online machines. Currently, if one logs into onl02, then the information for the most recent cdev run (2013) can be found in


This will get one started with the code, and teach them how to start it running, barring any new issues. It should be updated yearly with any kind of issues that pop up, and could come up again in the future.

After the run the CDEV information will be used to retreive the spin information from the MySQL database, and then upload that to the database where it's used in MuDst, etc. This comes in two steps:

- Produce runlist and QA spin patterns
- Upload results to star database

2.) L2 Monitoring:
Necessary Files: L2 Monitoring and Pedestal Documentation

The URL above sends you to a blog post which has the most recent (as of 2/12/2015) version of the L2 monitoring and pedestal documentation. The relevant sections for the overall monitoring scripts are the introduction to Section 1, and then Section 1.1 will give the instructions on how to start the L2 monitoring code.

3.) L2 Pedestal Monitoring and Updates:
Necessary Files: L2 Monitoring and Pedestal Documentation

The URL above sends you to a blog post which has the most recent (as of 2/12/2015) version of the L2 monitoring and pedestal documentation. The relevant sections for the pedestal monitoring and updating are the introduction to Section 1, and then Section 1.2 will give the instructions on how to monitor and update the pedestals for L2.

4.) BEMC Trigger Database Monitoring:
Necessary Files: Documentation for setting up BEMC online trigger database monitoring code

The above documentation isn't extremely expansive, but does get the new user started with the code, and points them to the appropriate place to get an enumerated list of steps that gets the code going. In principle, this code is very simple to set up and keep running, so long as no issues arise.

This task shouldn't be ignored. It'll either be run by the EMC software coordinator or a dedicated monitor, and should be checked daily for validity and to ensure proper execution of the code (i.e. reviewing the log files.)

5.) BSMD Pedestal Monitoring:
Necessary Files: Documentation for setting up and running BSMD monitoring code

This monitoring tasks produces the BSMD Pedestal PDFs, as well as monitoring plots run-by-run, and can be found here. The code is simple to set up and run, barring any issues. There are, though, a few issues that persist and are noted in the above documentation. The code will compile and run with these issues, but should be investigated in the future when time allows.

One should update these instructions on another blog accordingly for the appropriate year.

6.) BEMC Status/Pedestal Tables
Necessary Files: Documentation on creation and QA of offline BEMC status/pedestal tables

This task is particularly important to get done before production because the vertex finder uses tower statuses. Thus, they should be up to date before production begins. This code can be executed during the running period, but as it uses files from each fill it's best to wait till the end of the run to make the status and pedestal tables (but before production!). This task will be headed up by the BEMC software coordinator, but will involve an "assistant" which will be from the spin-pwg community (in general) if the most recent run was dedicated to spin (i.e. pp running). Thus the instructions for the usage of the code is included here for those helpers as well as future software coordinators.

Runtime Documentation

L2 monitoring Scripts:

All L2 algorithms are run via a single script monitorDaemon.  You can reach the monitoring account through the following steps (use -X -A flags on all steps):

  • ssh -X -A -Y
  • ssh -X -A (you may need to specify or
  • ssh -X -A

If you do not have permission to log on as onlmon, you can request access to onlmon at

If the scripts need to be restarted do these commands as onlmon on onl02:

  • cd L2algo2012
  • monitorDaemon >& /dev/null ; bg ; exit

If this starts garbling data, the cleanUp command in the same directory will clear out all data and monitorDaemon can then be started completely from scratch.  Files to be accessed online are stored in /onlineweb/www/L2algo2012/l2jetHigh.

Original L2 doumentation
Maxance's L2 Monitoring "How To"  is attached at the bottom.

 L2 Tower Masks

Here is how to update the towerMask out of the L2 algorithms:

  • ssh -X -A (pw4*DAQops)
  • cd /RTS/conf/L2/emc_setup/
  • cp towerMask.pp_mo_day_yr towerMask.pp_mo_day_yr (copy last/latest file to today's date)
  • emacs -nw towerMask.pp_mo_day_yr (do not use 'vi' editor, add new tower mask to new file)
  • rm towerMask.pp_mo_day_yr (remove old file)
  • ls -s towerMask.pp_mo_day_yr towerMask_current (link to new file to current mask)

BTOW people use SoftID to indicate channels in hardware and software. To us, this is a number from 1-4800,
which is the number of PMTs in the BTOW. When Oleg or I mask a new tower, we will announce it in the electronic
run log, giving the SoftID number and alerting L2 that they have to mask it as well. If we mask it in hardware/L0,
you still have to mask it in L2.

However, Jan Balewski uses a different numbering system than SoftID. His notation, uses the form ##ll##.
If the letters are CAPITAL, this indicates an ENDCAP tower. If lower case, it indicates the barrel. Let us call
this the 'towerID'.

If you obtain the hot tower towerID in his format, then you simply go to the towerMask#### file and edit it as before.
If you obtain the SoftID as a number from 1-4800, then you can find the number in his format by 'grepping'
one of his L2ped files. In this file, the first column is the towerID in his notation, the SoftID is the
6th column. So for example, if Oleg tells you that he masked SoftID 1382. You can then type:

cat run12042002.l2ped.log|grep 1382

Looking at the (2) instances that this number appears in the text, we find the line:

12td27 45 1.7 0x10 0x2e id267-057-07 1382 ,.......*.......>...............

The 6th column is 1382, so that is the SoftID. The 1st column is 12td27. So to mask out this tower
you would edit the towerMask.pp_mo_day_yr file and add the line
12td27 0x0f 0x0f

if you put a '#' a the start of the line, it ignores that line.

L2ped - output linked to

Pedestals are calculated for every run and residuals are monitored. These pedestals serve as input to physics L2 algos and the HLT.  When necessary new pedestals need to be updated.  This is done by updating links that live in daqman:/RTS/conf/L2/emc_setup/ so that they point at the most recent log files from L2.

From stargw or another node on starp network login to daqman and move to L2 setup file directory:

Copy and link new reference peds (where DDD=day and RRR=runNumber,EVB=eventBuilder# 1-4):

  • cp /data/l2AlgoPlots/YYYY/DDD/runRRR.l2ped.EVB.log ./YYYYhistory/
  • $ ln -sf YYYYhistory/runRRR.l2ped.EVB.log pedestal.current

Print first few lines from new peds to check the run number is correct:

  • $head pedestal.current

email emc2-hn hypernews to alert crew and emc group that pedestals have been changed.  

BSMD Online Monitoring -

Monitors BSMD pedestals (obviously before zero suppression)

Log on to


The file 'tellme' gives the command to execute

If it is not working, then kill the Bsmd process already running.

python -n 1000000 -m /evp/ &