Run plan for Year 10 data processing

Quick links


Run plan

The data list would be as listed below, in order of priorities.

  • 7.7 GeV and 11 GeV - Production of the 11 GeV finishing ... [+]

    • All events target (all streams)

    • Pt and dE/dx resolutions and TOF all fine as it stands. Current numbers are OK for the 7.7 and 11.5 GeV

    • Bulk-corr would like to revisit the definition of good events – 1 week time to get back on event selection criteria.

      • Contact person: Paul (TBC / Jerome will follow up as unclear if this means we hold)

      • Follow-up:

        • Main current issues are related to the dN/dEta of charged particles in East and West not only negative but showing some very negative – when those are looked into, a V2 analysis would exhibit problems (no eta/y dependencies as expected). See below:

        • General idea is that the vertex may still not be selected appropriately. Had a long discussion with Paul Sorensen on this topic and suggested (again) to have someone look at the Minuit VF. The VF is un-tuned and it is not at all clear it is running optimally. For example, could we move the first “preferred” vertex toward a real collision event by applying a weight depending on particle production asymmetry (eliminate splash or weight based on asymmetry). Are the current weight appropriate (match to EMC? any trigger detector in which should not be? minimum number of track to consider a vertex relevant? etc …) and without looking into it, all are good questions remaining unanswered. It would be easy to run over one DAQ file though, using a local database and tuning the parameters or extending the VF with additional selection …

        • Reminder: Rosi not back (perhaps next week). Planned to at least start with looking at vertex finding efficiency and tune from there.

      • Additional consideration:

        • PMD 7 GeV calibration not done but planned for the end-of-the-week

    • Status: Likely to take another week before a GO, code is ready (VF TBC)

    • Updates:

      • 20100701 - Proposed at Thursday's July 1st management meeting to delay this production and swap with 39 GeV now ready. Agreed.
      • 20100707 - PMD calibrations for Au+Au 39, 62 and 11 GeV declared ready but still work being done on the 7 GeV. Code updated needed for StPmdReadMaker.
      • 20100708 - vertex finder tuning effort summary sent to vertex-hn.
      • 20100718 - Vertex tuning cuts suggested by the VF team approved by PWGC
      • 20100718  - final decision for the PMD was to use the 39 GeV & 11 GeV for the 7 GeV (see this post).
      • 20100803 - First wave of 7.7 done, pending the second time period requiring TPC calibration
      • 20100812 - TPC calibrations for remaining 7.7 ready (see also Software and Computing phone meeting).
                              We decidded to try to use a Effect of stream data on database performance, a 2010 study and it worked (no stream slowdown issue).
      • 20100819 - Second wave of 7.7 done, remaining data incurs excessive CPU usage. This may be due to issues indentified in RT # 1989.
                             S&C executive decision was to abort and re-address the looping jobs later (1/3rd of stat pending).
      • 20100820 - TPC calibrations for 11 GeV ready.
      • 20100825 - st_physics from 11 done, other streams pending DB setup (may use snapshot again).

  • 39 GeV - production begins, Phase I (25-30 M) - first wave of runs as described completed in the 20100702 follow-up is complete [+]

    • 25 M events target (selected)

    • Convener also agree the resolution numbers are OK for the Physics program for the 25 M events targeted.

    • LFSpectra would like to wait until the final calibrations before producing the full samples which is 100 Million+

    • Note: Not clear when the full sample is planned to be scheduled and its relative priority in the production planning but event selections was then discussed

    • Selection 1 – st_physics – 25-30 M events request with the discussion below

      • A list of runNumber and basic trigger information was produced and sent it to the PWGC as information ...

      • PWG looked into the possibility to select runs with minimum of bad RDO at first but also all runs having PMD and FTPC (39 GeV).

      • A run list was determined for this 25 M pass

      • Additional considerations:

        • Bulk-correlation PWG need some time (10 days) to discuss within the PWG the runNumber set which would give the minbias sample they need (i.e. a trimming of the run list above)

        • JetCorr would like to look into the offline QA logs and the online QA root files (now on disks) and confirm the list of runs

      • Overall status of selection 1: (thought to need +10 days on 20100630)

      • Update (20100702): We have a GO.
        PMD calibration confirmed for this energy.
        Run list was sorted out by the PWG. List follows:

        11105011, 11105012, 11100044, 11100055, 11100056, 11100057, 11100058, 11100061, 11100062, 11100063, 11100064, 11100066, 11100069, 11100070, 11100071, 11100076, 11100077, 11100078, 11100079, 11100081, 11100082, 11100083, 11100084, 11100085, 11100090, 11100091, 11100092, 11100093, 11100094, 11100095, 11100096, 11100097, 11100098, 11101001, 11101007, 11101008, 11101010, 11101011, 11101012, 11101013, 11101014, 11101015, 11101016, 11101017, 11101018, 11101023, 11101024, 11101025,

        11101104, 11101106, 11101107, 11101108, 11101109, 11102001, 11102002, 11102003, 11102004, 11102005, 11102006, 11102011, 11102012, 11102013, 11102015, 11102017, 11102018, 11102019, 11102020, 11102021, 11102022, 11102023, 11102027, 11102033, 11102034, 11102035, 11102038, 11102039, 11102040, 11102051, 11102052, 11102053, 11102054, 11102055, 11102056, 11102057, 11102058, 11102065, 11105018, 11105019, 11105020, 11105021, 11105029, 11105033,
        11102075, 11102085, 11102086, 11102092, 11102093, 11102094, 11102095, 11102096, 11102098, 11102106, 11102107, 11102108, 11102110, 11103001, 11103002, 11103003, 11103007, 11103008, 11103009, 11103010, 11103011, 11103012, 11103013, 11103019, 11105037, 11105038, 11105040, 11105041, 11105042, 11105043, 11105044, 11105045, 11105050, 11105051, 11105052, 11105053, 11105054, 11105055, 11105056, 11105057, 11105058, 11105059, 11105065, 11105066, 11105067, 11105068, 11106001, 11106002
      • Status (20100723): all done.
    • Selection 2 – st_upc – Also to be considered in this first pass of production of 39 GeV

      • All events to be produced

      • Status: Jerome pointed that precision is needed: The UPC PWG requested for PPVF (noCTB) to be used in ALL UPC events selection and run. Should this be considered for the 39 GeV? Will consider a no GO until we get confirmation.

      • Update (20100702): PPVFnoCTB confirmed needed. beamLine constraint being checked by Gene and Alex.

      • Update (20100810): Not all st_upc produced as pointed by the PWGC - new wave started on 20100809 to finish all upc. Separated into files with > 20 events and the rest on the 10th.

      • 20100811 - st_upc completed and bootstrapped.

  • 200 GeV

    • Target includes HLT, UPC, MTD and a combo of minbias (100 M) and central (50 M) triggers detailed below. Unless specified. This category should be st_physics.
    • General feel is to wait for the full resolution before proceeding forward.
    • Calibrations
      • HF mentioned they would provide help on EMC calibrations if needed.
      • The UPC beamLine constraint topic was mentioned in this category (see note above)
    • Dataset priorities should be the HLT (all events) then a set of UPC (14 and 23 M events) followed by MTD (2 M), PMDFTP (??), a set of runs with the HT (NPE_18 trigger, 6.5 M) trigger and finally, the famous (100/50 M )events of minbias/central
      • Note: unclear how many events from PMDFTP are needed – precisions needed. Is this all events?
    • Some details
      • UPC (there are two versions of it: a upc main and upc topo. The request is to have the "main" first and we should find 2 versions of those, one sent with trigger id 1 and one with trigger id 260750. Total will be 37.52 M
        • Note: Any run list to remove ambiguities?
      • MTD: 1 M events from two runNumber ranges: from 11010013 to 11031088 and from 11055095 to 11071047
      • PMDFTP: having PMD anf FTPC (need clarification as pointed in the note above)
      • HT – this is included in the minbias and central samples
        • Note: we may not sub-select if not clearly identified. However, it was noted that this is not resolved and discussion is ongoing in the HF PWG (within a week, will let us know more details on high tower processing)
      • Additional stream: Then 5 M events of atomcules.
        • Note: unclear which run number
        • Answer (20100630): runs from 11021000 to 11075999
      • 100 M of the minbias and 50 M central – PWG selected the ranges 11004040 – 11011007, 11014041 – 11020077 and 11028018 – 11029079.
    • Additional work and discussions
      • LFSpectra would like to see if there are issues with TOF before giving the OK to go.
      • LFSpectra would like to help with sorting out the TPC Anode table before the final runnumber selection occurs
    • Full dataset for Au+Au (and other streams) was also discussed with the PWGC but NOT yet a priority not made so no point to provide details here (TBC).
    • General status and updates:
      • 201007XX -  Discussion, several calibration (EMCs, beamLine, TPC) remains to be done 
                              Several checks (TOF, run selection) neeeded before a run list/OK to go is finalized/given.
                              There is NO GO for the 200 GeV for now.
      • 20100811 -  Still some uncertainties related to calibration, progress given in Software and Computing phone meeting as Run 10 Calibration Status (2010-08-11)
                              Discussed possibility to re-process the 39 GeV pending TOF update.

  • 62.4 GeV
    • There was no BUR request about this energy so there was not much planning so far. The physics priorities will be discussed at a later time and no a consideration for now.

    • One question asked: in case the BES, 39 are covered before QM 2011 (before April) and there is some time to produce some data, what would it be?


Time estimates

Time estimates made by Lidia Didenko. The assumptions include a 80% farm efficiency and no DB connection problems or HPSS issues. Timing were taken from timing from a typical test library.

Low energy runs

  •  AuAu7_production, all datastream, 127M events; avg CPU / RealTime - 1.9/2.11 sec/evt in optimized for st_physics.
    Time: 4 to 12 days (we have 48K jobs for this dataset and I don't expect that we could proceed more than 4K jobs for one day)
    This dataset  should be processed in parallel with some other data, perhaps auau 39GeV otherwise it will be difficult to get farm saturation.
  • AuAu11_production, all datastream, 68M events; avg CPU / RealTime - 3.1/3.3 sec/evt in optimized for st_physics,
    Time: it will require ~4 days to a week depending on whether or not staging will be smooth. Recommendation is to mix with another stream.
  • AuAu39_production, 30M events st_physics and (I understood) st_pmdftpc stream data; avg CPU / RealTime - 4.35/4.58 sec/evt in optimized for st_physics
    Time: it will require < 3 days (based on CPU). If throttled due to HPSS, it could take up to 5 days
    Note: st_upc for  AuAu39_production is only 390K, so insignificant time

Summary: Taken into account that our assumptions are correct, the low energy production request should be done within ~ 1 month if we don't have any major problems.

Other energies

  • AuAu200_production
    • 16M st_hlt will take ~12 days to process (timing from test production)
    • 2M st_mtd expected to take the same CPU as st_hlt = 45 sec/evt => will take ~2 days
    • Selected runnumbers for AuAu200_production have only 160M events in st_physics files => Production should take up to 4 months but if the selection is done right, ~60 days could be possible (see S&C meeting talk about the overlap and selection of what "central" means)
    • st_upc, ~38M events,  don't have yet CPU estimation but expect that it will take < 2 weeks with CPU ~25sec  (on the high side for upc)
    • Note: we have datastream st_ht (high tower), 60M events, not clear if we are going to process them or they will be inside the planning auau 200GeV runnumbers selected for minbias & central trigger events.
    • 5M st_atomcules events => expectation ~3-4  days (but exact timing need fine tuning)
    • st_pmdftp - no clear idea at this stage

Summary: In all, the production could take from 3 to 5 months.