- genevb's home page
- Posts
- 2024
- 2023
- 2022
- September (1)
- 2021
- 2020
- 2019
- December (1)
- October (4)
- September (2)
- August (6)
- July (1)
- June (2)
- May (4)
- April (2)
- March (3)
- February (3)
- 2018
- 2017
- December (1)
- October (3)
- September (1)
- August (1)
- July (2)
- June (2)
- April (2)
- March (2)
- February (1)
- 2016
- November (2)
- September (1)
- August (2)
- July (1)
- June (2)
- May (2)
- April (1)
- March (5)
- February (2)
- January (1)
- 2015
- December (1)
- October (1)
- September (2)
- June (1)
- May (2)
- April (2)
- March (3)
- February (1)
- January (3)
- 2014
- December (2)
- October (2)
- September (2)
- August (3)
- July (2)
- June (2)
- May (2)
- April (9)
- March (2)
- February (2)
- January (1)
- 2013
- December (5)
- October (3)
- September (3)
- August (1)
- July (1)
- May (4)
- April (4)
- March (7)
- February (1)
- January (2)
- 2012
- December (2)
- November (6)
- October (2)
- September (3)
- August (7)
- July (2)
- June (1)
- May (3)
- April (1)
- March (2)
- February (1)
- 2011
- November (1)
- October (1)
- September (4)
- August (2)
- July (4)
- June (3)
- May (4)
- April (9)
- March (5)
- February (6)
- January (3)
- 2010
- December (3)
- November (6)
- October (3)
- September (1)
- August (5)
- July (1)
- June (4)
- May (1)
- April (2)
- March (2)
- February (4)
- January (2)
- 2009
- November (1)
- October (2)
- September (6)
- August (4)
- July (4)
- June (3)
- May (5)
- April (5)
- March (3)
- February (1)
- 2008
- 2005
- October (1)
- My blog
- Post new blog entry
- All blogs
Run 22 pp500 production estimates - more streams
Updated on Wed, 2024-06-05 21:48. Originally created by genevb on 2024-05-29 16:48.
Here are some very rough projections for processing Run 22 data:
NB: Run 22 data volume acquired was roughly ~50% above projected in prior Computing Plan and BUR.
Other factors:
- st_physics stream, ~650k files
- P23if production in February-March completed ~2500 jobs per day
- => Estimated total = 650k/2.5k = 260 days
- st_fwd stream, ~1100k files
- P24ia production without tracking in April-May completed ~4000 jobs per day
- tape-drive limited, but not using all drives for just this...
- ...concurrently processing BES-II data
- P23if production with tracking in January completed ~1600 jobs per day...
- ...but there were some memory issues slowing down jobs
- True dedicated rate is perhaps around ~3000 jobs per day
- => Estimated total = 1100k/3k = 370 days
- P24ia production without tracking in April-May completed ~4000 jobs per day
- st_W stream, ~50k files
- => Estimated total = 50k/2.5k = 20 days
NB: Run 22 data volume acquired was roughly ~50% above projected in prior Computing Plan and BUR.
Other factors:
- SDCC Farm OS transition to Alma 9 will happen mid-production, which may force our hand on moving (finally) to 64-bit (the above assumed currently-used 32-bit), and that should be ~20% faster
- We must be very careful to avoid and/or understand any impacts of OS change!
- Last September, I made an estimate of CPU time per job for st_physics in 64-bit, and it was ~1.35 days per job (typically), which is consistent with bringing the ~2500 jobs per day number up to ~3000 per day on 4000 production CPU cores
- Could possibly allocate more nodes for production (taken from analysis & embedding ) to reduce this by perhaps 25% at best??? Very unlikely to reduce by 50%
- BES-II re-productions will continue to use some CPU
- Run 23 AuAu200 has to be accounted in priorities
-Gene
»
- genevb's blog
- Login or register to post comments