Embedding instructions for Embedding Helpers
This instructions provide for embedding helpers how to prepare/submit the embedding jobs at PDSF
You can also find the documentation about the embedding QA in the "Links" section.
NOTE: This is specific instructions at PDSF, some procedures may not work at RCF
Current Embedding Coordinator (EC): Terence Tarnowsky (tarnowsk@nscl.msu.edu)
Current PDSF Point Of Contact (POC): Jeff Porter (RJPorter@lbl.gov)
Last updated on Nov/8/2011 by T. Tarnowsky
Revised history (Most recent first)
-
Nov/8/2011: Updated EC and ED information to bring it in line with other webpages.
-
Apr/12/2011: Update Section 3.1, remove obsolete procedure for StPrepEmbedMaker
-
Feb/17/2011: Update Section 3.2
-
Nov/17/2010: Update to get afs token to access afs area
-
Sep/08/2010: Update memory limit in the batch at PDSF (Section 1)
-
Aug/25/2010: Update Section 3.2
-
May/25/2010: Update Section 3 (include klog, library and NOTE about VFMCE)
-
May/21/2010: Added "How to setup your environment at PDSF ?" and "Links"
-
May/20/2010: Updated instructions
Contents
-
-
-
-
-
-
Please go to the PDSF-wiki page to set up your environment at PDSF,
in case you have any troubles.
Important note:
Please have a look at the "common issue: memory limit in batch"
and follow the procedure how to increase the memory limit in the batch jobs.
All EH should make this change before submitting any embedding production jobs.
Please contact POC at PDSF to locate proper daq and tags files at eliza disks at PDSF.
3-0. Get afs token to access CVS
Below is the copy from the link above what you need to do in order to get afs token to access CVS
PDSF does not belong to the rhic.bnl.gov afs cell (the default cell is nersc),
so you have to specify the rhic.bnl.gov cell explicitly.
Additionally, your PDSF username may be different than on RACF.
If so, you need to specify your afs account name explicitly as well.
> klog -cell rhic -principal YourRCFUserName
3-1. Check out relevant macros/scripts and Makers from CVS
> cvs co StRoot/macros/embedding
> cvs co StRoot/St_geant_Maker
****** THIS PROCEDURE IS OBSOLETE, SO YOU DON'T NEED TO FOLLOW ANYMORE ******
NOTE: Under the current environment at PDSF, you will not be able to compile St_geant_Maker by cons. So you need move "StRoot/St_geant_Maker/Embed/StPrepEmbedMaker.*" one directory up, then compile by cons
> mv StRoot/St_geant_Maker/Embed/StPrepEmbedMaker* StRoot/St_geant_Maker/ > starver ${library} > cons
****************************************************************************************************************
Please don't forget to set the proper library "${library}" before compiling the source codes.
In following sections, we assume you have already did "starver ${library}".
If you are not clear which library should be used for the request, please contact EC or ED.
NOTE: Any productions prior to P07ie should be simulated under SL07e due to the lack of "VFMCE"
chain options in older libraries, even if PA's request a specific library (like SL06d).
3-2. Set up input MC particle
The input MC particle will be specified by using the geantid (see GEANT3 manual around page number 61 for pre-defined particle).
Take a look at "$STAR/pams/sim/gstar/gstar_part.g", if you cannot find the geantid for the requested particle
> cvs co pams/sim/gstar
> cons
Note that you need to check out the whole gstar directory in order to properly load gstar library.
The particle you need to simulate must be in either "$STAR/pams/sim/gstar/gstar_part.g" or
3-3. Set up bfcMixer macro
Please contact EC or ED whether bfcMixer (either bfcMixer_TpcSvtSsd.C or bfcMixer_Tpx.C) is ready to submit or not, and confirm which bfcMixer should be used for the current request.
3-4. Set up xml file for job submission
Please contact EC or ED to obtain a proper xml file for your job submission.
3-5. Other code modifications
For other code modifications needed, please check the libraries installed locally under the star embedding production account (starofl), and contact POC at PDSF for assistance.
4-1. Local sandbox
Local compiled codes/libraries are archived as "Localmakerlibs.zip" and used for each job submission.
Default local sandbox should be
<!-- Put any locally-compiled stuffs into a sand-box -->
<SandBox installer="ZIP">
<Package name="Localmakerlibs">
<File>file:./.sl44_gcc346/</File>
<File>file:./StRoot/</File>
<File>file:./pams/</File>
</Package>
</SandBox>
in your xml file. If you have anything other than above codes, please include them.
Please contact EC or ED if you are not clear enough which codes you need to include.
4-2. Submit jobs by scheduler
Once everything is prepared, you can submit jobs like
> star-submit-template -template embed_template.xml -entities FSET=200
where "embed_template.xml" is the xml file, and FSET is the unique 3 digit number
for a given request (typically starting from 100 or 101).
The embedding templates at PDSF are set up to write the output files to HPSS at the end of each job.
When all jobs in one FSET are finished, the outputs need to be retrieved from HPSS.
Please contact POC at PDSF which eliza disk can be used for your outputs.
The xml file "schedTemplateExp.xml", which will be automatically generated once you submit job by scheduler,
should be kept for each FSET job under "setup" directory.
And please don't submit the same FSET jobs, that will overwrite your previous outputs in HPSS.
4-3. Re-submitting jobs
Sometime you may need to modify something under "StRoot" or "pams", and recompile to fix some problems.
Each time you recompiled your local codes, you should clean up the current "Localmakerlibs.zip" and
"Localmakerlibs.package/" before starting resubmission. If you forgot to clean up the older "Localmakerlibs",
then the modification in the local codes will not reflect in the resubmitting jobs.
Please contact EC or ED for backing up all relevant macros, scripts and codes in the local starofl area as well as HPSS at PDSF. Please also remind to clean up all relevant LOG and LIST files that have been generated by scheduler after PWG QA is done by PA.
