Credit goes to Kevin Adkins for the User's guide below:
With the current state of storage on RCF, several users are becoming regular users of PDSF. Anselm and others have requested that I write a short introduction to PDSF. So this blog will hold the keys for successful operation on PDSF. It will expand as issues are broght forth and addressed.
Getting started at PDSF:
1. Get your username at this website: https://nim.nersc.gov/nersc_account_request.php
Once you submit this form you will receive an email that includes a link. This link will only be valid for 72 hours, and will point you to a location where you can set your password. So don't postpone! If you have trouble the email will include a phone number to call, the staff is very helpful so don't hesitate to contact them.
2. Get logged in using the same terminal command as RCF:
ssh -Y username@pdsf.nersc.gov : where username, of course, will be your username.
3. When entering your password, you only have three chances. After your third chance you'll be "locked out" and you must call to have your password reset. To avoid the hassle, make your password something you can remember!
Storage disks at PDSF:
There are two disks that STAR-spin has access to on PDSF:
/eliza14/star/pwg/starspin/
/eliza17/star/pwg/starspin/
You must email Jeff Porter ( rjporter@bnl.gov ) with your username once you can log in. He will give you access to write on these disks. Once you have the access you can create yourself a folder to write your data to on one or both of the above disks.
Transferring data to PDSF:
PDSF has two Data Transfer Nodes (DTN) that are dedicated to the transfer of data at a high rate. These are
pdsfdtn1.nersc.gov
pdsfdtn2.nersc.gov
Transferring data is best with the rsync command. As an example, assume we have several subdirectories of jets stored in /star/data05/scratch/jkadkins/run12_Jets/ on RCF. To transfer this directory as is to the directory /eliza17/star/pwg/starspin/jkadkins/ on PDSF we would use:
rsync -r -v /star/data05/scratch/jkadkins/run12_Jets jkadkins@pdsfdtn1.nersc.gov:/eliza17/star/pwg/starspin/jkadkins/
Note that I left off the "/" at the end of the run12_Jets directory above. This means that we will copy all subdirectories to PDSF in the same structure. If the directory at PDSF doesn't exist, it will be created. If there is data already in a directory of the same name on PDSF then the new data will simply be added. If we had left "/" on the end of run12_Jets then we would have copied all files and subdirectories to /eliza17/star/pwg/starspin/jkadkins and not group it into a directory named run12_Jets on PDSF. Give it a test with a few files in a directory to see exactly how this works.
Note: Transferring large volumes of data takes time. To transfer ~90 gigs of data it will take ~60 minutes. So transferring large jet trees or something similar can take a really long time. It may be best to break it up into smaller segments that are more time manageable.
Running code on PDSF:
Code runs EXACTLY the same on PDSF as it does on RCF. PDSF has the same CVS code up to date as on RCF (I'm not sure how often it's updated, but it's all there). So if you use code in CVS on RCF, then you can also use it on PDSF. The only thing that changes is that PDSF doesn't support is the STAR development library. So when running you'll need to use "starpro" or another library.
Submitting jobs is also EXACTLY the same. You'll need an XML (if it works on RCF, it'll work on PDSF without changes) and you'll use the same star-submit command that you use on RCF. The changes come when you want to check the status of your jobs. The two most common commands to manage jobs are:
qstat -u username : Check the status of all jobs you have submitted
qdel -u username : Remove all jobs you currently have submitted
A full list of queue commands can be found here: http://www.nersc.gov/users/computational-systems/pdsf/using-the-sge-batch-system/monitoring-and-managing-jobs/
Finally, problems should be reported to the PDSF hypernews ( pdsf-hn@sun.star.bnl.gov ).