HPSS I/O

HTAR : save/restore private directory (large set of small files) in HPSS

For saving all files in directory 'aaa' on disk to your HPSS target bbb/ccc.tar and keep log file in ~/0x/ddd.log1 (needed if you want to kill/quit the window) do:

cd aaa

du -hs .

htar -cf bbb/ccc.tar file1 file2 fileX* ... >& ~/0x/ddd.log1 &

(wait until job is done, for 50GB make take severa hours)

hsi (goes directly to HPSS, be careful!)

?ls -l bbb/

?quit (exit hsi)

You should see 2 files : ccc.tar & ccc.tar.idx, the size of file ccc.tar should be close to the size of your oryginal directory.

The .tar file shows up immediately but sinking is NOT finished until you see .idx file.

The log file should contain the string: HTAR: HTAR SUCCESSFUL , no warnings.

However the most reliable way to verify the storage did not failed is to:

retrieving files from HPSS to disk directory eee do:

mkdir eee

cd eee

htar -xf bbb/ccc.tar >& ~/0x/ddd.log2 &

Other options of HTAR command are described here:
http://drupal.star.bnl.gov/STAR/comp/sofi/hpss/htar

Known problems:

WARNING: htar_PreallocateSpace: HPSS OUT-OF-SPACE error preallocating

**** bytes for file=[/home/salur/myfile.tar]

This is a directory with 350G. Do I have to divide my files into smaller.

there is a 60GB max file size limit [when creating tarballs], Another

limit to be aware of, however, is an 8GB limit on member files

Mass restoring of ~100GB files from HPSS using Data Carusel

Prepare input file xxx.hpss with list of source +destination paths for all files you need, one line per file

Execute the command (at any location at rcas6nnn)

hpss_user.pl -f xxx.hpss

Wait 10-1000 minutes fro the files to show up at your destination (if all paths are correct)