Workaround for star-submit for large filelists on pdsf

On pdsf, especially when the disk is very full, it takes a very long time to enumerate a large file list (example, my file list of 181k files took ~5 hours to enumerate when eliza 17 was 94% full).  Therefore, I have written a small customizable perl script to star submit over a for loop, which defaults to submitting chunks of 1000 files.

Be sure to change the filelist and chunk_size arguments on the first few lines to get what you want.

This also requires a subdirectory called "fl" in the working directory where the generated filelists are saved.

The program is designed to pass along the macro, along with the startfile, endfile, filelist, and a unique identifier (to prevent overwriting the same file) to a csh script, but of course, you can always use it for whatever you like.

 On pdsf:  /home/kjung/dielec/macro/submit2.pl

 

-Kurt