CMS Analysis Facility

MIT’s CMS Analysis Facility is a large Tier-2 computing center built for CMS user analyses. We’re looking into the viability of using it for STAR computing.

Database Mirror

MIT has a local slave connected to the STAR master database server.  A dbServers.xml with the following content will allow you to connect to it:

For more information on selecting database mirrors please visit this page.  You can also view a heartbeat of all the STAR database slaves here.  Finally, if you're interested in setting up your own database slave, Michael DePhillips has put some preliminary instructions on the

Guidelines For MIT Tier2 Job Requests

In order to facilitate the submission of jobs, all requests for the Tier2 must contain the following information.  Note that, because we cannot maintain stardev on Tier2, all jobs must be run from a tagged release.  It is the users responsibility to ensure that the requested job runs from a tagged release, with any necessary updates from CVS made explicit.

1.  Tagged release of the STAR environment from which the job will be run, e.g. SL08a.

2.  Link to all custom macros and/or  kumacs.

3.  Link to pams/ and StRoot/ directories containing any custom code, including all necessary CVS updates of the tagged release.

5.  List of commands to be executed, i.e. the contents of the <job></job> in your submission XML.

One is also free to include a custom log4j.xml, but this is not necessary.

MIT Simulation Productions

STAR environment on OS X

This page is obsolete -- please see Mac port of STAR offline software for the current status

In order of decreasing importance:

1. pams - still can't get too far here.  No idea how the whole gcc -> agetof -> g77 works to compile "Mortran".  I know VMC is the future and all that, but I think we really do need pams in order to have a useful STAR cluster.
2. dynamic library paths - specifying a relative object pathname to g++ -o means that the created dylib always looks for itself in the current directory on OS X.  In other words, the repository is useless.  Need to figure out how to tell cons to send the absolute path when linking.  Executables work just fine; it's only the .dylibs that have this problem.
3. starsim - crashing on startup (!!!!! ZFATAL called from MZIOCH) Hopefully this is related to pams problems, although I do remember having some trouble linking.
4. root4star - StarRoot,StUtilities,StarClassLibrary,St_base do not load automatically as I thought they were supposed to.  How do we do this at BNL?
5. QtRoot - has it's own build system that didn't work out of the box for me.  Disabled StEventDisplayMaker and St_geom_Maker until I figure this out.

Contents of $OPTSTAR

I went through the list of required packages in /afs/rhic.bnl.gov/star/common/AAAREADME and figured out which ones were installed by default in an Intel OS X 10.4.8 client.  Here's what I found:

• perl 5.8.6:  /usr/bin/perl (slightly newer than requested 5.8.4)
• make 3.8.0:  /usr/bin/make -> gnumake
• tar (??):  /usr/bin/tar
• flex 2.5.4:  /usr/bin/flex
• libXpm 4.11:  /usr/X11R6/lib/libXpm.dylib
  
• libpng:  not found
• mysql:  not found
• gcc 4.0.1: /usr/bin/gcc -> gcc-4.0 (yeah, I know.  Apple does not support gcc 3.x in 10.4 for Intel!  We can do gcc_select to go back to 3.3 on ppc though.)
• dejagnu:  not found
• gdb 6.3.50:  /usr/bin/gdb (instead of 5.2)
• texinfo:  not found
• emacs 21.2.1:  /usr/bin/emacs (instead of 20.7)
• findutils:  not found
• fileutils:  not found
• cvs 1.11:  /usr/bin/cvs
• grep 2.5.1:  /bin/grep (instead of 2.5.1a)
• m4 1.4.2:  /usr/bin/m4 (instead of 1.4.1)
• autoconf 2.59:  /usr/bin/autoconf (2.53)
• automake 1.6.3:  /usr/bin/automake
• libtool (??):  /usr/bin/libtool (1.5.8)

I was able to find nearly all of the missing packages in the unstable branch for Fink (Intel machine).  I wouldn't worry about the "unstable" moniker; as long as you don't do a blind update-all it's certainly possible to stick to a solid config, and there are several packages on the list that are only available in unstable (only because they haven't yet gotten the votes to move them over to stable).  I've gone ahead and installed some of the missing packages in a fresh Fink installation and will serve it up over NFS at /Volumes/star1.lns.mit.edu/STAR/opt/star/osx48_i386_gcc401 (with a power_macintosh_gcc401 to match, although a more consistent $STAR_HOST_SYS would probably have been osx48_ppc_gcc401).

Here's a summary table of the packages installed in $OPTSTAR for the two OS X architectures at MIT.  Note that many of these packages have additional dependencies, so the full list of installed packages on each system (attached at the bottom of the page) is actually much longer.

I also looked for required perlmods in Fink.  I stuck with the default Perl 5.86, so the modules that say e.g. pm588 required I did not install.  I found that some of the modules are already part of core.  If the older ones hosted by STAR are still needed, let me know.  Virtual package means that it came with the OS already:

There were some additional perlmods that install_perlmods listed as "Linux only" but Fink offered to install:

Questions:

• what was with all those soft-links (/usr/bin/sed -> /bin/sed, etc.) that Jerome had me make?  Will they be needed on every machine running STAR environment (that's a problem), or just on the one he was compiling on?
• is perl in /usr/bin sufficient or do we need to put it in $OPTSTAR as directed in AAAREADME?
• what to do about mysql? Is 5.0 back-compatible, or do we only need development headers and shared libraries?

Building PYTHIA dylibs with gfortran

The default makePythia6.macosx won't work out of the box for 10.4, since it requires g77.  Here's what I did to get the libraries built for Pythia 5:
$ gfortran -c jetset74.f $ gfortran -c pythia5707.f $ echo 'void MAIN__() {}' > main.c $ gcc -c main.c $ gcc -dynamiclib -flat_namespace -single_module -undefined dynamic_lookup -install_name $OPTSTAR/lib/libPythia.dylib -o libPythia.dylib *.o $ sudo cp libPythia.dylib $OPTSTAR/lib/. and for Pythia 6: $ export MACOSX_DEPLOYMENT_TARGET=10.4 $ gfortran -c pythia6319.f In file pythia6319.f:50551 IF (AAMAX.EQ.0D0) PAUSE 'SINGULAR MATRIX IN PYLDCM' 1 Warning: Obsolete: PAUSE statement at (1) $ gfortran -fno-second-underscore -c tpythia6_called_from_cc.F $ echo 'void MAIN__() {}' > main.c $ gcc -c main.c $ gcc -c pythia6_common_address.c $ gcc -dynamiclib -flat_namespace -single_module -undefined dynamic_lookup -install_name $OPTSTAR/lib/libPythia6.dylib -o libPythia6.dylib main.o tpythia6_called_from_cc.o pythia6*.o $ ln -s libPythia6.dylib libPythia6.so $ sudo cp libPythia6.* $OPTSTAR/lib/.

CERNLIB notes

All the CERNLIB libraries are static and the binaries depend only on system libraries, so the whole installation should be portable.  For PowerPC I had a CERNLIB 2005 build left over from a different Fink installation, so I just copied those binaries and libraries to the new location and downloaded the headers from CERN.  Fink doesn't support CERNLIB on Intel Macs, so for this build I used Robert Hatcher's excellent shell script:

http://home.fnal.gov/~rhatcher/macosx/readme.html

Hatcher's binaries link against the gfortran dylib, so I made sure to build them with gfortran from $OPTSTAR.

CERNLIB 2005 doesn't include libshift.a, but STAR really wants to link against it.  Here's a hack from Robert Hatcher to build your own cat > fakeshift.c < eof int rshift_(int* in, int* ishft) { return *in >> *ishft; } int ishft_(int* in, int* ishft) { if (*ishft == 0) return *in; if (*ishft > 0) return *in << *ishft; else return *in >> *ishft; } EOF gcc -O -fPIC -c fakeshift.c fi g77 -fPIC -c getarg_stub.f ar cr libshift.a fakeshift.o eof

ROOT build notes

Following the instructions at http://www.star.bnl.gov/STAR/comp/root/building_root.html was basically fine.  Here was my configure command for rootdeb:
./configure macosx --build=debug --enable-qt --enable-table --enable-pythia6 --enable-pythia --with-pythia-libdir=$OPTSTAR/lib --with-pythia6-libdir=$OPTSTAR/lib --with-qt-incdir=$OPTSTAR/include/qt which resulted in the final list Enabled support for asimage, astiff, builtin_afterimage, builtin_freetype, builtin_pcre, builtin_zlib, cern, cintex, exceptions, krb5, ldap, mathcore, mysql, odbc, opengl, pch, pythia, pythia6, python, qt, qtgsi, reflex, shared, ssl, table, thread, winrtdebug, xml, xrootd. I did run into a few snags:

• MakeRootDir.pl didn't find my /usr/X11R6/bin/lndir automatically (even though that was in my $PATH) so I had edit the script and do it manually.
• Had to run MakeRootDir.pl twice to get root and rootdeb directory structures in place, editing the script in between.
• CVS was a mess.  I had to drill down into each subdirectory that needed updating, and even then it puked out conflicts instead of patching the files, so I had to trash the originals first.  Also, I'm fairly sure that root5/qt/inc/TQtWidget.h should have been included in the v5-12-00f tag, since my first attempt at compiling failed without the HEAD version of that file.

Hacking the environment scripts

• set rhflavor = "osx48_" in STAR_SYS to get the name I chose for $STAR_HOST_SYS
• I installed Qt in $OPTSTAR, so group_env.csh fails to find it

Building STAR software

I'm working with a checked out copy of the STAR software and modifying codes when necessary if the fix is obvious.  So far I've got the following cons working: cons %QtRoot %StEventDisplayMaker %pams %St_dst_Maker %St_geom_Maker St_dst_Maker tries to subtract an int and a struct!  Pams is a crazy mess of VAX-style Fortran STRUCTURES, but we really need it in order to run starsim.  I haven't delved too deeply into the QtRoot-related stuff; I'm sure Valeri can help when the time comes.  Hopefully we can get these things fixed without too much delay.

Power PC notes

• why does everything insist on linking with libshift?  It's not a part of CERNLIB 2005, so I used Hatcher's hack to get around it and stuck libshift.a in $OPTSTAR/lib
• libnsl is not needed on OS X, so we don't link against it anymore
• remove -dynamiclib and -single_module for executables
• cfortran.h can't identify our Fortran compiler -- define it as f2c
• asps/Simulation/starsim/deccc/fputools.c won't compile under power pc (contains assembly code!) -- skip it for now
• g++ root4star brings out lots of linking issues; one killer seems to be that libpacklib from Fink is missing fzicv symbol.
  • one very hack solution:  install gfortran, use it to build CERNLIB with Hatcher script, replace libpacklib.a, copy libgcc.a and libgfortran.a from gcc 4.2.0 into $OPTSTAR/lib or other, then link against them explicitly
  • needed to -lstarsim to get gufile, srndmc symbols defined
• <malloc.h> -- on Mac they decided to put this in /usr/include/malloc, so we add this to path in ConsDefs.pm
• cons wanted to link starsim using gcc and statically include libstdc++; on Mac we'll let g++ do the work.  Also, -lstarsim seems to be included too early in the chain.  Need to talk to Jerome about proper way to fix this, but for now I can hack a fix.
• PAMS -- ACK!

Problems requiring changes to codes:

• struct mallinfo isn't available on OS X
  • for now we surround any mallinfo with #ifndef __APPLE__; Frank Laue says there may be a workaround
• 'fabs' was not declared in this scope
  • add <cmath> in header
• TCL.h from ROOT conflicts with system tcl.h because of case-insensitive FS
  • TCL.h renamed to TCernLib.h in newer ROOT versions (ROOT bug 19313)
  • copied TCL.h to TCernLib.h myself and added #ifdef __APPLE__ #include "TCernLib.h"
  • this problem will go away when we patch/upgrade ROOT
• passing U_Int to StMatrix::inverse() when it wants a size_t
  • changed input to size_t (only affected StFtpcTrackingParams)
• abs(float) is not legal
  • change to fabs(float) and #include <cmath>

Intel notes

Basic problem here is (im)maturity of gfortran.  Current Fink unstable version 4.2.0-20060617 still does not include some instrinsic symbols (lshift, lstat) that we expect to be there.  Newer versions do have these symbols, and as soon as Fink updates I'll give it another go.  I may try installing gcc 4.3 from source in the meantime, but it's not a high priority.  Note that Intel machines should be able to run the Power PC build in translated mode with some hacking of the paths (force $STAR_HOST_SYS = osx48_power_macintosh_gcc401).

Xgrid