Differences between SL09g and SL10j for run9 pp500 - II

 This is the second installment of investigations into the differences in the number of Ws that we see in the SL09g production and the SL10j production. Right now, the current hypothesis is that the difference is caused by some shift in tracking at the global level because the loss of the W's was spread fairly evenly across the length of the run as well as in kinematic phase space. This could either be caused by a software problem or a DB problem. The easier one to investigate is the software problem so I will investigate the impact of any software changes first. So working with this hypothesis I ran a single W daq file using every library from SL09g to SL10j (with the DB timestamp used in the SL09g production, DbV20091225). Major changes were seen when going from SL10f to SL10g. If I look at the first 200 events in the MuDst, I find the following for SL10f

 

Number of global tracks (for pT > 20) = 228
Primary pT track list (for pT > 20) - (there are 14 of them)
StGMaker:INFO  - Joe : high pT = 41.332
StGMaker:INFO  - Joe : high pT = 68.0035
StGMaker:INFO  - Joe : high pT = 32.1336
StGMaker:INFO  - Joe : high pT = 21.1924
StGMaker:INFO  - Joe : high pT = 27.3568
StGMaker:INFO  - Joe : high pT = 50.8374
StGMaker:INFO  - Joe : high pT = 39.8534
StGMaker:INFO  - Joe : high pT = 21.5699
StGMaker:INFO  - Joe : high pT = 32.4973
StGMaker:INFO  - Joe : high pT = 72.7028
StGMaker:INFO  - Joe : high pT = 33.3604
StGMaker:INFO  - Joe : high pT = 29.7052
StGMaker:INFO  - Joe : high pT = 21.1069
StGMaker:INFO  - Joe : high pT = 24.1049

 

 
And for SL10g I find the following
 
Number of global tracks (for pT > 20) = 173
Primary pT track list (for pT > 20) - (there are 11 of them)
StGMaker:INFO  - Joe : high pT = 32.1336
StGMaker:INFO  - Joe : high pT = 21.1924
StGMaker:INFO  - Joe : high pT = 27.3568
StGMaker:INFO  - Joe : high pT = 105.599
StGMaker:INFO  - Joe : high pT = 21.5699
StGMaker:INFO  - Joe : high pT = 32.4973
StGMaker:INFO  - Joe : high pT = 30.8659
StGMaker:INFO  - Joe : high pT = 31.2732
StGMaker:INFO  - Joe : high pT = 27.7631
StGMaker:INFO  - Joe : high pT = 28.0248
StGMaker:INFO  - Joe : high pT = 24.1049
 
 
Here are some observations from looking at the data. This is a large reduction in high pT tracks when going from SL10f to SL10g. Some of the tracks are in the same event, but with very different pTs and some are in different events. There are global tracks in SL10f that were not found in SL10g that had a large number of hit points (on the order of 40 out of 45). So at first glance, the differences in these libraries seem to be a good candidate for the reduction in the number of tracks. The next thing is to look into what changed in the library. So looking at http://drupal.star.bnl.gov/STAR/comp/sofi/soft-n-libs/library-release-history/2010 we can see what the changes were between SL10f and SL10g and we can see that there were changes to Sti. Now to test whether it is the changes to Sti that cause the problem, I compiled a hybrid library where I used the Sti code from SL10f but compiled in SL10g and then ran the BFC over the daq file with this library and when I do, I see identical output to what I see above in SL10f so this code change is the cause of the difference between the two libraries. If I then do a diff of all the Sti code between SL10f and SL10g I get the following relevant differences (one of which has to be the culprit). There are a number of changes mainly in a number of hardcoded numbers for the Kalman filter. Right now it is uncertain which of these (or perhaps more than one) is causing the reduction. This will be investigated next. 
 
 /afs/rhic.bnl.gov/star/packages/SL10f/StRoot/Sti/StiKalmanTrack.cxx
3,4c3,4
<  * $Id: StiKalmanTrack.cxx,v 2.117 2010/02/17 14:28:07 fisyak Exp $
<  * $Id: StiKalmanTrack.cxx,v 2.117 2010/02/17 14:28:07 fisyak Exp $
---
>  * $Id: StiKalmanTrack.cxx,v 2.118 2010/04/03 04:04:57 perev Exp $
>  * $Id: StiKalmanTrack.cxx,v 2.118 2010/04/03 04:04:57 perev Exp $
8a9,11
>  * Revision 2.118  2010/04/03 04:04:57  perev
>  * Account field=0
>  *
1591c1594
<     P._ptin = (hh)? curv*hh:1e-6;
---
>     P._ptin = (hh)? curv*hh:1e-3;
 
/afs/rhic.bnl.gov/star/packages/SL10f/StRoot/Sti/StiKalmanTrackNode.cxx
3c3
<  * $Id: StiKalmanTrackNode.cxx,v 2.128 2010/04/01 20:25:34 perev Exp $
---
>  * $Id: StiKalmanTrackNode.cxx,v 2.129 2010/04/03 04:02:29 perev Exp $
7a8,10
>  * Revision 2.129  2010/04/03 04:02:29  perev
>  * Account field=0
>  *
474c477
<
---
>   mHz = 999;
552c555
<   return (fabs(mFP._ptin)<1e-6) ? 1e6: 1./fabs(mFP._ptin);
---
>   return (fabs(mFP._ptin)<1e-3) ? 1e3: 1./fabs(mFP._ptin);
572c575
<    if (mHz) return mHz;
---
>    if (mHz<999) return mHz;
576c579
<    if (fabs(h[2]) < ZEROHZ) h[2]=3e-33;
---
>    if (fabs(h[2]) < ZEROHZ) h[2]=0;
876c879
<   if (StiKalmanTrackFinderParameters::instance()->mcsCalculated() && fabs(getHz())>1e-5 )
 
---
>   if (StiKalmanTrackFinderParameters::instance()->mcsCalculated() && getHz())
1002c1005
<     if (fabs(dsin) < 0.02 && mgP.cosCA1 >0 && mgP.cosCA2 >0) { //tiny angle
---
>     if (fabs(dsin) < 0.02 ) { //tiny angle
1003a1007
>
1024c1028,1030
<   mFP._curv    = mFP._hz*mFP._ptin;
---
>   if (fabs(mFP._hz) > 1e-10)  { mFP._curv = mFP._hz*mFP._ptin;}
>   else                                { mFP._curv = 1e-6 ;}
>1319c1325
< #if 0
---> #if 1
1533a1540
>   if (fabs(pti) < 1e-3) pti=1e-3;
1687c1694
<       << " p3:" << n.mFP._ptin
---
>      << " p3:" << n.mFP._ptin
 
/afs/rhic.bnl.gov/star/packages/SL10f/StRoot/Sti/StiNodePars.h
6c6,7
< void ready(){_cosCA=cos(_eta);_sinCA=sin(_eta);_curv = _hz*_ptin;}
---
> void ready(){_cosCA=cos(_eta);_sinCA=sin(_eta);_curv = _hz*_ptin;
>              if (fabs(_curv) < 1e-6) _curv=1e-6;}
 
/afs/rhic.bnl.gov/star/packages/SL10f/StRoot/Sti/StiTrackNode.cxx
20c20
< static const double MIN1ERR[]={1e-5,1e-5,1e-5,1e-7,1e-7,1e-7};
---
> static const double MIN1ERR[]={1e-5,1e-5,1e-5,1e-7,0,1e-7};
 
/afs/rhic.bnl.gov/star/packages/SL10f/StRoot/Sti/StiTrackNodeHelper.cxx
651a652
>   mMcs._ptinCorr =  0;
 
/afs/rhic.bnl.gov/star/packages/SL10f/StRoot/StiMaker/StiMaker.h
49c49
<     {static const char cvs[]="Tag $Name: SL10f $ $Id: StiMaker.h,v 2.29 2010/01/27 21:43:49 perev Exp $ built "__DATE__" "__TIME__; return cvs;}
---
>     {static const char cvs[]="Tag $Name: SL10g $ $Id: StiMaker.h,v 2.29 2010/01/27 21:43:49 perev Exp $ built "__DATE__" "__TIME__; return cvs;}
 
 
 
Further investigation has tracked the problem down to a single one of these files and 1 line of code
 
/afs/rhic.bnl.gov/star/packages/SL10f/StRoot/Sti/StiNodePars.h
6c6,7
< void ready(){_cosCA=cos(_eta);_sinCA=sin(_eta);_curv = _hz*_ptin;}
---
> void ready(){_cosCA=cos(_eta);_sinCA=sin(_eta);_curv = _hz*_ptin;
>              if (fabs(_curv) < 1e-6) _curv=1e-6;}
 

While it is now thought this change is the issue, it isn't understood why it is the issue.