2009

Requirements and resource planing for 2009.

CPU and bulk storage purchase 2009

The assumed CPU profile will be:

  • 2 GB of memory per core
  • Nearly 6 TB of disk space per node
  • Several CPU model will be investigated for best price/performance ratio (bulk purchase pricing matters in this purchase hence coordination between STAR/Phenix is likely needed) - currentely being considered are
    • Xeon 5550 @ 3350 SI2k (scenario A)
    • Xeon 5560 @ 3526 SI2k (scenario B)

The share between space and CPU is as below within the following caveats:

  • THe additional massive amount of storage (+170 TB for production) requires a secondary Titan head and the proper network switches. The total cost is projected to be ~  50k$ and we agreed to leave a ~ 20k$ unspent fund to move in this direction (cost shared with facility budget)

 

Experiment Parameters

Scenario A

Scenario B

Central Disk (TB) - Institution

20.00

20.00

Type Institution (Index from C&C)

11

11

Cost of Central Disk for Institution

$62,441.47

$62,441.47

Central Disk (TB) - NexSan-Production

0.00

0.00

Type NS-Prod (Index from C&C)

13

13

Cost of NexSan-Production

$0.00

$0.00

Central Disk (TB) - Production

170.00

170.00

Type of Production (Index from C&C)

12

12

Cost of Production Disk

$136,374.27

$136,374.27

Total Size of new Central Disk (TB)

190.00

190.00

Total Cost of Central Disk

$198,815.74

$198,815.74

Cost of Servers to support Central Disk

 

 

 

 

 

Compensation Disk entitled (TB)

0.00

0.00

Amount (up to entitlement) (TB)

0.00

0.00

Cost of Compensation Disk

$0

$0

Remaining Funds

$0

$0

 

 

 

Compensation count (1U, 4 GB below)

0

0

Compensation count (1U, 8 GB below)

0

0

CPU Cost

$0

$0

Distributed Disk

0.0

0.0

kSI2k

0.0

0.0

 

 

 

CPU Type (Index from Constants&Costs)

2

5

# 2U, 55xx, 5700 GB disk, 24 GB

74

72

CPU Alternative (not used)

0

0

CPU Cost

$429,126

$427,680

Distrib. Disk on new machines (TB)

421.8

410.4

kSI2k new

1983.2

2031.0

Total Disk (TB)

1393.8

1382.4

Total CPU (kSI2000)

4303.2

4351.0

Total Cost

$627,942

$626,496

Outside Funds Available

$62,441

$62,441

Funds Available

$588,000

$588,000

Unspent Funds

$22,500

$23,946

 

 

Disk space for FY09

Institution disk space

The below is what was gathered as the call sent to starsoft "Inquiry - institutional disk space for FY09" (with delay, a copy was sent to starmail on the 14th of April 2009). The deadline was provided as the end of Tuesday the 14th 2009, feedback was accepted until Wednesday the 15th (anything afterward could have been ignored).

 

Institution # TB confirmed
LBNL 5 April 21st 17:30
BNL hi 2 [self]
BNL me 1  [self]
NPI/ASCR 3 April 22nd 05:54
UCLA 1  
Rice 4 April 21st 18:47
Purdue 1 April 22nd 15:12
Valpo 1 April 22nd 17:59
MIT 2 April 22nd 15:56
Total 20  

The pricing on the table is as initially advertised i.e. a BlueArc Titan 3200 based solution at 4.3 k$/ TB for fiber channel based storage. For a discussion of fiber channel versus SATA, please consult this posting in starsofi. A quick performance overview of the Titan 3200 is showed below:

  Titan 3200
IOPS 200,000
Throughput Up to 20Gbps (2.5 GB/sec)
Scalability Up to 4PB in a single namespace
Ethernet Ports 2 x 10GbE or 6 x GbE
Fibre Channel Ports Eight 4Gb
Clustering Ports

Two 10GbE

Solution enables over 60,000 user sessions and thousands of compute nodes to be served concurrently.

The first scalability statement is over the top comparing to RHIC/STAR need but the second is by far reached at the RCF environment.

Production space

SATA based solution will be priced at 2.2 k$ / TB. While the price is lower than the fiber channel solution (and may be tempting), this solution is NOT recommended for institutional disk as the scalability for read IO at the level we are accustom to is doubtful (doubtful is probably an under-statement as we know by 5 years ago experience we will have to apply IO throttling).

As a space for production however (and considering resource constrained demanding cheaper solutions coupled with a Xrootd fast IO based aggregation solution which will remain the primary source of data access to users), the bet is that it will work if used as a buffer space (production jobs write locallyto the worker nodes, move files to central disk at the end as an additional copy along an HPSS data migration). There will be minimal guarantees of read performance access for analysis on those "production reserved" storage.

One unit of Thumper at 20k$ / 33 TB usable will be also purchased and tried out in special context. This solution is even less scalable and hence, requires a reduced amount of users and IO. The space targeted for this lower end may include (TBC):

  • data06 & data07 (2 TB) - reserved for specific projects and not meant for analysis, performance would not an issue
  • data08                (2 TB) - meant for Grid, IO is minimal there but we may need to measure data transfers compatible with KISTI based production
  • /star/rcf               (5 TB) - production log space (delayed IO, mostly a one time saving and will be fine)

Final breakdown

 

Post procurement 1 space topology

Following the Disk space for FY09, here is the new space topology and space allocation. 

BlueArc01   BlueArc02   BlueArc04  
           
STAR-FS01  Space STAR-FS03 Space STAR-FS05 Space
star_institutions_emn  2.0 star_data05  3.0 star_grid  0.5
star_institutions_lbl  14.0 star_data13  2.5 star_starlib  0.25
star_institutions_lbl_prod  5.0 star_data34  2.5 star_u  1.6
star_institutions_mit  3.0 star_data35  2.5    
star_institutions_rice  5.0 star_data53  2.5 STAR-FS06  Space
    star_data54  2.5 star_data01  2.2
STAR-FS02  Space star_data55  2.5 star_data02  2.2
star_data03  2.5     star_data06  1.0
star_data04  2.0 STAR-FS04  Space star_data14  1.0
star_data08  2.0 star_data22  2.0 star_data15  1.0
star_data09  2.0 star_data27  1.5 star_data16  2.0
star_institutions_bnl  6.0 star_institutions_psu  1.0 star_data38  2.0
star_institutions_bnl_me  1.0 star_institutions_purdue  1.0 star_data39  2.0
star_institutions_iucf  1.0 star_institutions_ucla  4.0 star_simu  3.0
star_institutions_ksu  1.0 star_institutions_uky  1.0    
star_institutions_npiascr  3.0 star_institutions_uta  1.0 STAR-FS07  Space
star_institutions_valpo  1.0 star_institutions_vecc  2.0 star_data07  0.89
    star_rcf  3.0 star_data10  0.89
        star_data12  0.76
        star_data17  0.89
        star_data24  0.89
        star_data28  0.89
        star_data29  0.89
        star_data30  0.89
        star_data32  1.75
        star_data33  0.89
        star_data37  1.66
        star_data42  1.66
        star_data44  1.79
        star_data45  1.66

 

Projects & proposals

This page is under constructions. Most projects are stil under the Projects and proposals page and not revised.

  • Supplemental funds were requested from DOE to help with infrastructure issues for both STAR & Phenix (and in predicion of a difficult FY10 funding cycle). The document is attached below as Supplemental-justification-v0 7.jl_.pdf
  • CloudSpan: Enabling Scientific Computing Across Cloud and Grid Platforms proposal was granted a Phase-I SBIR. This proposal is made in collaboration with Virkaz Tech.
  • Customizable Web Service for Efficient Access to Distributed Nuclear Physics Relational Databases proposal was granted a Phase-II award.