Home » Groups » STAR Protected

I/O Comparison between SSD, SAS and RAM-based filesystems

Updated on Wed, 2010-04-28 17:40. Originally created by wbetts on 2010-04-27 18:03.

Primary test node:

Scientific Linux 5.3, kernel 2.6.18-164.15.1.el5 #1 SMP (x86_64)

Dell PowerEdge R610

12 GB of RAM (RAM type/speed?)

Dell PERC 6/i RAID controller (SAS/SATA).  All disks are attached to this controller.  This introduces several additional variables to the performance testing:  read policy, write policy, IO Policy (which I don't understand) and disk cache policy (strictly speaking, the disk cache policy is a disk setting, not a RAID controller setting, but it is adjustable through the RAID controller interface so I'm lumping it in here.) 

Starting with inidividual disks (no RAID):

/dev/sdb1 on /db01/ssd1 type ext3 (rw)
/dev/sdc1 on /db01/ssd2 type ext3 (rw)
/dev/sdd1 on /db01/sas1 type ext3 (rw)
/dev/sde1 on /db01/sas2 type ext3 (rw)
 

(All file systems created with 100% default/auto-determined parameters - no tuning whatsoever attempted initially.  For instance, simply "mke2fs -j /dev/sdb1" to create ext3 filesystem (see for instance output of "tune2fs -l /dev/sdb1" - block size is 4096, for instance)).  There are many options for tuning performance here that can be explored later.

 

There is also by default on SL 5.x systems a tmpfs with a size up to 1/2 of RAM (so 6GB in this case) at /dev/shm.  This is similar to a RAMdisk, but a RAMdisk's memory usage is constant, while a tmpfs only uses as much RAM as is needed to hold the files, and it automatically frees the RAM for system use when files are deleted.  It does have the slight complication though that things in a tmpfs can be swapped out to disk(!), just like anything else in RAM -- it isn't guaranteed to be in RAM.  

 

Miscellaneous notes:

Feature added in kernel 2.6.16: "/proc/sys/vm/drop_caches":

To free pagecache:

    * echo 1 > /proc/sys/vm/drop_caches

To free dentries and inodes:

    * echo 2 > /proc/sys/vm/drop_caches

To free pagecache, dentries and inodes:

    * echo 3 > /proc/sys/vm/drop_caches

As this is a non-destructive operation, and dirty objects are not freeable, the user should run "sync" first in order to make sure all cached objects are freed.

This is essentially instantaneous, and should be done before any read performance tests are done to ensure the read is not pulled from in-RAM cache (but it has no effect on RAID controller caching or disk caching).

 

 

Groups: