Home » Blogs » veprbl's blog

Disk performance

Updated on Mon, 2020-09-14 12:44. Originally created by veprbl on 2018-04-24 17:56.
/star/u on RACF 
veprbl@rcas6006 ~ % w | wc -l
25
veprbl@rcas6006 ~ % fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=4G --readwrite=randrw
test: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=64
fio-3.5
Starting 1 process
Jobs: 1 (f=1): [m(1)][100.0%][r=2900KiB/s,w=2932KiB/s][r=725,w=733 IOPS][eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=17783: Tue Apr 24 18:41:33 2018
   read: IOPS=651, BW=2607KiB/s (2670kB/s)(2049MiB/804858msec)
   bw (  KiB/s): min=  311, max= 4896, per=100.00%, avg=2607.42, stdev=636.11, samples=1609
   iops        : min=   77, max= 1224, avg=651.81, stdev=159.02, samples=1609
  write: IOPS=650, BW=2604KiB/s (2666kB/s)(2047MiB/804858msec)
   bw (  KiB/s): min=  280, max= 4720, per=100.00%, avg=2604.16, stdev=638.25, samples=1609
   iops        : min=   70, max= 1180, avg=651.00, stdev=159.55, samples=1609
  cpu          : usr=0.79%, sys=1.73%, ctx=1549241, majf=0, minf=723
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwts: total=524625,523951,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=2607KiB/s (2670kB/s), 2607KiB/s-2607KiB/s (2670kB/s-2670kB/s), io=2049MiB (2149MB), run=804858-804858msec
  WRITE: bw=2604KiB/s (2666kB/s), 2604KiB/s-2604KiB/s (2666kB/s-2666kB/s), io=2047MiB (2146MB), run=804858-804858msec
/star/u on RACF from EIC 
veprbl@eic0006 ~ % fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=4G --readwrite=randrw
test: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=64
fio-3.5
Starting 1 process
Jobs: 1 (f=1): [m(1)][100.0%][r=45.8MiB/s,w=45.6MiB/s][r=11.7k,w=11.7k IOPS][eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=22813: Tue Apr 24 19:00:53 2018
   read: IOPS=8848, BW=34.6MiB/s (36.2MB/s)(2049MiB/59290msec)
   bw (  KiB/s): min= 3312, max=55048, per=99.90%, avg=35356.36, stdev=12689.72, samples=118
   iops        : min=  828, max=13762, avg=8839.08, stdev=3172.45, samples=118
  write: IOPS=8837, BW=34.5MiB/s (36.2MB/s)(2047MiB/59290msec)
   bw (  KiB/s): min= 3464, max=54520, per=99.88%, avg=35303.91, stdev=12649.22, samples=118
   iops        : min=  866, max=13630, avg=8825.95, stdev=3162.33, samples=118
  cpu          : usr=8.24%, sys=17.45%, ctx=1398346, majf=0, minf=491
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwts: total=524625,523951,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=34.6MiB/s (36.2MB/s), 34.6MiB/s-34.6MiB/s (36.2MB/s-36.2MB/s), io=2049MiB (2149MB), run=59290-59290msec
  WRITE: bw=34.5MiB/s (36.2MB/s), 34.5MiB/s-34.5MiB/s (36.2MB/s-36.2MB/s), io=2047MiB (2146MB), run=59290-59290msec
/eic/u on RACF from EIC 
veprbl2@eic0006 ~ % fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=4G --readwrite=randrw
test: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=64
fio-3.5
Starting 1 process
test: Laying out IO file (1 file / 4096MiB)
Jobs: 1 (f=1): [m(1)][100.0%][r=54.1MiB/s,w=53.8MiB/s][r=13.9k,w=13.8k IOPS][eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=23337: Tue Apr 24 19:03:50 2018
   read: IOPS=13.2k, BW=51.6MiB/s (54.1MB/s)(2049MiB/39706msec)
   bw (  KiB/s): min=33424, max=57160, per=100.00%, avg=52848.56, stdev=4728.07, samples=79
   iops        : min= 8356, max=14290, avg=13212.10, stdev=1182.03, samples=79
  write: IOPS=13.2k, BW=51.5MiB/s (54.0MB/s)(2047MiB/39706msec)
   bw (  KiB/s): min=32664, max=56952, per=99.98%, avg=52770.99, stdev=4799.35, samples=79
   iops        : min= 8166, max=14238, avg=13192.73, stdev=1199.83, samples=79
  cpu          : usr=9.28%, sys=21.76%, ctx=1322936, majf=0, minf=493
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwts: total=524625,523951,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=51.6MiB/s (54.1MB/s), 51.6MiB/s-51.6MiB/s (54.1MB/s-54.1MB/s), io=2049MiB (2149MB), run=39706-39706msec
  WRITE: bw=51.5MiB/s (54.0MB/s), 51.5MiB/s-51.5MiB/s (54.0MB/s-54.0MB/s), io=2047MiB (2146MB), run=39706-39706msec
$HOME on PDSF 
veprbl@pdsf8 ~ % w | wc -l
73
veprbl@pdsf8 ~ % fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=4G --readwrite=randrw
test: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=64
fio-2.99
Starting 1 process
Jobs: 1 (f=1): [m(1)][100.0%][r=7732KiB/s,w=7888KiB/s][r=1933,w=1972 IOPS][eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=18708: Tue Apr 24 15:25:11 2018
   read: IOPS=1800, BW=7204KiB/s (7377kB/s)(2049MiB/291302msec)
   bw (  KiB/s): min= 4079, max=15048, per=100.00%, avg=7210.26, stdev=1222.49, samples=582
   iops        : min= 1019, max= 3762, avg=1802.55, stdev=305.62, samples=582
  write: IOPS=1798, BW=7195KiB/s (7367kB/s)(2047MiB/291302msec)
   bw (  KiB/s): min= 3928, max=15480, per=100.00%, avg=7201.25, stdev=1259.82, samples=582
   iops        : min=  982, max= 3870, avg=1800.30, stdev=314.96, samples=582
  cpu          : usr=1.55%, sys=6.09%, ctx=887960, majf=0, minf=10
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwt: total=524625,523951,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=7204KiB/s (7377kB/s), 7204KiB/s-7204KiB/s (7377kB/s-7377kB/s), io=2049MiB (2149MB), run=291302-291302msec
  WRITE: bw=7195KiB/s (7367kB/s), 7195KiB/s-7195KiB/s (7367kB/s-7367kB/s), io=2047MiB (2146MB), run=291302-291302msec
"Seagate Backup Plus" archive drive formatted to HFS+ (USB 3.0, macOS) 
fio --randrepeat=1 --ioengine=posixaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=4G --readwrite=randrw
test: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=64
fio-3.16
Starting 1 process
test: Laying out IO file (1 file / 4096MiB)
Jobs: 1 (f=1): [m(1)][100.0%][r=380KiB/s,w=348KiB/s][r=95,w=87 IOPS][eta 00m:00s]     
test: (groupid=0, jobs=1): err= 0: pid=35061: Wed Nov 27 16:51:57 2019
  read: IOPS=52, BW=208KiB/s (213kB/s)(2049MiB/10075205msec)
   bw (  KiB/s): min=    7, max= 8440, per=100.00%, avg=389.03, stdev=1005.35, samples=10713
   iops        : min=    1, max= 2110, avg=96.77, stdev=251.40, samples=10713
  write: IOPS=52, BW=208KiB/s (213kB/s)(2047MiB/10075205msec)
   bw (  KiB/s): min=    7, max= 7832, per=100.00%, avg=392.87, stdev=1005.99, samples=10594
   iops        : min=    1, max= 1958, avg=97.74, stdev=251.56, samples=10594
  cpu          : usr=0.17%, sys=0.13%, ctx=547457, majf=0, minf=23
  IO depths    : 1=0.1%, 2=0.1%, 4=0.5%, 8=60.4%, 16=39.1%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=97.0%, 8=2.7%, 16=0.2%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=524625,523951,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=208KiB/s (213kB/s), 208KiB/s-208KiB/s (213kB/s-213kB/s), io=2049MiB (2149MB), run=10075205-10075205msec
  WRITE: bw=208KiB/s (213kB/s), 208KiB/s-208KiB/s (213kB/s-213kB/s), io=2047MiB (2146MB), run=10075205-10075205msec
MacBook Pro (Mid 2014) SSD 
fio --randrepeat=1 --ioengine=posixaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=4G --readwrite=randrw
test: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=64
fio-3.16
Starting 1 process
Jobs: 1 (f=1): [m(1)][100.0%][r=159MiB/s,w=159MiB/s][r=40.8k,w=40.6k IOPS][eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=35781: Wed Nov 27 17:03:35 2019
  read: IOPS=41.9k, BW=164MiB/s (172MB/s)(2049MiB/12527msec)
   bw (  KiB/s): min=136144, max=185056, per=99.84%, avg=167245.60, stdev=15137.70, samples=25
   iops        : min=34036, max=46264, avg=41811.28, stdev=3784.38, samples=25
  write: IOPS=41.8k, BW=163MiB/s (171MB/s)(2047MiB/12527msec)
   bw (  KiB/s): min=138552, max=184056, per=99.83%, avg=167023.08, stdev=14833.34, samples=25
   iops        : min=34638, max=46014, avg=41755.68, stdev=3708.30, samples=25
  cpu          : usr=43.16%, sys=37.34%, ctx=418983, majf=0, minf=24
  IO depths    : 1=0.1%, 2=0.1%, 4=1.5%, 8=64.6%, 16=33.8%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=96.2%, 8=3.2%, 16=0.6%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=524625,523951,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=164MiB/s (172MB/s), 164MiB/s-164MiB/s (172MB/s-172MB/s), io=2049MiB (2149MB), run=12527-12527msec
  WRITE: bw=163MiB/s (171MB/s), 163MiB/s-163MiB/s (171MB/s-171MB/s), io=2047MiB (2146MB), run=12527-12527msec
Samsung SSD in USB3.0 enclosure (macOS 10.14) 
sudo fio --randrepeat=1 --ioengine=posixaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=4G --readwrite=randrw --filename=/dev/disk2
Password:
test: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=64
fio-3.16
Starting 1 process
Jobs: 1 (f=2): [m(1)][100.0%][r=78.2MiB/s,w=78.4MiB/s][r=20.0k,w=20.1k IOPS][eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=41886: Sun Dec  1 11:51:22 2019
  read: IOPS=19.7k, BW=76.9MiB/s (80.7MB/s)(2049MiB/26635msec)
   bw (  KiB/s): min=53336, max=82688, per=99.73%, avg=78576.17, stdev=5132.29, samples=53
   iops        : min=13334, max=20672, avg=19643.85, stdev=1283.04, samples=53
  write: IOPS=19.7k, BW=76.8MiB/s (80.6MB/s)(2047MiB/26635msec)
   bw (  KiB/s): min=53576, max=85624, per=99.72%, avg=78462.70, stdev=5371.87, samples=53
   iops        : min=13394, max=21406, avg=19615.45, stdev=1343.01, samples=53
  cpu          : usr=30.97%, sys=27.63%, ctx=738350, majf=7, minf=17
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=49.8%, 16=50.2%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=98.9%, 8=1.1%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=524625,523951,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=76.9MiB/s (80.7MB/s), 76.9MiB/s-76.9MiB/s (80.7MB/s-80.7MB/s), io=2049MiB (2149MB), run=26635-26635msec
  WRITE: bw=76.8MiB/s (80.6MB/s), 76.8MiB/s-76.8MiB/s (80.6MB/s-80.6MB/s), io=2047MiB (2146MB), run=26635-26635msec
MD RAID 1 on rhicspin 
test: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=64
fio-3.7
Starting 1 process
test: Laying out IO file (1 file / 4096MiB)
Jobs: 1 (f=1): [m(1)][100.0%][r=868KiB/s,w=416KiB/s][r=217,w=104 IOPS][eta 00m:00s]  
test: (groupid=0, jobs=1): err= 0: pid=2822: Mon Sep 14 13:26:26 2020
   read: IOPS=173, BW=695KiB/s (712kB/s)(2049MiB/3017823msec)
   bw (  KiB/s): min=   24, max= 1224, per=100.00%, avg=695.66, stdev=196.42, samples=6032
   iops        : min=    6, max=  306, avg=173.89, stdev=49.11, samples=6032
  write: IOPS=173, BW=694KiB/s (711kB/s)(2047MiB/3017823msec)
   bw (  KiB/s): min=   24, max= 1584, per=100.00%, avg=694.86, stdev=257.74, samples=6031
   iops        : min=    6, max=  396, avg=173.69, stdev=64.44, samples=6031
  cpu          : usr=0.44%, sys=1.70%, ctx=1006553, majf=0, minf=26
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwts: total=524625,523951,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=695KiB/s (712kB/s), 695KiB/s-695KiB/s (712kB/s-712kB/s), io=2049MiB (2149MB), run=3017823-3017823msec
  WRITE: bw=694KiB/s (711kB/s), 694KiB/s-694KiB/s (711kB/s-711kB/s), io=2047MiB (2146MB), run=3017823-3017823msec

Disk stats (read/write):
    md124: ios=525247/529024, merge=0/0, ticks=0/0, in_queue=0, util=0.00%, aggrios=263278/527607, aggrmerge=343/3901, aggrticks=58952745/44152197, aggrin_queue=103104600, aggrutil=98.34%
  sda: ios=263678/527426, merge=441/4083, ticks=65108159/54480020, in_queue=119587477, util=98.34%
  sdb: ios=262879/527789, merge=245/3720, ticks=52797331/33824374, in_queue=86621724, util=96.04%
»