PC の HDD を SSD に変更したので、I/Oスケジューラを noop に変更してみた。
SSD は HDD のようにシーク（目的のレコードが記憶されたシリンダにヘッドを移動）したり、サーチ（目的のセクタの先頭がヘッドの位置に来るのを待つ）したり、物理的な動きはない。なので、OSのI/OスケジューラでソートなんかせずにI/Oリクエストを渡してくれたほうが速いとという話だと思う。
簡単に noop が速いことを検証した上で、noop に変更した。
cfq
- I/Oスケジューラを cfq に設定する
[root@yazekats-linux queue]# echo cfq > /sys/block/sda/queue/scheduler [root@yazekats-linux queue]# cat /sys/block/sda/queue/scheduler noop deadline [cfq]
- ランダム書込みで約1万4千IOPS
[yazekats@yazekats-linux ~]$ fio -rw=randwrite -bs=4k -size=100m -directory=/tmp -direct=1 -numjobs=50 -name=file1 -ioengine=libaio -group_reporting file1: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1 ... file1: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1 fio-2.0.13 Starting 50 processes Jobs: 32 (f=32): [wwwwwwwwwwwww__wwwwwww__w________w____ww_wwwwww_ww] [94.6% done] [0K/61548K/0K /s] [0 /15.4K/0 iops] [eta 00m:05s] file1: (groupid=0, jobs=50): err= 0: pid=18696: Sat Oct 5 22:22:26 2013 write: io=4972.2MB, bw=57929KB/s, iops=14482 , runt= 87891msec slat (usec): min=3 , max=628117 , avg=30.84, stdev=3232.73 clat (usec): min=63 , max=248454 , avg=3377.95, stdev=5661.48 lat (usec): min=71 , max=634383 , avg=3409.05, stdev=6521.86 clat percentiles (usec): | 1.00th=[ 362], 5.00th=[ 564], 10.00th=[ 716], 20.00th=[ 980], | 30.00th=[ 1224], 40.00th=[ 1496], 50.00th=[ 1800], 60.00th=[ 2224], | 70.00th=[ 2768], 80.00th=[ 3728], 90.00th=[ 6752], 95.00th=[12736], | 99.00th=[25216], 99.50th=[34560], 99.90th=[65280], 99.95th=[75264], | 99.99th=[107008] bw (KB/s) : min= 6, max= 6336, per=2.02%, avg=1168.49, stdev=325.18 lat (usec) : 100=0.01%, 250=0.19%, 500=3.16%, 750=7.95%, 1000=9.60% lat (msec) : 2=33.98%, 4=27.02%, 10=11.33%, 20=4.84%, 50=1.66% lat (msec) : 100=0.25%, 250=0.01% cpu : usr=0.19%, sys=0.55%, ctx=1273391, majf=0, minf=1366 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=0/w=1272867/d=0, short=r=0/w=0/d=0 Run status group 0 (all jobs): WRITE: io=4972.2MB, aggrb=57929KB/s, minb=57929KB/s, maxb=57929KB/s, mint=87891msec, maxt=87891msec Disk stats (read/write): dm-1: ios=0/1271214, merge=0/0, ticks=0/4238575, in_queue=4255946, util=100.00%, aggrios=0/1273218, aggrmerge=0/0, aggrticks=0/4239334, aggrin_queue=4252968, aggrutil=100.00% dm-0: ios=0/1273218, merge=0/0, ticks=0/4239334, in_queue=4252968, util=100.00%, aggrios=0/1272981, aggrmerge=0/237, aggrticks=0/4128661, aggrin_queue=4128001, aggrutil=99.93% sda: ios=0/1272981, merge=0/237, ticks=0/4128661, in_queue=4128001, util=99.93%
deadline
- I/Oスケジューラを deadline に設定する
[root@yazekats-linux queue]# echo deadline > /sys/block/sda/queue/scheduler [root@yazekats-linux queue]# cat /sys/block/sda/queue/scheduler noop [deadline] cfq
- ランダム書込みで約2万4千IOPS
[yazekats@yazekats-linux ~]$ fio -rw=randwrite -bs=4k -size=100m -directory=/tmp -direct=1 -numjobs=50 -name=file1 -ioengine=libaio -group_reporting file1: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1 ... file1: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1 fio-2.0.13 Starting 50 processes Jobs: 1 (f=1): [w_________________________________________________] [92.9% done] [0K/22717K/0K /s] [0 /5679 /0 iops] [eta 00m:04s]s] file1: (groupid=0, jobs=50): err= 0: pid=18753: Sat Oct 5 22:24:07 2013 write: io=4972.2MB, bw=99060KB/s, iops=24764 , runt= 51398msec slat (usec): min=3 , max=5227 , avg=13.18, stdev=22.32 clat (usec): min=64 , max=70703 , avg=1831.53, stdev=1629.30 lat (usec): min=70 , max=70733 , avg=1844.97, stdev=1629.54 clat percentiles (usec): | 1.00th=[ 306], 5.00th=[ 580], 10.00th=[ 772], 20.00th=[ 996], | 30.00th=[ 1192], 40.00th=[ 1384], 50.00th=[ 1576], 60.00th=[ 1816], | 70.00th=[ 2128], 80.00th=[ 2512], 90.00th=[ 3120], 95.00th=[ 3696], | 99.00th=[ 5152], 99.50th=[ 5856], 99.90th=[ 8256], 99.95th=[55040], | 99.99th=[60672] bw (KB/s) : min= 1011, max=16888, per=2.17%, avg=2152.73, stdev=576.77 lat (usec) : 100=0.02%, 250=0.52%, 500=3.14%, 750=5.65%, 1000=10.73% lat (msec) : 2=46.17%, 4=30.20%, 10=3.51%, 20=0.01%, 100=0.05% cpu : usr=0.34%, sys=0.95%, ctx=1273218, majf=0, minf=1357 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=0/w=1272867/d=0, short=r=0/w=0/d=0 Run status group 0 (all jobs): WRITE: io=4972.2MB, aggrb=99059KB/s, minb=99059KB/s, maxb=99059KB/s, mint=51398msec, maxt=51398msec Disk stats (read/write): dm-1: ios=0/1272045, merge=0/0, ticks=0/2289492, in_queue=2323351, util=99.92%, aggrios=0/1273075, aggrmerge=0/0, aggrticks=0/2288244, aggrin_queue=2311349, aggrutil=99.43% dm-0: ios=0/1273075, merge=0/0, ticks=0/2288244, in_queue=2311349, util=99.43%, aggrios=0/1272938, aggrmerge=0/137, aggrticks=0/2193051, aggrin_queue=2192778, aggrutil=98.55% sda: ios=0/1272938, merge=0/137, ticks=0/2193051, in_queue=2192778, util=98.55%
noop
- I/Oスケジューラを noop に設定する
[root@yazekats-linux queue]# echo noop > /sys/block/sda/queue/scheduler [root@yazekats-linux queue]# cat /sys/block/sda/queue/scheduler [noop] deadline cfq
- ランダム書込みで約2万6千IOPS
[yazekats@yazekats-linux ~]$ fio -rw=randwrite -bs=4k -size=100m -directory=/tmp -direct=1 -numjobs=50 -name=file1 -ioengine=libaio -group_reporting file1: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1 ... file1: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1 fio-2.0.13 Starting 50 processes Jobs: 40 (f=40): [wwwwwwwww_wwww_wwww_wwwwwwww_www_wwwwwww_ww__ww__w] [98.0% done] [0K/106.7M/0K /s] [0 /27.4K/0 iops] [eta 00m:01s] file1: (groupid=0, jobs=50): err= 0: pid=18808: Sat Oct 5 22:25:31 2013 write: io=4972.2MB, bw=104733KB/s, iops=26183 , runt= 48614msec slat (usec): min=3 , max=3314 , avg=13.18, stdev=24.50 clat (usec): min=69 , max=66421 , avg=1875.02, stdev=1603.59 lat (usec): min=77 , max=66445 , avg=1888.45, stdev=1603.80 clat percentiles (usec): | 1.00th=[ 588], 5.00th=[ 836], 10.00th=[ 948], 20.00th=[ 1112], | 30.00th=[ 1256], 40.00th=[ 1416], 50.00th=[ 1608], 60.00th=[ 1832], | 70.00th=[ 2128], 80.00th=[ 2512], 90.00th=[ 3056], 95.00th=[ 3600], | 99.00th=[ 4960], 99.50th=[ 5664], 99.90th=[ 8256], 99.95th=[55552], | 99.99th=[60160] bw (KB/s) : min= 1657, max= 3282, per=2.01%, avg=2105.35, stdev=153.88 lat (usec) : 100=0.01%, 250=0.11%, 500=0.53%, 750=2.20%, 1000=9.91% lat (msec) : 2=53.61%, 4=30.58%, 10=2.99%, 20=0.01%, 50=0.01% lat (msec) : 100=0.05% cpu : usr=0.33%, sys=0.93%, ctx=1273283, majf=0, minf=1363 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=0/w=1272867/d=0, short=r=0/w=0/d=0 Run status group 0 (all jobs): WRITE: io=4972.2MB, aggrb=104732KB/s, minb=104732KB/s, maxb=104732KB/s, mint=48614msec, maxt=48614msec Disk stats (read/write): dm-1: ios=2/1269752, merge=0/0, ticks=4/2344641, in_queue=2380245, util=100.00%, aggrios=2/1273184, aggrmerge=0/0, aggrticks=4/2344988, aggrin_queue=2368234, aggrutil=100.00% dm-0: ios=2/1273184, merge=0/0, ticks=4/2344988, in_queue=2368234, util=100.00%, aggrios=2/1272982, aggrmerge=0/202, aggrticks=4/2241698, aggrin_queue=2241401, aggrutil=99.84% sda: ios=2/1272982, merge=0/202, ticks=4/2241698, in_queue=2241401, util=99.84%
結論
やはり noop が向いてそうなので、noop に変更。/boot/grub/grub.conf に elevator=noop を追記して、
title Oracle Linux Server Unbreakable Enterprise Kernel (2.6.39-400.17.1.el6uek.x86_64) root (hd0,0) kernel /vmlinuz-2.6.39-400.17.1.el6uek.x86_64 ... quiet <span class="deco" style="color:#FF0000;">elevator=noop</span> rhgb quiet
再起動すると
[root@yazekats-linux ~]# cat /sys/block/sda/queue/scheduler [noop] deadline cfq
OK
