ablog

不器用で落着きのない技術者のメモ

libaioでの非同期I/O発行時にプロセスはブロックされるか

fio を実行して、

[yoheia@www****** ~]$ fio -rw=randwrite -bs=4k -size=100m -directory=/tmp -direct=1 -numjobs=2 -name=file1 -ioengine=libaio
file1: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
file1: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
fio-2.0.13
Starting 2 processes
Jobs: 2 (f=2): [ww] [100.0% done] [0K/23276K/0K /s] [0 /5819 /0  iops] [eta 00m:00s]
file1: (groupid=0, jobs=1): err= 0: pid=7657: Fri Sep 27 21:53:59 2013
  write: io=102400KB, bw=12088KB/s, iops=3022 , runt=  8471msec
    slat (usec): min=11 , max=2807 , avg=36.99, stdev=39.82
    clat (usec): min=1 , max=173352 , avg=288.07, stdev=1354.32
     lat (usec): min=110 , max=173376 , avg=326.13, stdev=1355.00
    clat percentiles (usec):
     |  1.00th=[  118],  5.00th=[  141], 10.00th=[  157], 20.00th=[  181],
     | 30.00th=[  199], 40.00th=[  217], 50.00th=[  235], 60.00th=[  255],
     | 70.00th=[  278], 80.00th=[  310], 90.00th=[  366], 95.00th=[  434],
     | 99.00th=[  732], 99.50th=[ 1256], 99.90th=[ 7584], 99.95th=[18560],
     | 99.99th=[37632]
    bw (KB/s)  : min= 6288, max=14424, per=49.78%, avg=12025.50, stdev=2526.34
    lat (usec) : 2=0.01%, 4=0.10%, 20=0.01%, 50=0.02%, 100=0.15%
    lat (usec) : 250=57.70%, 500=39.20%, 750=1.85%, 1000=0.28%
    lat (msec) : 2=0.44%, 4=0.07%, 10=0.07%, 20=0.05%, 50=0.04%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=3.36%, sys=17.18%, ctx=27386, majf=0, minf=25
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=25600/d=0, short=r=0/w=0/d=0
file1: (groupid=0, jobs=1): err= 0: pid=7658: Fri Sep 27 21:53:59 2013
  write: io=102400KB, bw=12080KB/s, iops=3019 , runt=  8477msec
    slat (usec): min=11 , max=1623 , avg=36.58, stdev=33.44
    clat (usec): min=1 , max=173516 , avg=288.41, stdev=1363.08
     lat (usec): min=116 , max=173555 , avg=326.22, stdev=1364.14
    clat percentiles (usec):
     |  1.00th=[  117],  5.00th=[  141], 10.00th=[  157], 20.00th=[  179],
     | 30.00th=[  199], 40.00th=[  217], 50.00th=[  233], 60.00th=[  253],
     | 70.00th=[  278], 80.00th=[  310], 90.00th=[  370], 95.00th=[  438],
     | 99.00th=[  740], 99.50th=[ 1304], 99.90th=[ 8768], 99.95th=[17536],
     | 99.99th=[43776]
    bw (KB/s)  : min= 6344, max=14616, per=49.74%, avg=12015.50, stdev=2565.50
    lat (usec) : 2=0.01%, 4=0.06%, 10=0.01%, 20=0.01%, 50=0.02%
    lat (usec) : 100=0.16%, 250=58.10%, 500=38.70%, 750=2.00%, 1000=0.24%
    lat (msec) : 2=0.47%, 4=0.07%, 10=0.07%, 20=0.05%, 50=0.04%
    lat (msec) : 100=0.01%, 250=0.01%
  cpu          : usr=3.10%, sys=17.30%, ctx=27293, majf=0, minf=23
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=25600/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=204800KB, aggrb=24159KB/s, minb=12079KB/s, maxb=12088KB/s, mint=8471msec, maxt=8477msec

Disk stats (read/write):
  vda: ios=0/50678, merge=0/1, ticks=0/12806, in_queue=12739, util=82.55%

vmstat で様子を見ると、

[yoheia@www****** ~]$ vmstat 2
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 2  1    312 572960  91724 227320    0    0     0  7982 2061 4624  2  7 66 24  1        
 2  2    312 572960  91724 227320    0    0     0 22618 5738 13167  2 14 35 47  2       
 2  1    312 572960  91732 227312    0    0     0 26750 6600 15472  3 16 35 42  3       
 2  1    312 572960  91732 227320    0    0     0 27452 6743 15791  3 16 36 42  2       
 1  0    312 622436  91732 178520    0    0     0 17606 4321 10099  2 10 55 31  2       

あれ、思ってたのと違った。