ablog

不器用で落着きのない技術者のメモ

YCSBで DynamoDB に負荷をかけたメモ

AWS

設定

  • YCSB/dynamodb/conf/AWSCredentials.properties
accessKey = <アクセスキー>
secretKey = <シークレットキー>
  • YCSB/dynamodb/conf/dynamodb.properties
dynamodb.awsCredentialsFile = dynamodb/conf/AWSCredentials.properties
dynamodb.primaryKey = firstname
dynamodb.endpoint = http://dynamodb.ap-northeast-1.amazonaws.com
  • YCSB/workloads/workload02
recordcount=10000
table=ycsb02
operationcount=10000
workload=com.yahoo.ycsb.workloads.CoreWorkload
readallfields=true
readproportion=0.33
updateproportion=0.33
scanproportion=0
insertproportion=0.33

実行手順

  • データロード
$ ./bin/ycsb load dynamodb -P workloads/workload02 -P dynamodb/conf/dynamodb.properties
Command line: -db com.yahoo.ycsb.db.DynamoDBClient -P workloads/workload02 -P dynamodb/conf/dynamodb.properties -load
YCSB Client 0.16.0-SNAPSHOT
（中略）
Loading workload...
Starting test.
0    [Thread-1] INFO  com.yahoo.ycsb.db.DynamoDBClient  -dynamodb connection created with http://dynamodb.ap-northeast-1.amazonaws.com
DBWrapper: report latency for each error is false and specific error codes to track for latency are: []
[OVERALL], RunTime(ms), 98442
[OVERALL], Throughput(ops/sec), 101.58265780865891
[TOTAL_GCS_PS_Scavenge], Count, 17
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 52
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 0.05282298206050263
[TOTAL_GCS_PS_MarkSweep], Count, 0
[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 0
[TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.0
[TOTAL_GCs], Count, 17
[TOTAL_GC_TIME], Time(ms), 52
[TOTAL_GC_TIME_%], Time(%), 0.05282298206050263
[CLEANUP], Operations, 1
[CLEANUP], AverageLatency(us), 4.0
[CLEANUP], MinLatency(us), 4
[CLEANUP], MaxLatency(us), 4
[CLEANUP], 95thPercentileLatency(us), 4
[CLEANUP], 99thPercentileLatency(us), 4
[INSERT], Operations, 10000
[INSERT], AverageLatency(us), 9763.9544
[INSERT], MinLatency(us), 8960
[INSERT], MaxLatency(us), 255359
[INSERT], 95thPercentileLatency(us), 10071
[INSERT], 99thPercentileLatency(us), 12527
[INSERT], Return=OK, 10000
  • 負荷をかける
$ ./bin/ycsb run dynamodb -P workloads/workload02 -P dynamodb/conf/dynamodb.properties -threads 4
（中略）
[OVERALL], RunTime(ms), 27116
[OVERALL], Throughput(ops/sec), 368.7859566307715
[TOTAL_GCS_PS_Scavenge], Count, 13
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 62
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 0.22864729311107834
[TOTAL_GCS_PS_MarkSweep], Count, 0
[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 0
[TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.0
[TOTAL_GCs], Count, 13
[TOTAL_GC_TIME], Time(ms), 62
[TOTAL_GC_TIME_%], Time(%), 0.22864729311107834
[READ], Operations, 3326
[READ], AverageLatency(us), 8019.719783523752
[READ], MinLatency(us), 5880
[READ], MaxLatency(us), 227199
[READ], 95thPercentileLatency(us), 9455
[READ], 99thPercentileLatency(us), 16039
[READ], Return=OK, 3326
[CLEANUP], Operations, 4
[CLEANUP], AverageLatency(us), 1.5
[CLEANUP], MinLatency(us), 1
[CLEANUP], MaxLatency(us), 3
[CLEANUP], 95thPercentileLatency(us), 3
[CLEANUP], 99thPercentileLatency(us), 3
[INSERT], Operations, 3315
[INSERT], AverageLatency(us), 11822.618401206637
[INSERT], MinLatency(us), 10560
[INSERT], MaxLatency(us), 223487
[INSERT], 95thPercentileLatency(us), 12527
[INSERT], 99thPercentileLatency(us), 16655
[INSERT], Return=OK, 3315
[UPDATE], Operations, 3359
[UPDATE], AverageLatency(us), 11647.422447156892
[UPDATE], MinLatency(us), 10360
[UPDATE], MaxLatency(us), 108095
[UPDATE], 95thPercentileLatency(us), 12463
[UPDATE], 99thPercentileLatency(us), 19071
[UPDATE], Return=OK, 3359
$ ./bin/ycsb run dynamodb -P workloads/workload02 -P dynamodb/conf/dynamodb.properties -threads 4
（中略）
[OVERALL], RunTime(ms), 27247
[OVERALL], Throughput(ops/sec), 367.0128821521635
[TOTAL_GCS_PS_Scavenge], Count, 13
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 62
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 0.22754798693434142
[TOTAL_GCS_PS_MarkSweep], Count, 0
[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 0
[TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.0
[TOTAL_GCs], Count, 13
[TOTAL_GC_TIME], Time(ms), 62
[TOTAL_GC_TIME_%], Time(%), 0.22754798693434142
[READ], Operations, 3331
[READ], AverageLatency(us), 8085.699789852897
[READ], MinLatency(us), 5840
[READ], MaxLatency(us), 231039
[READ], 95thPercentileLatency(us), 9495
[READ], 99thPercentileLatency(us), 19775
[READ], Return=OK, 3331
[CLEANUP], Operations, 4
[CLEANUP], AverageLatency(us), 1.75
[CLEANUP], MinLatency(us), 1
[CLEANUP], MaxLatency(us), 4
[CLEANUP], 95thPercentileLatency(us), 4
[CLEANUP], 99thPercentileLatency(us), 4
[UPDATE], Operations, 3336
[UPDATE], AverageLatency(us), 11837.420863309353
[UPDATE], MinLatency(us), 10312
[UPDATE], MaxLatency(us), 227711
[UPDATE], 95thPercentileLatency(us), 12503
[UPDATE], 99thPercentileLatency(us), 33151
[UPDATE], Return=OK, 3336
[INSERT], Operations, 3333
[INSERT], AverageLatency(us), 11616.116411641164
[INSERT], MinLatency(us), 10504
[INSERT], MaxLatency(us), 65919
[INSERT], 95thPercentileLatency(us), 12415
[INSERT], 99thPercentileLatency(us), 16463
[INSERT], Return=OK, 3333
$ ./bin/ycsb run dynamodb -P workloads/workload02 -P dynamodb/conf/dynamodb.properties -threads 4
（中略）
[OVERALL], RunTime(ms), 26427
[OVERALL], Throughput(ops/sec), 378.4008778900367
[TOTAL_GCS_PS_Scavenge], Count, 13
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 61
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 0.23082453551292237
[TOTAL_GCS_PS_MarkSweep], Count, 0
[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 0
[TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.0
[TOTAL_GCs], Count, 13
[TOTAL_GC_TIME], Time(ms), 61
[TOTAL_GC_TIME_%], Time(%), 0.23082453551292237
[READ], Operations, 3335
[READ], AverageLatency(us), 8170.629085457272
[READ], MinLatency(us), 6060
[READ], MaxLatency(us), 239103
[READ], 95thPercentileLatency(us), 9703
[READ], 99thPercentileLatency(us), 12135
[READ], Return=OK, 3335
[CLEANUP], Operations, 4
[CLEANUP], AverageLatency(us), 1.25
[CLEANUP], MinLatency(us), 1
[CLEANUP], MaxLatency(us), 2
[CLEANUP], 95thPercentileLatency(us), 2
[CLEANUP], 99thPercentileLatency(us), 2
[INSERT], Operations, 3357
[INSERT], AverageLatency(us), 10820.53142686923
[INSERT], MinLatency(us), 9104
[INSERT], MaxLatency(us), 245375
[INSERT], 95thPercentileLatency(us), 11463
[INSERT], 99thPercentileLatency(us), 31871
[INSERT], Return=OK, 3357
[UPDATE], Operations, 3308
[UPDATE], AverageLatency(us), 10581.397823458283
[UPDATE], MinLatency(us), 8872
[UPDATE], MaxLatency(us), 231551
[UPDATE], 95thPercentileLatency(us), 11287
[UPDATE], 99thPercentileLatency(us), 19039
[UPDATE], Return=OK, 3308

環境

  • DynamoDB
    • テーブル名: ycsb02
    • パーティションキー: firstname
    • Item件数：1万件
    • Provisioned capacity
      • Read: 3000
      • Write: 1000
  • EC2インスタンス
    • Instance type: m4.xlarge
    • OS: Amazon Linux AMI release 2018.03(kernel 4.14.77-70.59.amzn1.x86_64)
    • AMI: amzn-ami-hvm-2018.03.0.20181129-x86_64-gp2 (ami-00a5245b4816c38e6)
    • Availability zone: ap-northeast-1a