CloudTrail の S3 オブジェクトレベルでロギングできる項目

手順

CloudTrail の[証跡情報]-[データイベント]でバケット名を選択して、[読み込み]と[書き込み]にチェックして[ストレージの場所]でログの出力先S3バケットを指定する。

CloudTrail の[イベント履歴]から[Amazon Athena で高度なクエリを実行します]をクリックし、分析したい Trail が出力されている S3 バケットを選択して、[テーブルを作成する]をクリックする。

検証結果

実行コマンド

$ aws s3 cp s3://az-test-src/test.txt s3://az-test-dst/

Athena でクエリをかける

SELECT eventtime,
  eventsource,
  eventname,
  awsregion,
  sourceipaddress,
  useragent,
  requestparameters,
  resources
FROM cloudtrail_logs_s3_trail_az_test_src
WHERE eventsource ='s3.amazonaws.com'
AND awsregion     = 'ap-northeast-1'
AND eventtime    >= '2018-08-26T05:59:00Z'
AND eventtime    <= '2018-08-26T06:11:00Z';

結果

eventtime	eventsource	eventname	awsregion	sourceipaddress	useragent	requestparameters	resources
2018-08-26T06:10:09Z	s3.amazonaws.com	CopyObject	ap-northeast-1	13.*.*.210	[aws-cli/1.14.9 Python/2.7.14 Linux/4.14.62-65.117.amzn1.x86_64 botocore/1.8.13]	{"x-amz-copy-source":"az-test-dst/test.txt","bucketName":"az-test-src","key":"test.txt"}	[{arn=arn:aws:s3:::az-test-src/test.txt, accountid=null, type=AWS::S3::Object}, {arn=arn:aws:s3:::az-test-src, accountid=012345678901, type=AWS::S3::Bucket}, {arn=arn:aws:s3:::az-test-dst, accountid=14AmazonCustomer*************56, type=AWS::S3::Bucket}, {arn=arn:aws:s3:::az-test-dst/test.txt, accountid=null, type=AWS::S3::Object}]

012345678901_CloudTrail_ap-northeast-1_20180826T0615Z_5jYgE98fX9T5Fw3F.json
- s3-trail-az-test-src/AWSLogs/012345678901/CloudTrail/ap-northeast-1/2018/08/26/

{
    "Records": [
        {
            "eventVersion": "1.05",
            "userIdentity": {
                "type": "AWSAccount",
                "principalId": "AI****************RX4",
                "accountId": "123456789012"
            },
            "eventTime": "2018-08-26T06:10:09Z", ★タイムスタンプ
            "eventSource": "s3.amazonaws.com",
            "eventName": "CopyObject", ★実行した API
            "awsRegion": "ap-northeast-1",
            "sourceIPAddress": "13.***.***.210", ★API 発行元IPアドレス
            "userAgent": "[aws-cli/1.14.9 Python/2.7.14 Linux/4.14.62-65.117.amzn1.x86_64 botocore/1.8.13]",
            "requestParameters": {
                "x-amz-copy-source": "az-test-dst/test.txt", ★コピー元
                "bucketName": "az-test-src", ★コピー先
                "key": "test.txt" ★オブジェクト名
            },
            "responseElements": null,
            "additionalEventData": {
                "x-amz-id-2": "wBh**********************************************************************zM="
            },
            "requestID": "06F6E609A776F262",
            "eventID": "64f61181-5be6-4c4c-a21c-f7707b0057f8",
            "readOnly": false,
            "resources": [
                {
                    "type": "AWS::S3::Object",
                    "ARN": "arn:aws:s3:::az-test-src/test.txt"
                },
                {
                    "accountId": "012345678901",
                    "type": "AWS::S3::Bucket",
                    "ARN": "arn:aws:s3:::az-test-src"
                },
                {
                    "accountId": "14AmazonCustomer*************56",
                    "type": "AWS::S3::Bucket",
                    "ARN": "arn:aws:s3:::az-test-dst"
                },
                {
                    "type": "AWS::S3::Object",
                    "ARN": "arn:aws:s3:::az-test-dst/test.txt"
                }
            ],
            "eventType": "AwsApiCall",
            "recipientAccountId": "012345678901",
            "sharedEventID": "86eacdb2-075b-4d11-9072-3b6dfcf91e74"
        }
    ]
}

Athena の CREATE TABLE 文

CREATE EXTERNAL TABLE [TABLE_NAME] (
    eventVersion STRING,
    userIdentity STRUCT<
        type: STRING,
        principalId: STRING,
        arn: STRING,
        accountId: STRING,
        invokedBy: STRING,
        accessKeyId: STRING,
        userName: STRING,
        sessionContext: STRUCT<
            attributes: STRUCT<
                mfaAuthenticated: STRING,
                creationDate: STRING>,
            sessionIssuer: STRUCT<
                type: STRING,
                principalId: STRING,
                arn: STRING,
                accountId: STRING,
                userName: STRING>>>,
    eventTime STRING,
    eventSource STRING,
    eventName STRING,
    awsRegion STRING,
    sourceIpAddress STRING,
    userAgent STRING,
    errorCode STRING,
    errorMessage STRING,
    requestParameters STRING,
    responseElements STRING,
    additionalEventData STRING,
    requestId STRING,
    eventId STRING,
    resources ARRAY<STRUCT<
        arn: STRING,
        accountId: STRING,
        type: STRING>>,
    eventType STRING,
    apiVersion STRING,
    readOnly STRING,
    recipientAccountId STRING,
    serviceEventDetails STRING,
    sharedEventID STRING,
    vpcEndpointId STRING
)
COMMENT 'CloudTrail table for [S3_BUCKET_NAME] bucket'
ROW FORMAT SERDE 'com.amazon.emr.hive.serde.CloudTrailSerde'
STORED AS INPUTFORMAT 'com.amazon.emr.cloudtrail.CloudTrailInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION '[S3_BUCKET_URL]'
TBLPROPERTIES ('classification'='cloudtrail');