Lambda で Secrets Manager から取得した情報をキャッシュする

Secrets Manager は Rate limit for DescribeSecret/GetSecretValue API requests の上限が2000（上限緩和不可）という場合、Lambda の実行が2000回/秒を超える場合どうするかという問題がある。そこで、Lambda の handler 外に書いたコードは、コールドスタート時の1回だけ実行されるため、Lambda の handler の外で Secrets Manager から接続情報を取得してキャッシュするコードを書いてみた。

SecretsManagerSingletonSample
- ランタイム: Python 3.8
- メモリ: 128 MB
- タイムアウト: 15分
- カスタム VPC

import json
import ast
import boto3
import base64
from botocore.exceptions import ClientError
import pymysql
import sys

secret_name = "aurora-mysql-57-2072"
region_name = "ap-northeast-1"

session = boto3.session.Session()
client = session.client(
    service_name='secretsmanager',
    region_name=region_name
)

try:
    get_secret_value_response = client.get_secret_value(
        SecretId=secret_name
    )
except ClientError as e:
    if e.response['Error']['Code'] == 'DecryptionFailureException':
        raise e
    elif e.response['Error']['Code'] == 'InternalServiceErrorException':
        raise e
    elif e.response['Error']['Code'] == 'InvalidParameterException':
        raise e
    elif e.response['Error']['Code'] == 'InvalidRequestException':
        raise e
    elif e.response['Error']['Code'] == 'ResourceNotFoundException':
        raise e
else:
    print("get_secrets")
    if 'SecretString' in get_secret_value_response:
        secret = get_secret_value_response['SecretString']
    else:
        secret = base64.b64decode(get_secret_value_response['SecretBinary'])

params = ast.literal_eval(secret)
host = params["host"]
username = params["username"]
password = params["password"]
dbname = params["dbname"]

def lambda_handler(event, context):

    try:
        conn = pymysql.connect(host, user=username, passwd=password, db=dbname, connect_timeout=10)
    except:
        print("ERROR: Unexpected error: Could not connect to MySql instance.")
        sys.exit()
    
    item_count = 0

    with conn.cursor() as cur:
        cur.execute("show databases")

        for row in cur:
            item_count += 1
            print(row)

    return {
        'statusCode': 200,
        'body': json.dumps('lambda executed successfully.')
    }

CloudWatch Logs(CloudWatch-CloudWatch Logs-Log groups-/aws/lambda/SecretsManagerSingletonSample)

2020-08-15T20:12:24.253+09:00
get_secrets

2020-08-15T20:28:19.459+09:00
START RequestId: a0de7f36-3731-447d-a59f-23e23b214ff8 Version: $LATEST

2020-08-15T20:28:19.523+09:00
('information_schema',)

2020-08-15T20:28:19.523+09:00
('mydb',)

2020-08-15T20:28:19.523+09:00
('mysql',)

2020-08-15T20:28:19.523+09:00
('performance_schema',)

2020-08-15T20:28:19.523+09:00
('sys',)

2020-08-15T20:28:19.523+09:00
('tmp',)

2020-08-15T20:28:19.524+09:00
END RequestId: a0de7f36-3731-447d-a59f-23e23b214ff8

2020-08-15T20:28:19.524+09:00
REPORT RequestId: a0de7f36-3731-447d-a59f-23e23b214ff8 Duration: 61.07 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 75 MB

2020-08-15T20:28:21.274+09:00
START RequestId: 3010f253-566b-4beb-a15d-ab8b2c87bd77 Version: $LATEST

2020-08-15T20:28:21.299+09:00
('information_schema',)

2020-08-15T20:28:21.299+09:00
('mydb',)

2020-08-15T20:28:21.299+09:00
('mysql',)

2020-08-15T20:28:21.299+09:00
('performance_schema',)

2020-08-15T20:28:21.299+09:00
('sys',)

2020-08-15T20:28:21.299+09:00
('tmp',)

2020-08-15T20:28:21.299+09:00
END RequestId: 3010f253-566b-4beb-a15d-ab8b2c87bd77

2020-08-15T20:28:21.299+09:00
REPORT RequestId: 3010f253-566b-4beb-a15d-ab8b2c87bd77 Duration: 19.68 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 75 MB

2020-08-15T20:28:22.174+09:00
START RequestId: 693a0cfc-8dd8-48be-ae68-578c66def410 Version: $LATEST

2020-08-15T20:28:22.193+09:00
('information_schema',)

2020-08-15T20:28:22.193+09:00
('mydb',)

2020-08-15T20:28:22.193+09:00
('mysql',)

2020-08-15T20:28:22.193+09:00
('performance_schema',)

2020-08-15T20:28:22.193+09:00
('sys',)

2020-08-15T20:28:22.193+09:00
('tmp',)

2020-08-15T20:28:22.194+09:00
END RequestId: 693a0cfc-8dd8-48be-ae68-578c66def410

2020-08-15T20:28:22.194+09:00
REPORT RequestId: 693a0cfc-8dd8-48be-ae68-578c66def410 Duration: 16.95 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 75 MB

2020-08-15T20:28:23.077+09:00
START RequestId: 8d662fed-9875-4997-87c4-d99193eaa3c1 Version: $LATEST