java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
$ cd /etc/pki/ca-trust/extracted/java $ sudo cp -p cacerts cacerts.org
$ sudo keytool -delete -noprompt -alias baltimorecybertrustroot -keystore $JAVA_HOME/jre/lib/security/cacerts -storepass changeit
$ ./run_example.sh ListBuckets ## Running ListBuckets... ## arguments ... [INFO] Scanning for projects... [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building Amazon S3 Examples 1.0 [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- exec-maven-plugin:1.6.0:java (default-cli) @ s3examples --- [WARNING] com.amazonaws.SdkClientException: Unable to execute HTTP request: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleRetryableException (AmazonHttpClient.java:1175) （中略） at java.lang.Thread.run (Thread.java:748) Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.ssl.Alerts.getSSLException (Alerts.java:192) （中略） at java.lang.Thread.run (Thread.java:748) Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.validator.PKIXValidator.doBuild (PKIXValidator.java:397) （中略） at java.lang.Thread.run (Thread.java:748) Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
$ keytool -list -keystore $JAVA_HOME/jre/lib/security/cacerts -storepass changeit|perl -F, -lane '/^Certificate/ or print $F[0]' > alias_list.txt
Keystore type: jks Keystore provider: SUN Your keystore contains 132 entries
$ cat alias_list.txt|while read LINE do sudo keytool -delete -noprompt -alias ${LINE} -keystore $JAVA_HOME/jre/lib/security/cacerts -storepass changeit done
$ keytool -list -keystore $JAVA_HOME/jre/lib/security/cacerts -storepass changeit Keystore type: jks Keystore provider: SUN Your keystore contains 0 entries ★
$ ./run_example.sh ListBuckets ## Running ListBuckets... ## arguments ... [INFO] Scanning for projects... [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building Amazon S3 Examples 1.0 [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- exec-maven-plugin:1.6.0:java (default-cli) @ s3examples --- [WARNING] com.amazonaws.SdkClientException: Unable to execute HTTP request: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleRetryableException (AmazonHttpClient.java:1175) （中略） at java.lang.Thread.run (Thread.java:748) Caused by: javax.net.ssl.SSLException: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty at sun.security.ssl.Alerts.getSSLException (Alerts.java:208) （中略） at java.lang.Thread.run (Thread.java:748) Caused by: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty at sun.security.validator.PKIXValidator.<init> (PKIXValidator.java:91) （中略） at java.lang.Thread.run (Thread.java:748) Caused by: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty at java.security.cert.PKIXParameters.setTrustAnchors (PKIXParameters.java:200)
$ sudo rm cacerts
$ ./run_example.sh ListBuckets ## Running ListBuckets... ## arguments ... [INFO] Scanning for projects... [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building Amazon S3 Examples 1.0 [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- exec-maven-plugin:1.6.0:java (default-cli) @ s3examples --- [WARNING] com.amazonaws.SdkClientException: Unable to execute HTTP request: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleRetryableException (AmazonHttpClient.java:1175) （中略） at java.lang.Thread.run (Thread.java:748) Caused by: javax.net.ssl.SSLException: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty at sun.security.ssl.Alerts.getSSLException (Alerts.java:208) （中略） at java.lang.Thread.run (Thread.java:748) Caused by: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty at sun.security.validator.PKIXValidator.<init> (PKIXValidator.java:91) （中略） at java.lang.Thread.run (Thread.java:748) Caused by: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty at java.security.cert.PKIXParameters.setTrustAnchors (PKIXParameters.java:200)
$ cat /etc/system-release Amazon Linux release 2 (Karoo) $ uname -r 4.14.114-105.126.amzn2.x86_64
$ java -version openjdk version "1.8.0_201" OpenJDK Runtime Environment (build 1.8.0_201-b09) OpenJDK 64-Bit Server VM (build 25.201-b09, mixed mode)
aws-doc-sdk-examples/java at master · awsdocs/aws-doc-sdk-examples · GitHub をビルドして、S3 バケットをリスト表示する Java サンプルプログラムを動かしてみた。
$ sudo yum -y install git
$ sudo yum -y install java-1.8.0-openjdk-devel.x86_64 openjdk version "1.8.0_201" OpenJDK Runtime Environment (build 1.8.0_201-b09) OpenJDK 64-Bit Server VM (build 25.201-b09, mixed mode)
$ curl -OL https://archive.apache.org/dist/maven/maven-3/3.5.2/binaries/apache-maven-3.5.2-bin.tar.gz $ tar xzvf apache-maven-3.5.2-bin.tar.gz $ sudo mv apache-maven-3.5.2 /opt/ $ sudo ln -s /opt/apache-maven-3.5.2 /opt/apache-maven
$ wget https://sdk-for-java.amazonwebservices.com/latest/aws-java-sdk.zip $ unzip aws-java-sdk.zip
$ vi .bash_profile PATH=$PATH:$HOME/.local/bin:$HOME/bin:/opt/apache-maven/bin JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.201.b09-0.amzn2.x86_64 export JAVA_HOME CLASSPATH=.:/home/ec2-user/aws-java-sdk-1.11.568/lib/aws-java-sdk-1.11.568.jar export CLASSPATH $ source .bash_profile
$ mvn --version Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) Maven home: /opt/apache-maven Java version: 1.8.0_201, vendor: Oracle Corporation Java home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.201.b09-0.amzn2.x86_64/jre Default locale: en_US, platform encoding: UTF-8 OS name: "linux", version: "4.14.114-105.126.amzn2.x86_64", arch: "amd64", family: "unix"
$ git clone https://github.com/awsdocs/aws-doc-sdk-examples.git
<dependency> <groupId>com.amazonaws</groupId> <artifactId>aws-java-sdk</artifactId> <version>1.11.543</version> </dependency>
$ cd ~/aws-doc-sdk-examples/java/example_code/s3/src/main/java $ mv CrossRegionReplication.java CrossRegionReplication.java.org
$ cd ~/aws-doc-sdk-examples/java/example_code/s3
$ mvn package
$ export AWS_ACCESS_KEY_ID=... $ export AWS_SECRET_ACCESS_KEY=...
$ ./run_example.sh ListBuckets ## Running ListBuckets... ## arguments ... [INFO] Scanning for projects... [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building Amazon S3 Examples 1.0 [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- exec-maven-plugin:1.6.0:java (default-cli) @ s3examples --- Your Amazon S3 buckets are: * aws-athena-query-results-123456789012-ap-northeast-1 * aws-athena-query-results-123456789012-us-east-1 * aws-athena-query-results-us-east-1-123456789012 （中略） [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 1.835 s [INFO] Finished at: 2019-06-09T16:55:16Z [INFO] Final Memory: 27M/897M [INFO] ------------------------------------------------------------------------
$ sudo update-alternatives --config java There are 2 programs which provide 'java'. Selection Command ----------------------------------------------- * 1 /usr/lib/jvm/jre-1.7.0-openjdk.x86_64/bin/java + 2 /usr/lib/jvm/jre-1.8.0-openjdk.x86_64/bin/java Enter to keep the current selection[+], or type selection number:
$ sudo update-alternatives --config javac There is 1 program that provides 'javac'. Selection Command ----------------------------------------------- *+ 1 /usr/lib/jvm/java-1.8.0-openjdk.x86_64/bin/javac
$ sudo mkfs -t ext4 /dev/sdb $ sudo mkdir /ebs $ sudo chmod o+w /ebs $ sudo vi /etc/fstab $ sudo mount -a $ df -h
$ nohup time dd if=/dev/urandom of=/ebs/1tb.dat bs=1M count=1048576 & $ ls -l 1tb.dat -rw-rw-r-- 1 ec2-user ec2-user 1099511627776 Jun 9 11:23 1tb.dat $ ls -lh 1tb.dat -rw-rw-r-- 1 ec2-user ec2-user 1.0T Jun 9 11:23 1tb.dat
$ aws configure set default.s3.multipart_chunksize 128MB $ aws configure set default.s3.max_concurrent_requests 32 $ nohup time aws s3 cp 1tb.dat s3://az-20190609/ &
$ aws s3 ls s3://az-20190609/1tb.dat 2019-06-09 11:36:01 1099511627776★ 1tb.dat $ aws s3 ls --human-readable s3://az-20190609/1tb.dat 2019-06-09 11:36:01 1.0 TiB★ 1tb.dat
class ListCommand(S3Command): NAME = 'ls' DESCRIPTION = ("List S3 objects and common prefixes under a prefix or " "all S3 buckets. Note that the --output and --no-paginate " "arguments are ignored for this command.") USAGE = "<S3Uri> or NONE" ARG_TABLE = [{'name': 'paths', 'nargs': '?', 'default': 's3://', 'positional_arg': True, 'synopsis': USAGE}, RECURSIVE, PAGE_SIZE, HUMAN_READABLE, SUMMARIZE, REQUEST_PAYER] def _run_main(self, parsed_args, parsed_globals): super(ListCommand, self)._run_main(parsed_args, parsed_globals) self._empty_result = False self._at_first_page = True self._size_accumulator = 0 self._total_objects = 0 self._human_readable = parsed_args.human_readable path = parsed_args.paths if path.startswith('s3://'): path = path[5:] bucket, key = find_bucket_key(path) if not bucket: self._list_all_buckets() elif parsed_args.dir_op: # Then --recursive was specified. self._list_all_objects_recursive( bucket, key, parsed_args.page_size, parsed_args.request_payer) else: self._list_all_objects( ★ bucket, key, parsed_args.page_size, parsed_args.request_payer) if parsed_args.summarize: self._print_summary() if key: # User specified a key to look for. We should return an rc of one # if there are no matching keys and/or prefixes or return an rc # of zero if there are matching keys or prefixes. return self._check_no_objects() else: # This covers the case when user is trying to list all of of # the buckets or is trying to list the objects of a bucket # (without specifying a key). For both situations, a rc of 0 # should be returned because applicable errors are supplied by # the server (i.e. bucket not existing). These errors will be # thrown before reaching the automatic return of rc of zero. return 0 def _list_all_objects(self, bucket, key, page_size=None, ★ request_payer=None): paginator = self.client.get_paginator('list_objects_v2') paging_args = { 'Bucket': bucket, 'Prefix': key, 'Delimiter': '/', 'PaginationConfig': {'PageSize': page_size} } if request_payer is not None: paging_args['RequestPayer'] = request_payer iterator = paginator.paginate(**paging_args) for response_data in iterator: self._display_page(response_data) ★ def _display_page(self, response_data, use_basename=True): common_prefixes = response_data.get('CommonPrefixes', []) contents = response_data.get('Contents', []) ★ if not contents and not common_prefixes: self._empty_result = True return for common_prefix in common_prefixes: prefix_components = common_prefix['Prefix'].split('/') prefix = prefix_components[-2] pre_string = "PRE".rjust(30, " ") print_str = pre_string + ' ' + prefix + '/\n' uni_print(print_str) for content in contents: last_mod_str = self._make_last_mod_str(content['LastModified']) self._size_accumulator += int(content['Size']) self._total_objects += 1 size_str = self._make_size_str(content['Size']) if use_basename: filename_components = content['Key'].split('/') filename = filename_components[-1] else: filename = content['Key'] print_str = last_mod_str + ' ' + size_str + ' ' + \ ★ filename + '\n' uni_print(print_str) self._at_first_page = False （中略） def _make_size_str(self, size): """ This function creates the size string when objects are being listed. """ if self._human_readable: size_str = human_readable_size(size) else: size_str = str(size) ★ return size_str.rjust(10, ' ') ★
str.rjust(width[, fillchar])
width の長さをもつ右寄せした文字列を返します。パディングには fillchar で指定された文字(デフォルトではスペース)が使われます。 width が len(s) 以下の場合、元の文字列が返されます。バージョン 2.4 で変更: 引数 fillchar に対応.
5. 組み込み型 — Python 2.7.16 ドキュメント
オブジェクトのアップロード - Amazon Simple Storage Service
- 1 回のオペレーションでオブジェクトをアップロードする — 1 回の PUT オペレーションでアップロードできるオブジェクトの最大サイズは 5 GB です。詳細については、「1 回のオペレーションでのオブジェクトのアップロード」を参照してください。
- オブジェクトをいくつかに分けてアップロードする — マルチパートアップロード API を使用すると、最大 5 TB の大容量オブジェクトをアップロードできます。
$ JAVA_HOME/bin/java -version openjdk version "1.8.0_191" OpenJDK Runtime Environment (build 1.8.0_191-b12) OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)
$ keytool -list -keystore $JAVA_HOME/jre/lib/security/cacerts > cacerts_list.txt Enter keystore password: changeit
$ grep amazon cacerts_list.txt amazonrootca4, Nov 16, 2018, trustedCertEntry, amazonrootca3, Nov 16, 2018, trustedCertEntry, amazonrootca2, Nov 16, 2018, trustedCertEntry, amazonrootca1, Nov 16, 2018, trustedCertEntry,
$ ls -l $JAVA_HOME/jre/lib/security/cacerts lrwxrwxrwx 1 root root 41 Feb 27 12:44 /usr/lib/jvm/java/jre/lib/security/cacerts -> ../../../../../../../etc/pki/java/cacerts $ ls -l /etc/pki/java/cacerts lrwxrwxrwx 1 root root 40 Nov 16 2018 /etc/pki/java/cacerts -> /etc/pki/ca-trust/extracted/java/cacerts $ ls -l /etc/pki/ca-trust/extracted/java/cacerts -r--r--r-- 1 root root 197507 Nov 16 2018 /etc/pki/ca-trust/extracted/java/cacerts
$ sudo cp -p cacerts cacerts.org
$ wget https://sdk-for-java.amazonwebservices.com/latest/aws-java-sdk.zip $ unzip aws-java-sdk.zip
$ git clone https://github.com/awsdocs/aws-doc-sdk-examples.git
$ cd aws-doc-sdk-examples/java/example_code/s3
$ javac -classpath .:/home/ec2-user/aws-java-sdk-1.11.568/lib/aws-java-sdk-1.11.568.jar ListBuckets.java
# curl --silent --location https://rpm.nodesource.com/setup_6.x | bash - # yum install -y nodejs
$ npm install aws-sdk
2020/6/11 追記：
# cat /etc/system-release Amazon Linux release 2 (Karoo) # uname -r 4.14.177-139.254.amzn2.x86_64 # curl -sL https://rpm.nodesource.com/setup_12.x | bash - # yum install -y nodejs # node -v v12.18.0
2021/5/9 追記：
$ curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.34.0/install.sh | bash $ . ~/.nvm/nvm.sh $ nvm install node $ node -e "console.log('Running Node.js ' + process.version)"
