How to export a DynamoDB table to S3 bucket using AWS CLI?
Hello Everyone
Welcome to CloudAffaire and this is Debjeet.
Today we are going to discuss how to export a DynamoDB table to S3 bucket using AWS CLI.
You can now export a DynamoDB table to an S3 bucket. To export data from an Amazon DynamoDB
table to an Amazon S3 bucket, point-in-time recovery (PITR) must be enabled on the source table.
You can export table data from any point in time within the PITR window, up to 35 days.
Exporting a table does not consume read capacity on the table, and has no impact on table
performance and availability. You can export table data to an S3 bucket owned by another AWS
account, and to a different region than the one your table is in. You can choose to export your data
in either DynamoDB JSON format or Amazon Ion text format.
How to export a DynamoDB table to S3 bucket using AWS CLI?
Prerequisites:
AWS CLI installed and configured with proper access.
You can use below link to install and configure AWS CLI.
https://cloudaffaire.com/how-to-install-aws-cli/
https://cloudaffaire.com/how-to-configure-aws-cli/
Step 1: Create a DynamoDB table and insert some data that can be exported.
## Create a dynamodb table
aws dynamodb create-table \
--table-name mytable \
--attribute-definitions AttributeName=id,AttributeType=S \
--key-schema AttributeName=id,KeyType=HASH \
--provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5
## Insert an item into the dynamodb table
cat << EOF > item.json
{
"id": {"S": "10001"},
"Name": {"S": "Debjeet"},
"Email": {"S": "debjeettoni@gmail.com"}
}
EOF
aws dynamodb put-item \
--table-name mytable \
--item file://item.json
## Get the dynamodb table arn
TABLE_ARN=$(aws dynamodb describe-table \
--table-name mytable | jq -r .Table.TableArn) &&
echo $TABLE_ARN
Step 2: Enable point in time recover for your DynamoDB table.
## Enable dynamodb point in time recovery
aws dynamodb update-continuous-backups \
--table-name mytable \
--point-in-time-recovery-specification PointInTimeRecoveryEnabled=true
Step 3: Create an S3 bucket where your DynamoDB table will get exported.
## Create a S3 bucket to export dynamodb table
aws s3api create-bucket \
--bucket s3-cloudaffaire-dynamodb-table \
--region ap-south-1 \
--create-bucket-configuration LocationConstraint=ap-south-1
## Create a bucket policy definition file
IAM_USER_ARN=$(aws sts get-caller-identity | jq -r .Arn) &&
cat << EOF > bucket_policy.json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "ExportToS3",
"Effect": "Allow",
"Principal": {
"AWS": "$IAM_USER_ARN"
},
"Action": [
"s3:AbortMultipartUpload",
"s3:PutObject",
"s3:PutObjectAcl"
],
"Resource": "arn:aws:s3:::s3-cloudaffaire-dynamodb-table/*"
}
]
}
EOF
## Create a S3 bucket policy for CloudTrail
aws s3api put-bucket-policy \
--bucket s3-cloudaffaire-dynamodb-table \
--policy file://bucket_policy.json
Step 4: Export DynamoDB table to an S3 bucket using API:
## Export dynamodb table to S3 bucket
aws dynamodb export-table-to-point-in-time \
--table-arn $TABLE_ARN \
--s3-bucket s3-cloudaffaire-dynamodb-table \
--s3-prefix myexport \
--export-format DYNAMODB_JSON
Step 5: Check if your DynamoDB table was exported to S3 bucket.
## Check the export status
EXPORT_ARN=$(aws dynamodb list-exports \
--table-arn $TABLE_ARN | jq -r .ExportSummaries[0].ExportArn) &&
aws dynamodb describe-export \
--export-arn $EXPORT_ARN
## Once the ("ExportStatus": "COMPLETED") in the above output
## Check if dynamodb table is exported
aws s3api list-objects \
--bucket s3-cloudaffaire-dynamodb-table \
--prefix "myexport/AWSDynamoDB/$(basename $EXPORT_ARN)"
## data that has been exported
aws s3api list-objects \
--bucket s3-cloudaffaire-dynamodb-table \
--prefix "myexport/AWSDynamoDB/$(basename $EXPORT_ARN)/data"
Note: It may take some time (generally 1 minutes) for AWS to export your DynamoDB table to S3
bucket.
Step 6: Clean up.
## Delete the S3 bucket with objects
aws s3 rb \
s3://s3-cloudaffaire-dynamodb-table --force
## Disable point in time recovery
aws dynamodb update-continuous-backups \
--table-name mytable \
--point-in-time-recovery-specification PointInTimeRecoveryEnabled=false
## Delete the dynamodb table
aws dynamodb delete-table \
--table-name mytable
Hope you have enjoyed this article, to get more details on DynamoDB table, please follow below
link.
https://docs.aws.amazon.com/dynamodb/index.html
