Moving data between S3 buckets/accounts in AWS

To copy Amazon S3 objects from one AWS account to another by using the S3 COPY operation. You must give the destination AWS account access to the source AWS account’s resources by using Amazon S3 Access Control Lists (ACLs) or bucket polices. For example, the following steps describe how a source AWS account can create a bucket policy to grant another AWS account access to one or more Amazon S3 resources.

Note: The following bucket policy, created in the source AWS account, grants full permissions to the destintation AWS account for each specified Amazon S3 resource. In a production environment, consider specifying more restrictive Action parameters to follow the principle of least privileged access. Then add resources to the Resource section that the destination AWS account should have permissions to. Make sure to separate each resource entry with a comma.

First, get the 12-digit account ID for the destination account. Here is one way to find the account number:

  1. Sign in to the AWS Management Console for the destination AWS account.
  2. In the navigation bar, click Support, and then click Support Center.
    The account number (for example, 222222222222) is displayed in the top-right corner of the Support Center.

In the source account, attach the following policy to the bucket you want to copy. For detailed instructions, see Editing Bucket Permissions.

  1. #Bucket policy in the source AWS account
  2. {
  3. “Version”: “2012-10-17”,
  4. “Statement”: [
  5. {
  6. “Sid”: “DelegateS3Access”,
  7. “Effect”: “Allow”,
  8. “Principal”: {“AWS”: “222222222222”},
  9. “Action”: “s3:*”,
  10. “Resource”: [
  11. “arn:aws:s3:::sourcebucket/*”,
  12. “arn:aws:s3:::sourcebucket”
  13. ]
  14. }
  15. ]
  16. }

Attach a policy to a user or group in the destination AWS account to delegate access to the bucket in the source AWS account. If you attach the policy to a group, make sure that the IAM user is a member of the group.

  1. #User or group policy in the destination AWS account
  2. {
  3. “Version”: “2012-10-17”,
  4. “Statement”: {
  5. “Effect”: “Allow”,
  6. “Action”: “s3:*”,
  7. “Resource”: [
  8. “arn:aws:s3:::sourcebucket”,
  9. “arn:aws:s3:::sourcebucket/*”,
  10. “arn:aws:s3:::destinationbucket”,
  11. “arn:aws:s3:::destinationbucket/*”
  12. ]
  13. }
  14. }

When these steps are completed, you can copy objects by using the AWS Command Line Interface (CLI) commands cp or sync. For example, the following aws s3 sync command could be used to copy the contents from a bucket in the source AWS account to a bucket in the destination AWS account:

  1. aws s3 sync s3://sourcebucket s3://destinationbucket

Note: Successful execution of this command assumes that the AWS CLI has been correctly configured for the user in the destination AWS account, and that the source and destination buckets are in the same region (there is a command-line option for specifiying a different region). For more information about configuring the AWS CLI, see Configuring the AWS Command Line Interface. The user in the destination AWS account must have appropriate permissions to copy files to s3://destinationbucket

For more information about delegating access to a bucket in another account, see Example: Using a resource-based policy to delegate access to an Amazon S3 bucket in another account.

For information about delegating access to resources in different AWS accounts with IAM roles, see Walkthrough: Delegating Access Across AWS Accounts Using IAM Roles.

For a detailed walkthrough that describes how a bucket owner can grant cross-account bucket permissions, see Example 2: Bucket Owner Granting Cross-Account Bucket Permissions


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s