In this article, we will see the basic aws s3 CLI commands used to manage buckets and objects.
Before that, kindly install aws CLI commands in your system, you can refer our previous article for this.(Install aws commad line interferace).

1. Create New S3 Bucket
Use mb option for creating bucket through CLI. mb stands for Make Bucket.

$ aws s3 mb s3://techrunnrbucket
make_bucket: techrunnrbucket

The bucket is created in the ap-southeast-1 region, as that is what is specified in the user’s config file as shown below.

$ cat ~/.aws/config
[profile techrunnr]
region = ap-southeast-1

2. Create Bucket in a different region
Creating a bucket in a particular region(different than the one we created in our config file), then we will use –region option

$ aws s3 mb s3://techrunnrbucket --region us-west-2
make_bucket: techrunnrbucket

3. List all s3 buckets
View all the buckets for our account/user,we can use aws s3 ls

$ aws s3 ls
2019-07-06 17:13:42 techrunnrbucket
2019-03-05 02:10:56 techrunnrbucket2
2019-02-01 06:15:48 backupbucket

4. List All Objects in a Bucket

The following command displays all objects under the techrunnrbucket.

$ aws s3 ls s3://techrunnrbucket
                           PRE sampledata/
                           PRE sampletechunnrfiles/
2019-05-06 05:12:19         14 testdrive.html
2019-03-01 10:15:13       2546 sample.txt

The above output doesn’t display the content of sub-folders sampledata and sampletechunnrfiles

5. List all Objects in a Bucket Recursively

Display all the objects recursively including the content of the sub-folders,

$ aws s3 ls s3://techrunnrbucket --recursive
2019-08-06 10:12:22       3567  sampledata/index.html
2019-07-05 13:08:17         46 sampletechunnrfiles/file2.txt
2019-05-06 05:12:19         14 testdrive.html
2019-03-01 10:15:13       2546 sample.txt

6.Delete S3 Bucket

Use rb option for deleting empty bucket. rb stands for remove bucket.

$ aws s3 rb s3://techrunnr1
remove_bucket: techrunnr1

Delete a bucket along with all objects,we can use –force
To delete a bucket along with all its objects, use the –force option as shown below.

$ aws s3 rb s3://techrunnrbucket --force
delete: s3://sampledata/index.html
delete: s3://sampletechunnrfiles/file2.txt
delete: s3://techrunnrbucket/testdrive.html
delete: s3://techrunnrbucket/sample.txt
remove_bucket: techrunnrbucket 

7. Copy Local File to S3 Bucket
Copy a file from the local system to S3 bucket.

$ aws s3 cp file1.txt s3://techrunnr1

Copy file with a different name:

$ aws s3 cp file1.txt s3://techrunnr1/newfile1.txt

8.Copy Local Folder with all Files to S3 Bucket

Copying all the files from the “projecttech” folder that is under /home/techrunnr directory to S3 bucket

$ cd /home/techrunnr

$ aws s3 cp projecttech s3://techrunnr1 --recursive
upload: projecttech/info1.txt to s3://techrunnr1/info1.txt
upload: projecttech/info2.txt to s3://techrunnr1/info2.txt

If we need to upload projecttech folder from local to s3 as projecttech folder, then we will specify the projecttech folder in s3 bucket.

$ aws s3 cp projecttech s3://techrunnr1/projecttech --recursive
upload: projecttech/info1.txt to s3://techrunnr1/projecttech/info1.txt
upload: projecttech/info2.txt to s3://techrunnr1/projecttech/info2.txt

9.Download a File from S3 Bucket

Download a specific file from an s3 bucket.

$ aws s3 cp s3://techrunnr1/info1.txt  /home/techrunnr/

Download all files from s3 Bucket

$ aws s3 cp s3://techrunnr1/  /home/techrunnr/ --recursive

10.Copy a File from One Bucket to Another Bucket
Copying a file from bucket to another bucket.

$ aws s3 cp s3://techrunnr1/info1.txt s3://techrunnrnewbucket/info1.txt

11. Copy All Files Recursively from One Bucket to Another

copy multiple files from one bucket to another bucket

After moving the file,check the file in local.

12. Move a File from Local to S3 Bucket
If we move from local to s3, from local the file will be completely moved from loacl. After moving the file won’t be available in the local system.

$ ls
samplefile1.txt
samplefile2.txt
samplefile3.txt
$ aws s3 mv samplefile1.txt s3://techrunnr

After moving the file,check the file in local.

$ ls
samplefile2.txt
samplefile3.txt

13. Move a File from S3 Bucket to Local
If we move a file from s3 bucket to local, then the file won’t be available in s3 after its move.

$ aws s3 ls s3://techrunnr1/script.sh
2019-03-04 08:14:38       1678 script.sh

Move the file from S3 bucket to /home/techrunnr directory on the local machine.

$ aws s3 mv s3://techrunnr1/script.sh /home/techrunnr

After the move, the file doesn’t exists on S3 bucketanymore.

14. Move a File from One S3 Bucket to Another S3 Bucket
Before the move, the file executionhub.js is in techrunnr1.

$ aws s3 ls s3://techrunnr1/executionhub.js
2019-01-04 04:15:32       1202 executionhub.js

This file is not in techrunnr2-bucket.

$ aws s3 ls s3://techrunnr2-bucket/executionhub.js

Move the file from techrunnr1 to techrunnr2-bucket.

$ aws s3 mv s3://techrunnr1/executionhub.js s3://techrunnr2-bucket
move: s3://techrunnr1/executionhub.js to s3://techrunnr2-bucket/executionhub.js

Now, the file is only on the techrunnr2-bucket.

$ aws s3 ls s3://techrunnr1/executionhub.js

$ aws s3 ls s3://techrunnr2-bucket/executionhub.js
2019-07-13 20:45:07       1202 executionhub.js

15. Move All Files from a Local Folder to S3 Bucket

Moving all the files from Local folder to s3 Bucket.

$ aws s3 mv project s3://techrunnr1/project --recursive

16. Move All Files from S3 Bucket to Local Folder

Move folder from s3 Bucket to Local system.

$ aws s3 mv s3://techrunnr1/project/  localproject --recursive
move: s3://techrunnr1/project/file1.txt to localproject/file1.txt
move: s3://techrunnr1/project/file2.txt to localproject/file2.txt
move: s3://techrunnr1/project/file3.txt to localproject/file3.txt
move: s3://techrunnr1/project/file4.txt to localproject/file4.txt

Here is the output after the above move.

$ aws s3 ls s3://techrunnr1/project/
$

$ ls -1  localproject
file1.txt
file2.txt
file3.txt
file4.txt

17.Move All Files from One S3 Bucket to Another S3 Bucket

Use the recursive option to move all files from one bucket to another as shown below.

$ aws s3 mv s3://techrunnr s3://techrunnr2 --recursive
move: s3://techrunnr/file1.txt to s3://techrunnr2/file1.txt
move: s3://techrunnr/file2.txt to s3://techrunnr2/file2.txt
move: s3://techrunnr/file3.txt to s3://techrunnr2/file3.txt
move: s3://techrunnr/file4.txt to s3://techrunnr2/file4.txt

18.Delete a File from S3 Bucket

To delete a specific file from a S3 bucket, use the rm option as shown below. The following will delete the index1.txt file from the given S3 bucket.

$ aws s3 rm s3://techrunnr/index1.txt
delete: s3://techrunnr/index1.txt

19. Delete All Objects from S3 buckets

When you specify rm option just with a bucket name, it doesn’t do anything. This will not delete any file from the bucket.

$ aws s3 rm s3://techrunnr

To delete all the files from a S3 bucket, use the –recursive option as shown below.

$ aws s3 rm s3://techrunnr --recursive
delete: s3://techrunnr/file2.txt
delete: s3://techrunnr/common1.txt
delete: s3://techrunnr/file4.txt
delete: s3://techrunnr/config/intex.xml
..

20. Sync files from Laptop to S3 Bucket

When we use sync command, it will recursively copy only the new or updated files from the source directory to the destination.

The following will sync the files from projectfolder directory in local machine to the techrunnr.

$ aws s3 sync projectfolder s3://techrunnr
upload: projectfolder/profile1.sh to s3://techrunnr/profile1.sh
upload: projectfolder/profile2.txt to s3://techrunnr/profile2.txt
upload: projectfolder/profile3.sh to s3://techrunnr/profile3.sh
upload: projectfolder/profile4.txt to s3://techrunnr/profile4.txt

If you want to sync it to a subfolder called projectfolder on the S3 bucket, then include the folder name in the s3 bucket as shown below.

$ aws s3 sync projectfolder s3://techrunnr/projectfolder
upload: projectfolder/profile1.sh to s3://techrunnr/projectfolder/profile1.sh
upload: projectfolder/profile2.txt to s3://techrunnr/projectfolder/profile2.txt
upload: projectfolder/profile3.sh to s3://techrunnr/projectfolder/profile3.sh
upload: projectfolder/profile4.txt to s3://techrunnr/projectfolder/profile4.txt

Once we do the sync once, if you run the command immediately again, it will not do anything, as there is no new or updated files on the local projectfolder directory.

$ aws s3 sync projectfolder s3://techrunnr/projectfolder

Let us create a new file on the local machine for testing.

echo "New file" > projectfolder/projectfile.txt

Now when you execute the sync, it will sync only this new file to the S3 bucket.

$ aws s3 sync projectfolder s3://techrunnr/projectfolder
upload: projectfolder/projectfile.txt to s3://techrunnr/projectfolder/projectfile.txt

 

 

© 2019, Techrunnr. All rights reserved.

#1
#2
#3
Questions Answered
Articles Written
Overall Points

0 Comments

Leave a Reply

Please wait...

Subscribe to our newsletter

Want to be notified when our article is published? Enter your email address and name below to be the first to know.