How to back up our MongoDB Database and upload it to AWS?

It's always better to do the backup of the database, instead of saving it to our server save to s3 storage. So let’s start firstly with the backup and then upload the same to AWS.

Let’s create a script folder but before need to make sure that we’re in the home directory

cd ~
mkdir scripts
sudo nano scripts/db_backup.sh

Now Paste the below scode in the db_backup.sh file

backup_name=db_backups/`date +%Y-%m-%d`
#use password in single commas if you’re using special characters in password
mongodump -h localhost:port -u username -p ‘password’ -d database -o $backup_name
Example:- mongodump -h localhost:27017 -u mydbuser -p ‘#password’ -d mydb -o $backup_name

Now run the script to check whether Mongo connects and dumps correctly. After that, create the db_backups folder where we dump the database

cd ~
mkdir db_backups

Now run the script to check whether Mongo connects and dumps correctly

/bin/bash ~/scripts/db_backup.sh or/bin/bash /home/ubuntu/scripts/db_backup.sh

Now we have to configure the AWS Command Line Interface so that we can upload the dump to the AWS S3 Storage

sudo apt-get update
sudo apt-get -y install python-pip
sudo pip install awscli
sudo pip install — upgrade pip

Or you can follow the steps in the AWS documentation
https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html

Now type the below command in the terminal, it asks you the following information

like access key, secret key, region name, and format

aws configure
AWS Access Key ID [None]: ACCESS_KEY
AWS Secret Access Key [None]: SECRET_KEY
Default region name [None]: region
Default output format [None]: json

Now we need to upload the MongoDB dump to AWS S3. For this, open the db_backup.sh and add the following lines

tar -cvzf $backup_name.tar.gz $backup_name -> Compressing folder and conver to tar
aws s3 cp $backup_name.tar.gz s3://bucket_name/db_backups/ -> Copying the tar file to aws using aws console
rm -rf $backup_name -> Now remove the backup folder
rm $backup_name.tar.gz -> Finally remove the tar file too as this file uploads to s3

Now we’ll run the script again to check whether all things are going fine
/bin/bash ~/scripts/db_backup.sh

After everything works, we run this script by using the Cron

Open Terminal(Ctrl+Alt+T), type

crontab -e

It asks you to choose your editor, you can choose any of your favorite ones. let’s type 1

It will open the nano editor, add your script command

#At 00:00 on day-of-month 7, next at 2019-08-07 00:00:00 and so on
0 0 7 * * /bin/bash /home/ubuntu/scripts/db_backup.shFor checking now, change the crontime to current time so that we can check whether its working fine or not

Now Check cron logs in /var/log/syslog is hit or not, It will see if your script command runs or not

If MTA problem shows, then we need to install postfix
sudo apt-get install postfixThen after script hitting from the cron, we have got mail in /home/mail/ubuntu where we will see the error in scriptBut from cron aws not found, so instead of aws use /usr/local/bin/awsAws command for copy will be like this
/usr/local/bin/aws s3 cp $backup_name.tar.gz s3://bucket_name/db_backups/

Now the final commands in the db_backup.sh

#!/bin/bash
backup_name=db_backups/`date +%Y-%m-%d`#use password in single commas if you're using special characters in password
mongodump -h localhost:27097 -u prodUsr -p '#@pass$#6%' -d proddb -o $backup_name tar -cvzf $backup_name.tar.gz $backup_name /usr/local/bin/aws s3 cp $backup_name.tar.gz s3://bucket_name/db_backups/ -> This all in the one line#aws s3 cp $backup_name.tar.gz s3://bucket_name/db_backups/ -> This not working we can remove from hererm -rf $backup_name
rm $backup_name.tar.gz

2. We can also upload the dump file using s3cmd

Firstly Install the s3cmd

sudo su
wget -O- -q http://s3tools.org/repo/deb-all/stable/s3tools.key | sudo apt-key add -
wget -O/etc/apt/sources.list.d/s3tools.list http://s3tools.org/repo/deb-all/stable/s3tools.list
apt-get install s3cmd

Configure s3cmd ith was was s3

s3cmd --configure

After was s3 configuration we will write the s2cmd upload command in the Mongo backup sh file

#!/bin/bash

MONGODUMP_PATH="/usr/bin/mongodump"
MONGO_DATABASE="db_name" #replace with your database name

TIMESTAMP=`date +%F-%H%M`
S3_BUCKET_NAME="bucket-name" #replace with your bucket name on Amazon S3
S3_BUCKET_PATH="mongodb-backups"
HOST="localhost"

# Create backup
$MONGODUMP_PATH -h $HOST --port portno -d $MONGO_DATABASE -u dbusr -p 'db password'

# Add timestamp to backup
mv dump mongodb-$HOSTNAME-$TIMESTAMP
tar cf mongodb-$HOSTNAME-$TIMESTAMP.tar mongodb-$HOSTNAME-$TIMESTAMP

# Upload to S3
s3cmd put mongodb-$HOSTNAME-$TIMESTAMP.tar s3://$S3_BUCKET_NAME/$S3_BUCKET_PATH/mongodb-$HOSTNAME-$TIMESTAMP.tar

rm -rf mongodb-*

We can instantly run the bash file bash mongo_backup.sh

After successfully running the above command we can add the bash command in the crontab by crontab -e, choose nano as an editor, add the below in that file, and save.

#db script
0 4 * * * /bin/bash /home/ubuntu/mongo_backup.sh

Let’s back up our database :)