Click here to Skip to main content
Click here to Skip to main content

Automating backup for MongoDB using CRON and S3CMD

, 19 Feb 2013
Rate this:
Please Sign up or sign in to vote.
This paper attempts to pass all the steps to create an automated backup for a MongoDB Server on a Linux Server where all backups are sent to Amazon AWS S3 service scheduled through crontab.

Table of Contents

Introduction

The tip below describes how to create an automatic backup routine for a MongoDB server in a Linux environment using the Cron scheduler and Amazon S3 to store all backup data.

Background

To complete the steps, you must have:

  • MongoDB 2.2.3 or higher configured in a Linux environment (preferably in an instance on Amazon EC2
  • Amazon S3 account to store all backup data 

Install and Configure S3CMD

S3cmd is a program that allows you to perform all operations in an Amazon S3 using the shell

To install and configure:

sudo su
sudo yum --enablerepo epel install s3cmd
s3cmd --configure

To test s3cmd, display all buckets in your Amazon S3 account.

s3cmd ls

Backup Script 

Create a new file shell, named as mongodb_to_s3_backup.sh using any Linux editor (I use vi)

vi mongodb_to_s3_backup.sh
#!/bin/bash
 
#Force file syncronization and lock writes
mongo admin --eval "printjson(db.fsyncLock())"
 
MONGODUMP_PATH="/usr/bin/mongodump"
MONGO_HOST="SERVER_IP_HERE" #replace with your server ip
MONGO_PORT="27017"
MONGO_DATABASE="dbname_here" #replace with your database name
 
TIMESTAMP=`date +%F-%H%M`
S3_BUCKET_NAME="bucketname_here" #replace with your bucket name on Amazon S3
S3_BUCKET_PATH="mongodb-backups"
 
# Create backup
$MONGODUMP_PATH -h $MONGO_HOST:$MONGO_PORT -d $MONGO_DATABASE
 
# Add timestamp to backup
mv dump mongodb-$HOSTNAME-$TIMESTAMP
tar cf mongodb-$HOSTNAME-$TIMESTAMP.tar mongodb-$HOSTNAME-$TIMESTAMP
 
# Upload to S3
s3cmd put mongodb-$HOSTNAME-$TIMESTAMP.tar 
   s3://$S3_BUCKET_NAME/$S3_BUCKET_PATH/mongodb-$HOSTNAME-$TIMESTAMP.tar
 
#Unlock database writes
mongo admin --eval "printjson(db.fsyncUnlock())"

Test the script and verify your Amazon S3 bucket.

bash mongodb_to_s3_backup.sh

Backup Automatization with CRON

To schedule the mongodb_to_s3_backup.sh use the cron scheduler.

sudo su
crontab -e

Each line (without comments) represents a schedule

#every day at 01h 00m
00 01 * * * /bin/bash /home/ec2-user/mongodb_to_s3_backup.sh

Just this. Any questions or problems, please, ask me and I'll help you. Be happy ;]

All codes are in my gist account at: https://gist.github.com/lazarofl/4961746

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

About the Author

Lazaro Lima
Architect America-NET Ltda.
Brazil Brazil
Lazaro is a brazilian software architect, live in São Paulo, focusing on web development, NoSql and agile practices for startups.

In his personal life he is a part time runner, speaker, amateur astronomer, gamer and an absorbing knowledge ambulant.
Follow on   Twitter   Google+

Comments and Discussions

 
QuestionProblem with script (a few suggestions) PinmemberMember 1033540714-Oct-13 5:36 
AnswerRe: Problem with script (a few suggestions) PinmemberLazaro Lima22-Nov-13 1:21 
QuestionWell explained documented stuff PinmemberBijendra Singh3-Jul-13 12:18 
QuestionNot an aritcle. PinmvpDave Kreskowiak19-Feb-13 6:25 
You simply don't have a broad enough topic nor discussion. This is a tip/trick at best.

AnswerRe: Not an aritcle. PinmemberLazaro Lima19-Feb-13 6:34 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web02 | 2.8.140709.1 | Last Updated 19 Feb 2013
Article Copyright 2013 by Lazaro Lima
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid