Community:SplunkBackupScript Linux

From Splunk Wiki

Jump to: navigation, search

Backing up Splunk in RHEL Linux


This script will backup most of the specific splunk customizations. This script will be run as a cron job and create a nightly full backup of the Splunk settings. We also included a flag to do a quick one time backup useful before or after you make major changes and want to fall back to a known working model.


Splunk version: 3.4.10

Current Setup

  1. Red Hat Enterprise Linux 4
  2. Default Splunk 3.4.10 RPM install to

File Locations Used Splunk settings. Most user created settings are in ../etc/system/local/


Store our backup logs in a new file under this directory.


Location of the actual backups themselves


Script Narrative

The script can be run using the standard linux execution


. This is the equivalent of running a "quick" backup that is executed before an administrator performs any actions. If you run the standard execution string with


appended to the end the script will assume you are running the script nightly as part of a cron job. The only difference is if the script will append a time stamp into the filename string. I chose to omit the string to help indicate if the system performed and automated cron backup versus a quick backup.

Once the script is executed with or without the flag it performs the following

  1. Determine if using --cron and append a timestamp if you are not using cron
  2. Generate a tar file with the appropriate day, time of day.
    1. The script will only backup directories or files specified in the BACKUPLIST variable.
    2. The script will ignore directories or files specified in the IGNORELIST variable.
  3. After the tar file is made it echos the contents into the splunk log file
  4. If compression is enabled the tar file is gzip'd to a smaller file size
  5. Regardless of compression we md5sum the file to ensure if you look at it later it has not been altered. The sum is stored in the splunk log file.
  6. If you want the script can remove backups over X number of day old to prevent your disk from filling up.

Supporting Files


This can be any file you choose as long as you update the location in the variables section in the beginning of the script.
Mine looks like this:


As you can see I want to gather all the files under /opt/splunk/etc as well as the script and supporting files.


This can be any file you choose as long as you update the location in the variables section in the beginning of the script.
Mine looks like this:


As you can see I want to forget backing up the README's and a few other things.


#Backup of the "local" files for Splunk application.
#These are the "brains" of the splunk application

# Setup commmon script options
# These can be modified to fit your needs
# Where should we put the backed up files?
# Default is /var/backup
# Should this script compress the backups?
# Default is ON (YES!)
# Should this script delete old backups?
# Default is OFF (NO!)
# If the script is going to delete old backups
# How old should the backups be before we delete them?
# Default is 60 Days
# Where should we output the logs of activity?
# Default is /var/log/backup/splunk.log
# Where can we find the backupfile list?
# Default is /var/backup/backupfiles.txt
# Where can we find the ignored files list?
# Default is /var/backup/ignorefiles.txt
# Setup our naming convetions based upon date & time formats
# The dates will look like this: YYYYMMDD
# Example: 20090101 | January 1st 2009
PDATE=`date -u +%Y%m%d`
# The times will look like this: HH:MM
# Example: 13:51 | 1PM 51 Minutes
PTIME=`date -u +%H:%M`

# Setup script runtime options
if [ "$1" = "--cron" ]
elif [ "$1" != "" ]
                echo "Invalid script option!"
                                echo "Only valid script option(s) are:"
                                echo "--cron | sets the script to assume once a day cron run"
                                echo "No script options will assume script needs to backup recent changes"
                                echo "This will append a timestamp to the end of the generated backup"
                exit 1

# Setup script file names
# Ok if this is cron job lets remove the time stamp.
# Ok if this is a normal execution by a human lets add a time stamp.

if [ "$CRON" = "1" ]

# Lets make a tar-ball


# List date and of backup into backup log
echo >> $LOGFILE
echo Archive date: $PDATE $PTIME >> $LOGFILE

# List files in tar-ball into backup log

# Should we gzip the file to reduce space?
# If yes then lets do IT!
# Also lets md5sum the file

if [ "$COMPRESS" = "ON" ]
                gzip $BACKUPFILE
                md5sum $BACKUPFILE.gz >> $LOGFILE
                md5sum $BACKUPFILE >> $LOGFILE

# Look for old files to remove
if [ $ROLL = ON ]
                echo "Deleting these files" >> $LOGFILE
                find $BACKUPDIR/  -mtime +$ROLLAGE >> $LOGFILE
                find $BACKUPDIR/  -mtime +$ROLLAGE -exec rm -f {} \;
                                du -h $BACKUPDIR/ >> $LOGFILE
                echo "No backup files removed" >> $LOGFILE
                echo "Current Sizes" >> $LOGFILE
                du -h $BACKUPDIR/ >> $LOGFILE

if [ $? != 0 ]
                echo "ERRORS!"
                exit 1
                exit 0

Jasonnadeau 08:06, 13 August 2009 (PDT)

Personal tools
Hot Wiki Topics

About Splunk >
  • Search and navigate IT data from applications, servers and network devices in real-time.
  • Download Splunk