* Add FTP upload, requires lftp.
* Correct a few bugs:
- Don't use -a option with tar, since older versions don't understand it.
* Small improvements
- Use functions to set the final status (SUCCESS, WARNING, ERROR, CRITICAL)
* Update documentation:
- Document the FTP part.
- Add a section about requirements.
- Add a warning about REPODIR being on a backed up device.
- Remove examples from the script itself, everything is in README.
- Some cosmetic changes.
- Renamed briochetab to briochetab.example.
:Author: Amand Tihon
:Contact: <amand.tihon@alrj.org>
-:Version: 1.0
+:Version: 1.1
:Date: Jan 5th, 2008
:Copyright: GNU GPL, see copyright file.
+.. sectnum::
+
+.. contents:: Table of contents
+ :backlinks: none
+
Abstract
========
- Full and differential backups
- LVM snapshots
-- Xen+paravirt oriented (somehow)
-- Soon: Upload to a distant FTP server.
+- Xen oriented (somehow)
+- Upload to a distant FTP server.
Rationale
===============
You can download the tarball from http://www.alrj.org/projects/brioche
-or get the latest development version with the following git command::
+or get the latest development version with the following git command: ::
git clone http://git.alrj.org/git/brioche.git
+A gitweb interface is also available at http://git.alrj.org/
+
+
+Requirements
+============
+
+Brioche relies on a few easily available free software :
+
+- In any case, Brioche will require GNU tar, which is able to deal with
+ incremental backups. Don't even try it with any other tar implementation.
+ Tested with version 1.16.
+- For LVM snapshots, lvm2 will obviously be needed.
+- FTP backups, if they're used, will require lftp.
+
+See the `References`_ section for links to the aforementioned softwares.
+
Installation
============
Copy the three files ``brioche``, ``brioche.conf`` and ``briochetab`` where
you like, and set the ``CONFIG_FILE`` variable in the ``brioche`` script
-accordingly. If needed, run ``chmod +x /path/to/brioche``.
+accordingly. If needed, run ``chmod +x /path/to/brioche``. In a typical setup,
+the ``brioche`` script will be put in ``/usr/local/bin`` with the other two
+files under ``/etc``.
Using Brioche
TAR_OPTS
Additionnal options that you may want to pass to tar. A typical value could
- be ``"--one-filesystem -S"``. The first option will skip all other mointpoints
- (very usefull if you have /dev, /proc, /sys or REPODIR mounted under a device
- that must be archived). The second one will try to deal with sparse files.
+ be ``"--one-file-system -S"``. The first option will skip all other
+ mointpoints (very usefull if you have /dev, /proc, /sys or REPODIR mounted
+ under a device that must be archived). The second one will try to deal with
+ sparse files.
SNAPSHOT_MOUNTPOINT
Sets the directory where the temporary LVM snapshots must be mounted.
are available.
USE_FTP
- Still TODO.
+ If set to "yes", Brioche will upload the backups on an FTP server.
+ See the `Using FTP`_ section for more information about this feature.
+
+FTP_HOST
+ The address of the FTP server.
+
+FTP_DIR
+ The base directory on the FTP server under which all the archives will be
+ stored. Brioche will never touch anything that is not below this directory.
+
+FTP_KEEP
+ Tells Brioche to keep a certain amount of older runs on the FTP. A *run* is
+ a full backup plus all its subsequent differential backups. See the `Using
+ FTP`_ section for a more detailed explanation.
Defining backups
The backups are defined in the file ``birochetab``. Here is a typical example
for a Xen config where cottman is the dom0 and syrtis, kadarin, valeron are
-domUs ::
+domUs: ::
# Partition or LV Snapshot Host name Volume name
# ---------------------------------------------------------------
files will be stored, and how they'll be named. The destination directory will
be created under the REPODIR_, and its name will be the value on the third
column. Inside this directory, archive files will be named from the value given
-in the fourth column. For instance, a full and a differential backup for
-valeron would lead to the following structure ::
+in the fourth column. Grouping by domUs' hostnames is only a suggestion, it can
+be completely different and adapted to suit your needs.
+
+For instance, a full and a differential backup for the host valeron of the
+previous example would lead to the following structure: ::
user:/REPODIR$ ls -l valeron/
total 356544
-rw-r--r-- 1 root root 504725 Jan 4 04:06 root.incr.20090104.snar
-rw-r--r-- 1 root root 160542 Jan 4 04:06 root.incr.20090104.tar.bz2
++--------------------------------------------------------------------------+
+| **WARNING !** |
+| |
+| In case your REPODIR_ is not on a distinct device, it will be included |
+| in the backup if you've included the device in your ``briochetab``. |
+| |
+| To avoid this issue, you can either exclude it explicitely by adding an |
+| "``--exclude=...``" option in TAR_OPTS, or simply specify in your |
+| ``briochetab`` file which directories need to be archived. |
++--------------------------------------------------------------------------+
+
+
+Using FTP
+---------
+
+With the help of lftp, Brioche is able to store an history of backups on an FTP
+server. This is mainly usefull when no other distant repository is available.
+If possible, consider using an CIFS, NFS, sshfs or any other kind of remote
+mountpoint for your REPODIR_.
+
+The archives present in the local REPODIR_ will be mirrored to the FTP server
+after each backup, be it a full or a differential one.
+
+Since there is no way to hide the credentials if they are passed to lftp on the
+command line, the authentication relies on your ``.netrc`` file. See ``man
+netrc(5)`` for more information. In the home directory of the user that runs
+Brioche (typically root's), create the ``.netrc`` file with the following
+lines: ::
+
+ machine ftp.example.com
+ login username
+ password SikRet
+
+Don't forget to secure it with ``chmod 600 .netrc`` or lftp will refuse to use
+it. The machine name must match the FTP_HOST configuration directive in
+``brioche.conf``.
+
+On the FTP server, Brioche will keep a configurable amount of *runs*.
+Each *run* consists of a full backup and all the differential backups that are
+based on it. Before doing a full backup, Brioche will rotate the
+*runs* and keep only the configured number of older backups. The current
+backups can always be found under ``/FTP_DIR/hostname/latest/``. Older ones
+will be under ``/FTP_DIR/hostname/run-X/`` with *X* equal to 1 for the previous
+run, 2 for the one before and so forth, up to the value of FTP_KEEP.
+
+Here's what happens during the rotation:
+
+- the oldest run is removed
+- all the ``run-X/`` directories are shifted (``run-3/`` becomes ``run-4/``,
+ etc)
+- the ``latest/`` directory is renamed to ``run-1/``
+- a new, empty, ``latest/`` directory is created, ready to accept the new files.
+
+
Running Brioche
---------------
Brioche understands the following arguments:
--f, --full Do a full backup (by default, brioche will do a differential).
+-f, --full Do a full backup (by default, brioche will try to do a
+ differential).
-h, --help Show a very limited help.
When everything is ready, execute the ``brioche`` script as root. The
will gracefully fall back and do a full backup if none is available.
If all is fine, it can be added in the system crontab. Here's a suggestion for
-weekly full backup on Sunday, with differential during the weekdays ::
+weekly full backup on Sunday, with differential during the weekdays: ::
# Daily incremental backup
30 3 * * 1-6 /usr/local/bin/brioche > /var/log/backup.`date "+%a"`.log 2>&1
- GNU tar documentation : http://www.gnu.org/software/tar/manual/
- LVM documentation and links : http://sourceware.org/lvm2/
-
+- lftp homepage : http://lftp.yar.ru/
<tr><th class="docinfo-name">Contact:</th>
<td><<a class="reference external" href="mailto:amand.tihon@alrj.org">amand.tihon@alrj.org</a>></td></tr>
<tr><th class="docinfo-name">Version:</th>
-<td>1.0</td></tr>
+<td>1.1</td></tr>
<tr><th class="docinfo-name">Date:</th>
<td>Jan 5th, 2008</td></tr>
<tr><th class="docinfo-name">Copyright:</th>
</tbody>
</table>
<!-- HTML version generated with rst2html -t README > README.html -->
+<div class="contents topic" id="table-of-contents">
+<p class="topic-title first">Table of contents</p>
+<ul class="auto-toc simple">
+<li><a class="reference internal" href="#abstract" id="id2">1 Abstract</a></li>
+<li><a class="reference internal" href="#rationale" id="id3">2 Rationale</a></li>
+<li><a class="reference internal" href="#getting-brioche" id="id4">3 Getting Brioche</a></li>
+<li><a class="reference internal" href="#requirements" id="id5">4 Requirements</a></li>
+<li><a class="reference internal" href="#installation" id="id6">5 Installation</a></li>
+<li><a class="reference internal" href="#using-brioche" id="id7">6 Using Brioche</a><ul class="auto-toc">
+<li><a class="reference internal" href="#configuration" id="id8">6.1 Configuration</a></li>
+<li><a class="reference internal" href="#defining-backups" id="id9">6.2 Defining backups</a></li>
+<li><a class="reference internal" href="#using-ftp" id="id10">6.3 Using FTP</a></li>
+<li><a class="reference internal" href="#running-brioche" id="id11">6.4 Running Brioche</a></li>
+</ul>
+</li>
+<li><a class="reference internal" href="#bug-reporting" id="id12">7 Bug reporting</a></li>
+<li><a class="reference internal" href="#references" id="id13">8 References</a></li>
+</ul>
+</div>
<div class="section" id="abstract">
-<h1>Abstract</h1>
+<h1>1 Abstract</h1>
<p>Brioche is yet another backup shell script. Its main features are</p>
<ul class="simple">
<li>Full and differential backups</li>
<li>LVM snapshots</li>
-<li>Xen+paravirt oriented (somehow)</li>
-<li>Soon: Upload to a distant FTP server.</li>
+<li>Xen oriented (somehow)</li>
+<li>Upload to a distant FTP server.</li>
</ul>
</div>
<div class="section" id="rationale">
-<h1>Rationale</h1>
+<h1>2 Rationale</h1>
<p>Large numbers of backup solutions are freely available today, but when playing
with incremental or differential backups, most of them rely on some filesystem
capabilities, like <em>hard-linking</em>. If the only remote location available to
</table>
</div>
<div class="section" id="getting-brioche">
-<h1>Getting Brioche</h1>
+<h1>3 Getting Brioche</h1>
<p>You can download the tarball from <a class="reference external" href="http://www.alrj.org/projects/brioche">http://www.alrj.org/projects/brioche</a>
or get the latest development version with the following git command:</p>
<pre class="literal-block">
git clone http://git.alrj.org/git/brioche.git
</pre>
+<p>A gitweb interface is also available at <a class="reference external" href="http://git.alrj.org/">http://git.alrj.org/</a></p>
+</div>
+<div class="section" id="requirements">
+<h1>4 Requirements</h1>
+<p>Brioche relies on a few easily available free software :</p>
+<ul class="simple">
+<li>In any case, Brioche will require GNU tar, which is able to deal with
+incremental backups. Don't even try it with any other tar implementation.
+Tested with version 1.16.</li>
+<li>For LVM snapshots, lvm2 will obviously be needed.</li>
+<li>FTP backups, if they're used, will require lftp.</li>
+</ul>
+<p>See the <a class="reference internal" href="#references">References</a> section for links to the aforementioned softwares.</p>
</div>
<div class="section" id="installation">
-<h1>Installation</h1>
+<h1>5 Installation</h1>
<p>Copy the three files <tt class="docutils literal"><span class="pre">brioche</span></tt>, <tt class="docutils literal"><span class="pre">brioche.conf</span></tt> and <tt class="docutils literal"><span class="pre">briochetab</span></tt> where
you like, and set the <tt class="docutils literal"><span class="pre">CONFIG_FILE</span></tt> variable in the <tt class="docutils literal"><span class="pre">brioche</span></tt> script
-accordingly. If needed, run <tt class="docutils literal"><span class="pre">chmod</span> <span class="pre">+x</span> <span class="pre">/path/to/brioche</span></tt>.</p>
+accordingly. If needed, run <tt class="docutils literal"><span class="pre">chmod</span> <span class="pre">+x</span> <span class="pre">/path/to/brioche</span></tt>. In a typical setup,
+the <tt class="docutils literal"><span class="pre">brioche</span></tt> script will be put in <tt class="docutils literal"><span class="pre">/usr/local/bin</span></tt> with the other two
+files under <tt class="docutils literal"><span class="pre">/etc</span></tt>.</p>
</div>
<div class="section" id="using-brioche">
-<h1>Using Brioche</h1>
+<h1>6 Using Brioche</h1>
<div class="section" id="configuration">
-<h2>Configuration</h2>
+<h2>6.1 Configuration</h2>
<p>Edit the file <tt class="docutils literal"><span class="pre">brioche.conf</span></tt> to suit your needs. Each option is commented
inline and will be detailed here.</p>
<dl class="docutils">
not be available with older versions of GNU tar.</dd>
<dt>TAR_OPTS</dt>
<dd>Additionnal options that you may want to pass to tar. A typical value could
-be <tt class="docutils literal"><span class="pre">"--one-filesystem</span> <span class="pre">-S"</span></tt>. The first option will skip all other mointpoints
-(very usefull if you have /dev, /proc, /sys or REPODIR mounted under a device
-that must be archived). The second one will try to deal with sparse files.</dd>
+be <tt class="docutils literal"><span class="pre">"--one-file-system</span> <span class="pre">-S"</span></tt>. The first option will skip all other
+mointpoints (very usefull if you have /dev, /proc, /sys or REPODIR mounted
+under a device that must be archived). The second one will try to deal with
+sparse files.</dd>
<dt>SNAPSHOT_MOUNTPOINT</dt>
<dd>Sets the directory where the temporary LVM snapshots must be mounted.</dd>
<dt>SNAPSHOT_NAME</dt>
<dd>Set the size of the snapshot volume. The same suffix than for lvcreate(8)
are available.</dd>
<dt>USE_FTP</dt>
-<dd>Still TODO.</dd>
+<dd>If set to "yes", Brioche will upload the backups on an FTP server.
+See the <a class="reference internal" href="#using-ftp">Using FTP</a> section for more information about this feature.</dd>
+<dt>FTP_HOST</dt>
+<dd>The address of the FTP server.</dd>
+<dt>FTP_DIR</dt>
+<dd>The base directory on the FTP server under which all the archives will be
+stored. Brioche will never touch anything that is not below this directory.</dd>
+<dt>FTP_KEEP</dt>
+<dd>Tells Brioche to keep a certain amount of older runs on the FTP. A <em>run</em> is
+a full backup plus all its subsequent differential backups. See the <a class="reference internal" href="#using-ftp">Using
+FTP</a> section for a more detailed explanation.</dd>
</dl>
</div>
<div class="section" id="defining-backups">
-<h2>Defining backups</h2>
+<h2>6.2 Defining backups</h2>
<p>The backups are defined in the file <tt class="docutils literal"><span class="pre">birochetab</span></tt>. Here is a typical example
for a Xen config where cottman is the dom0 and syrtis, kadarin, valeron are
-domUs</p>
+domUs:</p>
<pre class="literal-block">
# Partition or LV Snapshot Host name Volume name
# ---------------------------------------------------------------
files will be stored, and how they'll be named. The destination directory will
be created under the <a class="reference internal" href="#repodir">REPODIR</a>, and its name will be the value on the third
column. Inside this directory, archive files will be named from the value given
-in the fourth column. For instance, a full and a differential backup for
-valeron would lead to the following structure</p>
+in the fourth column. Grouping by domUs' hostnames is only a suggestion, it can
+be completely different and adapted to suit your needs.</p>
+<p>For instance, a full and a differential backup for the host valeron of the
+previous example would lead to the following structure:</p>
<pre class="literal-block">
user:/REPODIR$ ls -l valeron/
total 356544
-rw-r--r-- 1 root root 504725 Jan 4 04:06 root.incr.20090104.snar
-rw-r--r-- 1 root root 160542 Jan 4 04:06 root.incr.20090104.tar.bz2
</pre>
+<table border="1" class="docutils">
+<colgroup>
+<col width="100%" />
+</colgroup>
+<tbody valign="top">
+<tr><td><p class="first"><strong>WARNING !</strong></p>
+<p>In case your <a class="reference internal" href="#repodir">REPODIR</a> is not on a distinct device, it will be included
+in the backup if you've included the device in your <tt class="docutils literal"><span class="pre">briochetab</span></tt>.</p>
+<p class="last">To avoid this issue, you can either exclude it explicitely by adding an
+"<tt class="docutils literal"><span class="pre">--exclude=...</span></tt>" option in TAR_OPTS, or simply specify in your
+<tt class="docutils literal"><span class="pre">briochetab</span></tt> file which directories need to be archived.</p>
+</td>
+</tr>
+</tbody>
+</table>
+</div>
+<div class="section" id="using-ftp">
+<h2>6.3 Using FTP</h2>
+<p>With the help of lftp, Brioche is able to store an history of backups on an FTP
+server. This is mainly usefull when no other distant repository is available.
+If possible, consider using an CIFS, NFS, sshfs or any other kind of remote
+mountpoint for your <a class="reference internal" href="#repodir">REPODIR</a>.</p>
+<p>The archives present in the local <a class="reference internal" href="#repodir">REPODIR</a> will be mirrored to the FTP server
+after each backup, be it a full or a differential one.</p>
+<p>Since there is no way to hide the credentials if they are passed to lftp on the
+command line, the authentication relies on your <tt class="docutils literal"><span class="pre">.netrc</span></tt> file. See <tt class="docutils literal"><span class="pre">man</span>
+<span class="pre">netrc(5)</span></tt> for more information. In the home directory of the user that runs
+Brioche (typically root's), create the <tt class="docutils literal"><span class="pre">.netrc</span></tt> file with the following
+lines:</p>
+<pre class="literal-block">
+machine ftp.example.com
+login username
+password SikRet
+</pre>
+<p>Don't forget to secure it with <tt class="docutils literal"><span class="pre">chmod</span> <span class="pre">600</span> <span class="pre">.netrc</span></tt> or lftp will refuse to use
+it. The machine name must match the FTP_HOST configuration directive in
+<tt class="docutils literal"><span class="pre">brioche.conf</span></tt>.</p>
+<p>On the FTP server, Brioche will keep a configurable amount of <em>runs</em>.
+Each <em>run</em> consists of a full backup and all the differential backups that are
+based on it. Before doing a full backup, Brioche will rotate the
+<em>runs</em> and keep only the configured number of older backups. The current
+backups can always be found under <tt class="docutils literal"><span class="pre">/FTP_DIR/hostname/latest/</span></tt>. Older ones
+will be under <tt class="docutils literal"><span class="pre">/FTP_DIR/hostname/run-X/</span></tt> with <em>X</em> equal to 1 for the previous
+run, 2 for the one before and so forth, up to the value of FTP_KEEP.</p>
+<p>Here's what happens during the rotation:</p>
+<ul class="simple">
+<li>the oldest run is removed</li>
+<li>all the <tt class="docutils literal"><span class="pre">run-X/</span></tt> directories are shifted (<tt class="docutils literal"><span class="pre">run-3/</span></tt> becomes <tt class="docutils literal"><span class="pre">run-4/</span></tt>,
+etc)</li>
+<li>the <tt class="docutils literal"><span class="pre">latest/</span></tt> directory is renamed to <tt class="docutils literal"><span class="pre">run-1/</span></tt></li>
+<li>a new, empty, <tt class="docutils literal"><span class="pre">latest/</span></tt> directory is created, ready to accept the new files.</li>
+</ul>
</div>
<div class="section" id="running-brioche">
-<h2>Running Brioche</h2>
+<h2>6.4 Running Brioche</h2>
<p>Brioche understands the following arguments:</p>
<table class="docutils option-list" frame="void" rules="none">
<col class="option" />
<tbody valign="top">
<tr><td class="option-group">
<kbd><span class="option">-f</span>, <span class="option">--full</span></kbd></td>
-<td>Do a full backup (by default, brioche will do a differential).</td></tr>
+<td>Do a full backup (by default, brioche will try to do a
+differential).</td></tr>
<tr><td class="option-group">
<kbd><span class="option">-h</span>, <span class="option">--help</span></kbd></td>
<td>Show a very limited help.</td></tr>
your terminal. By default, Brioche will try to make differential backups, but
will gracefully fall back and do a full backup if none is available.</p>
<p>If all is fine, it can be added in the system crontab. Here's a suggestion for
-weekly full backup on Sunday, with differential during the weekdays</p>
+weekly full backup on Sunday, with differential during the weekdays:</p>
<pre class="literal-block">
# Daily incremental backup
30 3 * * 1-6 /usr/local/bin/brioche > /var/log/backup.`date "+%a"`.log 2>&1
</div>
</div>
<div class="section" id="bug-reporting">
-<h1>Bug reporting</h1>
+<h1>7 Bug reporting</h1>
<p>There's no bugtracker for this project, your bug reports should be sent to the
author : Amand Tihon <<a class="reference external" href="mailto:amand.tihon@alrj.org">amand.tihon@alrj.org</a>>. Please include as much information
as possible in your report.</p>
</div>
<div class="section" id="references">
-<h1>References</h1>
+<h1>8 References</h1>
<ul class="simple">
<li>GNU tar documentation : <a class="reference external" href="http://www.gnu.org/software/tar/manual/">http://www.gnu.org/software/tar/manual/</a></li>
<li>LVM documentation and links : <a class="reference external" href="http://sourceware.org/lvm2/">http://sourceware.org/lvm2/</a></li>
+<li>lftp homepage : <a class="reference external" href="http://lftp.yar.ru/">http://lftp.yar.ru/</a></li>
</ul>
</div>
</div>
<div class="footer">
<hr class="footer" />
-Generated on: 2009-01-05 20:08 UTC.
+Generated on: 2009-01-06 15:56 UTC.
</div>
</body>
-* Implement FTP storage.
-* Don't keep undo directory if FTP backup is used.
-* Better yet, don't keep any archive and snar, except the full.snar if FTP backup is used.
+Brioche to-do list, in random order.
+
+* No need to keep undo directory if FTP backup is used.
+* Better yet, don't keep any archive and snar locally, except the full.snar if
+ FTP backup is used.
+* Usage of FTP could be specified in briochetab for each backup.
* Snapshot size specified in briochetab, or automatically determined.
+* Implement hooks, in various parts of the script, that would allow to run
+ external commands (for instance, to freeze a database just before making a
+ snapshot).
#
# Note: This script relies on GNU tar specific options,
# do not attempt to use it as-is with other tar implementations.
-#
-#
-# Example of use, from crontab
-# # Daily incremental backup
-# 30 3 * * 1-6 /usr/local/bin/brioche > /var/log/backup.`date "+%a"`.log 2>&1
-# # Weekly full backup on Sunday
-# 30 3 * * 0 /usr/local/bin/brioche -f > /var/log/backup.`date "+%a"`.log 2>&1
# Mandatory
SNAPSHOT_SIZE="5G"
USAGE_WARN="80"
-# TODO: Implement the storage on a distant FTP
USE_FTP="no"
-#FTP_HOST="ftpback.example.com"
-#FTP_USER="username"
-#FTP_PASS="password"
-#FTP_KEEP="4"
+FTP_HOST="ftpback.example.com"
+FTP_DIR="/"
+FTP_KEEP="4"
# Ensure that we have a minimal PATH
PATH=/sbin:/bin:/usr/sbin:/usr/bin
echo $@ >> $SUMMARY
}
+# Set final status to CRITICAL
+set_critical()
+{
+ FINAL_STATUS="CRITICAL"
+}
+
+set_error()
+{
+ if [ "$FINAL_STATUS" != "CRITICAL" ]; then
+ FINAL_STATUS="ERROR"
+ fi
+}
+
+set_warning()
+{
+ if [ "$FINAL_STATUS" == "SUCCESS" ]; then
+ FINAL_STATUS="WARNING"
+ fi
+}
+
finish()
{
# Check free space
log "${REPODIR} usage after backup: ${usage}%."
if [ "$usage" -ge "$USAGE_WARN" ]; then
- if [ "$FINAL_STATUS" = "SUCCESS" ]; then
- FINAL_STATUS="WARNING"
- fi
+ set_warning
summary "Warning : Filesystem ${REPODIR} is ${usage}% full."
fi
fi
# Prepare the copy of the snar file
- cp $fullsnar $destsnar
+ cp "$fullsnar" "$destsnar"
# Do the actual backup. Destination file name and snar are like
# /backup/valeron/root.incr.20090105.tar.bz2
# /backup/valeron/root.incr.20090105.snar
log "Running tar..."
- tar -caf ${destfile} ${TAR_OPTS} ${COMPRESS_OPT} -g ${destsnar} $1
+ tar -cf ${destfile} ${TAR_OPTS} ${COMPRESS_OPT} -g ${destsnar} $1
if [ ! "$?" = "0" ]; then
- log "Error $?: Could not archive $hostname - $volumename"
+ log "Error $?: Could not archive $2 - $3"
return 1
fi
return 0
}
+#######################################################################
+# FTP functions
+#######################################################################
+# Push everything in the directory given in $1 to the FTP server, under
+# /FTP_DIR/$2/latest
+ftp_push()
+{
+ log "Mirror $1 on FTP (${FTP_HOST})."
+ local source="${1}"
+ local target="${FTP_DIR}/${2}/latest"
+ local command="mkdir -p ${target}; cd ${target}"
+ command="${command}; mirror --reverse --only-newer --verbose ${source}"
+
+ lftp -e "${command}; exit" ${FTP_HOST}
+}
+
+# Rotate old files on FTP
+# Usage: ftp_rotate group
+ftp_rotate()
+{
+ log "Rotating backups of $1 on FTP (${FTP_HOST})."
+ local lastrun="run-${FTP_KEEP}"
+ local target="${FTP_DIR}/${1}"
+ local commands="mkdir -p ${target}; cd ${target}"
+
+ # Build commands
+ # Remove oldest run
+ if [ "$FTP_KEEP" != "0" ]; then
+ commands="$commands; rmdir ${lastrun}"
+
+ # Move everything back
+ for run in `seq $FTP_KEEP -1 2`; do
+ local newer=$run
+ let "newer -= 1"
+ commands="$commands; mv run-$newer run-$run"
+ done
+ # Move "old latest" to run-1
+ commands="$commands; mv latest run-1"
+ else
+ commands="$commands; rmdir latest"
+ fi
+
+ # Create "new latest" directory
+ commands="$commands; mkdir latest; exit"
+
+ # Run the commands on the FTP server
+ lftp -e "$commands" $FTP_HOST
+}
#######################################################################
# Start here
#######################################################################
source "${CONFIG_FILE}"
else
summary "Error: Unable to read configuration file ${CONFIG_FILE}. Aborting."
- FINAL_STATUS="CRITICAL"
+ set_critical
finish
fi
if [ ! -r "${BACKUPTAB}" ]
then
summary "Error: Unable to read ${BACKUPTAB}. Aborting."
- FINAL_STATUS="CRITICAL"
+ set_critical
finish
fi
# Parse backuptab file, call backup functions for each line
#######################################################################
+# Rotate FTP groups on full
+if [ "$USE_FTP" = "yes" -a "$DO_FULL_BACKUP" = "yes" ]; then
+ for group in `grep -v -E '^[[:space:]]*(#.*)?$' $BACKUPTAB | awk '{print $3}' | sort -u`
+ do
+ ftp_rotate $group
+ done
+fi
+
# Ignore empty and commented lines
grep -v -E '^[[:space:]]*(#.*)?$' $BACKUPTAB | tr -s [:space:]| while read line
do
make_snapshot $vg $lv
if [ "$?" != "0" ]; then
summary "Could not take a snapshot of $device"
- FINAL_STATUS="ERROR"
+ set_error
continue
# Next one
fi
RETVAL="$?"
if [ "$RETVAL" != "0" ]; then
summary "Could not mount the snapshot of $device"
- FINAL_STATUS="ERROR"
+ set_error
if [ "$RETVAL" = "100" ]; then
summary "Could not remove the snapshot !"
- FINAL_STATUS="CRITICAL"
+ set_critical
finish
fi
continue
summary "INCREMENTAL backup of $device done on `NOW`."
elif [ "$RETVAL" = "1" ]; then
summary "Error during incremental backup of $device"
- FINAL_STATUS="ERROR"
+ set_error
elif [ "$RETVAL" = "2" ]; then
summary "Can't do an incremental backup without a full one being present."
summary "Switching to full backup for $device"
summary "FULL backup of $device done on `NOW`."
else
summary "Error during full backup of $device"
- FINAL_STATUS="ERROR"
+ set_error
fi
else
summary "Unknown error during incremental backup of $device"
- FINAL_STATUS="ERROR"
+ set_error
fi
else # Do a full backup
make_full_backup ${BACKUP_SOURCE} $group $volume
summary "FULL backup of $device done on `NOW`."
else
summary "Error during full backup of $device"
- FINAL_STATUS="ERROR"
+ set_error
fi
fi
unmount_snapshot $vg
if [ "$?" != "0" ]; then
summary "Could not unmount snapshot !"
- FINAL_STATUS="CRITICAL"
+ set_critical
finish
fi
remove_snapshot $vg
if [ "$?" != "0" ]; then
summary "Could not destroy the snapshot !"
- FINAL_STATUS="CRITICAL"
+ set_critical
finish
fi
fi
+
done
-finish
+# Push everything on the FTP
+if [ "$USE_FTP" = "yes" ]; then
+ summary ""
+ for group in `grep -v -E '^[[:space:]]*(#.*)?$' $BACKUPTAB | awk '{print $3}' | sort -u`
+ do
+ ftp_push "$REPODIR/$group" $group
+ summary "Mirrored ${group} to ${FTP_HOST}."
+ done
+fi
-## Example Backup description table
-##
-## Can handle logical volumes, with snapshots, or plain mountpoints.
-## Priority is not used yet.
-#
-## Partition or LV Snapshot Host name Volume name Priority
-## ----------------------------------------------------------------------------
-#/ no cottman root 1
-#/dev/vg00/kadarin-root yes kadarin root 1
-#/dev/vg00/valeron-root yes valeron root 1
-#/dev/vg00/syrtis-root yes syrtis root 1
-#/dev/vg00/syrtis-home yes syrtis home 1
-#/dev/vg00/syrtis-usr yes syrtis usr 1
-#/dev/vg00/syrtis-var yes syrtis var 1
-#/dev/vg00/syrtis-srv yes syrtis srv 1
+
+finish
COMPRESS="gz"
# TAR_OPTS: Additionnal options to the tar invocation.
-TAR_OPTS="--one-filesystem -S"
+TAR_OPTS="--one-file-system -S"
# SNAPSHOT_MOUNTPOINT: Where to mount the temporary LVM snaphot
SNAPSHOT_MOUNTPOINT="/mnt/backup-snapshot"
# automatically, rendering the backup unusable.
SNAPSHOT_SIZE="5G"
-# USE_FTP: Not yet implemented.
-USE_FTP="no"
+# USE_FTP: Mirror the local backup directory on a remote FTP
+USE_FTP="yes"
-# FTP_HOST="ftpback.example.com"
-# FTP_USER="username"
-# FTP_PASS="password"
-# FTP_KEEP="4"
+# FTP_HOST: address of the FTP server
+FTP_HOST="ftp.example.com"
+
+# FTP_DIR: Target directory on the FTP server
+FTP_DIR="/"
+
+# Authentication must be performed with .netrc to avoid giving login/password
+# on the command line.
+
+# How many old full runs sould be kept on the FTP server
+FTP_KEEP="4"
+++ /dev/null
-# Backup description table
-#
-# Can handle logical volumes, with snapshots
-# In this example, the first partition to backup is the root of the dom0.
-# The following lines are the different LVM volumes created for the domUs.
-
-# Partition or LV Snapshot Host name Volume name Priority
-# ----------------------------------------------------------------------------
-/ no cottman root 1
-/dev/vg00/valeron-root yes valeron root 1
-/dev/vg00/kadarin-root yes kadarin root 1
-/dev/vg00/syrtis-root yes syrtis root 1
-/dev/vg00/syrtis-home yes syrtis home 1
-/dev/vg00/syrtis-usr yes syrtis usr 1
-/dev/vg00/syrtis-var yes syrtis var 1
-/dev/vg00/syrtis-srv yes syrtis srv 1
--- /dev/null
+# Backup description table
+#
+# Can handle logical volumes, with snapshots
+# In this example, the first partition to backup is the root of the dom0.
+# The following lines are the different LVM volumes created for the domUs.
+
+# Partition or LV Snapshot Host name Volume name Priority
+# ----------------------------------------------------------------------------
+/ no cottman root 1
+/dev/vg00/valeron-root yes valeron root 1
+/dev/vg00/kadarin-root yes kadarin root 1
+/dev/vg00/syrtis-root yes syrtis root 1
+/dev/vg00/syrtis-home yes syrtis home 1
+/dev/vg00/syrtis-usr yes syrtis usr 1
+/dev/vg00/syrtis-var yes syrtis var 1
+/dev/vg00/syrtis-srv yes syrtis srv 1
+brioche 1.1
+
+ [ Amand Tihon ]
+ * Add FTP upload, requires lftp.
+ * Correct a few bugs:
+ - Don't use -a option with tar, since older versions don't understand it.
+ * Small improvements
+ - Use functions to set the final status (SUCCESS, WARNING, ERROR, CRITICAL)
+ * Update documentation:
+ - Document the FTP part.
+ - Add a section about requirements.
+ - Add a warning about REPODIR being on a backed up device.
+ - Remove examples from the script itself, everything is in README.
+ - Some cosmetic changes.
+ - Renamed briochetab to briochetab.example.
+
+ -- Amand Tihon <amand.tihon@alrj.org> Mon, 6 Jan 2008 17:00:00 +0100
+
brioche 1.0
[ Amand Tihon ]
projects / brioche.git / commitdiff