<meta name="generator" content="Docutils 0.5: http://docutils.sourceforge.net/" />
<title>Brioche Backup</title>
<meta name="author" content="Amand Tihon" />
-<meta name="date" content="Jan 5th, 2008" />
+<meta name="date" content="Jan 9, 2009" />
<meta name="copyright" content="GNU GPL, see copyright file." />
<style type="text/css">
<tr><th class="docinfo-name">Contact:</th>
<td><<a class="reference external" href="mailto:amand.tihon@alrj.org">amand.tihon@alrj.org</a>></td></tr>
<tr><th class="docinfo-name">Version:</th>
-<td>1.0</td></tr>
+<td>1.2</td></tr>
<tr><th class="docinfo-name">Date:</th>
-<td>Jan 5th, 2008</td></tr>
+<td>Jan 9, 2009</td></tr>
<tr><th class="docinfo-name">Copyright:</th>
<td>GNU GPL, see copyright file.</td></tr>
</tbody>
</table>
<!-- HTML version generated with rst2html -t README > README.html -->
+<div class="contents topic" id="table-of-contents">
+<p class="topic-title first">Table of contents</p>
+<ul class="auto-toc simple">
+<li><a class="reference internal" href="#abstract" id="id2">1 Abstract</a></li>
+<li><a class="reference internal" href="#rationale" id="id3">2 Rationale</a></li>
+<li><a class="reference internal" href="#getting-brioche" id="id4">3 Getting Brioche</a></li>
+<li><a class="reference internal" href="#requirements" id="id5">4 Requirements</a></li>
+<li><a class="reference internal" href="#installation" id="id6">5 Installation</a></li>
+<li><a class="reference internal" href="#using-brioche" id="id7">6 Using Brioche</a><ul class="auto-toc">
+<li><a class="reference internal" href="#configuration" id="id8">6.1 Configuration</a></li>
+<li><a class="reference internal" href="#defining-backups" id="id9">6.2 Defining backups</a></li>
+<li><a class="reference internal" href="#encryption-with-gnupg" id="id10">6.3 Encryption with GnuPG</a></li>
+<li><a class="reference internal" href="#using-ftp" id="id11">6.4 Using FTP</a></li>
+<li><a class="reference internal" href="#running-brioche" id="id12">6.5 Running Brioche</a></li>
+</ul>
+</li>
+<li><a class="reference internal" href="#bug-reporting" id="id13">7 Bug reporting</a></li>
+<li><a class="reference internal" href="#references" id="id14">8 References</a></li>
+</ul>
+</div>
<div class="section" id="abstract">
-<h1>Abstract</h1>
+<h1>1 Abstract</h1>
<p>Brioche is yet another backup shell script. Its main features are</p>
<ul class="simple">
<li>Full and differential backups</li>
<li>LVM snapshots</li>
-<li>Xen+paravirt oriented (somehow)</li>
-<li>Soon: Upload to a distant FTP server.</li>
+<li>Xen oriented (somehow)</li>
+<li>Upload to a distant FTP server.</li>
</ul>
</div>
<div class="section" id="rationale">
-<h1>Rationale</h1>
+<h1>2 Rationale</h1>
<p>Large numbers of backup solutions are freely available today, but when playing
with incremental or differential backups, most of them rely on some filesystem
capabilities, like <em>hard-linking</em>. If the only remote location available to
</table>
</div>
<div class="section" id="getting-brioche">
-<h1>Getting Brioche</h1>
+<h1>3 Getting Brioche</h1>
<p>You can download the tarball from <a class="reference external" href="http://www.alrj.org/projects/brioche">http://www.alrj.org/projects/brioche</a>
or get the latest development version with the following git command:</p>
<pre class="literal-block">
-git clone http://git.alrj.org/brioche.git
+git clone http://git.alrj.org/git/brioche.git
</pre>
+<p>A gitweb interface is also available at <a class="reference external" href="http://git.alrj.org/">http://git.alrj.org/</a></p>
+</div>
+<div class="section" id="requirements">
+<h1>4 Requirements</h1>
+<p>Brioche relies on a few easily available free software :</p>
+<ul class="simple">
+<li>In any case, Brioche will require GNU tar, which is able to deal with
+incremental backups. Don't even try it with any other tar implementation.
+Tested with version 1.16.</li>
+<li>For LVM snapshots, lvm2 will obviously be needed.</li>
+<li>FTP backups, if they're used, will require lftp.</li>
+</ul>
+<p>See the <a class="reference internal" href="#references">References</a> section for links to the aforementioned softwares.</p>
</div>
<div class="section" id="installation">
-<h1>Installation</h1>
+<h1>5 Installation</h1>
<p>Copy the three files <tt class="docutils literal"><span class="pre">brioche</span></tt>, <tt class="docutils literal"><span class="pre">brioche.conf</span></tt> and <tt class="docutils literal"><span class="pre">briochetab</span></tt> where
you like, and set the <tt class="docutils literal"><span class="pre">CONFIG_FILE</span></tt> variable in the <tt class="docutils literal"><span class="pre">brioche</span></tt> script
-accordingly. If needed, run <tt class="docutils literal"><span class="pre">chmod</span> <span class="pre">+x</span> <span class="pre">/path/to/brioche</span></tt>.</p>
+accordingly. If needed, run <tt class="docutils literal"><span class="pre">chmod</span> <span class="pre">+x</span> <span class="pre">/path/to/brioche</span></tt>. In a typical setup,
+the <tt class="docutils literal"><span class="pre">brioche</span></tt> script will be put in <tt class="docutils literal"><span class="pre">/usr/local/bin</span></tt> with the other two
+files under <tt class="docutils literal"><span class="pre">/etc</span></tt>.</p>
</div>
<div class="section" id="using-brioche">
-<h1>Using Brioche</h1>
+<h1>6 Using Brioche</h1>
<div class="section" id="configuration">
-<h2>Configuration</h2>
+<h2>6.1 Configuration</h2>
<p>Edit the file <tt class="docutils literal"><span class="pre">brioche.conf</span></tt> to suit your needs. Each option is commented
inline and will be detailed here.</p>
<dl class="docutils">
beyond the given threshold. The value must be an integer.</dd>
<dt>COMPRESS</dt>
<dd>This directive allows to specify the compression method to apply to the
-archives. Possible values are "gz", "bz2" and "lzma". <strong>Warning</strong> : lzma may
-not be available with older versions of GNU tar.</dd>
+archives. Possible values are "none", "gz", "bz2" and "lzma".
+<strong>Warning</strong> : lzma may not be available with older versions of GNU tar.</dd>
<dt>TAR_OPTS</dt>
<dd>Additionnal options that you may want to pass to tar. A typical value could
-be <tt class="docutils literal"><span class="pre">"--one-filesystem</span> <span class="pre">-S"</span></tt>. The first option will skip all other mointpoints
-(very usefull if you have /dev, /proc, /sys or REPODIR mounted under a device
-that must be archived). The second one will try to deal with sparse files.</dd>
+be <tt class="docutils literal"><span class="pre">"--one-file-system</span> <span class="pre">-S"</span></tt>. The first option will skip all other
+mointpoints (very usefull if you have /dev, /proc, /sys or REPODIR mounted
+under a device that must be archived). The second one will try to deal with
+sparse files.</dd>
<dt>SNAPSHOT_MOUNTPOINT</dt>
<dd>Sets the directory where the temporary LVM snapshots must be mounted.</dd>
<dt>SNAPSHOT_NAME</dt>
<dt>SNAPSHOT_SIZE</dt>
<dd>Set the size of the snapshot volume. The same suffix than for lvcreate(8)
are available.</dd>
+<dt>USE_GPG</dt>
+<dd>When set to "yes", Brioche will encrypt the archives with GnuPG.
+See the <a class="reference internal" href="#encryption-with-gnupg">Encryption with GnuPG</a> section below for a detailed explanation on
+how to use GnuPG with Brioche.</dd>
+<dt>GPG_KEY</dt>
+<dd>The identifier of the public GnuPG key to use when encrypting the archives.
+This is the key that will be needed in case of restore.</dd>
+<dt>GPG_PASSPHRASE</dt>
+<dd>The passphrase for the GnuPG private key used to encrypt the archives.</dd>
<dt>USE_FTP</dt>
-<dd>Still TODO.</dd>
+<dd>If set to "yes", Brioche will upload the backups on an FTP server.
+See the <a class="reference internal" href="#using-ftp">Using FTP</a> section for more information about this feature.</dd>
+<dt>FTP_HOST</dt>
+<dd>The address of the FTP server.</dd>
+<dt>FTP_DIR</dt>
+<dd>The base directory on the FTP server under which all the archives will be
+stored. Brioche will never touch anything that is not below this directory.</dd>
+<dt>FTP_KEEP</dt>
+<dd>Tells Brioche to keep a certain amount of older runs on the FTP. A <em>run</em> is
+a full backup plus all its subsequent differential backups. See the <a class="reference internal" href="#using-ftp">Using
+FTP</a> section for a more detailed explanation.</dd>
</dl>
</div>
<div class="section" id="defining-backups">
-<h2>Defining backups</h2>
+<h2>6.2 Defining backups</h2>
<p>The backups are defined in the file <tt class="docutils literal"><span class="pre">birochetab</span></tt>. Here is a typical example
for a Xen config where cottman is the dom0 and syrtis, kadarin, valeron are
-domUs</p>
+domUs:</p>
<pre class="literal-block">
# Partition or LV Snapshot Host name Volume name
# ---------------------------------------------------------------
files will be stored, and how they'll be named. The destination directory will
be created under the <a class="reference internal" href="#repodir">REPODIR</a>, and its name will be the value on the third
column. Inside this directory, archive files will be named from the value given
-in the fourth column. For instance, a full and a differential backup for
-valeron would lead to the following structure</p>
+in the fourth column. Grouping by domUs' hostnames is only a suggestion, it can
+be completely different and adapted to suit your needs.</p>
+<p>For instance, a full and a differential backup for the host valeron of the
+previous example would lead to the following structure:</p>
<pre class="literal-block">
user:/REPODIR$ ls -l valeron/
total 356544
-rw-r--r-- 1 root root 504725 Jan 4 04:06 root.incr.20090104.snar
-rw-r--r-- 1 root root 160542 Jan 4 04:06 root.incr.20090104.tar.bz2
</pre>
+<table border="1" class="docutils">
+<colgroup>
+<col width="100%" />
+</colgroup>
+<tbody valign="top">
+<tr><td><p class="first"><strong>WARNING !</strong></p>
+<p>In case your <a class="reference internal" href="#repodir">REPODIR</a> is not on a distinct device, it will be included
+in the backup if you've included the device in your <tt class="docutils literal"><span class="pre">briochetab</span></tt>.</p>
+<p class="last">To avoid this issue, you can either exclude it explicitely by adding an
+"<tt class="docutils literal"><span class="pre">--exclude=...</span></tt>" option in TAR_OPTS, or simply specify in your
+<tt class="docutils literal"><span class="pre">briochetab</span></tt> file which directories need to be archived.</p>
+</td>
+</tr>
+</tbody>
+</table>
+</div>
+<div class="section" id="encryption-with-gnupg">
+<h2>6.3 Encryption with GnuPG</h2>
+</div>
+<div class="section" id="using-ftp">
+<h2>6.4 Using FTP</h2>
+<p>With the help of lftp, Brioche is able to store an history of backups on an FTP
+server. This is mainly usefull when no other distant repository is available.
+If possible, consider using an CIFS, NFS, sshfs or any other kind of remote
+mountpoint for your <a class="reference internal" href="#repodir">REPODIR</a>.</p>
+<p>The archives present in the local <a class="reference internal" href="#repodir">REPODIR</a> will be mirrored to the FTP server
+after each backup, be it a full or a differential one.</p>
+<p>Since there is no way to hide the credentials if they are passed to lftp on the
+command line, the authentication relies on your <tt class="docutils literal"><span class="pre">.netrc</span></tt> file. See <tt class="docutils literal"><span class="pre">man</span>
+<span class="pre">netrc(5)</span></tt> for more information. In the home directory of the user that runs
+Brioche (typically root's), create the <tt class="docutils literal"><span class="pre">.netrc</span></tt> file with the following
+lines:</p>
+<pre class="literal-block">
+machine ftp.example.com
+login username
+password SikRet
+</pre>
+<p>Don't forget to secure it with <tt class="docutils literal"><span class="pre">chmod</span> <span class="pre">600</span> <span class="pre">.netrc</span></tt> or lftp will refuse to use
+it. The machine name must match the FTP_HOST configuration directive in
+<tt class="docutils literal"><span class="pre">brioche.conf</span></tt>.</p>
+<p>On the FTP server, Brioche will keep a configurable amount of <em>runs</em>.
+Each <em>run</em> consists of a full backup and all the differential backups that are
+based on it. Before doing a full backup, Brioche will rotate the
+<em>runs</em> and keep only the configured number of older backups. The current
+backups can always be found under <tt class="docutils literal"><span class="pre">/FTP_DIR/hostname/latest/</span></tt>. Older ones
+will be under <tt class="docutils literal"><span class="pre">/FTP_DIR/hostname/run-X/</span></tt> with <em>X</em> equal to 1 for the previous
+run, 2 for the one before and so forth, up to the value of FTP_KEEP.</p>
+<p>Here's what happens during the rotation:</p>
+<ul class="simple">
+<li>the oldest run is removed</li>
+<li>all the <tt class="docutils literal"><span class="pre">run-X/</span></tt> directories are shifted (<tt class="docutils literal"><span class="pre">run-3/</span></tt> becomes <tt class="docutils literal"><span class="pre">run-4/</span></tt>,
+etc)</li>
+<li>the <tt class="docutils literal"><span class="pre">latest/</span></tt> directory is renamed to <tt class="docutils literal"><span class="pre">run-1/</span></tt></li>
+<li>a new, empty, <tt class="docutils literal"><span class="pre">latest/</span></tt> directory is created, ready to accept the new files.</li>
+</ul>
</div>
<div class="section" id="running-brioche">
-<h2>Running Brioche</h2>
+<h2>6.5 Running Brioche</h2>
<p>Brioche understands the following arguments:</p>
<table class="docutils option-list" frame="void" rules="none">
<col class="option" />
<tbody valign="top">
<tr><td class="option-group">
<kbd><span class="option">-f</span>, <span class="option">--full</span></kbd></td>
-<td>Do a full backup (by default, brioche will do a differential).</td></tr>
+<td>Do a full backup (by default, brioche will try to do a
+differential).</td></tr>
<tr><td class="option-group">
<kbd><span class="option">-h</span>, <span class="option">--help</span></kbd></td>
<td>Show a very limited help.</td></tr>
your terminal. By default, Brioche will try to make differential backups, but
will gracefully fall back and do a full backup if none is available.</p>
<p>If all is fine, it can be added in the system crontab. Here's a suggestion for
-weekly full backup on Sunday, with differential during the weekdays</p>
+weekly full backup on Sunday, with differential during the weekdays:</p>
<pre class="literal-block">
# Daily incremental backup
-30 3 * * 1-6 /usr/local/bin/brioche > /var/log/backup.`date "+%a"`.log 2>&1
+30 3 * * 1-6 /usr/local/bin/brioche > /var/log/backup.`dow`.log 2>&1
# Weekly full backup on Sunday
-30 3 * * 0 /usr/local/bin/brioche -f > /var/log/backup.`date "+%a"`.log 2>&1
+30 3 * * 0 /usr/local/bin/brioche -f > /var/log/backup.`dow`.log 2>&1
</pre>
<p>The output of the script will be saved in /var/log/backup.DOW.log with DOW
-being the abbreviated day of the week (see man date(1) for the format).</p>
+being the abbreviated day of the week (see man date(1) for the format).
+The <tt class="docutils literal"><span class="pre">dow</span></tt> bash script is provided alongside Brioche.</p>
</div>
</div>
<div class="section" id="bug-reporting">
-<h1>Bug reporting</h1>
+<h1>7 Bug reporting</h1>
<p>There's no bugtracker for this project, your bug reports should be sent to the
author : Amand Tihon <<a class="reference external" href="mailto:amand.tihon@alrj.org">amand.tihon@alrj.org</a>>. Please include as much information
as possible in your report.</p>
</div>
<div class="section" id="references">
-<h1>References</h1>
+<h1>8 References</h1>
<ul class="simple">
<li>GNU tar documentation : <a class="reference external" href="http://www.gnu.org/software/tar/manual/">http://www.gnu.org/software/tar/manual/</a></li>
<li>LVM documentation and links : <a class="reference external" href="http://sourceware.org/lvm2/">http://sourceware.org/lvm2/</a></li>
+<li>lftp homepage : <a class="reference external" href="http://lftp.yar.ru/">http://lftp.yar.ru/</a></li>
</ul>
</div>
</div>
<div class="footer">
<hr class="footer" />
-Generated on: 2009-01-05 19:27 UTC.
+Generated on: 2009-01-09 14:26 UTC.
</div>
</body>
projects / brioche.git / blobdiff