<?xml version="1.0" encoding="utf-8" standalone="no"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Maintaining Berkeley DB Repositories</title>
<link rel="stylesheet" type="text/css" href="styles.css" />
<meta name="generator" content="DocBook XSL Stylesheets V1.79.1" />
<style type="text/css">
body { background-image: url('images/draft.png');
background-repeat: no-repeat;
background-position: top left;
/* The following properties make the watermark "fixed" on the page. */
/* I think that's just a bit too distracting for the reader... */
/* background-attachment: fixed; */
/* background-position: center center; */
}</style>
<link rel="home" href="index.html" title="Version Control with Subversion [DRAFT]" />
<link rel="up" href="svn.berkeleydb.html" title="Appendix D. The Berkeley DB Legacy Filesystem" />
<link rel="prev" href="svn.berkeleydb.limitations.html" title="Limitations of Berkeley DB" />
<link rel="next" href="svn.copyright.html" title="Appendix E. Copyright" />
</head>
<body>
<div xmlns="" id="vcws-version-notice">
<p>This text is a work in progress—highly subject to
change—and may not accurately describe any released
version of the Apache™ Subversion® software.
Bookmarking or otherwise referring others to this page is
probably not such a smart idea. Please visit
<a href="http://www.svnbook.com/">http://www.svnbook.com/</a>
for stable versions of this book.</p>
</div>
<div class="navheader">
<table width="100%" summary="Navigation header">
<tr>
<th colspan="3" align="center">Maintaining Berkeley DB Repositories</th>
</tr>
<tr>
<td width="20%" align="left"><a accesskey="p" href="svn.berkeleydb.limitations.html">Prev</a> </td>
<th width="60%" align="center">Appendix D. The Berkeley DB Legacy Filesystem</th>
<td width="20%" align="right"> <a accesskey="n" href="svn.copyright.html">Next</a></td>
</tr>
</table>
<hr />
</div>
<div class="sect1">
<div class="titlepage">
<div>
<div>
<h2 class="title" style="clear: both"><a id="svn.berkeleydb.maintenance"></a>Maintaining Berkeley DB Repositories</h2>
</div>
</div>
</div>
<p>In theory, the maintenance of a BDB-backed repository
involves essentially the same steps used to maintain an
FSFS-backed repository. Historically, though, Berkeley DB
repositories have need a little extra TLC<a href="#ftn.idm17144" class="footnote" id="idm17144"><sup class="footnote">[89]</sup></a> in order to stay
operational. This section will cover some of the unique aspects
of Berkeley DB administration.</p>
<div class="sect2">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a id="svn.berkeleydb.maintenance.recovery"></a>Berkeley DB Recovery</h3>
</div>
</div>
</div>
<p>As mentioned in
<a class="xref" href="svn.berkeleydb.limitations.html#svn.berkeleydb.limitations.faulttolerance" title="Fault Tolerance and the Need for Recovery">the section called “Fault Tolerance and the Need for Recovery”</a>,
a Berkeley DB repository can sometimes be left in a frozen
state if not closed properly. When this happens, an
administrator needs to rewind the database back into a
consistent state. This is unique to BDB-backed repositories,
though—if you are using FSFS-backed ones instead, this
won't apply to you. And for those of you using Subversion 1.4
with Berkeley DB 4.4 or later, you should find that Subversion
has become much more resilient in these types of situations.
Still, wedged Berkeley DB repositories do occur, and an
administrator needs to know how to safely deal with this
circumstance.</p>
<p>To protect the data in your repository, Berkeley
DB uses a locking mechanism. This mechanism ensures that
portions of the database are not simultaneously modified by
multiple database accessors, and that each process sees the
data in the correct state when that data is being read from
the database. When a process needs to change something in the
database, it first checks for the existence of a lock on the
target data. If the data is not locked, the process locks the
data, makes the change it wants to make, and then unlocks the
data. Other processes are forced to wait until that lock is
removed before they are permitted to continue accessing that
section of the database. (This has nothing to do with the
locks that you, as a user, can apply to versioned files within
the repository; we try to clear up the confusion caused by
this terminology collision in the sidebar <a class="xref" href="svn.advanced.locking.html#svn.advanced.locking.meanings" title="The Three Meanings of “Lock”">The Three Meanings of <span class="quote">“<span class="quote">Lock</span>”</span></a>.)</p>
<p>In the course of using your Subversion repository, fatal
errors or interruptions can prevent a process from having the
chance to remove the locks it has placed in the database. The
result is that the backend database system gets
<span class="quote">“<span class="quote">wedged.</span>”</span> When this happens, any attempts to
access the repository hang indefinitely (since each new
accessor is waiting for a lock to go away—which isn't
going to happen).</p>
<p>If this happens to your repository, don't panic. The
Berkeley DB filesystem takes advantage of database
transactions, checkpoints, and prewrite journaling to ensure
that only the most catastrophic of events<a href="#ftn.idm17155" class="footnote" id="idm17155"><sup class="footnote">[90]</sup></a> can permanently destroy a database
environment. A sufficiently paranoid repository administrator
will have made off-site backups of the repository data in some
fashion, but don't head off to the tape backup storage closet
just yet.</p>
<p>Instead, use the following recipe to attempt to
<span class="quote">“<span class="quote">unwedge</span>”</span> your repository:</p>
<div class="orderedlist">
<ol class="orderedlist" type="1">
<li class="listitem">
<p>Make sure no processes are accessing (or
attempting to access) the repository. For networked
repositories, this also means shutting down the Apache HTTP
Server or svnserve daemon.</p>
</li>
<li class="listitem">
<p>Become the user who owns and manages the repository.
This is important, as recovering a repository while
running as the wrong user can tweak the permissions of the
repository's files in such a way that your repository will
still be inaccessible even after it is
<span class="quote">“<span class="quote">unwedged.</span>”</span></p>
</li>
<li class="listitem">
<p>Run the <span class="command"><strong>svnadmin recover</strong></span> command:</p>
<div class="informalexample">
<pre class="screen">
$ svnadmin recover /var/svn/repos
Repository lock acquired.
Please wait; recovering the repository may take some time...
Recovery completed.
The latest repos revision is 19.
$
</pre>
</div>
<p>This command may take many minutes to complete.</p>
</li>
<li class="listitem">
<p>Restart the server process.</p>
</li>
</ol>
</div>
<p>This procedure fixes almost every case of repository
wedging. Make sure that you run this command as the user that
owns and manages the database, not just as
<code class="literal">root</code>. Part of the recovery process might
involve re-creating from scratch various database files (shared
memory regions, e.g.). Recovering as
<code class="literal">root</code> will create those files such that they
are owned by <code class="literal">root</code>, which means that even
after you restore connectivity to your repository, regular
users will be unable to access it.</p>
</div>
<div class="sect2">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a id="svn.berkeleydb.maintenance.bdblogs"></a>Purging unused Berkeley DB logfiles</h3>
</div>
</div>
</div>
<p>Prior to the release of Berkeley DB 4.2, the largest
offender of disk space usage with respect to BDB-backed
Subversion repositories were the logfiles in which Berkeley DB
performs its prewrites before modifying the actual database
files. These files capture all the actions taken along the
route of changing the database from one state to
another—while the database files, at any given time,
reflect a particular state, the logfiles contain all of the
many changes along the way
<span class="emphasis"><em>between</em></span> states. Thus, they can grow
and accumulate quite rapidly.</p>
<p>Fortunately, beginning with the 4.2 release of Berkeley
DB, the database environment has the ability to remove its
own unused logfiles automatically. Any
repositories created using <span class="command"><strong>svnadmin</strong></span>
when compiled against Berkeley DB version 4.2 or later
will be configured for this automatic logfile removal. If
you don't want this feature enabled, simply pass the
<code class="option">--bdb-log-keep</code> option to the
<span class="command"><strong>svnadmin create</strong></span> command. If you forget
to do this or change your mind at a later time, simply edit
the <code class="filename">DB_CONFIG</code> file found in your
repository's <code class="filename">db</code> directory, comment out
the line that contains the <code class="literal">set_flags
DB_LOG_AUTOREMOVE</code> directive, and then run
<span class="command"><strong>svnadmin recover</strong></span> on your repository to
force the configuration changes to take effect.</p>
<p>Without some sort of automatic logfile removal in
place, logfiles will accumulate as you use your repository.
This is actually somewhat of a feature of the database
system—you should be able to recreate your entire
database using nothing but the logfiles, so these files can
be useful for catastrophic database recovery. But
typically, you'll want to archive the logfiles that are no
longer in use by Berkeley DB, and then remove them from disk
to conserve space. Use the <span class="command"><strong>svnadmin
list-unused-dblogs</strong></span> command to list the unused
logfiles:</p>
<div class="informalexample">
<pre class="screen">
$ svnadmin list-unused-dblogs /var/svn/repos
/var/svn/repos/log.0000000031
/var/svn/repos/log.0000000032
/var/svn/repos/log.0000000033
…
$ rm `svnadmin list-unused-dblogs /var/svn/repos`
## disk space reclaimed!
</pre>
</div>
<div class="warning" style="margin-left: 0.5in; margin-right: 0.5in;">
<table border="0" summary="Warning">
<tr>
<td rowspan="2" align="center" valign="top" width="25">
<img alt="[Warning]" src="images/warning.png" />
</td>
<th align="left">Warning</th>
</tr>
<tr>
<td align="left" valign="top">
<p>BDB-backed repositories whose logfiles are used as
part of a backup or disaster recovery plan should
<span class="emphasis"><em>not</em></span> make use of the logfile
autoremoval feature. Reconstruction of a repository's
data from logfiles can only be accomplished only when
<span class="emphasis"><em>all</em></span> the logfiles are available. If
some of the logfiles are removed from disk before the
backup system has a chance to copy them elsewhere, the
incomplete set of backed-up logfiles is essentially
useless.</p>
</td>
</tr>
</table>
</div>
</div>
<div class="sect2">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a id="svn.berkeleydb.maintenance.bdbutil"></a>Berkeley DB Utilities</h3>
</div>
</div>
</div>
<p>If you're using a Berkeley DB repository, all of
your versioned filesystem's structure and data live in a set
of database tables within the <code class="filename">db/</code>
subdirectory of your repository. This subdirectory is a
regular Berkeley DB environment directory and can therefore
be used in conjunction with any of the Berkeley database
tools, typically provided as part of the Berkeley DB
distribution.</p>
<p>For day-to-day Subversion use, these tools are
unnecessary. Most of the functionality typically needed for
Subversion repositories has been duplicated in the
<span class="command"><strong>svnadmin</strong></span> tool. For example,
<span class="command"><strong>svnadmin list-unused-dblogs</strong></span> and
<span class="command"><strong>svnadmin list-dblogs</strong></span> perform a
subset of what is provided by the Berkeley
<span class="command"><strong>db_archive</strong></span> utility, and <span class="command"><strong>svnadmin
recover</strong></span> reflects the common use cases of the
<span class="command"><strong>db_recover</strong></span> utility.</p>
<p>However, there are still a few Berkeley DB utilities
that you might find useful. The <span class="command"><strong>db_dump</strong></span>
and <span class="command"><strong>db_load</strong></span> programs write and read,
respectively, a custom file format that describes the keys
and values in a Berkeley DB database. Since Berkeley
databases are not portable across machine architectures,
this format is a useful way to transfer those databases from
machine to machine, irrespective of architecture or
operating system. As we describe later in this chapter, you
can also use <span class="command"><strong>svnadmin dump</strong></span> and
<span class="command"><strong>svnadmin load</strong></span> for similar purposes, but
<span class="command"><strong>db_dump</strong></span> and <span class="command"><strong>db_load</strong></span>
can do certain jobs just as well and much faster. They can
also be useful if the experienced Berkeley DB hacker needs
to do in-place tweaking of the data in a BDB-backed
repository for some reason, which is something Subversion's
utilities won't allow. Also, the <span class="command"><strong>db_stat</strong></span>
utility can provide useful information about the status of
your Berkeley DB environment, including detailed statistics
about the locking and storage subsystems.</p>
<p>For more information on the Berkeley DB tool chain,
visit the documentation section of the Berkeley DB section
of Oracle's web site, located at <a class="ulink" href="http://www.oracle.com/technology/documentation/berkeley-db/db/" target="_top">http://www.oracle.com/technology/documentation/berkeley-db/db/</a>.</p>
</div>
<div class="footnotes">
<br />
<hr style="width:100; text-align:left;margin-left: 0" />
<div id="ftn.idm17144" class="footnote">
<p><a href="#idm17144" class="para"><sup class="para">[89] </sup></a>Tender
loving care, Baby.</p>
</div>
<div id="ftn.idm17155" class="footnote">
<p><a href="#idm17155" class="para"><sup class="para">[90] </sup></a>For
example, hard drive + huge electromagnet =
disaster.</p>
</div>
</div>
</div>
<div class="navfooter">
<hr />
<table width="100%" summary="Navigation footer">
<tr>
<td width="40%" align="left"><a accesskey="p" href="svn.berkeleydb.limitations.html">Prev</a> </td>
<td width="20%" align="center">
<a accesskey="u" href="svn.berkeleydb.html">Up</a>
</td>
<td width="40%" align="right"> <a accesskey="n" href="svn.copyright.html">Next</a></td>
</tr>
<tr>
<td width="40%" align="left" valign="top">Limitations of Berkeley DB </td>
<td width="20%" align="center">
<a accesskey="h" href="index.html">Home</a>
</td>
<td width="40%" align="right" valign="top"> Appendix E. Copyright</td>
</tr>
</table>
</div>
<div xmlns="" id="vcws-footer">
<hr />
<img src="images/cc-by.png" style="float: right;" />
<p>You are reading <em>Version Control with Subversion</em> (for
Subversion 1.8), by Ben Collins-Sussman, Brian W. Fitzpatrick,
and C. Michael Pilato.</p>
<p>This work is licensed under
the <a href="http://creativecommons.org/licenses/by/2.0/">Creative Commons Attribution License v2.0</a>.</p>
<p>To submit comments, corrections, or other contributions to the
text, please visit <a href="http://www.svnbook.com/">http://www.svnbook.com/</a>.</p>
</div>
</body>
</html>
distrib
>
Mageia
>
6
>
x86_64
>
media
>
core-updates
>
by-pkgid
>
31498a559ceee64333612647bef4b546
>
files
>
53
