<?xml version="1.0" encoding="utf-8" standalone="no"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <title>Layered Library Design</title> <link rel="stylesheet" type="text/css" href="styles.css" /> <meta name="generator" content="DocBook XSL Stylesheets V1.79.1" /> <style type="text/css"> body { background-image: url('images/draft.png'); background-repeat: no-repeat; background-position: top left; /* The following properties make the watermark "fixed" on the page. */ /* I think that's just a bit too distracting for the reader... */ /* background-attachment: fixed; */ /* background-position: center center; */ }</style> <link rel="home" href="index.html" title="Version Control with Subversion [DRAFT]" /> <link rel="up" href="svn.developer.html" title="Chapter 8. Embedding Subversion" /> <link rel="prev" href="svn.developer.html" title="Chapter 8. Embedding Subversion" /> <link rel="next" href="svn.developer.usingapi.html" title="Using the APIs" /> </head> <body> <div xmlns="" id="vcws-version-notice"> <p>This text is a work in progress—highly subject to change—and may not accurately describe any released version of the Apache™ Subversion® software. Bookmarking or otherwise referring others to this page is probably not such a smart idea. Please visit <a href="http://www.svnbook.com/">http://www.svnbook.com/</a> for stable versions of this book.</p> </div> <div class="navheader"> <table width="100%" summary="Navigation header"> <tr> <th colspan="3" align="center">Layered Library Design</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="svn.developer.html">Prev</a> </td> <th width="60%" align="center">Chapter 8. Embedding Subversion</th> <td width="20%" align="right"> <a accesskey="n" href="svn.developer.usingapi.html">Next</a></td> </tr> </table> <hr /> </div> <div class="sect1"> <div class="titlepage"> <div> <div> <h2 class="title" style="clear: both"><a id="svn.developer.layerlib"></a>Layered Library Design</h2> </div> </div> </div> <p> <a id="idm9876" class="indexterm"></a>Each of Subversion's core libraries can be said to exist in one of three main layers—the Repository layer, the Repository Access (RA) layer, or the Client layer (see <a class="xref" href="svn.intro.whatis.html#svn.intro.architecture.dia-1" title="Figure 1. Subversion's architecture">Figure 1, “Subversion's architecture”</a> in the Preface). We will examine these layers shortly, but first, let's briefly summarize Subversion's various libraries. For the sake of consistency, we will refer to the libraries by their extensionless Unix library names (<code class="filename">libsvn_fs</code>, <code class="filename">libsvn_wc</code>, <code class="filename">mod_dav_svn</code>, etc.).</p> <div class="variablelist"> <dl class="variablelist"> <dt> <span class="term">libsvn_client</span> </dt> <dd> <p>Primary interface for client programs</p> </dd> <dt> <span class="term">libsvn_delta</span> </dt> <dd> <p>Tree and byte-stream differencing routines</p> </dd> <dt> <span class="term">libsvn_diff</span> </dt> <dd> <p>Contextual differencing and merging routines</p> </dd> <dt> <span class="term">libsvn_fs</span> </dt> <dd> <p>Filesystem commons and module loader</p> </dd> <dt> <span class="term">libsvn_fs_base</span> </dt> <dd> <p>The Berkeley DB filesystem backend</p> </dd> <dt> <span class="term">libsvn_fs_fs</span> </dt> <dd> <p>The native filesystem (FSFS) backend</p> </dd> <dt> <span class="term">libsvn_ra</span> </dt> <dd> <p>Repository Access commons and module loader</p> </dd> <dt> <span class="term">libsvn_ra_local</span> </dt> <dd> <p>The local Repository Access module</p> </dd> <dt> <span class="term">libsvn_ra_serf</span> </dt> <dd> <p>Another (experimental) WebDAV Repository Access module</p> </dd> <dt> <span class="term">libsvn_ra_svn</span> </dt> <dd> <p>The custom protocol Repository Access module</p> </dd> <dt> <span class="term">libsvn_repos</span> </dt> <dd> <p>Repository interface</p> </dd> <dt> <span class="term">libsvn_subr</span> </dt> <dd> <p>Miscellaneous helpful subroutines</p> </dd> <dt> <span class="term">libsvn_wc</span> </dt> <dd> <p>The working copy management library</p> </dd> <dt> <span class="term">mod_authz_svn</span> </dt> <dd> <p>Apache authorization module for Subversion repositories access via WebDAV</p> </dd> <dt> <span class="term">mod_dav_svn</span> </dt> <dd> <p>Apache module for mapping WebDAV operations to Subversion ones</p> </dd> </dl> </div> <p>The fact that the word <span class="quote">“<span class="quote">miscellaneous</span>”</span> appears only once in the previous list is a good sign. The Subversion development team is serious about making sure that functionality lives in the right layer and libraries. Perhaps the greatest advantage of the modular design is its lack of complexity from a developer's point of view. As a developer, you can quickly formulate that kind of <span class="quote">“<span class="quote">big picture</span>”</span> that allows you to pinpoint the location of certain pieces of functionality with relative ease.</p> <p>Another benefit of modularity is the ability to replace a given module with a whole new library that implements the same API without affecting the rest of the code base. In some sense, this happens within Subversion already. The <code class="filename">libsvn_ra_local</code>, <code class="filename">libsvn_ra_serf</code>, and <code class="filename">libsvn_ra_svn</code> libraries each implement the same interface, all working as plug-ins to <code class="filename">libsvn_ra</code>. And all three communicate with the Repository layer—<code class="filename">libsvn_ra_local</code> connects to the repository directly; the others do so over a network. The <code class="filename">libsvn_fs_base</code> and <code class="filename">libsvn_fs_fs</code> libraries are another pair of libraries that implement the same functionality in different ways—both are plug-ins to the common <code class="filename">libsvn_fs</code> library.</p> <p>The client itself also highlights the benefits of modularity in the Subversion design. Subversion's <code class="filename">libsvn_client</code> library is a one-stop shop for most of the functionality necessary for designing a working Subversion client (see <a class="xref" href="svn.developer.layerlib.html#svn.developer.layerlib.client" title="Client Layer">the section called “Client Layer”</a>). So while the Subversion distribution provides only the <span class="command"><strong>svn</strong></span> command-line client program, several third-party programs provide various forms of graphical client UIs. These GUIs use the same APIs that the stock command-line client does. This type of modularity has played a large role in the proliferation of available Subversion clients and IDE integrations and, by extension, to the tremendous adoption rate of Subversion itself.</p> <div class="sect2"> <div class="titlepage"> <div> <div> <h3 class="title"><a id="svn.developer.layerlib.repos"></a>Repository Layer</h3> </div> </div> </div> <p> <a id="idm9963" class="indexterm"></a>When referring to Subversion's Repository layer, we're generally talking about two basic concepts—the versioned filesystem implementation (accessed via <code class="filename">libsvn_fs</code>, and supported by its <code class="filename">libsvn_fs_base</code> and <code class="filename">libsvn_fs_fs</code> plug-ins), and the repository logic that wraps it (as implemented in <code class="filename">libsvn_repos</code>). These libraries provide the storage and reporting mechanisms for the various revisions of your version-controlled data. This layer is connected to the Client layer via the Repository Access layer, and is, from the perspective of the Subversion user, the stuff at the <span class="quote">“<span class="quote">other end of the line.</span>”</span></p> <p> <a id="idm9973" class="indexterm"></a>The Subversion filesystem is not a kernel-level filesystem that one would install in an operating system (such as the Linux ext2 or NTFS), but instead is a virtual filesystem. Rather than storing <span class="quote">“<span class="quote">files</span>”</span> and <span class="quote">“<span class="quote">directories</span>”</span> as real files and directories (the kind you can navigate through using your favorite shell program), it uses one of two available abstract storage backends—either a Berkeley DB database environment or a flat-file representation. (To learn more about the two repository backends, see <a class="xref" href="svn.reposadmin.basics.html#svn.reposadmin.basics.backends" title="Speaking of Filesystems…">Speaking of Filesystems…</a>.) There has even been considerable interest by the development community in giving future releases of Subversion the ability to use other backend database systems, perhaps through a mechanism such as Open Database Connectivity (ODBC). In fact, Google did something similar to this before launching the Google Code Project Hosting service: they announced in mid-2006 that members of its open source team had written a new proprietary Subversion filesystem plug-in that used Google's ultra-scalable Bigtable database for its storage.</p> <p> <a id="idm9980" class="indexterm"></a>The filesystem API exported by <code class="filename">libsvn_fs</code> contains the kinds of functionality you would expect from any other filesystem API—you can create and remove files and directories, copy and move them around, modify file contents, and so on. It also has features that are not quite as common, such as the ability to add, modify, and remove metadata (<span class="quote">“<span class="quote">properties</span>”</span>) on each file or directory. Furthermore, the Subversion filesystem is a versioning filesystem, which means that as you make changes to your directory tree, Subversion remembers what your tree looked like before those changes. And before the previous changes. And the previous ones. And so on, all the way back through versioning time to (and just beyond) the moment you first started adding things to the filesystem.</p> <p>All the modifications you make to your tree are done within the context of a Subversion commit transaction. The following is a simplified general routine for modifying your filesystem:</p> <div class="orderedlist"> <ol class="orderedlist" type="1"> <li class="listitem"> <p>Begin a Subversion commit transaction.</p> </li> <li class="listitem"> <p>Make your changes (adds, deletes, property modifications, etc.).</p> </li> <li class="listitem"> <p>Commit your transaction.</p> </li> </ol> </div> <p>Once you have committed your transaction, your filesystem modifications are permanently stored as historical artifacts. Each of these cycles generates a single new revision of your tree, and each revision is forever accessible as an immutable snapshot of <span class="quote">“<span class="quote">the way things were.</span>”</span></p> <div class="sidebar"> <div class="titlepage"> <div> <div> <p class="title"> <strong>The Transaction Distraction</strong> </p> </div> </div> </div> <p>The notion of a Subversion transaction can become easily confused with the transaction support provided by the underlying database itself, especially given the former's close proximity to the Berkeley DB database code in <code class="filename">libsvn_fs_base</code>. Both types of transaction exist to provide atomicity and isolation. In other words, transactions give you the ability to perform a set of actions in an all-or-nothing fashion—either all the actions in the set complete with success, or they all get treated as though <span class="emphasis"><em>none</em></span> of them ever happened—and in a way that does not interfere with other processes acting on the data.</p> <p>Database transactions generally encompass small operations related specifically to the modification of data in the database itself (such as changing the contents of a table row). Subversion transactions are larger in scope, encompassing higher-level operations such as making modifications to a set of files and directories that are intended to be stored as the next revision of the filesystem tree. If that isn't confusing enough, consider the fact that Subversion uses a database transaction during the creation of a Subversion transaction (so that if the creation of a Subversion transaction fails, the database will look as though we had never attempted that creation in the first place)!</p> <p>Fortunately for users of the filesystem API, the transaction support provided by the database system itself is hidden almost entirely from view (as should be expected from a properly modularized library scheme). It is only when you start digging into the implementation of the filesystem itself that such things become visible (or interesting).</p> </div> <p>Most of the functionality the filesystem interface provides deals with actions that occur on individual filesystem paths. That is, from outside the filesystem, the primary mechanism for describing and accessing the individual revisions of files and directories comes through the use of path strings such as <code class="filename">/foo/bar</code>, just as though you were addressing files and directories through your favorite shell program. You add new files and directories by passing their paths-to-be to the right API functions. You query for information about them by the same mechanism.</p> <p>Unlike most filesystems, though, a path alone is not enough information to identify a file or directory in Subversion. Think of a directory tree as a two-dimensional system, where a node's siblings represent a sort of left-and-right motion, and navigating into the node's subdirectories represents a downward motion. <a class="xref" href="svn.developer.layerlib.html#svn.developer.layerlib.repos.dia-1" title="Figure 8.1. Files and directories in two dimensions">Figure 8.1, “Files and directories in two dimensions”</a> shows a typical representation of a tree as exactly that.</p> <div class="figure"> <a id="svn.developer.layerlib.repos.dia-1"></a> <p class="title"> <strong>Figure 8.1. Files and directories in two dimensions</strong> </p> <div class="figure-contents"> <div> <img src="images/ch08dia1.png" alt="Files and directories in two dimensions" /> </div> </div> </div> <br class="figure-break" /> <p>The difference here is that the Subversion filesystem has a nifty third dimension that most filesystems do not have—Time!<a href="#ftn.idm10010" class="footnote" id="idm10010"><sup class="footnote">[77]</sup></a> In the filesystem interface, nearly every function that has a <em class="parameter"><code>path</code></em> argument also expects a <em class="parameter"><code>root</code></em> argument. This <code class="literal">svn_fs_root_t</code> argument describes either a revision or a Subversion transaction (which is simply a revision in the making) and provides that third dimension of context needed to understand the difference between <code class="filename">/foo/bar</code> in revision 32, and the same path as it exists in revision 98. <a class="xref" href="svn.developer.layerlib.html#svn.developer.layerlib.repos.dia-2" title="Figure 8.2. Versioning time—the third dimension!">Figure 8.2, “Versioning time—the third dimension!”</a> shows revision history as an added dimension to the Subversion filesystem universe.</p> <div class="figure"> <a id="svn.developer.layerlib.repos.dia-2"></a> <p class="title"> <strong>Figure 8.2. Versioning time—the third dimension!</strong> </p> <div class="figure-contents"> <div> <img src="images/ch08dia2.png" alt="Versioning time—the third dimension!" /> </div> </div> </div> <br class="figure-break" /> <p>As we mentioned earlier, the <code class="filename">libsvn_fs</code> API looks and feels like any other filesystem, except that it has this wonderful versioning capability. It was designed to be usable by any program interested in a versioning filesystem. Not coincidentally, Subversion itself is interested in that functionality. But while the filesystem API should be sufficient for basic file and directory versioning support, Subversion wants more—and that is where <code class="filename">libsvn_repos</code> comes in.</p> <p>The Subversion repository library (<code class="filename">libsvn_repos</code>) sits (logically speaking) atop the <code class="filename">libsvn_fs</code> API, providing additional functionality beyond that of the underlying versioned filesystem logic. It does not completely wrap each and every filesystem function—only certain major steps in the general cycle of filesystem activity are wrapped by the repository interface. Some of these include the creation and commit of Subversion transactions and the modification of revision properties. These particular events are wrapped by the repository layer because they have hooks associated with them. A repository hook system is not strictly related to implementing a versioning filesystem, so it lives in the repository wrapper library.</p> <p>The hooks mechanism is but one of the reasons for the abstraction of a separate repository library from the rest of the filesystem code. The <code class="filename">libsvn_repos</code> API provides several other important utilities to Subversion. These include the abilities to:</p> <div class="itemizedlist"> <ul class="itemizedlist" style="list-style-type: disc; "> <li class="listitem"> <p>Create, open, destroy, and perform recovery steps on a Subversion repository and the filesystem included in that repository.</p> </li> <li class="listitem"> <p>Describe the differences between two filesystem trees.</p> </li> <li class="listitem"> <p>Query for the commit log messages associated with all (or some) of the revisions in which a set of files was modified in the filesystem.</p> </li> <li class="listitem"> <p>Generate a human-readable <span class="quote">“<span class="quote">dump</span>”</span> of the filesystem—a complete representation of the revisions in the filesystem.</p> </li> <li class="listitem"> <p>Parse that dump format, loading the dumped revisions into a different Subversion repository.</p> </li> </ul> </div> <p>As Subversion continues to evolve, the repository library will grow with the filesystem library to offer increased functionality and configurable option support.</p> </div> <div class="sect2"> <div class="titlepage"> <div> <div> <h3 class="title"><a id="svn.developer.layerlib.ra"></a>Repository Access Layer</h3> </div> </div> </div> <p> <a id="idm10045" class="indexterm"></a>If the Subversion Repository layer is at <span class="quote">“<span class="quote">the other end of the line,</span>”</span> the Repository Access (RA) layer is the line itself. Charged with marshaling data between the client libraries and the repository, this layer includes the <code class="filename">libsvn_ra</code> module loader library, the RA modules themselves (which currently includes <code class="filename">libsvn_ra_local</code>, <code class="filename">libsvn_ra_serf</code>, and <code class="filename">libsvn_ra_svn</code>), and any additional libraries needed by one or more of those RA modules (such as the <code class="filename">mod_dav_svn</code> Apache module or <code class="filename">libsvn_ra_svn</code>'s server, <span class="command"><strong>svnserve</strong></span>).</p> <p> <a id="idm10058" class="indexterm"></a>Since Subversion uses URLs to identify its repository resources, the protocol portion of the URL scheme (usually <code class="literal">file://</code>, <code class="literal">http://</code>, <code class="literal">https://</code>, <code class="literal">svn://</code>, or <code class="literal">svn+ssh://</code>) is used to determine which RA module will handle the communications. Each module registers a list of the protocols it knows how to <span class="quote">“<span class="quote">speak</span>”</span> so that the RA loader can, at runtime, determine which module to use for the task at hand. You can determine which RA modules are available to the Subversion command-line client, and what protocols they claim to support, by running <strong class="userinput"><code>svn --version</code></strong>:</p> <div class="informalexample"> <pre class="screen"> $ svn --version svn, version 1.8.0-dev (under development) compiled Jan 8 2013, 11:45:25 on i686-pc-linux-gnu Copyright (C) 2013 The Apache Software Foundation. This software consists of contributions made by many people; see the NOTICE file for more information. Subversion is open source software, see http://subversion.apache.org/ The following repository access (RA) modules are available: * ra_svn : Module for accessing a repository using the svn network protocol. - with Cyrus SASL authentication - handles 'svn' scheme * ra_local : Module for accessing a repository on local disk. - handles 'file' scheme * ra_serf : Module for accessing a repository via WebDAV protocol using serf. - handles 'http' scheme - handles 'https' scheme $ </pre> </div> <p>The public API exported by the RA layer contains functionality necessary for sending and receiving versioned data to and from the repository. And each of the available RA plug-ins is able to perform that task using a specific protocol—<code class="filename">libsvn_ra_serf</code> speaks HTTP/WebDAV (optionally using SSL encryption) with an Apache HTTP Server that is running the <code class="filename">mod_dav_svn</code> Subversion server module; <code class="filename">libsvn_ra_svn</code> speaks a custom network protocol with the <span class="command"><strong>svnserve</strong></span> program; and so on.</p> <p>For those who wish to access a Subversion repository using still another protocol, that is precisely why the Repository Access layer is modularized! Developers can simply write a new library that implements the RA interface on one side and communicates with the repository on the other. Your new library can use existing network protocols or you can invent your own. You could use interprocess communication (IPC) calls, or—let's get crazy, shall we?—you could even implement an email-based protocol. Subversion supplies the APIs; you supply the creativity.</p> </div> <div class="sect2"> <div class="titlepage"> <div> <div> <h3 class="title"><a id="svn.developer.layerlib.client"></a>Client Layer</h3> </div> </div> </div> <p> <a id="idm10078" class="indexterm"></a>On the client side, the Subversion working copy is where all the action takes place. The bulk of functionality implemented by the client-side libraries exists for the sole purpose of managing working copies—directories full of files and other subdirectories that serve as a sort of local, editable <span class="quote">“<span class="quote">reflection</span>”</span> of one or more repository locations—and propagating changes to and from the Repository Access layer.</p> <p> <a id="idm10084" class="indexterm"></a>Subversion's working copy library, <code class="filename">libsvn_wc</code>, is directly responsible for managing the data in the working copies. To accomplish this, the library stores administrative information about the working copy within a special subdirectory. This subdirectory, named <code class="filename">.svn</code>, is present in each working copy and contains various other files and directories that record state and provide a private workspace for administrative action. For those familiar with CVS, this <code class="filename">.svn</code> subdirectory is similar in purpose to the <code class="filename">CVS</code> administrative directories found in CVS working copies.</p> <p>The Subversion client library, <code class="filename">libsvn_client</code>, has the broadest responsibility; its job is to mingle the functionality of the working copy library with that of the Repository Access layer, and then to provide the highest-level API to any application that wishes to perform general revision control actions. For example, the function <code class="function">svn_client_checkout()</code> takes a URL as an argument. It passes this URL to the RA layer and opens an authenticated session with a particular repository. It then asks the repository for a certain tree, and sends this tree into the working copy library, which then writes a full working copy to disk (<code class="filename">.svn</code> directories and all).</p> <p>The client library is designed to be used by any application. While the Subversion source code includes a standard command-line client, it should be very easy to write any number of GUI clients on top of the client library. New GUIs (or any new client, really) for Subversion need not be clunky wrappers around the included command-line client—they have full access via the <code class="filename">libsvn_client</code> API to the same functionality, data, and callback mechanisms that the command-line client uses. In fact, the Subversion source code tree contains a small C program (which you can find at <code class="filename">tools/examples/minimal_client.c</code>) that exemplifies how to wield the Subversion API to create a simple client program.</p> <div class="sidebar"> <div class="titlepage"> <div> <div> <p class="title"> <strong>Binding Directly—A Word About Correctness</strong> </p> </div> </div> </div> <p>Why should your GUI program bind directly with a <code class="filename">libsvn_client</code> instead of acting as a wrapper around a command-line program? Besides simply being more efficient, it can be more correct as well. A command-line program (such as the one supplied with Subversion) that binds to the client library needs to effectively translate feedback and requested data bits from C types to some form of human-readable output. This type of translation can be lossy. That is, the program may not display all of the information harvested from the API or may combine bits of information for compact representation.</p> <p>If you wrap such a command-line program with yet another program, the second program has access only to already interpreted (and as we mentioned, likely incomplete) information, which it must <span class="emphasis"><em>again</em></span> translate into <span class="emphasis"><em>its</em></span> representation format. With each layer of wrapping, the integrity of the original data is potentially tainted more and more, much like the result of making a copy of a copy (of a copy…) of a favorite audio or video cassette.</p> <p>But the most compelling argument for binding directly to the APIs instead of wrapping other programs is that the Subversion project makes compatibility promises regarding its APIs. Across minor versions of those APIs (such as between 1.3 and 1.4), no function's prototype will change. In other words, you aren't forced to update your program's source code simply because you've upgraded to a new version of Subversion. Certain functions might be deprecated, but they still work, and this gives you a buffer of time to eventually embrace the newer APIs. These kinds of compatibility promises do not exist for Subversion command-line program output, which is subject to change from release to release.</p> </div> </div> <div class="footnotes"> <br /> <hr style="width:100; text-align:left;margin-left: 0" /> <div id="ftn.idm10010" class="footnote"> <p><a href="#idm10010" class="para"><sup class="para">[77] </sup></a>We understand that this may come as a shock to sci-fi fans who have long been under the impression that Time was actually the <span class="emphasis"><em>fourth</em></span> dimension, and we apologize for any emotional trauma induced by our assertion of a different theory.</p> </div> </div> </div> <div class="navfooter"> <hr /> <table width="100%" summary="Navigation footer"> <tr> <td width="40%" align="left"><a accesskey="p" href="svn.developer.html">Prev</a> </td> <td width="20%" align="center"> <a accesskey="u" href="svn.developer.html">Up</a> </td> <td width="40%" align="right"> <a accesskey="n" href="svn.developer.usingapi.html">Next</a></td> </tr> <tr> <td width="40%" align="left" valign="top">Chapter 8. Embedding Subversion </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> <td width="40%" align="right" valign="top"> Using the APIs</td> </tr> </table> </div> <div xmlns="" id="vcws-footer"> <hr /> <img src="images/cc-by.png" style="float: right;" /> <p>You are reading <em>Version Control with Subversion</em> (for Subversion 1.8), by Ben Collins-Sussman, Brian W. Fitzpatrick, and C. Michael Pilato.</p> <p>This work is licensed under the <a href="http://creativecommons.org/licenses/by/2.0/">Creative Commons Attribution License v2.0</a>.</p> <p>To submit comments, corrections, or other contributions to the text, please visit <a href="http://www.svnbook.com/">http://www.svnbook.com/</a>.</p> </div> </body> </html>