<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Concepts — Buildbot v0.8.5rc2-29-g80a524b documentation</title>
<link rel="stylesheet" href="../_static/agogo.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
VERSION: '0.8.5rc2-29-g80a524b',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="../_static/jquery.js"></script>
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="shortcut icon" href="../_static/buildbot.ico"/>
<link rel="top" title="Buildbot v0.8.5rc2-29-g80a524b documentation" href="../index.html" />
<link rel="up" title="Buildbot Manual" href="index.html" />
<link rel="next" title="Configuration" href="configuration.html" />
<link rel="prev" title="Installation" href="installation.html" />
</head>
<body>
<div class="header-wrapper">
<div class="header">
<p class="logo"><a href="../index.html">
<img class="logo" src="../_static/header-text-transparent.png" alt="Logo"/>
</a></p>
<h1><a href="../index.html">Buildbot v0.8.5rc2-29-g80a524b documentation</a></h1>
<div class="rel">
<a href="installation.html" title="Installation"
accesskey="P">previous</a> |
<a href="configuration.html" title="Configuration"
accesskey="N">next</a> |
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |
<a href="../genindex.html" title="General Index"
accesskey="I">index</a>
</div>
</div>
</div>
<div class="content-wrapper">
<div class="content">
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body">
<div class="section" id="concepts">
<h1>Concepts<a class="headerlink" href="#concepts" title="Permalink to this headline">¶</a></h1>
<p>This chapter defines some of the basic concepts that the Buildbot
uses. You'll need to understand how the Buildbot sees the world to
configure it properly.</p>
<div class="section" id="version-control-systems">
<span id="id1"></span><h2>Version Control Systems<a class="headerlink" href="#version-control-systems" title="Permalink to this headline">¶</a></h2>
<p>These source trees come from a Version Control System of some kind.
CVS and Subversion are two popular ones, but the Buildbot supports
others. All VC systems have some notion of an upstream
<cite>repository</cite> which acts as a server <a class="footnote-reference" href="#id20" id="id2">[1]</a>, from which clients
can obtain source trees according to various parameters. The VC
repository provides source trees of various projects, for different
branches, and from various points in time. The first thing we have to
do is to specify which source tree we want to get.</p>
<div class="section" id="generalizing-vc-systems">
<span id="id3"></span><h3>Generalizing VC Systems<a class="headerlink" href="#generalizing-vc-systems" title="Permalink to this headline">¶</a></h3>
<p>For the purposes of the Buildbot, we will try to generalize all VC
systems as having repositories that each provide sources for a variety
of projects. Each project is defined as a directory tree with source
files. The individual files may each have revisions, but we ignore
that and treat the project as a whole as having a set of revisions
(CVS is the only VC system still in widespread use that has
per-file revisions, as everything modern has moved to atomic tree-wide
changesets). Each time someone commits a change to the project, a new
revision becomes available. These revisions can be described by a
tuple with two items: the first is a branch tag, and the second is
some kind of revision stamp or timestamp. Complex projects may have
multiple branch tags, but there is always a default branch. The
timestamp may be an actual timestamp (such as the <em class="xref std std-option">-D</em> option to CVS),
or it may be a monotonically-increasing transaction number (such as
the change number used by SVN and P4, or the revision number used by
Bazaar, or a labeled tag used in CVS. <a class="footnote-reference" href="#id21" id="id4">[2]</a>)
The SHA1 revision ID used by Mercurial, and Git is
also a kind of revision stamp, in that it specifies a unique copy of
the source tree, as does a Darcs <tt class="docutils literal"><span class="pre">context</span></tt> file.</p>
<p>When we aren't intending to make any changes to the sources we check out
(at least not any that need to be committed back upstream), there are two
basic ways to use a VC system:</p>
<blockquote>
<div><ul class="simple">
<li>Retrieve a specific set of source revisions: some tag or key is used
to index this set, which is fixed and cannot be changed by subsequent
developers committing new changes to the tree. Releases are built from
tagged revisions like this, so that they can be rebuilt again later
(probably with controlled modifications).</li>
<li>Retrieve the latest sources along a specific branch: some tag is used
to indicate which branch is to be used, but within that constraint we want
to get the latest revisions.</li>
</ul>
</div></blockquote>
<p>Build personnel or CM staff typically use the first approach: the
build that results is (ideally) completely specified by the two
parameters given to the VC system: repository and revision tag. This
gives QA and end-users something concrete to point at when reporting
bugs. Release engineers are also reportedly fond of shipping code that
can be traced back to a concise revision tag of some sort.</p>
<p>Developers are more likely to use the second approach: each morning
the developer does an update to pull in the changes committed by the
team over the last day. These builds are not easy to fully specify: it
depends upon exactly when you did a checkout, and upon what local
changes the developer has in their tree. Developers do not normally
tag each build they produce, because there is usually significant
overhead involved in creating these tags. Recreating the trees used by
one of these builds can be a challenge. Some VC systems may provide
implicit tags (like a revision number), while others may allow the use
of timestamps to mean "the state of the tree at time X" as opposed
to a tree-state that has been explicitly marked.</p>
<p>The Buildbot is designed to help developers, so it usually works in
terms of <em>the latest</em> sources as opposed to specific tagged
revisions. However, it would really prefer to build from reproducible
source trees, so implicit revisions are used whenever possible.</p>
</div>
<div class="section" id="source-tree-specifications">
<span id="id5"></span><h3>Source Tree Specifications<a class="headerlink" href="#source-tree-specifications" title="Permalink to this headline">¶</a></h3>
<p>So for the Buildbot's purposes we treat each VC system as a server
which can take a list of specifications as input and produce a source
tree as output. Some of these specifications are static: they are
attributes of the builder and do not change over time. Others are more
variable: each build will have a different value. The repository is
changed over time by a sequence of Changes, each of which represents a
single developer making changes to some set of files. These Changes
are cumulative.</p>
<p>For normal builds, the Buildbot wants to get well-defined source trees
that contain specific <tt class="xref py py-class docutils literal"><span class="pre">Change</span></tt>s, and exclude other <tt class="xref py py-class docutils literal"><span class="pre">Change</span></tt>s that may have
occurred after the desired ones. We assume that the <tt class="xref py py-class docutils literal"><span class="pre">Change</span></tt>s arrive at
the buildbot (through one of the mechanisms described in
<a class="reference internal" href="cfg-changesources.html#change-sources"><em>Change Sources</em></a>) in the same order in which they are committed to the
repository. The Buildbot waits for the tree to become <tt class="docutils literal"><span class="pre">stable</span></tt>
before initiating a build, for two reasons. The first is that
developers frequently make multiple related commits in quick
succession, even when the VC system provides ways to make atomic
transactions involving multiple files at the same time. Running a
build in the middle of these sets of changes would use an inconsistent
set of source files, and is likely to fail (and is certain to be less
useful than a build which uses the full set of changes). The
tree-stable-timer is intended to avoid these useless builds that
include some of the developer's changes but not all. The second reason
is that some VC systems (i.e. CVS) do not provide repository-wide
transaction numbers, so that timestamps are the only way to refer to
a specific repository state. These timestamps may be somewhat
ambiguous, due to processing and notification delays. By waiting until
the tree has been stable for, say, 10 minutes, we can choose a
timestamp from the middle of that period to use for our source
checkout, and then be reasonably sure that any clock-skew errors will
not cause the build to be performed on an inconsistent set of source
files.</p>
<p>The <tt class="xref py py-class docutils literal"><span class="pre">Scheduler</span></tt>s always use the tree-stable-timer, with a timeout that
is configured to reflect a reasonable tradeoff between build latency
and change frequency. When the VC system provides coherent
repository-wide revision markers (such as Subversion's revision
numbers, or in fact anything other than CVS's timestamps), the
resulting <tt class="xref py py-class docutils literal"><span class="pre">Build</span></tt> is simply performed against a source tree defined by
that revision marker. When the VC system does not provide this, a
timestamp from the middle of the tree-stable period is used to
generate the source tree <a class="footnote-reference" href="#id22" id="id6">[3]</a>.</p>
</div>
<div class="section" id="how-different-vc-systems-specify-sources">
<span id="id7"></span><h3>How Different VC Systems Specify Sources<a class="headerlink" href="#how-different-vc-systems-specify-sources" title="Permalink to this headline">¶</a></h3>
<p>For CVS, the static specifications are <em>repository</em> and
<em>module</em>. In addition to those, each build uses a timestamp (or
omits the timestamp to mean <em>the latest</em>) and <em>branch tag</em>
(which defaults to <tt class="docutils literal"><span class="pre">HEAD</span></tt>). These parameters collectively specify a set
of sources from which a build may be performed.</p>
<p><a class="reference external" href="http://subversion.tigris.org">Subversion</a>, combines the
repository, module, and branch into a single <em>Subversion URL</em>
parameter. Within that scope, source checkouts can be specified by a
numeric <em>revision number</em> (a repository-wide
monotonically-increasing marker, such that each transaction that
changes the repository is indexed by a different revision number), or
a revision timestamp. When branches are used, the repository and
module form a static <tt class="docutils literal"><span class="pre">baseURL</span></tt>, while each build has a
<em>revision number</em> and a <em>branch</em> (which defaults to a
statically-specified <tt class="docutils literal"><span class="pre">defaultBranch</span></tt>). The <tt class="docutils literal"><span class="pre">baseURL</span></tt> and
<tt class="docutils literal"><span class="pre">branch</span></tt> are simply concatenated together to derive the
<tt class="docutils literal"><span class="pre">svnurl</span></tt> to use for the checkout.</p>
<p><a class="reference external" href="http://www.perforce.com/">Perforce</a> is similar. The server
is specified through a <tt class="docutils literal"><span class="pre">P4PORT</span></tt> parameter. Module and branch
are specified in a single depot path, and revisions are
depot-wide. When branches are used, the <tt class="docutils literal"><span class="pre">p4base</span></tt> and
<tt class="docutils literal"><span class="pre">defaultBranch</span></tt> are concatenated together to produce the depot
path.</p>
<p><a class="reference external" href="http://bazaar-vcs.org">Bzr</a> (which is a descendant of
Arch/Bazaar, and is frequently referred to as "Bazaar") has the same
sort of repository-vs-workspace model as Arch, but the repository data
can either be stored inside the working directory or kept elsewhere
(either on the same machine or on an entirely different machine). For
the purposes of Buildbot (which never commits changes), the repository
is specified with a URL and a revision number.</p>
<p>The most common way to obtain read-only access to a bzr tree is via
HTTP, simply by making the repository visible through a web server
like Apache. Bzr can also use FTP and SFTP servers, if the buildslave
process has sufficient privileges to access them. Higher performance
can be obtained by running a special Bazaar-specific server. None of
these matter to the buildbot: the repository URL just has to match the
kind of server being used. The <tt class="docutils literal"><span class="pre">repoURL</span></tt> argument provides the
location of the repository.</p>
<p>Branches are expressed as subdirectories of the main central
repository, which means that if branches are being used, the BZR step
is given a <tt class="docutils literal"><span class="pre">baseURL</span></tt> and <tt class="docutils literal"><span class="pre">defaultBranch</span></tt> instead of getting
the <tt class="docutils literal"><span class="pre">repoURL</span></tt> argument.</p>
<p><a class="reference external" href="http://darcs.net/">Darcs</a> doesn't really have the
notion of a single master repository. Nor does it really have
branches. In Darcs, each working directory is also a repository, and
there are operations to push and pull patches from one of these
<tt class="docutils literal"><span class="pre">repositories</span></tt> to another. For the Buildbot's purposes, all you
need to do is specify the URL of a repository that you want to build
from. The build slave will then pull the latest patches from that
repository and build them. Multiple branches are implemented by using
multiple repositories (possibly living on the same server).</p>
<p>Builders which use Darcs therefore have a static <tt class="docutils literal"><span class="pre">repourl</span></tt> which
specifies the location of the repository. If branches are being used,
the source Step is instead configured with a <tt class="docutils literal"><span class="pre">baseURL</span></tt> and a
<tt class="docutils literal"><span class="pre">defaultBranch</span></tt>, and the two strings are simply concatenated
together to obtain the repository's URL. Each build then has a
specific branch which replaces <tt class="docutils literal"><span class="pre">defaultBranch</span></tt>, or just uses the
default one. Instead of a revision number, each build can have a
<tt class="docutils literal"><span class="pre">context</span></tt>, which is a string that records all the patches that are
present in a given tree (this is the output of <tt class="docutils literal"><span class="pre">darcs</span> <span class="pre">changes</span>
<span class="pre">--context</span></tt>, and is considerably less concise than, e.g. Subversion's
revision number, but the patch-reordering flexibility of Darcs makes
it impossible to provide a shorter useful specification).</p>
<p><a class="reference external" href="http://selenic.com/mercurial">Mercurial</a> is like Darcs, in that
each branch is stored in a separate repository. The <tt class="docutils literal"><span class="pre">repourl</span></tt>,
<tt class="docutils literal"><span class="pre">baseURL</span></tt>, and <tt class="docutils literal"><span class="pre">defaultBranch</span></tt> arguments are all handled the
same way as with Darcs. The <em>revision</em>, however, is the hash
identifier returned by <tt class="docutils literal"><span class="pre">hg</span> <span class="pre">identify</span></tt>.</p>
<p><a class="reference external" href="http://git.or.cz/">Git</a> also follows a decentralized model, and
each repository can have several branches and tags. The source Step is
configured with a static <tt class="docutils literal"><span class="pre">repourl</span></tt> which specifies the location
of the repository. In addition, an optional <tt class="docutils literal"><span class="pre">branch</span></tt> parameter
can be specified to check out code from a specific branch instead of
the default <em>master</em> branch. The <em>revision</em> is specified as a SHA1
hash as returned by e.g. <tt class="docutils literal"><span class="pre">git</span> <span class="pre">rev-parse</span></tt>. No attempt is made
to ensure that the specified revision is actually a subset of the
specified branch.</p>
<p><a class="reference external" href="http://www.monotone.ca/">Monotone</a> is another that follows a
decentralized model where each repository can have several branches and
tags. The source Step is configured with static <tt class="docutils literal"><span class="pre">repourl</span></tt> and
<tt class="docutils literal"><span class="pre">branch</span></tt> parameters, which specifies the location of the
repository and the branch to use. The <em>revision</em> is specified as a
SHA1 hash as returned by e.g. <tt class="docutils literal"><span class="pre">mtn</span> <span class="pre">automate</span> <span class="pre">select</span> <span class="pre">w:</span></tt>. No
attempt is made to ensure that the specified revision is actually a
subset of the specified branch.</p>
</div>
<div class="section" id="attributes-of-changes">
<span id="id8"></span><h3>Attributes of Changes<a class="headerlink" href="#attributes-of-changes" title="Permalink to this headline">¶</a></h3>
<div class="section" id="who">
<span id="attr-who"></span><h4>Who<a class="headerlink" href="#who" title="Permalink to this headline">¶</a></h4>
<p>Each <tt class="xref py py-class docutils literal"><span class="pre">Change</span></tt> has a <tt class="xref py py-attr docutils literal"><span class="pre">who</span></tt> attribute, which specifies which developer is
responsible for the change. This is a string which comes from a namespace
controlled by the VC repository. Frequently this means it is a username on the
host which runs the repository, but not all VC systems require this. Each
<tt class="xref py py-class docutils literal"><span class="pre">StatusNotifier</span></tt> will map the <tt class="xref py py-attr docutils literal"><span class="pre">who</span></tt> attribute into something appropriate for
their particular means of communication: an email address, an IRC handle, etc.</p>
<p>This <tt class="docutils literal"><span class="pre">who</span></tt> attribute is also parsed and stored into Buildbot's database (see
<a class="reference internal" href="#user-objects"><em>User Objects</em></a>). Currently, only <tt class="docutils literal"><span class="pre">who</span></tt> attributes in Changes from
<tt class="docutils literal"><span class="pre">git</span></tt> repositories are translated into user objects, but in the future all
incoming Changes will have their <tt class="docutils literal"><span class="pre">who</span></tt> parsed and stored.</p>
</div>
<div class="section" id="files">
<span id="attr-files"></span><h4>Files<a class="headerlink" href="#files" title="Permalink to this headline">¶</a></h4>
<p>It also has a list of <tt class="xref py py-attr docutils literal"><span class="pre">files</span></tt>, which are just the tree-relative
filenames of any files that were added, deleted, or modified for this
<tt class="xref py py-class docutils literal"><span class="pre">Change</span></tt>. These filenames are used by the <tt class="xref py py-func docutils literal"><span class="pre">fileIsImportant</span></tt>
function (in the <tt class="xref py py-class docutils literal"><span class="pre">Scheduler</span></tt>) to decide whether it is worth triggering a
new build or not, e.g. the function could use the following function
to only run a build if a C file were checked in:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="k">def</span> <span class="nf">has_C_files</span><span class="p">(</span><span class="n">change</span><span class="p">):</span>
<span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">change</span><span class="o">.</span><span class="n">files</span><span class="p">:</span>
<span class="k">if</span> <span class="n">name</span><span class="o">.</span><span class="n">endswith</span><span class="p">(</span><span class="s">".c"</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">True</span>
<span class="k">return</span> <span class="bp">False</span>
</pre></div>
</div>
<p>Certain <tt class="xref py py-class docutils literal"><span class="pre">BuildStep</span></tt>s can also use the list of changed files
to run a more targeted series of tests, e.g. the
<tt class="docutils literal"><span class="pre">python_twisted.Trial</span></tt> step can run just the unit tests that
provide coverage for the modified .py files instead of running the
full test suite.</p>
</div>
<div class="section" id="comments">
<span id="attr-comments"></span><h4>Comments<a class="headerlink" href="#comments" title="Permalink to this headline">¶</a></h4>
<p>The Change also has a <tt class="xref py py-attr docutils literal"><span class="pre">comments</span></tt> attribute, which is a string
containing any checkin comments.</p>
</div>
<div class="section" id="project">
<span id="attr-project"></span><h4>Project<a class="headerlink" href="#project" title="Permalink to this headline">¶</a></h4>
<p>A change's <tt class="xref py py-attr docutils literal"><span class="pre">project</span></tt>, by default the empty string, describes the source code
that changed. It is a free-form string which the buildbot administrator can
use to flexibly discriminate among changes.</p>
<p>Generally, a project is an independently-buildable unit of source. This field
can be used to apply different build steps to different projects. For example,
an open-source application might build its Windows client from a separate
codebase than its POSIX server. In this case, the change sources should be
configured to attach an appropriate project string (say, "win-client" and
"server") to changes from each codebase. Schedulers would then examine these
strings and trigger the appropriate builders for each project.</p>
</div>
<div class="section" id="repository">
<span id="attr-repository"></span><h4>Repository<a class="headerlink" href="#repository" title="Permalink to this headline">¶</a></h4>
<p>A change occurs within the context of a specific repository. This is generally
specified with a string, and for most version-control systems, this string
takes the form of a URL.</p>
<p><tt class="xref py py-class docutils literal"><span class="pre">Change</span></tt>s can be filtered on repository, but more often this field is used as a
hint for the build steps to figure out which code to check out.</p>
</div>
<div class="section" id="revision">
<span id="attr-revision"></span><h4>Revision<a class="headerlink" href="#revision" title="Permalink to this headline">¶</a></h4>
<p>Each Change can have a <tt class="xref py py-attr docutils literal"><span class="pre">revision</span></tt> attribute, which describes how
to get a tree with a specific state: a tree which includes this Change
(and all that came before it) but none that come after it. If this
information is unavailable, the <tt class="xref py py-attr docutils literal"><span class="pre">revision</span></tt> attribute will be
<tt class="xref docutils literal"><span class="pre">None</span></tt>. These revisions are provided by the <tt class="xref py py-class docutils literal"><span class="pre">ChangeSource</span></tt>, and
consumed by the <tt class="xref py py-meth docutils literal"><span class="pre">computeSourceRevision</span></tt> method in the appropriate
<tt class="xref py py-class docutils literal"><span class="pre">source.Source</span></tt> class.</p>
<dl class="docutils">
<dt><cite>CVS</cite></dt>
<dd><tt class="xref py py-attr docutils literal"><span class="pre">revision</span></tt> is an int, seconds since the epoch</dd>
<dt><cite>SVN</cite></dt>
<dd><tt class="xref py py-attr docutils literal"><span class="pre">revision</span></tt> is an int, the changeset number (r%d)</dd>
<dt><cite>Darcs</cite></dt>
<dd><tt class="xref py py-attr docutils literal"><span class="pre">revision</span></tt> is a large string, the output of <strong class="command">darcs changes --context</strong></dd>
<dt><cite>Mercurial</cite></dt>
<dd><tt class="xref py py-attr docutils literal"><span class="pre">revision</span></tt> is a short string (a hash ID), the output of <strong class="command">hg identify</strong></dd>
<dt><cite>P4</cite></dt>
<dd><tt class="xref py py-attr docutils literal"><span class="pre">revision</span></tt> is an int, the transaction number</dd>
<dt><cite>Git</cite></dt>
<dd><tt class="xref py py-attr docutils literal"><span class="pre">revision</span></tt> is a short string (a SHA1 hash), the output of e.g.
<strong class="command">git rev-parse</strong></dd>
</dl>
<div class="section" id="branches">
<h5>Branches<a class="headerlink" href="#branches" title="Permalink to this headline">¶</a></h5>
<p>The Change might also have a <tt class="xref py py-attr docutils literal"><span class="pre">branch</span></tt> attribute. This indicates
that all of the Change's files are in the same named branch. The
Schedulers get to decide whether the branch should be built or not.</p>
<p>For VC systems like CVS, Git and Monotone the <tt class="xref py py-attr docutils literal"><span class="pre">branch</span></tt>
name is unrelated to the filename. (that is, the branch name and the
filename inhabit unrelated namespaces). For SVN, branches are
expressed as subdirectories of the repository, so the file's
<tt class="docutils literal"><span class="pre">svnurl</span></tt> is a combination of some base URL, the branch name, and the
filename within the branch. (In a sense, the branch name and the
filename inhabit the same namespace). Darcs branches are
subdirectories of a base URL just like SVN. Mercurial branches are the
same as Darcs.</p>
<dl class="docutils">
<dt><cite>CVS</cite></dt>
<dd>branch='warner-newfeature', files=['src/foo.c']</dd>
<dt><cite>SVN</cite></dt>
<dd>branch='branches/warner-newfeature', files=['src/foo.c']</dd>
<dt><cite>Darcs</cite></dt>
<dd>branch='warner-newfeature', files=['src/foo.c']</dd>
<dt><cite>Mercurial</cite></dt>
<dd>branch='warner-newfeature', files=['src/foo.c']</dd>
<dt><cite>Git</cite></dt>
<dd>branch='warner-newfeature', files=['src/foo.c']</dd>
<dt><cite>Monotone</cite></dt>
<dd>branch='warner-newfeature', files=['src/foo.c']</dd>
</dl>
</div>
<div class="section" id="build-properties">
<h5>Build Properties<a class="headerlink" href="#build-properties" title="Permalink to this headline">¶</a></h5>
<p>A Change may have one or more properties attached to it, usually specified
through the Force Build form or <a class="reference internal" href="cmdline.html#cmdline-sendchange" title="sendchange"><tt class="xref bb bb-cmdline docutils literal"><span class="pre">sendchange</span></tt></a>. Properties are discussed
in detail in the <a class="reference internal" href="#id18"><em>Build Properties</em></a> section.</p>
</div>
<div class="section" id="links">
<h5>Links<a class="headerlink" href="#links" title="Permalink to this headline">¶</a></h5>
<p>Finally, the Change might have a <tt class="xref py py-attr docutils literal"><span class="pre">links</span></tt> list, which is intended
to provide a list of URLs to a <em>viewcvs</em>-style web page that
provides more detail for this Change, perhaps including the full file
diffs.</p>
</div>
</div>
</div>
</div>
<div class="section" id="scheduling-builds">
<span id="id9"></span><h2>Scheduling Builds<a class="headerlink" href="#scheduling-builds" title="Permalink to this headline">¶</a></h2>
<p>Each Buildmaster has a set of <tt class="xref py py-class docutils literal"><span class="pre">Scheduler</span></tt> objects, each of which
gets a copy of every incoming <tt class="xref py py-class docutils literal"><span class="pre">Change</span></tt>. The Schedulers are responsible
for deciding when <tt class="xref py py-class docutils literal"><span class="pre">Build</span></tt>s should be run. Some Buildbot installations
might have a single <tt class="xref py py-class docutils literal"><span class="pre">Scheduler</span></tt>, while others may have several, each for
a different purpose.</p>
<p>For example, a <em>quick</em> scheduler might exist to give immediate
feedback to developers, hoping to catch obvious problems in the code
that can be detected quickly. These typically do not run the full test
suite, nor do they run on a wide variety of platforms. They also
usually do a VC update rather than performing a brand-new checkout
each time.</p>
<p>A separate <em>full</em> scheduler might run more comprehensive tests, to
catch more subtle problems. configured to run after the quick scheduler, to give
developers time to commit fixes to bugs caught by the quick scheduler before
running the comprehensive tests. This scheduler would also feed multiple
<tt class="xref py py-class docutils literal"><span class="pre">Builder</span></tt>s.</p>
<p>Many schedulers can be configured to wait a while after seeing a source-code
change - this is the <em>tree stable timer</em>. The timer allows multiple commits to
be "batched" together. This is particularly useful in distributed version
control systems, where a developer may push a long sequence of changes all at
once. To save resources, it's often desirable only to test the most recent
change.</p>
<p>Schedulers can also filter out the changes they are interested in, based on a
number of criteria. For example, a scheduler that only builds documentation
might skip any changes that do not affect the documentation. Schedulers can
also filter on the branch to which a commit was made.</p>
<p>There is some support for configuring dependencies between builds - for
example, you may want to build packages only for revisions which pass all of
the unit tests. This support is under active development in Buildbot, and is
referred to as "build coordination".</p>
<p>Periodic builds (those which are run every N seconds rather than after
new Changes arrive) are triggered by a special <tt class="xref py py-class docutils literal"><span class="pre">Periodic</span></tt>
Scheduler subclass.</p>
<p>Each Scheduler creates and submits <tt class="xref py py-class docutils literal"><span class="pre">BuildSet</span></tt> objects to the
<tt class="xref py py-class docutils literal"><span class="pre">BuildMaster</span></tt>, which is then responsible for making sure the
individual <tt class="xref py py-class docutils literal"><span class="pre">BuildRequests</span></tt> are delivered to the target
<tt class="xref py py-class docutils literal"><span class="pre">Builder</span></tt>s.</p>
<p><tt class="xref py py-class docutils literal"><span class="pre">Scheduler</span></tt> instances are activated by placing them in the
<tt class="docutils literal"><span class="pre">c['schedulers']</span></tt> list in the buildmaster config file. Each
<tt class="xref py py-class docutils literal"><span class="pre">Scheduler</span></tt> has a unique name.</p>
</div>
<div class="section" id="buildset">
<span id="id10"></span><h2>BuildSet<a class="headerlink" href="#buildset" title="Permalink to this headline">¶</a></h2>
<p>A <tt class="xref py py-class docutils literal"><span class="pre">BuildSet</span></tt> is the name given to a set of <tt class="xref py py-class docutils literal"><span class="pre">Build</span></tt>s that all
compile/test the same version of the tree on multiple <tt class="xref py py-class docutils literal"><span class="pre">Builder</span></tt>s. In
general, all these component <tt class="xref py py-class docutils literal"><span class="pre">Build</span></tt>s will perform the same sequence of
<tt class="xref py py-class docutils literal"><span class="pre">Step</span></tt>s, using the same source code, but on different platforms or
against a different set of libraries.</p>
<p>The <tt class="xref py py-class docutils literal"><span class="pre">BuildSet</span></tt> is tracked as a single unit, which fails if any of
the component <tt class="xref py py-class docutils literal"><span class="pre">Build</span></tt>s have failed, and therefore can succeed only if
<em>all</em> of the component <tt class="xref py py-class docutils literal"><span class="pre">Build</span></tt>s have succeeded. There are two kinds
of status notification messages that can be emitted for a <tt class="xref py py-class docutils literal"><span class="pre">BuildSet</span></tt>:
the <tt class="docutils literal"><span class="pre">firstFailure</span></tt> type (which fires as soon as we know the
<tt class="xref py py-class docutils literal"><span class="pre">BuildSet</span></tt> will fail), and the <tt class="docutils literal"><span class="pre">Finished</span></tt> type (which fires once
the <tt class="xref py py-class docutils literal"><span class="pre">BuildSet</span></tt> has completely finished, regardless of whether the
overall set passed or failed).</p>
<p>A <tt class="xref py py-class docutils literal"><span class="pre">BuildSet</span></tt> is created with a <em>source stamp</em> tuple of
<tt class="docutils literal"><span class="pre">(branch,</span> <span class="pre">revision,</span> <span class="pre">changes,</span> <span class="pre">patch)</span></tt>, some of which may be <tt class="xref docutils literal"><span class="pre">None</span></tt>, and a
list of <tt class="xref py py-class docutils literal"><span class="pre">Builder</span></tt>s on which it is to be run. They are then given to the
BuildMaster, which is responsible for creating a separate
<tt class="xref py py-class docutils literal"><span class="pre">BuildRequest</span></tt> for each <tt class="xref py py-class docutils literal"><span class="pre">Builder</span></tt>.</p>
<p>There are a couple of different likely values for the
<tt class="docutils literal"><span class="pre">SourceStamp</span></tt>:</p>
<dl class="docutils">
<dt><tt class="samp docutils literal"><span class="pre">(revision=None,</span> <span class="pre">changes=</span><em><span class="pre">CHANGES</span></em><span class="pre">,</span> <span class="pre">patch=None)</span></tt></dt>
<dd>This is a <tt class="xref py py-class docutils literal"><span class="pre">SourceStamp</span></tt> used when a series of <tt class="xref py py-class docutils literal"><span class="pre">Change</span></tt>s have
triggered a build. The VC step will attempt to check out a tree that
contains <em>CHANGES</em> (and any changes that occurred before <em>CHANGES</em>, but
not any that occurred after them.)</dd>
<dt><tt class="samp docutils literal"><span class="pre">(revision=None,</span> <span class="pre">changes=None,</span> <span class="pre">patch=None)</span></tt></dt>
<dd>This builds the most recent code on the default branch. This is the
sort of <tt class="xref py py-class docutils literal"><span class="pre">SourceStamp</span></tt> that would be used on a <tt class="xref py py-class docutils literal"><span class="pre">Build</span></tt> that was
triggered by a user request, or a <tt class="xref py py-class docutils literal"><span class="pre">Periodic</span></tt> scheduler. It is also
possible to configure the VC Source Step to always check out the
latest sources rather than paying attention to the <tt class="xref py py-class docutils literal"><span class="pre">Change</span></tt>s in the
<tt class="xref py py-class docutils literal"><span class="pre">SourceStamp</span></tt>, which will result in same behavior as this.</dd>
<dt><tt class="samp docutils literal"><span class="pre">(branch=</span><em><span class="pre">BRANCH</span></em><span class="pre">,</span> <span class="pre">revision=None,</span> <span class="pre">changes=None,</span> <span class="pre">patch=None)</span></tt></dt>
<dd>This builds the most recent code on the given <em>BRANCH</em>. Again, this is
generally triggered by a user request or <tt class="xref py py-class docutils literal"><span class="pre">Periodic</span></tt> build.</dd>
<dt><tt class="samp docutils literal"><span class="pre">(revision=</span><em><span class="pre">REV</span></em><span class="pre">,</span> <span class="pre">changes=None,</span> <span class="pre">patch=(</span><em><span class="pre">LEVEL</span></em><span class="pre">,</span> <em><span class="pre">DIFF</span></em><span class="pre">,</span> <em><span class="pre">SUBDIR_ROOT</span></em><span class="pre">))</span></tt></dt>
<dd>This checks out the tree at the given revision <em>REV</em>, then applies a
patch (using <tt class="docutils literal"><span class="pre">patch</span> <span class="pre">-pLEVEL</span> <span class="pre"><DIFF</span></tt>) from inside the relative
directory <em>SUBDIR_ROOT</em>. Item <em>SUBDIR_ROOT</em> is optional and defaults to the
builder working directory. The <a class="reference internal" href="cmdline.html#cmdline-try" title="try"><tt class="xref bb bb-cmdline docutils literal"><span class="pre">try</span></tt></a> command creates this kind of
<tt class="xref py py-class docutils literal"><span class="pre">SourceStamp</span></tt>. If <tt class="docutils literal"><span class="pre">patch</span></tt> is <tt class="xref docutils literal"><span class="pre">None</span></tt>, the patching step is
bypassed.</dd>
</dl>
<p>The buildmaster is responsible for turning the <tt class="xref py py-class docutils literal"><span class="pre">BuildSet</span></tt> into a
set of <tt class="xref py py-class docutils literal"><span class="pre">BuildRequest</span></tt> objects and queueing them on the
appropriate <tt class="xref py py-class docutils literal"><span class="pre">Builder</span></tt>s.</p>
</div>
<div class="section" id="buildrequest">
<span id="id11"></span><h2>BuildRequest<a class="headerlink" href="#buildrequest" title="Permalink to this headline">¶</a></h2>
<p>A <tt class="xref py py-class docutils literal"><span class="pre">BuildRequest</span></tt> is a request to build a specific set of source
code (spcified by a source stamp) on a single <tt class="xref py py-class docutils literal"><span class="pre">Builder</span></tt>. Each <tt class="xref py py-class docutils literal"><span class="pre">Builder</span></tt> runs the
<tt class="xref py py-class docutils literal"><span class="pre">BuildRequest</span></tt> as soon as it can (i.e. when an associated
buildslave becomes free). <tt class="xref py py-class docutils literal"><span class="pre">BuildRequest</span></tt>s are prioritized from
oldest to newest, so when a buildslave becomes free, the
<tt class="xref py py-class docutils literal"><span class="pre">Builder</span></tt> with the oldest <tt class="xref py py-class docutils literal"><span class="pre">BuildRequest</span></tt> is run.</p>
<p>The <tt class="xref py py-class docutils literal"><span class="pre">BuildRequest</span></tt> contains the <tt class="xref py py-class docutils literal"><span class="pre">SourceStamp</span></tt> specification.
The actual process of running the build (the series of <tt class="xref py py-class docutils literal"><span class="pre">Step</span></tt>s that will
be executed) is implemented by the <tt class="xref py py-class docutils literal"><span class="pre">Build</span></tt> object. In this future
this might be changed, to have the <tt class="xref py py-class docutils literal"><span class="pre">Build</span></tt> define <em>what</em>
gets built, and a separate <tt class="xref py py-class docutils literal"><span class="pre">BuildProcess</span></tt> (provided by the
Builder) to define <em>how</em> it gets built.</p>
<p>The <tt class="xref py py-class docutils literal"><span class="pre">BuildRequest</span></tt> may be mergeable with other compatible
<tt class="xref py py-class docutils literal"><span class="pre">BuildRequest</span></tt>s. Builds that are triggered by incoming <tt class="xref py py-class docutils literal"><span class="pre">Change</span></tt>s
will generally be mergeable. Builds that are triggered by user
requests are generally not, unless they are multiple requests to build
the <em>latest sources</em> of the same branch.</p>
</div>
<div class="section" id="builder">
<span id="id12"></span><h2>Builder<a class="headerlink" href="#builder" title="Permalink to this headline">¶</a></h2>
<p>The Buildmaster runs a collection of <tt class="xref py py-class docutils literal"><span class="pre">Builder</span></tt>s, each of which handles a single
type of build (e.g. full versus quick), on one or more build slaves. <tt class="xref py py-class docutils literal"><span class="pre">Builder</span></tt>s
serve as a kind of queue for a particular type of build. Each <tt class="xref py py-class docutils literal"><span class="pre">Builder</span></tt> gets a
separate column in the waterfall display. In general, each <tt class="xref py py-class docutils literal"><span class="pre">Builder</span></tt> runs
independently (although various kinds of interlocks can cause one <tt class="xref py py-class docutils literal"><span class="pre">Builder</span></tt> to
have an effect on another).</p>
<p>Each builder is a long-lived object which controls a sequence of <tt class="xref py py-class docutils literal"><span class="pre">Build</span></tt>s.
Each <tt class="xref py py-class docutils literal"><span class="pre">Builder</span></tt> is created when the config file is first parsed, and lives forever
(or rather until it is removed from the config file). It mediates the
connections to the buildslaves that do all the work, and is responsible for
creating the <tt class="xref py py-class docutils literal"><span class="pre">Build</span></tt> objects - <a class="reference internal" href="#concepts-build"><em>Build</em></a>.</p>
<p>Each builder gets a unique name, and the path name of a directory where it gets
to do all its work (there is a buildmaster-side directory for keeping status
information, as well as a buildslave-side directory where the actual
checkout/compile/test commands are executed).</p>
<div class="section" id="build-factories">
<span id="concepts-build-factories"></span><h3>Build Factories<a class="headerlink" href="#build-factories" title="Permalink to this headline">¶</a></h3>
<p>A builder also has a <tt class="xref py py-class docutils literal"><span class="pre">BuildFactory</span></tt>, which is responsible for creating new <tt class="xref py py-class docutils literal"><span class="pre">Build</span></tt>
instances: because the <tt class="xref py py-class docutils literal"><span class="pre">Build</span></tt> instance is what actually performs each build,
choosing the <tt class="xref py py-class docutils literal"><span class="pre">BuildFactory</span></tt> is the way to specify what happens each time a build
is done (<a class="reference internal" href="#concepts-build"><em>Build</em></a>).</p>
</div>
<div class="section" id="build-slaves">
<span id="concepts-build-slaves"></span><h3>Build Slaves<a class="headerlink" href="#build-slaves" title="Permalink to this headline">¶</a></h3>
<p>Each builder is associated with one of more <tt class="xref py py-class docutils literal"><span class="pre">BuildSlave</span></tt>s. A builder which is
used to perform Mac OS X builds (as opposed to Linux or Solaris builds) should
naturally be associated with a Mac buildslave.</p>
<p>If multiple buildslaves are available for any given builder, you will
have some measure of redundancy: in case one slave goes offline, the
others can still keep the <tt class="xref py py-class docutils literal"><span class="pre">Builder</span></tt> working. In addition, multiple
buildslaves will allow multiple simultaneous builds for the same
<tt class="xref py py-class docutils literal"><span class="pre">Builder</span></tt>, which might be useful if you have a lot of forced or <tt class="docutils literal"><span class="pre">try</span></tt>
builds taking place.</p>
<p>If you use this feature, it is important to make sure that the
buildslaves are all, in fact, capable of running the given build. The
slave hosts should be configured similarly, otherwise you will spend a
lot of time trying (unsuccessfully) to reproduce a failure that only
occurs on some of the buildslaves and not the others. Different
platforms, operating systems, versions of major programs or libraries,
all these things mean you should use separate Builders.</p>
</div>
</div>
<div class="section" id="build">
<span id="concepts-build"></span><h2>Build<a class="headerlink" href="#build" title="Permalink to this headline">¶</a></h2>
<p>A build is a single compile or test run of a particular version of the source
code, and is comprised of a series of steps. It is ultimately up to you what
constitutes a build, but for compiled software it is generally the checkout,
configure, make, and make check sequence. For interpreted projects like Python
modules, a build is generally a checkout followed by an invocation of the
bundled test suite.</p>
<p>A <tt class="xref py py-class docutils literal"><span class="pre">BuildFactory</span></tt> describes the steps a build will perform. The builder which
starts a build uses its configured build factory to determine the build's
steps.</p>
</div>
<div class="section" id="users">
<span id="concepts-users"></span><h2>Users<a class="headerlink" href="#users" title="Permalink to this headline">¶</a></h2>
<p>Buildbot has a somewhat limited awareness of <em>users</em>. It assumes
the world consists of a set of developers, each of whom can be
described by a couple of simple attributes. These developers make
changes to the source code, causing builds which may succeed or fail.</p>
<p>Users also may have different levels of authorization when issuing Buildbot
commands, such as forcing a build from the web interface or from an IRC channel
(see <a class="reference internal" href="cfg-statustargets.html#status-WebStatus" title="WebStatus"><tt class="xref bb bb-status docutils literal"><span class="pre">WebStatus</span></tt></a> and <a class="reference internal" href="cfg-statustargets.html#status-IRC" title="IRC"><tt class="xref bb bb-status docutils literal"><span class="pre">IRC</span></tt></a>).</p>
<p>Each developer is primarily known through the source control system. Each
<tt class="xref py py-class docutils literal"><span class="pre">Change</span></tt> object that arrives is tagged with a <tt class="xref py py-attr docutils literal"><span class="pre">who</span></tt> field that
typically gives the account name (on the repository machine) of the user
responsible for that change. This string is displayed on the HTML status
pages and in each <tt class="xref py py-class docutils literal"><span class="pre">Build</span></tt>'s <em>blamelist</em>.</p>
<p>To do more with the User than just refer to them, this username needs to be
mapped into an address of some sort. The responsibility for this mapping is
left up to the status module which needs the address. In the future, the
responsbility for managing users will be transferred to User Objects.</p>
<p>The <tt class="docutils literal"><span class="pre">who</span></tt> fields in <tt class="docutils literal"><span class="pre">git</span></tt> Changes are used to create <a class="reference internal" href="#user-objects"><em>User Objects</em></a>,
which allows for more control and flexibility in how Buildbot manages users.</p>
<div class="section" id="user-objects">
<span id="id13"></span><h3>User Objects<a class="headerlink" href="#user-objects" title="Permalink to this headline">¶</a></h3>
<p>User Objects allow Buildbot to better manage users throughout its various
interactions with users (see <a class="reference internal" href="cfg-changesources.html#change-sources"><em>Change Sources</em></a> and <a class="reference internal" href="cfg-statustargets.html#status-targets"><em>Status Targets</em></a>).
The User Objects are stored in the Buildbot database and correlate the various
attributes that a user might have: irc, git, etc.</p>
<div class="section" id="changes">
<h4>Changes<a class="headerlink" href="#changes" title="Permalink to this headline">¶</a></h4>
<p>Incoming Changes all have a <tt class="docutils literal"><span class="pre">who</span></tt> attribute attached to them that specifies
which developer is responsible for that Change. When a Change is first
rendered, the <tt class="docutils literal"><span class="pre">who</span></tt> attribute is parsed and added to the database if it
doesn't exist or checked against an existing user. The <tt class="docutils literal"><span class="pre">who</span></tt> attribute is
formatted in different ways depending on the version control system that the
Change came from.</p>
<dl class="docutils">
<dt><tt class="docutils literal"><span class="pre">git</span></tt></dt>
<dd><tt class="docutils literal"><span class="pre">who</span></tt> attributes take the form <tt class="docutils literal"><span class="pre">Full</span> <span class="pre">Name</span> <span class="pre"><Email></span></tt>.</dd>
<dt><tt class="docutils literal"><span class="pre">svn</span></tt></dt>
<dd><tt class="docutils literal"><span class="pre">who</span></tt> attributes are of the form <tt class="docutils literal"><span class="pre">Username</span></tt>.</dd>
<dt><tt class="docutils literal"><span class="pre">hg</span></tt></dt>
<dd><tt class="docutils literal"><span class="pre">who</span></tt> attributes are free-form strings, but usually adhere to similar
conventions as <tt class="docutils literal"><span class="pre">git</span></tt> attributes (<tt class="docutils literal"><span class="pre">Full</span> <span class="pre">Name</span> <span class="pre"><Email></span></tt>).</dd>
<dt><tt class="docutils literal"><span class="pre">cvs</span></tt></dt>
<dd><tt class="docutils literal"><span class="pre">who</span></tt> attributes are of the form <tt class="docutils literal"><span class="pre">Username</span></tt>.</dd>
<dt><tt class="docutils literal"><span class="pre">darcs</span></tt></dt>
<dd><tt class="docutils literal"><span class="pre">who</span></tt> attributes contain an <tt class="docutils literal"><span class="pre">Email</span></tt> and may also include a <tt class="docutils literal"><span class="pre">Full</span> <span class="pre">Name</span></tt>
like <tt class="docutils literal"><span class="pre">git</span></tt> attributes.</dd>
<dt><tt class="docutils literal"><span class="pre">bzr</span></tt></dt>
<dd><tt class="docutils literal"><span class="pre">who</span></tt> attributes are free-form strings like <tt class="docutils literal"><span class="pre">hg</span></tt>, and can include a
<tt class="docutils literal"><span class="pre">Username</span></tt>, <tt class="docutils literal"><span class="pre">Email</span></tt>, and/or <tt class="docutils literal"><span class="pre">Full</span> <span class="pre">Name</span></tt>.</dd>
</dl>
</div>
<div class="section" id="tools">
<h4>Tools<a class="headerlink" href="#tools" title="Permalink to this headline">¶</a></h4>
<p>For managing users manually, use the <tt class="docutils literal"><span class="pre">buildbot</span> <span class="pre">user</span></tt> command, which allows
you to add, remove, update, and show various attributes of users in the Buildbot
database (see <a class="reference internal" href="cmdline.html#command-line-tool"><em>Command-line Tool</em></a>).</p>
<p>To show all of the users in the database in a more pretty manner, use the users page in
the <a class="reference internal" href="cfg-statustargets.html#status-WebStatus" title="WebStatus"><tt class="xref bb bb-status docutils literal"><span class="pre">WebStatus</span></tt></a>.</p>
</div>
<div class="section" id="uses">
<h4>Uses<a class="headerlink" href="#uses" title="Permalink to this headline">¶</a></h4>
<p>Correlating the various bits and pieces that Buildbot views as users also means
that one attribute of a user can be translated into another. This provides a
more complete view of users throughout Buildbot.</p>
<p>One such use is being able to find email addresses based on a set of Builds
to notify users through the <tt class="docutils literal"><span class="pre">MailNotifier</span></tt>. This process is explained
more clearly in :ref:<tt class="docutils literal"><span class="pre">Email-Addresses</span></tt>.</p>
<p>Another way to utilize <cite>User Objects</cite> is through <cite>UsersAuth</cite> for web authentication
(see <a class="reference internal" href="cfg-statustargets.html#status-WebStatus" title="WebStatus"><tt class="xref bb bb-status docutils literal"><span class="pre">WebStatus</span></tt></a>). To use <cite>UsersAuth</cite>, you need to
set a <cite>bb_username</cite> and <cite>bb_password</cite> via the <tt class="docutils literal"><span class="pre">buildbot</span> <span class="pre">user</span></tt> command line tool
to check against. The password will be encrypted before storing in the database
along with other user attributes.</p>
</div>
</div>
<div class="section" id="doing-things-with-users">
<span id="id14"></span><h3>Doing Things With Users<a class="headerlink" href="#doing-things-with-users" title="Permalink to this headline">¶</a></h3>
<p>Each change has a single user who is responsible for it. Most builds have a set
of changes: the build generally represents the first time these changes have
been built and tested by the Buildbot. The build has a <em>blamelist</em> that is
the union of the users responsible for all the build's changes. If the build
was created by a <a class="reference internal" href="cfg-schedulers.html#try-schedulers"><em>Try Schedulers</em></a> this list will include the submitter of the try
job, if known.</p>
<p>The build provides a list of users who are interested in the build -- the
<em>interested users</em>. Usually this is equal to the blamelist, but may also be
expanded, e.g., to include the current build sherrif or a module's maintainer.</p>
<p>If desired, the buildbot can notify the interested users until the problem is
resolved.</p>
</div>
<div class="section" id="email-addresses">
<span id="id15"></span><h3>Email Addresses<a class="headerlink" href="#email-addresses" title="Permalink to this headline">¶</a></h3>
<p>The <a class="reference internal" href="cfg-statustargets.html#status-MailNotifier" title="MailNotifier"><tt class="xref bb bb-status docutils literal"><span class="pre">MailNotifier</span></tt></a> is a status target which can send email
about the results of each build. It accepts a static list of email
addresses to which each message should be delivered, but it can also
be configured to send mail to the <tt class="xref py py-class docutils literal"><span class="pre">Build</span></tt>'s Interested Users. To do
this, it needs a way to convert User names into email addresses.</p>
<p>For many VC systems, the User Name is actually an account name on the
system which hosts the repository. As such, turning the name into an
email address is a simple matter of appending
<tt class="docutils literal"><span class="pre">@repositoryhost.com</span></tt>. Some projects use other kinds of mappings
(for example the preferred email address may be at <tt class="docutils literal"><span class="pre">project.org</span></tt>
despite the repository host being named <tt class="docutils literal"><span class="pre">cvs.project.org</span></tt>), and some
VC systems have full separation between the concept of a user and that
of an account on the repository host (like Perforce). Some systems
(like Git) put a full contact email address in every change.</p>
<p>To convert these names to addresses, the <tt class="xref py py-class docutils literal"><span class="pre">MailNotifier</span></tt> uses an <tt class="xref py py-class docutils literal"><span class="pre">EmailLookup</span></tt>
object. This provides a <tt class="xref py py-meth docutils literal"><span class="pre">getAddress</span></tt> method which accepts a name and
(eventually) returns an address. The default <tt class="xref py py-class docutils literal"><span class="pre">MailNotifier</span></tt>
module provides an <tt class="xref py py-class docutils literal"><span class="pre">EmailLookup</span></tt> which simply appends a static string,
configurable when the notifier is created. To create more complex behaviors
(perhaps using an LDAP lookup, or using <tt class="docutils literal"><span class="pre">finger</span></tt> on a central host to
determine a preferred address for the developer), provide a different object
as the <tt class="docutils literal"><span class="pre">lookup</span></tt> argument.</p>
<p>If an EmailLookup object isn't given to the MailNotifier, the MailNotifier
will try to find emails through <a class="reference internal" href="#user-objects"><em>User Objects</em></a>. This will work the
same as if an EmailLookup object was used if every user in the Build's
Interested Users list has an email in the database for them. If a user
whose change led to a Build doesn't have an email attribute, that user
will not receive an email. If <tt class="docutils literal"><span class="pre">extraRecipients</span></tt> is given, those users
are still sent mail when the EmailLookup object is not specified.</p>
<p>In the future, when the Problem mechanism has been set up, the Buildbot
will need to send mail to arbitrary Users. It will do this by locating a
<tt class="xref py py-class docutils literal"><span class="pre">MailNotifier</span></tt>-like object among all the buildmaster's status targets, and
asking it to send messages to various Users. This means the User-to-address
mapping only has to be set up once, in your <tt class="xref py py-class docutils literal"><span class="pre">MailNotifier</span></tt>, and every email
message the buildbot emits will take advantage of it.</p>
</div>
<div class="section" id="irc-nicknames">
<span id="id16"></span><h3>IRC Nicknames<a class="headerlink" href="#irc-nicknames" title="Permalink to this headline">¶</a></h3>
<p>Like <tt class="xref py py-class docutils literal"><span class="pre">MailNotifier</span></tt>, the <a class="reference internal" href="cfg-statustargets.html#buildbot.status.words.IRC" title="buildbot.status.words.IRC"><tt class="xref py py-class docutils literal"><span class="pre">buildbot.status.words.IRC</span></tt></a> class
provides a status target which can announce the results of each build. It
also provides an interactive interface by responding to online queries
posted in the channel or sent as private messages.</p>
<p>In the future, the buildbot can be configured map User names to IRC
nicknames, to watch for the recent presence of these nicknames, and to
deliver build status messages to the interested parties. Like
<tt class="xref py py-class docutils literal"><span class="pre">MailNotifier</span></tt> does for email addresses, the <tt class="xref py py-class docutils literal"><span class="pre">IRC</span></tt> object
will have an <tt class="xref py py-class docutils literal"><span class="pre">IRCLookup</span></tt> which is responsible for nicknames. The
mapping can be set up statically, or it can be updated by online users
themselves (by claiming a username with some kind of <tt class="docutils literal"><span class="pre">buildbot:</span> <span class="pre">i</span> <span class="pre">am</span>
<span class="pre">user</span> <span class="pre">warner</span></tt> commands).</p>
<p>Once the mapping is established, the rest of the buildbot can ask the
<tt class="xref py py-class docutils literal"><span class="pre">IRC</span></tt> object to send messages to various users. It can report on
the likelihood that the user saw the given message (based upon how long the
user has been inactive on the channel), which might prompt the Problem
Hassler logic to send them an email message instead.</p>
<p>These operations and authentication of commands issued by particular
nicknames will be implemented in <a class="reference internal" href="#user-objects"><em>User Objects</em></a>.</p>
</div>
<div class="section" id="live-status-clients">
<span id="id17"></span><h3>Live Status Clients<a class="headerlink" href="#live-status-clients" title="Permalink to this headline">¶</a></h3>
<p>The Buildbot also offers a desktop status client interface which can display
real-time build status in a GUI panel on the developer's desktop.</p>
</div>
</div>
<div class="section" id="id18">
<span id="index-0"></span><span id="id19"></span><h2>Build Properties<a class="headerlink" href="#id18" title="Permalink to this headline">¶</a></h2>
<p>Each build has a set of <em>Build Properties</em>, which can be used by its
build steps to modify their actions. These properties, in the form of
key-value pairs, provide a general framework for dynamically altering
the behavior of a build based on its circumstances.</p>
<p>Properties form a simple kind of variable in a build. Some properties are set
when the build starts, and properties can be changed as a build progresses --
properties set or changed in one step may be accessed in subsequent steps.
Property values can be numbers, strings, lists, or dictionaries - basically,
anything that can be represented in JSON.</p>
<p>Properties are very flexible, and can be used to implement all manner
of functionality. Here are some examples:</p>
<p>Most Source steps record the revision that they checked out in
the <tt class="docutils literal"><span class="pre">got_revision</span></tt> property. A later step could use this
property to specify the name of a fully-built tarball, dropped in an
easily-acessible directory for later testing.</p>
<p>Some projects want to perform nightly builds as well as bulding in response to
committed changes. Such a project would run two schedulers, both pointing to
the same set of builders, but could provide an <tt class="docutils literal"><span class="pre">is_nightly</span></tt> property so
that steps can distinguish the nightly builds, perhaps to run more
resource-intensive tests.</p>
<p>Some projects have different build processes on different systems.
Rather than create a build factory for each slave, the steps can use
buildslave properties to identify the unique aspects of each slave
and adapt the build process dynamically.</p>
<p class="rubric">Footnotes</p>
<table class="docutils footnote" frame="void" id="id20" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#id2">[1]</a></td><td>Except Darcs, but since the Buildbot never modifies its local source tree we can ignore
the fact that Darcs uses a less centralized model</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="id21" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#id4">[2]</a></td><td>Many VC systems provide more complexity than this: in particular the local
views that P4 and ClearCase can assemble out of various source
directories are more complex than we're prepared to take advantage of
here</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="id22" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#id6">[3]</a></td><td>This <tt class="docutils literal"><span class="pre">checkoutDelay</span></tt> defaults
to half the tree-stable timer, but it can be overridden with an
argument to the <tt class="xref py py-class docutils literal"><span class="pre">Source</span></tt> Step</td></tr>
</tbody>
</table>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="sidebar">
<h3>Table Of Contents</h3>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../tutorial/index.html">Buildbot Tutorial</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="index.html">Buildbot Manual</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="introduction.html">Introduction</a></li>
<li class="toctree-l2"><a class="reference internal" href="installation.html">Installation</a></li>
<li class="toctree-l2 current"><a class="current reference internal" href="">Concepts</a><ul class="simple">
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="configuration.html">Configuration</a></li>
<li class="toctree-l2"><a class="reference internal" href="customization.html">Customization</a></li>
<li class="toctree-l2"><a class="reference internal" href="cmdline.html">Command-line Tool</a></li>
<li class="toctree-l2"><a class="reference internal" href="resources.html">Resources</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../developer/index.html">Buildbot Development</a></li>
</ul>
<h3 style="margin-top: 1.5em;">Search</h3>
<form class="search" action="../search.html" method="get">
<input type="text" name="q" size="18" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div>
<div class="clearer"></div>
</div>
</div>
<div class="footer-wrapper">
<div class="footer">
<div class="left">
<a href="installation.html" title="Installation"
>previous</a> |
<a href="configuration.html" title="Configuration"
>next</a> |
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |
<a href="../genindex.html" title="General Index"
>index</a>
<br/>
<a href="../_sources/manual/concepts.txt"
rel="nofollow">Show Source</a>
</div>
<div class="right">
<div class="footer">
© Copyright Buildbot Team Members.
Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 1.0.7.
</div>
</div>
<div class="clearer"></div>
</div>
</div>
</body>
</html>
