<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Configuring Buildbot — Buildbot 0.8.12 documentation</title>
<link rel="stylesheet" href="../_static/agogo.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
VERSION: '0.8.12',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="../_static/jquery.js"></script>
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="shortcut icon" href="../_static/buildbot.ico"/>
<link rel="top" title="Buildbot 0.8.12 documentation" href="../index.html" />
<link rel="up" title="Configuration" href="configuration.html" />
<link rel="next" title="Global Configuration" href="cfg-global.html" />
<link rel="prev" title="Configuration" href="configuration.html" />
</head>
<body role="document">
<div class="header-wrapper" role="banner">
<div class="header">
<p class="logo"><a href="../index.html">
<img class="logo" src="../_static/header-text-transparent.png" alt="Logo"/>
</a></p>
<div class="headertitle"><a
href="../index.html">Buildbot 0.8.12 documentation</a></div>
<div class="rel" role="navigation" aria-label="related navigation">
<a href="configuration.html" title="Configuration"
accesskey="P">previous</a> |
<a href="cfg-global.html" title="Global Configuration"
accesskey="N">next</a> |
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |
<a href="../genindex.html" title="General Index"
accesskey="I">index</a>
</div>
</div>
</div>
<div class="content-wrapper">
<div class="content">
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<div class="section" id="configuring-buildbot">
<h1>Configuring Buildbot<a class="headerlink" href="#configuring-buildbot" title="Permalink to this headline">¶</a></h1>
<p>The buildbot's behavior is defined by the <em>config file</em>, which normally lives in the <code class="file docutils literal"><span class="pre">master.cfg</span></code> file in the buildmaster's base directory (but this can be changed with an option to the <strong class="command">buildbot create-master</strong> command).
This file completely specifies which <code class="xref py py-class docutils literal"><span class="pre">Builder</span></code>s are to be run, which slaves they should use, how <code class="xref py py-class docutils literal"><span class="pre">Change</span></code>s should be tracked, and where the status information is to be sent.
The buildmaster's <code class="file docutils literal"><span class="pre">buildbot.tac</span></code> file names the base directory; everything else comes from the config file.</p>
<p>A sample config file was installed for you when you created the buildmaster, but you will need to edit it before your buildbot will do anything useful.</p>
<p>This chapter gives an overview of the format of this file and the various sections in it.
You will need to read the later chapters to understand how to fill in each section properly.</p>
<div class="section" id="config-file-format">
<span id="id1"></span><h2>Config File Format<a class="headerlink" href="#config-file-format" title="Permalink to this headline">¶</a></h2>
<p>The config file is, fundamentally, just a piece of Python code which defines a dictionary named <code class="docutils literal"><span class="pre">BuildmasterConfig</span></code>, with a number of keys that are treated specially.
You don't need to know Python to do basic configuration, though, you can just copy the syntax of the sample file.
If you <em>are</em> comfortable writing Python code, however, you can use all the power of a full programming language to achieve more complicated configurations.</p>
<p>The <code class="docutils literal"><span class="pre">BuildmasterConfig</span></code> name is the only one which matters: all other names defined during the execution of the file are discarded.
When parsing the config file, the Buildmaster generally compares the old configuration with the new one and performs the minimum set of actions necessary to bring the buildbot up to date: <code class="xref py py-class docutils literal"><span class="pre">Builder</span></code>s which are not changed are left untouched, and <code class="xref py py-class docutils literal"><span class="pre">Builder</span></code>s which are modified get to keep their old event history.</p>
<p>The beginning of the <code class="file docutils literal"><span class="pre">master.cfg</span></code> file typically starts with something like:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">BuildmasterConfig</span> <span class="o">=</span> <span class="n">c</span> <span class="o">=</span> <span class="p">{}</span>
</pre></div>
</div>
<p>Therefore a config key like <a class="reference internal" href="cfg-changesources.html#cfg-change_source" title="change_source"><code class="xref bb bb-cfg docutils literal"><span class="pre">change_source</span></code></a> will usually appear in <code class="file docutils literal"><span class="pre">master.cfg</span></code> as <code class="docutils literal"><span class="pre">c['change_source']</span></code>.</p>
<p>See <a class="reference internal" href="../bb-cfg.html" title="Buildmaster Configuration Index"><em>Buildmaster Configuration Index</em></a> for a full list of <code class="docutils literal"><span class="pre">BuildMasterConfig</span></code> keys.</p>
<div class="section" id="basic-python-syntax">
<h3>Basic Python Syntax<a class="headerlink" href="#basic-python-syntax" title="Permalink to this headline">¶</a></h3>
<p>The master configuration file is interpreted as Python, allowing the full flexibility of the language.
For the configurations described in this section, a detailed knowledge of Python is not required, but the basic syntax is easily described.</p>
<p>Python comments start with a hash character <code class="docutils literal"><span class="pre">#</span></code>, tuples are defined with <code class="docutils literal"><span class="pre">(parenthesis,</span> <span class="pre">pairs)</span></code>, and lists (arrays) are defined with <code class="docutils literal"><span class="pre">[square,</span> <span class="pre">brackets]</span></code>.
Tuples and lists are mostly interchangeable.
Dictionaries (data structures which map <em>keys</em> to <em>values</em>) are defined with curly braces: <code class="docutils literal"><span class="pre">{'key1':</span> <span class="pre">value1,</span> <span class="pre">'key2':</span> <span class="pre">value2}</span></code>.
Function calls (and object instantiation) can use named parameters, like <code class="docutils literal"><span class="pre">w</span> <span class="pre">=</span> <span class="pre">html.Waterfall(http_port=8010)</span></code>.</p>
<p>The config file starts with a series of <code class="docutils literal"><span class="pre">import</span></code> statements, which make various kinds of <code class="xref py py-class docutils literal"><span class="pre">Step</span></code>s and <code class="xref py py-class docutils literal"><span class="pre">Status</span></code> targets available for later use.
The main <code class="docutils literal"><span class="pre">BuildmasterConfig</span></code> dictionary is created, then it is populated with a variety of keys, described section-by-section in subsequent chapters.</p>
</div>
</div>
<div class="section" id="predefined-config-file-symbols">
<span id="id2"></span><h2>Predefined Config File Symbols<a class="headerlink" href="#predefined-config-file-symbols" title="Permalink to this headline">¶</a></h2>
<p>The following symbols are automatically available for use in the configuration file.</p>
<dl class="docutils">
<dt><code class="docutils literal"><span class="pre">basedir</span></code></dt>
<dd><p class="first">the base directory for the buildmaster.
This string has not been expanded, so it may start with a tilde.
It needs to be expanded before use. The config file is located in:</p>
<div class="last highlight-default"><div class="highlight"><pre><span></span><span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">expanduser</span><span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">basedir</span><span class="p">,</span> <span class="s1">'master.cfg'</span><span class="p">))</span>
</pre></div>
</div>
</dd>
<dt><code class="docutils literal"><span class="pre">__file__</span></code></dt>
<dd>the absolute path of the config file.
The config file's directory is located in <code class="docutils literal"><span class="pre">os.path.dirname(__file__)</span></code>.</dd>
</dl>
</div>
<div class="section" id="testing-the-config-file">
<span id="id3"></span><h2>Testing the Config File<a class="headerlink" href="#testing-the-config-file" title="Permalink to this headline">¶</a></h2>
<p>To verify that the config file is well-formed and contains no deprecated or invalid elements, use the <code class="docutils literal"><span class="pre">checkconfig</span></code> command, passing it either a master directory or a config file.</p>
<div class="highlight-bash"><div class="highlight"><pre><span></span>% buildbot checkconfig master.cfg
Config file is good!
<span class="c1"># or</span>
% buildbot checkconfig /tmp/masterdir
Config file is good!
</pre></div>
</div>
<p>If the config file has deprecated features (perhaps because you've upgraded the buildmaster and need to update the config file to match), they will be announced by checkconfig.
In this case, the config file will work, but you should really remove the deprecated items and use the recommended replacements instead:</p>
<div class="highlight-none"><div class="highlight"><pre><span></span>% buildbot checkconfig master.cfg
/usr/lib/python2.4/site-packages/buildbot/master.py:559: DeprecationWarning: c['sources'] is
deprecated as of 0.7.6 and will be removed by 0.8.0 . Please use c['change_source'] instead.
Config file is good!
</pre></div>
</div>
<p>If you have errors in your configuration file, checkconfig will let you know:</p>
<div class="highlight-none"><div class="highlight"><pre><span></span>% buildbot checkconfig master.cfg
Configuration Errors:
c['slaves'] must be a list of BuildSlave instances
no slaves are configured
builder 'smoketest' uses unknown slaves 'linux-002'
</pre></div>
</div>
<p>If the config file is simply broken, that will be caught too:</p>
<div class="highlight-none"><div class="highlight"><pre><span></span>% buildbot checkconfig master.cfg
error while parsing config file:
Traceback (most recent call last):
File "/home/buildbot/master/bin/buildbot", line 4, in <module>
runner.run()
File "/home/buildbot/master/buildbot/scripts/runner.py", line 1358, in run
if not doCheckConfig(so):
File "/home/buildbot/master/buildbot/scripts/runner.py", line 1079, in doCheckConfig
return cl.load(quiet=quiet)
File "/home/buildbot/master/buildbot/scripts/checkconfig.py", line 29, in load
self.basedir, self.configFileName)
--- <exception caught here> ---
File "/home/buildbot/master/buildbot/config.py", line 147, in loadConfig
exec f in localDict
exceptions.SyntaxError: invalid syntax (master.cfg, line 52)
Configuration Errors:
error while parsing config file: invalid syntax (master.cfg, line 52) (traceback in logfile)
</pre></div>
</div>
</div>
<div class="section" id="loading-the-config-file">
<h2>Loading the Config File<a class="headerlink" href="#loading-the-config-file" title="Permalink to this headline">¶</a></h2>
<p>The config file is only read at specific points in time.
It is first read when the buildmaster is launched.</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">If the configuration is invalid, the master will display the errors in the console output, but will not exit.</p>
</div>
<div class="section" id="reloading-the-config-file-reconfig">
<h3>Reloading the Config File (reconfig)<a class="headerlink" href="#reloading-the-config-file-reconfig" title="Permalink to this headline">¶</a></h3>
<p>If you are on the system hosting the buildmaster, you can send a <code class="docutils literal"><span class="pre">SIGHUP</span></code> signal to it: the <strong class="command">buildbot</strong> tool has a shortcut for this:</p>
<div class="highlight-none"><div class="highlight"><pre><span></span>buildbot reconfig BASEDIR
</pre></div>
</div>
<p>This command will show you all of the lines from <code class="file docutils literal"><span class="pre">twistd.log</span></code> that relate to the reconfiguration.
If there are any problems during the config-file reload, they will be displayed in these lines.</p>
<p>When reloading the config file, the buildmaster will endeavor to change as little as possible about the running system.
For example, although old status targets may be shut down and new ones started up, any status targets that were not changed since the last time the config file was read will be left running and untouched.
Likewise any <code class="xref py py-class docutils literal"><span class="pre">Builder</span></code>s which have not been changed will be left running.
If a <code class="xref py py-class docutils literal"><span class="pre">Builder</span></code> is modified (say, the build process is changed) while a <code class="xref py py-class docutils literal"><span class="pre">Build</span></code> is currently running, that <code class="xref py py-class docutils literal"><span class="pre">Build</span></code> will keep running with the old process until it completes.
Any previously queued <code class="xref py py-class docutils literal"><span class="pre">Build</span></code>s (or <code class="xref py py-class docutils literal"><span class="pre">Build</span></code>s which get queued after the reconfig) will use the new process.</p>
<div class="admonition warning">
<p class="first admonition-title">Warning</p>
<p>Buildbot's reconfiguration system is fragile for a few difficult-to-fix reasons:</p>
<ul class="simple">
<li>Any modules imported by the configuration file are not automatically reloaded.
Python modules such as <a class="reference external" href="http://pypi.python.org/pypi/lazy-reload">http://pypi.python.org/pypi/lazy-reload</a> may help here, but reloading modules is fraught with subtleties and difficult-to-decipher failure cases.</li>
<li>During the reconfiguration, active internal objects are divorced from the service hierarchy, leading to tracebacks in the web interface and other components.
These are ordinarily transient, but with HTTP connection caching (either by the browser or an intervening proxy) they can last for a long time.</li>
<li>If the new configuration file is invalid, it is possible for Buildbot's internal state to be corrupted, leading to undefined results.
When this occurs, it is best to restart the master.</li>
<li>For more advanced configurations, it is impossible for Buildbot to tell if the configuration for a <code class="xref py py-class docutils literal"><span class="pre">Builder</span></code> or <code class="xref py py-class docutils literal"><span class="pre">Scheduler</span></code> has changed, and thus the <code class="xref py py-class docutils literal"><span class="pre">Builder</span></code> or <code class="xref py py-class docutils literal"><span class="pre">Scheduler</span></code> will always be reloaded.
This occurs most commonly when a callable is passed as a configuration parameter.</li>
</ul>
<p class="last">The bbproto project (at <a class="reference external" href="https://github.com/dabrahams/bbproto">https://github.com/dabrahams/bbproto</a>) may help to construct large (multi-file) configurations which can be effectively reloaded and reconfigured.</p>
</div>
</div>
<div class="section" id="reconfig-by-debug-client">
<h3>Reconfig by Debug Client<a class="headerlink" href="#reconfig-by-debug-client" title="Permalink to this headline">¶</a></h3>
<p>The <a class="reference internal" href="cmdline.html#cmdline-debugclient" title="debugclient"><code class="xref bb bb-cmdline docutils literal"><span class="pre">debug</span> <span class="pre">tool</span></code></a> (<code class="samp docutils literal"><span class="pre">buildbot</span> <span class="pre">debugclient</span> <span class="pre">--master</span> <em><span class="pre">HOST</span></em><span class="pre">:</span><em><span class="pre">PORT</span></em></code>) has a <span class="guilabel">Reload .cfg</span> button which will also trigger a reload.</p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="sidebar">
<h3>Table Of Contents</h3>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../tutorial/index.html">Buildbot Tutorial</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="index.html">Buildbot Manual</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="introduction.html">Introduction</a></li>
<li class="toctree-l2"><a class="reference internal" href="installation.html">Installation</a></li>
<li class="toctree-l2"><a class="reference internal" href="concepts.html">Concepts</a></li>
<li class="toctree-l2 current"><a class="reference internal" href="configuration.html">Configuration</a><ul class="current">
<li class="toctree-l3 current"><a class="current reference internal" href="#">Configuring Buildbot</a><ul class="simple">
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="cfg-global.html">Global Configuration</a></li>
<li class="toctree-l3"><a class="reference internal" href="cfg-changesources.html">Change Sources</a></li>
<li class="toctree-l3"><a class="reference internal" href="cfg-schedulers.html">Schedulers</a></li>
<li class="toctree-l3"><a class="reference internal" href="cfg-buildslaves.html">Buildslaves</a></li>
<li class="toctree-l3"><a class="reference internal" href="cfg-builders.html">Builder Configuration</a></li>
<li class="toctree-l3"><a class="reference internal" href="cfg-buildfactories.html">Build Factories</a></li>
<li class="toctree-l3"><a class="reference internal" href="cfg-properties.html">Properties</a></li>
<li class="toctree-l3"><a class="reference internal" href="cfg-buildsteps.html">Build Steps</a></li>
<li class="toctree-l3"><a class="reference internal" href="cfg-interlocks.html">Interlocks</a></li>
<li class="toctree-l3"><a class="reference internal" href="cfg-statustargets.html">Status Targets</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="customization.html">Customization</a></li>
<li class="toctree-l2"><a class="reference internal" href="new-style-steps.html">New-Style Build Steps</a></li>
<li class="toctree-l2"><a class="reference internal" href="cmdline.html">Command-line Tool</a></li>
<li class="toctree-l2"><a class="reference internal" href="resources.html">Resources</a></li>
<li class="toctree-l2"><a class="reference internal" href="optimization.html">Optimization</a></li>
<li class="toctree-l2"><a class="reference internal" href="plugins.html">Plugin Infrastructure in Buildbot</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../developer/index.html">Buildbot Development</a></li>
<li class="toctree-l1"><a class="reference internal" href="../relnotes/index.html">Release Notes for Buildbot 0.8.12</a></li>
</ul>
<div role="search">
<h3 style="margin-top: 1.5em;">Search</h3>
<form class="search" action="../search.html" method="get">
<input type="text" name="q" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div>
<div class="clearer"></div>
</div>
</div>
<div class="footer-wrapper">
<div class="footer">
<div class="left">
<div role="navigation" aria-label="related navigaton">
<a href="configuration.html" title="Configuration"
>previous</a> |
<a href="cfg-global.html" title="Global Configuration"
>next</a> |
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |
<a href="../genindex.html" title="General Index"
>index</a>
</div>
<div role="note" aria-label="source link">
<br/>
<a href="../_sources/manual/cfg-intro.txt"
rel="nofollow">Show Source</a>
</div>
</div>
<div class="right">
<div class="footer" role="contentinfo">
© Copyright Buildbot Team Members.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.4.1.
</div>
</div>
<div class="clearer"></div>
</div>
</div>
</body>
</html>
