<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <title>Build Steps — Buildbot 0.8.8 documentation</title> <link rel="stylesheet" href="../_static/agogo.css" type="text/css" /> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" /> <script type="text/javascript"> var DOCUMENTATION_OPTIONS = { URL_ROOT: '../', VERSION: '0.8.8', COLLAPSE_INDEX: false, FILE_SUFFIX: '.html', HAS_SOURCE: true }; </script> <script type="text/javascript" src="../_static/jquery.js"></script> <script type="text/javascript" src="../_static/underscore.js"></script> <script type="text/javascript" src="../_static/doctools.js"></script> <link rel="shortcut icon" href="../_static/buildbot.ico"/> <link rel="top" title="Buildbot 0.8.8 documentation" href="../index.html" /> <link rel="up" title="Configuration" href="configuration.html" /> <link rel="next" title="Interlocks" href="cfg-interlocks.html" /> <link rel="prev" title="Properties" href="cfg-properties.html" /> </head> <body> <div class="header-wrapper"> <div class="header"> <p class="logo"><a href="../index.html"> <img class="logo" src="../_static/header-text-transparent.png" alt="Logo"/> </a></p> <div class="headertitle"><a href="../index.html">Buildbot 0.8.8 documentation</a></div> <div class="rel"> <a href="cfg-properties.html" title="Properties" accesskey="P">previous</a> | <a href="cfg-interlocks.html" title="Interlocks" accesskey="N">next</a> | <a href="../py-modindex.html" title="Python Module Index" >modules</a> | <a href="../genindex.html" title="General Index" accesskey="I">index</a> </div> </div> </div> <div class="content-wrapper"> <div class="content"> <div class="document"> <div class="documentwrapper"> <div class="bodywrapper"> <div class="body"> <div class="section" id="build-steps"> <span id="id1"></span><h1>Build Steps<a class="headerlink" href="#build-steps" title="Permalink to this headline">¶</a></h1> <p><tt class="xref py py-class docutils literal"><span class="pre">BuildStep</span></tt>s are usually specified in the buildmaster's configuration file, in a list that goes into the <tt class="xref py py-class docutils literal"><span class="pre">BuildFactory</span></tt>. The <tt class="xref py py-class docutils literal"><span class="pre">BuildStep</span></tt> instances in this list are used as templates to construct new independent copies for each build (so that state can be kept on the <tt class="xref py py-class docutils literal"><span class="pre">BuildStep</span></tt> in one build without affecting a later build). Each <tt class="xref py py-class docutils literal"><span class="pre">BuildFactory</span></tt> can be created with a list of steps, or the factory can be created empty and then steps added to it using the <tt class="xref py py-meth docutils literal"><span class="pre">addStep</span></tt> method:</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps</span> <span class="kn">import</span> <span class="n">source</span><span class="p">,</span> <span class="n">shell</span> <span class="kn">from</span> <span class="nn">buildbot.process</span> <span class="kn">import</span> <span class="n">factory</span> <span class="n">f</span> <span class="o">=</span> <span class="n">factory</span><span class="o">.</span><span class="n">BuildFactory</span><span class="p">()</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">SVN</span><span class="p">(</span><span class="n">svnurl</span><span class="o">=</span><span class="s">"http://svn.example.org/Trunk/"</span><span class="p">))</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">shell</span><span class="o">.</span><span class="n">ShellCommand</span><span class="p">(</span><span class="n">command</span><span class="o">=</span><span class="p">[</span><span class="s">"make"</span><span class="p">,</span> <span class="s">"all"</span><span class="p">]))</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">shell</span><span class="o">.</span><span class="n">ShellCommand</span><span class="p">(</span><span class="n">command</span><span class="o">=</span><span class="p">[</span><span class="s">"make"</span><span class="p">,</span> <span class="s">"test"</span><span class="p">]))</span> </pre></div> </div> <p>The basic behavior for a <tt class="xref py py-class docutils literal"><span class="pre">BuildStep</span></tt> is to:</p> <blockquote> <div><ul class="simple"> <li>run for a while, then stop</li> <li>possibly invoke some RemoteCommands on the attached build slave</li> <li>possibly produce a set of log files</li> <li>finish with a status described by one of four values defined in <tt class="xref py py-mod docutils literal"><span class="pre">buildbot.status.builder</span></tt>: <tt class="docutils literal"><span class="pre">SUCCESS</span></tt>, <tt class="docutils literal"><span class="pre">WARNINGS</span></tt>, <tt class="docutils literal"><span class="pre">FAILURE</span></tt>, <tt class="docutils literal"><span class="pre">SKIPPED</span></tt></li> <li>provide a list of short strings to describe the step</li> </ul> </div></blockquote> <p>The rest of this section describes all the standard <tt class="xref py py-class docutils literal"><span class="pre">BuildStep</span></tt> objects available for use in a <tt class="xref py py-class docutils literal"><span class="pre">Build</span></tt>, and the parameters which can be used to control each. A full list of build steps is available in the <a class="reference internal" href="../bb-step.html" title="Build Step Index"><em>Build Step Index</em></a>.</p> <div class="section" id="common-parameters"> <span id="buildstep-common-parameters"></span><span id="index-0"></span><h2>Common Parameters<a class="headerlink" href="#common-parameters" title="Permalink to this headline">¶</a></h2> <p>All <tt class="xref py py-class docutils literal"><span class="pre">BuildStep</span></tt>s accept some common parameters. Some of these control how their individual status affects the overall build. Others are used to specify which <cite>Locks</cite> (see <a class="reference internal" href="cfg-interlocks.html#interlocks"><em>Interlocks</em></a>) should be acquired before allowing the step to run.</p> <p>Arguments common to all <tt class="xref py py-class docutils literal"><span class="pre">BuildStep</span></tt> subclasses:</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">name</span></tt></dt> <dd>the name used to describe the step on the status display. It is also used to give a name to any <tt class="xref py py-class docutils literal"><span class="pre">LogFile</span></tt>s created by this step.</dd> </dl> <dl class="docutils" id="index-1"> <dt><tt class="docutils literal"><span class="pre">haltOnFailure</span></tt></dt> <dd>if <tt class="docutils literal"><span class="pre">True</span></tt>, a <tt class="docutils literal"><span class="pre">FAILURE</span></tt> of this build step will cause the build to halt immediately. Steps with <tt class="docutils literal"><span class="pre">alwaysRun=True</span></tt> are still run. Generally speaking, <tt class="docutils literal"><span class="pre">haltOnFailure</span></tt> implies <tt class="docutils literal"><span class="pre">flunkOnFailure</span></tt> (the default for most <tt class="xref py py-class docutils literal"><span class="pre">BuildStep</span></tt>s). In some cases, particularly series of tests, it makes sense to <tt class="docutils literal"><span class="pre">haltOnFailure</span></tt> if something fails early on but not <tt class="docutils literal"><span class="pre">flunkOnFailure</span></tt>. This can be achieved with <tt class="docutils literal"><span class="pre">haltOnFailure=True</span></tt>, <tt class="docutils literal"><span class="pre">flunkOnFailure=False</span></tt>.</dd> </dl> <dl class="docutils" id="index-2"> <dt><tt class="docutils literal"><span class="pre">flunkOnWarnings</span></tt></dt> <dd>when <tt class="docutils literal"><span class="pre">True</span></tt>, a <tt class="docutils literal"><span class="pre">WARNINGS</span></tt> or <tt class="docutils literal"><span class="pre">FAILURE</span></tt> of this build step will mark the overall build as <tt class="docutils literal"><span class="pre">FAILURE</span></tt>. The remaining steps will still be executed.</dd> </dl> <dl class="docutils" id="index-3"> <dt><tt class="docutils literal"><span class="pre">flunkOnFailure</span></tt></dt> <dd>when <tt class="docutils literal"><span class="pre">True</span></tt>, a <tt class="docutils literal"><span class="pre">FAILURE</span></tt> of this build step will mark the overall build as a <tt class="docutils literal"><span class="pre">FAILURE</span></tt>. The remaining steps will still be executed.</dd> </dl> <dl class="docutils" id="index-4"> <dt><tt class="docutils literal"><span class="pre">warnOnWarnings</span></tt></dt> <dd>when <tt class="docutils literal"><span class="pre">True</span></tt>, a <tt class="docutils literal"><span class="pre">WARNINGS</span></tt> or <tt class="docutils literal"><span class="pre">FAILURE</span></tt> of this build step will mark the overall build as having <tt class="docutils literal"><span class="pre">WARNINGS</span></tt>. The remaining steps will still be executed.</dd> </dl> <dl class="docutils" id="index-5"> <dt><tt class="docutils literal"><span class="pre">warnOnFailure</span></tt></dt> <dd>when <tt class="docutils literal"><span class="pre">True</span></tt>, a <tt class="docutils literal"><span class="pre">FAILURE</span></tt> of this build step will mark the overall build as having <tt class="docutils literal"><span class="pre">WARNINGS</span></tt>. The remaining steps will still be executed.</dd> </dl> <dl class="docutils" id="index-6"> <dt><tt class="docutils literal"><span class="pre">alwaysRun</span></tt></dt> <dd>if <tt class="docutils literal"><span class="pre">True</span></tt>, this build step will always be run, even if a previous buildstep with <tt class="docutils literal"><span class="pre">haltOnFailure=True</span></tt> has failed.</dd> </dl> <dl class="docutils" id="index-7"> <dt><tt class="docutils literal"><span class="pre">doStepIf</span></tt></dt> <dd>A step can be configured to only run under certain conditions. To do this, set the step's <tt class="docutils literal"><span class="pre">doStepIf</span></tt> to a boolean value, or to a function that returns a boolean value or Deferred. If the value or function result is false, then the step will return <tt class="docutils literal"><span class="pre">SKIPPED</span></tt> without doing anything. Otherwise, the step will be executed normally. If you set <tt class="docutils literal"><span class="pre">doStepIf</span></tt> to a function, that function should accept one parameter, which will be the <tt class="xref py py-class docutils literal"><span class="pre">Step</span></tt> object itself.</dd> </dl> <dl class="docutils" id="index-8"> <dt><tt class="docutils literal"><span class="pre">hideStepIf</span></tt></dt> <dd><p class="first">A step can be optionally hidden from the waterfall and build details web pages. To do this, set the step's <tt class="docutils literal"><span class="pre">hideStepIf</span></tt> to a boolean value, or to a function that takes two parameters -- the results and the <tt class="xref py py-class docutils literal"><span class="pre">BuildStep</span></tt> -- and returns a boolean value. Steps are always shown while they execute, however after the step as finished, this parameter is evaluated (if a function) and if the value is True, the step is hidden. For example, in order to hide the step if the step has been skipped,</p> <div class="last highlight-python"><div class="highlight"><pre><span class="n">factory</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">Foo</span><span class="p">(</span><span class="o">...</span><span class="p">,</span> <span class="n">hideStepIf</span><span class="o">=</span><span class="k">lambda</span> <span class="n">results</span><span class="p">,</span> <span class="n">s</span><span class="p">:</span> <span class="n">results</span><span class="o">==</span><span class="n">SKIPPED</span><span class="p">))</span> </pre></div> </div> </dd> </dl> <dl class="docutils" id="index-9"> <dt><tt class="docutils literal"><span class="pre">locks</span></tt></dt> <dd>a list of <tt class="docutils literal"><span class="pre">Locks</span></tt> (instances of <tt class="xref py py-class docutils literal"><span class="pre">buildbot.locks.SlaveLock</span></tt> or <tt class="xref py py-class docutils literal"><span class="pre">buildbot.locks.MasterLock</span></tt>) that should be acquired before starting this <tt class="xref py py-class docutils literal"><span class="pre">Step</span></tt>. The <tt class="docutils literal"><span class="pre">Locks</span></tt> will be released when the step is complete. Note that this is a list of actual <tt class="xref py py-class docutils literal"><span class="pre">Lock</span></tt> instances, not names. Also note that all Locks must have unique names. See <a class="reference internal" href="cfg-interlocks.html#interlocks"><em>Interlocks</em></a>.</dd> </dl> </div> <div class="section" id="module-buildbot.steps.source"> <span id="id2"></span><span id="source-checkout"></span><h2>Source Checkout<a class="headerlink" href="#module-buildbot.steps.source" title="Permalink to this headline">¶</a></h2> <p>At the moment, Buildbot contains two implementations of most source steps. The new implementation handles most of the logic on the master side, and has a simpler, more unified approach. The older implementation (<a class="reference internal" href="#source-checkout-slave-side"><em>Source Checkout (Slave-Side)</em></a>) handles the logic on the slave side, and some of the classes have a bewildering array of options.</p> <div class="admonition caution"> <p class="first admonition-title">Caution</p> <p>Master-side source checkout steps are recently developed and not stable yet. If you find any bugs please report them on the <a class="reference external" href="http://trac.buildbot.net/newticket">Buildbot Trac</a>. The older Slave-side described source steps are <a class="reference internal" href="#source-checkout-slave-side"><em>Source Checkout (Slave-Side)</em></a>.</p> <p>The old source steps are imported like this:</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.source</span> <span class="kn">import</span> <span class="n">Git</span> </pre></div> </div> <p>while new source steps are in separate source-packages for each version-control system:</p> <div class="last highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.source.git</span> <span class="kn">import</span> <span class="n">Git</span> </pre></div> </div> </div> <p>New users should, where possible, use the new implementations. The old implementations will be deprecated in a later release. Old users should take this opportunity to switch to the new implementations while both are supported by Buildbot.</p> <p>Some version control systems have not yet been implemented as master-side steps. If you are interested in continued support for such a version control system, please consider helping the Buildbot developers to create such an implementation. In particular, version-control systems with proprietary licenses will not be supported without access to the version-control system for development.</p> <div class="section" id="id3"> <h3>Common Parameters<a class="headerlink" href="#id3" title="Permalink to this headline">¶</a></h3> <p>All source checkout steps accept some common parameters to control how they get the sources and where they should be placed. The remaining per-VC-system parameters are mostly to specify where exactly the sources are coming from.</p> <p><tt class="docutils literal"><span class="pre">mode</span></tt> <tt class="docutils literal"><span class="pre">method</span></tt></p> <blockquote> <div><p>These two parameters specify the means by which the source is checked out. <tt class="docutils literal"><span class="pre">mode</span></tt> specifies the type of checkout and <tt class="docutils literal"><span class="pre">method</span></tt> tells about the way to implement it.</p> <div class="highlight-python"><div class="highlight"><pre><span class="n">factory</span> <span class="o">=</span> <span class="n">BuildFactory</span><span class="p">()</span> <span class="kn">from</span> <span class="nn">buildbot.steps.source.mercurial</span> <span class="kn">import</span> <span class="n">Mercurial</span> <span class="n">factory</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">Mercurial</span><span class="p">(</span><span class="n">repourl</span><span class="o">=</span><span class="s">'path/to/repo'</span><span class="p">,</span> <span class="n">mode</span><span class="o">=</span><span class="s">'full'</span><span class="p">,</span> <span class="n">method</span><span class="o">=</span><span class="s">'fresh'</span><span class="p">))</span> </pre></div> </div> <p>The <tt class="docutils literal"><span class="pre">mode</span></tt> parameter a string describing the kind of VC operation that is desired, defaulting to <tt class="docutils literal"><span class="pre">incremental</span></tt>. The options are</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">incremental</span></tt></dt> <dd>Update the source to the desired revision, but do not remove any other files generated by previous builds. This allows compilers to take advantage of object files from previous builds. This mode is exactly same as the old <tt class="docutils literal"><span class="pre">update</span></tt> mode.</dd> <dt><tt class="docutils literal"><span class="pre">full</span></tt></dt> <dd>Update the source, but delete remnants of previous builds. Build steps that follow will need to regenerate all object files.</dd> </dl> <p>Methods are specific to the version-control system in question, as they may take advantage of special behaviors in that version-control system that can make checkouts more efficient or reliable.</p> </div></blockquote> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">workdir</span></tt></dt> <dd>like all Steps, this indicates the directory where the build will take place. Source Steps are special in that they perform some operations outside of the workdir (like creating the workdir itself).</dd> <dt><tt class="docutils literal"><span class="pre">alwaysUseLatest</span></tt></dt> <dd>if True, bypass the usual behavior of checking out the revision in the source stamp, and always update to the latest revision in the repository instead.</dd> <dt><tt class="docutils literal"><span class="pre">retry</span></tt></dt> <dd>If set, this specifies a tuple of <tt class="docutils literal"><span class="pre">(delay,</span> <span class="pre">repeats)</span></tt> which means that when a full VC checkout fails, it should be retried up to <tt class="docutils literal"><span class="pre">repeats</span></tt> times, waiting <tt class="docutils literal"><span class="pre">delay</span></tt> seconds between attempts. If you don't provide this, it defaults to <tt class="docutils literal"><span class="pre">None</span></tt>, which means VC operations should not be retried. This is provided to make life easier for buildslaves which are stuck behind poor network connections.</dd> <dt><tt class="docutils literal"><span class="pre">repository</span></tt></dt> <dd><p class="first">The name of this parameter might vary depending on the Source step you are running. The concept explained here is common to all steps and applies to <tt class="docutils literal"><span class="pre">repourl</span></tt> as well as for <tt class="docutils literal"><span class="pre">baseURL</span></tt> (when applicable).</p> <p class="last">A common idiom is to pass <tt class="docutils literal"><span class="pre">Property('repository',</span> <span class="pre">'url://default/repo/path')</span></tt> as repository. This grabs the repository from the source stamp of the build. This can be a security issue, if you allow force builds from the web, or have the <tt class="xref py py-class docutils literal"><span class="pre">WebStatus</span></tt> change hooks enabled; as the buildslave will download code from an arbitrary repository.</p> </dd> <dt><tt class="docutils literal"><span class="pre">codebase</span></tt></dt> <dd>This specifies which codebase the source step should use to select the right source stamp. The default codebase value is ''. The codebase must correspond to a codebase assigned by the <a class="reference internal" href="cfg-global.html#cfg-codebaseGenerator" title="codebaseGenerator"><tt class="xref bb bb-cfg docutils literal"><span class="pre">codebaseGenerator</span></tt></a>. If there is no codebaseGenerator defined in the master then codebase doesn't need to be set, the default value will then match all changes.</dd> <dt><tt class="docutils literal"><span class="pre">timeout</span></tt></dt> <dd>Specifies the timeout for slave-side operations, in seconds. If your repositories are particularly large, then you may need to increase this value from its default of 1200 (20 minutes).</dd> <dt><tt class="docutils literal"><span class="pre">logEnviron</span></tt></dt> <dd>If this option is true (the default), then the step's logfile will describe the environment variables on the slave. In situations where the environment is not relevant and is long, it may be easier to set logEnviron=False.</dd> <dt><tt class="docutils literal"><span class="pre">env</span></tt></dt> <dd>a dictionary of environment strings which will be added to the child command's environment. The usual property interpolations can be used in environment variable names and values - see <a class="reference internal" href="cfg-properties.html#properties"><em>Properties</em></a>.</dd> </dl> </div> <div class="section" id="mercurial"> <span id="step-mercurial"></span><span id="step-Mercurial"></span><h3>Mercurial<a class="headerlink" href="#mercurial" title="Permalink to this headline">¶</a></h3> <dl class="class"> <dt id="buildbot.steps.source.buildbot.steps.source.mercurial.Mercurial"> <em class="property">class </em><tt class="descclassname">buildbot.steps.source.mercurial.</tt><tt class="descname">Mercurial</tt><a class="headerlink" href="#buildbot.steps.source.buildbot.steps.source.mercurial.Mercurial" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <p>The <a class="reference internal" href="#step-Mercurial" title="Mercurial"><tt class="xref bb bb-step docutils literal"><span class="pre">Mercurial</span></tt></a> build step performs a <a class="reference external" href="http://selenic.com/mercurial">Mercurial</a> (aka <tt class="docutils literal"><span class="pre">hg</span></tt>) checkout or update.</p> <p>Branches are available in two modes: <tt class="docutils literal"><span class="pre">dirname</span></tt>, where the name of the branch is a suffix of the name of the repository, or <tt class="docutils literal"><span class="pre">inrepo</span></tt>, which uses Hg's named-branches support. Make sure this setting matches your changehook, if you have that installed.</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.source.mercurial</span> <span class="kn">import</span> <span class="n">Mercurial</span> <span class="n">factory</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">Mercurial</span><span class="p">(</span><span class="n">repourl</span><span class="o">=</span><span class="s">'path/to/repo'</span><span class="p">,</span> <span class="n">mode</span><span class="o">=</span><span class="s">'full'</span><span class="p">,</span> <span class="n">method</span><span class="o">=</span><span class="s">'fresh'</span><span class="p">,</span> <span class="n">branchType</span><span class="o">=</span><span class="s">'inrepo'</span><span class="p">))</span> </pre></div> </div> <p>The Mercurial step takes the following arguments:</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">repourl</span></tt></dt> <dd>where the Mercurial source repository is available.</dd> <dt><tt class="docutils literal"><span class="pre">defaultBranch</span></tt></dt> <dd>this specifies the name of the branch to use when a Build does not provide one of its own. This will be appended to <tt class="docutils literal"><span class="pre">repourl</span></tt> to create the string that will be passed to the <tt class="docutils literal"><span class="pre">hg</span> <span class="pre">clone</span></tt> command.</dd> <dt><tt class="docutils literal"><span class="pre">branchType</span></tt></dt> <dd>either 'dirname' (default) or 'inrepo' depending on whether the branch name should be appended to the <tt class="docutils literal"><span class="pre">repourl</span></tt> or the branch is a Mercurial named branch and can be found within the <tt class="docutils literal"><span class="pre">repourl</span></tt>.</dd> <dt><tt class="docutils literal"><span class="pre">clobberOnBranchChange</span></tt></dt> <dd>boolean, defaults to <tt class="docutils literal"><span class="pre">True</span></tt>. If set and using inrepos branches, clobber the tree at each branch change. Otherwise, just update to the branch.</dd> </dl> <p><tt class="docutils literal"><span class="pre">mode</span></tt> <tt class="docutils literal"><span class="pre">method</span></tt></p> <blockquote> <div><p>Mercurial's incremental mode does not require a method. The full mode has three methods defined:</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">clobber</span></tt></dt> <dd>It removes the build directory entirely then makes full clone from repo. This can be slow as it need to clone whole repository</dd> <dt><tt class="docutils literal"><span class="pre">fresh</span></tt></dt> <dd>This remove all other files except those tracked by VCS. First it does <strong class="command">hg purge --all</strong> then pull/update</dd> <dt><tt class="docutils literal"><span class="pre">clean</span></tt></dt> <dd>All the files which are tracked by Mercurial and listed ignore files are not deleted. Remaining all other files will be deleted before pull/update. This is equivalent to <strong class="command">hg purge</strong> then pull/update.</dd> </dl> </div></blockquote> </div> <div class="section" id="git"> <span id="step-git"></span><span id="step-Git"></span><h3>Git<a class="headerlink" href="#git" title="Permalink to this headline">¶</a></h3> <dl class="class"> <dt id="buildbot.steps.source.buildbot.steps.source.git.Git"> <em class="property">class </em><tt class="descclassname">buildbot.steps.source.git.</tt><tt class="descname">Git</tt><a class="headerlink" href="#buildbot.steps.source.buildbot.steps.source.git.Git" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <p>The <tt class="docutils literal"><span class="pre">Git</span></tt> build step clones or updates a <a class="reference external" href="http://git.or.cz/">Git</a> repository and checks out the specified branch or revision. Note that the buildbot supports Git version 1.2.0 and later: earlier versions (such as the one shipped in Ubuntu 'Dapper') do not support the <strong class="command">git init</strong> command that the buildbot uses.</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.source.git</span> <span class="kn">import</span> <span class="n">Git</span> <span class="n">factory</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">Git</span><span class="p">(</span><span class="n">repourl</span><span class="o">=</span><span class="s">'git://path/to/repo'</span><span class="p">,</span> <span class="n">mode</span><span class="o">=</span><span class="s">'full'</span><span class="p">,</span> <span class="n">method</span><span class="o">=</span><span class="s">'clobber'</span><span class="p">,</span> <span class="n">submodules</span><span class="o">=</span><span class="bp">True</span><span class="p">))</span> </pre></div> </div> <p>The Git step takes the following arguments:</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">repourl</span></tt></dt> <dd>(required): the URL of the upstream Git repository.</dd> <dt><tt class="docutils literal"><span class="pre">branch</span></tt></dt> <dd>(optional): this specifies the name of the branch to use when a Build does not provide one of its own. If this this parameter is not specified, and the Build does not provide a branch, the default branch of the remote repository will be used.</dd> <dt><tt class="docutils literal"><span class="pre">submodules</span></tt></dt> <dd>(optional): when initializing/updating a Git repository, this decides whether or not buildbot should consider Git submodules. Default: <tt class="docutils literal"><span class="pre">False</span></tt>.</dd> <dt><tt class="docutils literal"><span class="pre">shallow</span></tt></dt> <dd>(optional): instructs git to attempt shallow clones (<tt class="docutils literal"><span class="pre">--depth</span> <span class="pre">1</span></tt>). This option can be used only in full builds with clobber method.</dd> <dt><tt class="docutils literal"><span class="pre">progress</span></tt></dt> <dd>(optional): passes the (<tt class="docutils literal"><span class="pre">--progress</span></tt>) flag to (<strong class="command">git fetch</strong>). This solves issues of long fetches being killed due to lack of output, but requires Git 1.7.2 or later.</dd> <dt><tt class="docutils literal"><span class="pre">retryFetch</span></tt></dt> <dd>(optional): this value defaults to <tt class="docutils literal"><span class="pre">False</span></tt>. In any case if fetch fails buildbot retries to fetch again instead of failing the entire source checkout.</dd> <dt><tt class="docutils literal"><span class="pre">clobberOnFailure</span></tt></dt> <dd>(optional): defaults to <tt class="docutils literal"><span class="pre">False</span></tt>. If a fetch or full clone fails we can checkout source removing everything. This way new repository will be cloned. If retry fails it fails the source checkout step.</dd> </dl> <p><tt class="docutils literal"><span class="pre">mode</span></tt></p> <blockquote> <div><p>(optional): defaults to <tt class="docutils literal"><span class="pre">'incremental'</span></tt>. Specifies whether to clean the build tree or not.</p> <blockquote> <div><dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">incremental</span></tt></dt> <dd>The source is update, but any built files are left untouched.</dd> <dt><tt class="docutils literal"><span class="pre">full</span></tt></dt> <dd>The build tree is clean of any built files. The exact method for doing this is controlled by the <tt class="docutils literal"><span class="pre">method</span></tt> argument.</dd> </dl> </div></blockquote> </div></blockquote> <p><tt class="docutils literal"><span class="pre">method</span></tt></p> <blockquote> <div><p>(optional): defaults to <tt class="docutils literal"><span class="pre">fresh</span></tt> when mode is <tt class="docutils literal"><span class="pre">full</span></tt>. Git's incremental mode does not require a method. The full mode has four methods defined:</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">clobber</span></tt></dt> <dd>It removes the build directory entirely then makes full clone from repo. This can be slow as it need to clone whole repository. To make faster clones enable <tt class="docutils literal"><span class="pre">shallow</span></tt> option. If shallow options is enabled and build request have unknown revision value, then this step fails.</dd> <dt><tt class="docutils literal"><span class="pre">fresh</span></tt></dt> <dd>This remove all other files except those tracked by Git. First it does <strong class="command">git clean -d -f -x</strong> then fetch/checkout to a specified revision(if any). This option is equal to update mode with <tt class="docutils literal"><span class="pre">ignore_ignores=True</span></tt> in old steps.</dd> <dt><tt class="docutils literal"><span class="pre">clean</span></tt></dt> <dd>All the files which are tracked by Git and listed ignore files are not deleted. Remaining all other files will be deleted before fetch/checkout. This is equivalent to <strong class="command">git clean -d -f</strong> then fetch. This is equivalent to <tt class="docutils literal"><span class="pre">ignore_ignores=False</span></tt> in old steps.</dd> <dt><tt class="docutils literal"><span class="pre">copy</span></tt></dt> <dd>This first checkout source into source directory then copy the <tt class="docutils literal"><span class="pre">source</span></tt> directory to <tt class="docutils literal"><span class="pre">build</span></tt> directory then performs the build operation in the copied directory. This way we make fresh builds with very less bandwidth to download source. The behavior of source checkout follows exactly same as incremental. It performs all the incremental checkout behavior in <tt class="docutils literal"><span class="pre">source</span></tt> directory.</dd> </dl> </div></blockquote> <p><tt class="docutils literal"><span class="pre">getDescription</span></tt></p> <blockquote> <div><p>(optional) After checkout, invoke a <cite>git describe</cite> on the revision and save the result in a property; the property's name is either <tt class="docutils literal"><span class="pre">commit-description</span></tt> or <tt class="docutils literal"><span class="pre">commit-description-foo</span></tt>, depending on whether the <tt class="docutils literal"><span class="pre">codebase</span></tt> argument was also provided. The argument should either be a <tt class="docutils literal"><span class="pre">bool</span></tt> or <tt class="docutils literal"><span class="pre">dict</span></tt>, and will change how <cite>git describe</cite> is called:</p> <ul> <li><p class="first"><tt class="docutils literal"><span class="pre">getDescription=False</span></tt>: disables this feature explicitly</p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">getDescription=True</span></tt> or empty <tt class="docutils literal"><span class="pre">dict()</span></tt>: Run <cite>git describe</cite> with no args</p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">getDescription={...}</span></tt>: a dict with keys named the same as the Git option. Each key's value can be <tt class="docutils literal"><span class="pre">False</span></tt> or <tt class="docutils literal"><span class="pre">None</span></tt> to explicitly skip that argument.</p> <p>For the following keys, a value of <tt class="docutils literal"><span class="pre">True</span></tt> appends the same-named Git argument:</p> <blockquote> <div><ul class="simple"> <li><tt class="docutils literal"><span class="pre">all</span></tt> : <cite>--all</cite></li> <li><tt class="docutils literal"><span class="pre">always</span></tt>: <cite>--always</cite></li> <li><tt class="docutils literal"><span class="pre">contains</span></tt>: <cite>--contains</cite></li> <li><tt class="docutils literal"><span class="pre">debug</span></tt>: <cite>--debug</cite></li> <li><tt class="docutils literal"><span class="pre">long</span></tt>: <cite>--long`</cite></li> <li><tt class="docutils literal"><span class="pre">exact-match</span></tt>: <cite>--exact-match</cite></li> <li><tt class="docutils literal"><span class="pre">tags</span></tt>: <cite>--tags</cite></li> <li><tt class="docutils literal"><span class="pre">dirty</span></tt>: <cite>--dirty</cite></li> </ul> </div></blockquote> <p>For the following keys, an integer or string value (depending on what Git expects) will set the argument's parameter appropriately. Examples show the key-value pair:</p> <blockquote> <div><ul class="simple"> <li><tt class="docutils literal"><span class="pre">match=foo</span></tt>: <cite>--match foo</cite></li> <li><tt class="docutils literal"><span class="pre">abbrev=7</span></tt>: <cite>--abbrev=7</cite></li> <li><tt class="docutils literal"><span class="pre">candidates=7</span></tt>: <cite>--candidates=7</cite></li> <li><tt class="docutils literal"><span class="pre">dirty=foo</span></tt>: <cite>--dirty=foo</cite></li> </ul> </div></blockquote> </li> </ul> </div></blockquote> <p><tt class="docutils literal"><span class="pre">config</span></tt></p> <blockquote> <div>(optional) A dict of git configuration settings to pass to the remote git commands.</div></blockquote> </div> <div class="section" id="svn"> <span id="step-svn"></span><span id="step-SVN"></span><h3>SVN<a class="headerlink" href="#svn" title="Permalink to this headline">¶</a></h3> <dl class="class"> <dt id="buildbot.steps.source.buildbot.steps.source.svn.SVN"> <em class="property">class </em><tt class="descclassname">buildbot.steps.source.svn.</tt><tt class="descname">SVN</tt><a class="headerlink" href="#buildbot.steps.source.buildbot.steps.source.svn.SVN" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <p>The <a class="reference internal" href="#step-SVN" title="SVN"><tt class="xref bb bb-step docutils literal"><span class="pre">SVN</span></tt></a> build step performs a <a class="reference external" href="http://subversion.tigris.org">Subversion</a> checkout or update. There are two basic ways of setting up the checkout step, depending upon whether you are using multiple branches or not.</p> <p>The <a class="reference internal" href="#step-SVN" title="SVN"><tt class="xref bb bb-step docutils literal"><span class="pre">SVN</span></tt></a> step should be created with the <tt class="docutils literal"><span class="pre">repourl</span></tt> argument:</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">repourl</span></tt></dt> <dd>(required): this specifies the <tt class="docutils literal"><span class="pre">URL</span></tt> argument that will be given to the <strong class="command">svn checkout</strong> command. It dictates both where the repository is located and which sub-tree should be extracted. One way to specify the branch is to use <tt class="docutils literal"><span class="pre">Interpolate</span></tt>. For example, if you wanted to check out the trunk repository, you could use <tt class="docutils literal"><span class="pre">repourl=Interpolate("http://svn.example.com/repos/%(src::branch)s")</span></tt> Alternatively, if you are using a remote Subversion repository which is accessible through HTTP at a URL of <tt class="docutils literal"><span class="pre">http://svn.example.com/repos</span></tt>, and you wanted to check out the <tt class="docutils literal"><span class="pre">trunk/calc</span></tt> sub-tree, you would directly use <tt class="docutils literal"><span class="pre">repourl="http://svn.example.com/repos/trunk/calc"</span></tt> as an argument to your <a class="reference internal" href="#step-SVN" title="SVN"><tt class="xref bb bb-step docutils literal"><span class="pre">SVN</span></tt></a> step.</dd> </dl> <p>If you are building from multiple branches, then you should create the <a class="reference internal" href="#step-SVN" title="SVN"><tt class="xref bb bb-step docutils literal"><span class="pre">SVN</span></tt></a> step with the <tt class="docutils literal"><span class="pre">repourl</span></tt> and provide branch information with <tt class="docutils literal"><span class="pre">Interpolate</span></tt>:</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.source.svn</span> <span class="kn">import</span> <span class="n">SVN</span> <span class="n">factory</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">SVN</span><span class="p">(</span><span class="n">mode</span><span class="o">=</span><span class="s">'incremental'</span><span class="p">,</span> <span class="n">repourl</span><span class="o">=</span><span class="n">Interpolate</span><span class="p">(</span><span class="s">'svn://svn.example.org/svn/%(src::branch)s/myproject'</span><span class="p">)))</span> </pre></div> </div> <p>Alternatively, the <tt class="docutils literal"><span class="pre">repourl</span></tt> argument can be used to create the <a class="reference internal" href="#step-SVN" title="SVN"><tt class="xref bb bb-step docutils literal"><span class="pre">SVN</span></tt></a> step without <tt class="docutils literal"><span class="pre">Interpolate</span></tt>:</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.source.svn</span> <span class="kn">import</span> <span class="n">SVN</span> <span class="n">factory</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">SVN</span><span class="p">(</span><span class="n">mode</span><span class="o">=</span><span class="s">'full'</span><span class="p">,</span> <span class="n">repourl</span><span class="o">=</span><span class="s">'svn://svn.example.org/svn/myproject/trunk'</span><span class="p">))</span> </pre></div> </div> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">username</span></tt></dt> <dd>(optional): if specified, this will be passed to the <tt class="docutils literal"><span class="pre">svn</span></tt> binary with a <tt class="docutils literal"><span class="pre">--username</span></tt> option.</dd> <dt><tt class="docutils literal"><span class="pre">password</span></tt></dt> <dd>(optional): if specified, this will be passed to the <tt class="docutils literal"><span class="pre">svn</span></tt> binary with a <tt class="docutils literal"><span class="pre">--password</span></tt> option.</dd> <dt><tt class="docutils literal"><span class="pre">extra_args</span></tt></dt> <dd>(optional): if specified, an array of strings that will be passed as extra arguments to the <tt class="docutils literal"><span class="pre">svn</span></tt> binary.</dd> <dt><tt class="docutils literal"><span class="pre">keep_on_purge</span></tt></dt> <dd>(optional): specific files or directories to keep between purges, like some build outputs that can be reused between builds.</dd> <dt><tt class="docutils literal"><span class="pre">depth</span></tt></dt> <dd><p class="first">(optional): Specify depth argument to achieve sparse checkout. Only available if slave has Subversion 1.5 or higher.</p> <p class="last">If set to <tt class="docutils literal"><span class="pre">empty</span></tt> updates will not pull in any files or subdirectories not already present. If set to <tt class="docutils literal"><span class="pre">files</span></tt>, updates will pull in any files not already present, but not directories. If set to <tt class="docutils literal"><span class="pre">immediates</span></tt>, updates will pull in any files or subdirectories not already present, the new subdirectories will have depth: empty. If set to <tt class="docutils literal"><span class="pre">infinity</span></tt>, updates will pull in any files or subdirectories not already present; the new subdirectories will have depth-infinity. Infinity is equivalent to SVN default update behavior, without specifying any depth argument.</p> </dd> <dt><tt class="docutils literal"><span class="pre">preferLastChangedRev</span></tt></dt> <dd>(optional): By default, the <tt class="docutils literal"><span class="pre">got_revision</span></tt> property is set to the repository's global revision ("Revision" in the <cite>svn info</cite> output). Set this parameter to <tt class="docutils literal"><span class="pre">True</span></tt> to have it set to the "Last Changed Rev" instead.</dd> </dl> <p><tt class="docutils literal"><span class="pre">mode</span></tt> <tt class="docutils literal"><span class="pre">method</span></tt></p> <blockquote> <div><p>SVN's incremental mode does not require a method. The full mode has five methods defined:</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">clobber</span></tt></dt> <dd>It removes the working directory for each build then makes full checkout.</dd> <dt><tt class="docutils literal"><span class="pre">fresh</span></tt></dt> <dd>This always always purges local changes before updating. This deletes unversioned files and reverts everything that would appear in a <strong class="command">svn status --no-ignore</strong>. This is equivalent to the old update mode with <tt class="docutils literal"><span class="pre">always_purge</span></tt>.</dd> <dt><tt class="docutils literal"><span class="pre">clean</span></tt></dt> <dd>This is same as fresh except that it deletes all unversioned files generated by <strong class="command">svn status</strong>.</dd> <dt><tt class="docutils literal"><span class="pre">copy</span></tt></dt> <dd>This first checkout source into source directory then copy the <tt class="docutils literal"><span class="pre">source</span></tt> directory to <tt class="docutils literal"><span class="pre">build</span></tt> directory then performs the build operation in the copied directory. This way we make fresh builds with very less bandwidth to download source. The behavior of source checkout follows exactly same as incremental. It performs all the incremental checkout behavior in <tt class="docutils literal"><span class="pre">source</span></tt> directory.</dd> <dt><tt class="docutils literal"><span class="pre">export</span></tt></dt> <dd>Similar to <tt class="docutils literal"><span class="pre">method='copy'</span></tt>, except using <tt class="docutils literal"><span class="pre">svn</span> <span class="pre">export</span></tt> to create build directory so that there are no <tt class="docutils literal"><span class="pre">.svn</span></tt> directories in the build directory.</dd> </dl> </div></blockquote> <p>If you are using branches, you must also make sure your <tt class="docutils literal"><span class="pre">ChangeSource</span></tt> will report the correct branch names.</p> </div> <div class="section" id="cvs"> <span id="step-cvs"></span><span id="step-CVS"></span><h3>CVS<a class="headerlink" href="#cvs" title="Permalink to this headline">¶</a></h3> <dl class="class"> <dt id="buildbot.steps.source.buildbot.steps.source.cvs.CVS"> <em class="property">class </em><tt class="descclassname">buildbot.steps.source.cvs.</tt><tt class="descname">CVS</tt><a class="headerlink" href="#buildbot.steps.source.buildbot.steps.source.cvs.CVS" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <p>The <a class="reference internal" href="#step-CVS" title="CVS"><tt class="xref bb bb-step docutils literal"><span class="pre">CVS</span></tt></a> build step performs a <a class="reference external" href="http://www.nongnu.org/cvs/">CVS</a> checkout or update.</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.source.cvs</span> <span class="kn">import</span> <span class="n">CVS</span> <span class="n">factory</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">CVS</span><span class="p">(</span><span class="n">mode</span><span class="o">=</span><span class="s">'incremental'</span><span class="p">,</span> <span class="n">cvsroot</span><span class="o">=</span><span class="s">':pserver:me@cvs.sourceforge.net:/cvsroot/myproj'</span><span class="p">,</span> <span class="n">cvsmodule</span><span class="o">=</span><span class="s">'buildbot'</span><span class="p">))</span> </pre></div> </div> <p>This step takes the following arguments:</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">cvsroot</span></tt></dt> <dd>(required): specify the CVSROOT value, which points to a CVS repository, probably on a remote machine. For example, if Buildbot was hosted in CVS then the cvsroot value you would use to get a copy of the Buildbot source code might be <tt class="docutils literal"><span class="pre">:pserver:anonymous@cvs.sourceforge.net:/cvsroot/buildbot</span></tt>.</dd> <dt><tt class="docutils literal"><span class="pre">cvsmodule</span></tt></dt> <dd>(required): specify the cvs <tt class="docutils literal"><span class="pre">module</span></tt>, which is generally a subdirectory of the CVSROOT. The cvsmodule for the Buildbot source code is <tt class="docutils literal"><span class="pre">buildbot</span></tt>.</dd> <dt><tt class="docutils literal"><span class="pre">branch</span></tt></dt> <dd>a string which will be used in a <tt class="docutils literal"><span class="pre">-r</span></tt> argument. This is most useful for specifying a branch to work on. Defaults to <tt class="docutils literal"><span class="pre">HEAD</span></tt>.</dd> <dt><tt class="docutils literal"><span class="pre">global_options</span></tt></dt> <dd>a list of flags to be put before the argument <tt class="docutils literal"><span class="pre">checkout</span></tt> in the CVS command.</dd> <dt><tt class="docutils literal"><span class="pre">extra_options</span></tt></dt> <dd>a list of flags to be put after the <tt class="docutils literal"><span class="pre">checkout</span></tt> in the CVS command.</dd> </dl> <p><tt class="docutils literal"><span class="pre">mode</span></tt> <tt class="docutils literal"><span class="pre">method</span></tt></p> <blockquote> <div><p>No method is needed for incremental mode. For full mode, <tt class="docutils literal"><span class="pre">method</span></tt> can take the values shown below. If no value is given, it defaults to <tt class="docutils literal"><span class="pre">fresh</span></tt>.</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">clobber</span></tt></dt> <dd>This specifies to remove the <tt class="docutils literal"><span class="pre">workdir</span></tt> and make a full checkout.</dd> <dt><tt class="docutils literal"><span class="pre">fresh</span></tt></dt> <dd>This method first runs <tt class="docutils literal"><span class="pre">cvsdisard</span></tt> in the build directory, then updates it. This requires <tt class="docutils literal"><span class="pre">cvsdiscard</span></tt> which is a part of the cvsutil package.</dd> <dt><tt class="docutils literal"><span class="pre">clean</span></tt></dt> <dd>This method is the same as <tt class="docutils literal"><span class="pre">method='fresh'</span></tt>, but it runs <tt class="docutils literal"><span class="pre">cvsdiscard</span> <span class="pre">--ignore</span></tt> instead of <tt class="docutils literal"><span class="pre">cvsdiscard</span></tt>.</dd> <dt><tt class="docutils literal"><span class="pre">copy</span></tt></dt> <dd>This maintains a <tt class="docutils literal"><span class="pre">source</span></tt> directory for source, which it updates copies to the build directory. This allows Buildbot to start with a fresh directory, without downloading the entire repository on every build.</dd> </dl> </div></blockquote> </div> <div class="section" id="bzr"> <span id="step-bzr"></span><span id="step-Bzr"></span><h3>Bzr<a class="headerlink" href="#bzr" title="Permalink to this headline">¶</a></h3> <dl class="class"> <dt id="buildbot.steps.source.buildbot.steps.source.bzr.Bzr"> <em class="property">class </em><tt class="descclassname">buildbot.steps.source.bzr.</tt><tt class="descname">Bzr</tt><a class="headerlink" href="#buildbot.steps.source.buildbot.steps.source.bzr.Bzr" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <p>bzr is a descendant of Arch/Baz, and is frequently referred to as simply <cite>Bazaar</cite>. The repository-vs-workspace model is similar to Darcs, but it uses a strictly linear sequence of revisions (one history per branch) like Arch. Branches are put in subdirectories. This makes it look very much like Mercurial.</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.source.bzr</span> <span class="kn">import</span> <span class="n">Bzr</span> <span class="n">factory</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">Bzr</span><span class="p">(</span><span class="n">mode</span><span class="o">=</span><span class="s">'incremental'</span><span class="p">,</span> <span class="n">repourl</span><span class="o">=</span><span class="s">'lp:~knielsen/maria/tmp-buildbot-test'</span><span class="p">))</span> </pre></div> </div> <p>The step takes the following arguments:</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">repourl</span></tt></dt> <dd>(required unless <tt class="docutils literal"><span class="pre">baseURL</span></tt> is provided): the URL at which the Bzr source repository is available.</dd> <dt><tt class="docutils literal"><span class="pre">baseURL</span></tt></dt> <dd>(required unless <tt class="docutils literal"><span class="pre">repourl</span></tt> is provided): the base repository URL, to which a branch name will be appended. It should probably end in a slash.</dd> <dt><tt class="docutils literal"><span class="pre">defaultBranch</span></tt></dt> <dd>(allowed if and only if <tt class="docutils literal"><span class="pre">baseURL</span></tt> is provided): this specifies the name of the branch to use when a Build does not provide one of its own. This will be appended to <tt class="docutils literal"><span class="pre">baseURL</span></tt> to create the string that will be passed to the <tt class="docutils literal"><span class="pre">bzr</span> <span class="pre">checkout</span></tt> command.</dd> </dl> <p><tt class="docutils literal"><span class="pre">mode</span></tt> <tt class="docutils literal"><span class="pre">method</span></tt></p> <blockquote> <div><p>No method is needed for incremental mode. For full mode, <tt class="docutils literal"><span class="pre">method</span></tt> can take the values shown below. If no value is given, it defaults to <tt class="docutils literal"><span class="pre">fresh</span></tt>.</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">clobber</span></tt></dt> <dd>This specifies to remove the <tt class="docutils literal"><span class="pre">workdir</span></tt> and make a full checkout.</dd> <dt><tt class="docutils literal"><span class="pre">fresh</span></tt></dt> <dd>This method first runs <tt class="docutils literal"><span class="pre">bzr</span> <span class="pre">clean-tree</span></tt> to remove all the unversioned files then <tt class="docutils literal"><span class="pre">update</span></tt> the repo. This remove all unversioned files including those in .bzrignore.</dd> <dt><tt class="docutils literal"><span class="pre">clean</span></tt></dt> <dd>This is same as fresh except that it doesn't remove the files mentioned in .bzrginore i.e, by running <tt class="docutils literal"><span class="pre">bzr</span> <span class="pre">clean-tree</span> <span class="pre">--ignore</span></tt>.</dd> <dt><tt class="docutils literal"><span class="pre">copy</span></tt></dt> <dd>A local bzr repository is maintained and the repo is copied to <tt class="docutils literal"><span class="pre">build</span></tt> directory for each build. Before each build the local bzr repo is updated then copied to <tt class="docutils literal"><span class="pre">build</span></tt> for next steps.</dd> </dl> </div></blockquote> </div> <div class="section" id="p4"> <span id="step-P4"></span><h3>P4<a class="headerlink" href="#p4" title="Permalink to this headline">¶</a></h3> <dl class="class"> <dt id="buildbot.steps.source.buildbot.steps.source.p4.P4"> <em class="property">class </em><tt class="descclassname">buildbot.steps.source.p4.</tt><tt class="descname">P4</tt><a class="headerlink" href="#buildbot.steps.source.buildbot.steps.source.p4.P4" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <p>The <a class="reference internal" href="#step-P4" title="P4"><tt class="xref bb bb-step docutils literal"><span class="pre">P4</span></tt></a> build step creates a <a class="reference external" href="http://www.perforce.com/">Perforce</a> client specification and performs an update.</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.source.p4</span> <span class="kn">import</span> <span class="n">P4</span> <span class="n">factory</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">P4</span><span class="p">(</span><span class="n">p4port</span><span class="o">=</span><span class="n">p4port</span><span class="p">,</span> <span class="n">p4client</span><span class="o">=</span><span class="n">WithProperties</span><span class="p">(</span><span class="s">'</span><span class="si">%(P4USER)s</span><span class="s">-</span><span class="si">%(slavename)s</span><span class="s">-</span><span class="si">%(buildername)s</span><span class="s">'</span><span class="p">),</span> <span class="n">p4user</span><span class="o">=</span><span class="n">p4user</span><span class="p">,</span> <span class="n">p4base</span><span class="o">=</span><span class="s">'//depot'</span><span class="p">,</span> <span class="n">p4viewspec</span><span class="o">=</span><span class="n">p4viewspec</span><span class="p">,</span> <span class="n">mode</span><span class="o">=</span><span class="s">'incremental'</span><span class="p">,</span> <span class="p">))</span> </pre></div> </div> <p>You can specify the client spec in two different ways. You can use the <tt class="docutils literal"><span class="pre">p4base</span></tt>, <tt class="docutils literal"><span class="pre">p4branch</span></tt>, and (optionally) <tt class="docutils literal"><span class="pre">p4extra_views</span></tt> to build up the viewspec, or you can utilize the <tt class="docutils literal"><span class="pre">p4viewspec</span></tt> to specify the whole viewspec as a set of tuples.</p> <p>Using p4viewspec will allow you to add lines such as:</p> <div class="highlight-python"><pre>//depot/branch/mybranch/... //<p4client>/... -//depot/branch/mybranch/notthisdir/... //<p4client>/notthisdir/...</pre> </div> <p>If you specify <tt class="docutils literal"><span class="pre">p4viewspec</span></tt> and any of <tt class="docutils literal"><span class="pre">p4base</span></tt>, <tt class="docutils literal"><span class="pre">p4branch</span></tt>, and/or <tt class="docutils literal"><span class="pre">p4extra_views</span></tt> you will receive a configuration error exception.</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">p4base</span></tt></dt> <dd>A view into the Perforce depot without branch name or trailing "/...". Typically <tt class="docutils literal"><span class="pre">//depot/proj</span></tt>.</dd> <dt><tt class="docutils literal"><span class="pre">p4branch</span></tt></dt> <dd><dl class="first last docutils"> <dt>(optional): A single string, which is appended to the p4base as follows</dt> <dd><tt class="docutils literal"><span class="pre"><p4base>/<p4branch>/...</span></tt> to form the first line in the viewspec</dd> </dl> </dd> <dt><tt class="docutils literal"><span class="pre">p4extra_views</span></tt></dt> <dd>(optional): a list of <tt class="docutils literal"><span class="pre">(depotpath,</span> <span class="pre">clientpath)</span></tt> tuples containing extra views to be mapped into the client specification. Both will have <tt class="docutils literal"><span class="pre">/...</span></tt> appended automatically. The client name and source directory will be prepended to the client path.</dd> <dt><tt class="docutils literal"><span class="pre">p4viewspec</span></tt></dt> <dd><p class="first">This will override any p4branch, p4base, and/or p4extra_views specified. The viewspec will be an array of tuples as follows <tt class="docutils literal"><span class="pre">[('//depot/main/','')]</span></tt></p> <p>yields a viewspec with just</p> <p class="last"><tt class="docutils literal"><span class="pre">//depot/main/...</span> <span class="pre">//<p4client>/...</span></tt></p> </dd> <dt><tt class="docutils literal"><span class="pre">p4port</span></tt></dt> <dd>(optional): the <tt class="samp docutils literal"><em><span class="pre">host</span></em><span class="pre">:</span><em><span class="pre">port</span></em></tt> string describing how to get to the P4 Depot (repository), used as the <em class="xref std std-option">-p</em> argument for all p4 commands.</dd> <dt><tt class="docutils literal"><span class="pre">p4user</span></tt></dt> <dd>(optional): the Perforce user, used as the <em class="xref std std-option">-u</em> argument to all p4 commands.</dd> <dt><tt class="docutils literal"><span class="pre">p4passwd</span></tt></dt> <dd>(optional): the Perforce password, used as the <em class="xref std std-option">-p</em> argument to all p4 commands.</dd> <dt><tt class="docutils literal"><span class="pre">p4client</span></tt></dt> <dd>(optional): The name of the client to use. In <tt class="docutils literal"><span class="pre">mode='full'</span></tt> and <tt class="docutils literal"><span class="pre">mode='incremental'</span></tt>, it's particularly important that a unique name is used for each checkout directory to avoid incorrect synchronization. For this reason, Python percent substitution will be performed on this value to replace %(slave)s with the slave name and %(builder)s with the builder name. The default is <cite>buildbot_%(slave)s_%(build)s</cite>.</dd> <dt><tt class="docutils literal"><span class="pre">p4line_end</span></tt></dt> <dd>(optional): The type of line ending handling P4 should use. This is added directly to the client spec's <tt class="docutils literal"><span class="pre">LineEnd</span></tt> property. The default is <tt class="docutils literal"><span class="pre">local</span></tt>.</dd> </dl> </div> <div class="section" id="repo"> <span id="step-Repo"></span><h3>Repo<a class="headerlink" href="#repo" title="Permalink to this headline">¶</a></h3> <dl class="class"> <dt id="buildbot.steps.source.buildbot.steps.source.repo.Repo"> <em class="property">class </em><tt class="descclassname">buildbot.steps.source.repo.</tt><tt class="descname">Repo</tt><a class="headerlink" href="#buildbot.steps.source.buildbot.steps.source.repo.Repo" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <p>The <a class="reference internal" href="#step-Repo" title="Repo"><tt class="xref bb bb-step docutils literal"><span class="pre">Repo</span></tt></a> build step performs a <a class="reference external" href="http://lwn.net/Articles/304488/">Repo</a> init and sync.</p> <p>It is a drop-in replacement for <cite>Repo (Slave-Side)</cite>, which should not be used anymore for new and old projects.</p> <p>The Repo step takes the following arguments:</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">manifestURL</span></tt></dt> <dd>(required): the URL at which the Repo's manifests source repository is available.</dd> <dt><tt class="docutils literal"><span class="pre">manifestBranch</span></tt></dt> <dd>(optional, defaults to <tt class="docutils literal"><span class="pre">master</span></tt>): the manifest repository branch on which repo will take its manifest. Corresponds to the <tt class="docutils literal"><span class="pre">-b</span></tt> argument to the <strong class="command">repo init</strong> command.</dd> <dt><tt class="docutils literal"><span class="pre">manifestFile</span></tt></dt> <dd>(optional, defaults to <tt class="docutils literal"><span class="pre">default.xml</span></tt>): the manifest filename. Corresponds to the <tt class="docutils literal"><span class="pre">-m</span></tt> argument to the <strong class="command">repo init</strong> command.</dd> <dt><tt class="docutils literal"><span class="pre">tarball</span></tt></dt> <dd>(optional, defaults to <tt class="docutils literal"><span class="pre">None</span></tt>): the repo tarball used for fast bootstrap. If not present the tarball will be created automatically after first sync. It is a copy of the <tt class="docutils literal"><span class="pre">.repo</span></tt> directory which contains all the Git objects. This feature helps to minimize network usage on very big projects with lots of slaves.</dd> <dt><tt class="docutils literal"><span class="pre">jobs</span></tt></dt> <dd>(optional, defaults to <tt class="docutils literal"><span class="pre">None</span></tt>): Number of projects to fetch simultaneously while syncing. Passed to repo sync subcommand with "-j".</dd> <dt><tt class="docutils literal"><span class="pre">syncAllBranches</span></tt></dt> <dd>(optional, defaults to <tt class="docutils literal"><span class="pre">False</span></tt>): renderable boolean to control whether <tt class="docutils literal"><span class="pre">repo</span></tt> syncs all branches. i.e. <tt class="docutils literal"><span class="pre">repo</span> <span class="pre">sync</span> <span class="pre">-c</span></tt></dd> <dt><tt class="docutils literal"><span class="pre">updateTarballAge</span></tt></dt> <dd>(optional, defaults to "one week"): renderable to control the policy of updating of the tarball given properties Returns: max age of tarball in seconds, or None, if we want to skip tarball update The default value should be good trade off on size of the tarball, and update frequency compared to cost of tarball creation</dd> <dt><tt class="docutils literal"><span class="pre">repoDownloads</span></tt></dt> <dd><p class="first">(optional, defaults to None): list of <tt class="docutils literal"><span class="pre">repo</span> <span class="pre">download</span></tt> commands to perform at the end of the Repo step each string in the list will be prefixed <tt class="docutils literal"><span class="pre">repo</span> <span class="pre">download</span></tt>, and run as is. This means you can include parameter in the string. e.g:</p> <blockquote class="last"> <div><ul class="simple"> <li><tt class="docutils literal"><span class="pre">["-c</span> <span class="pre">project</span> <span class="pre">1234/4"]</span></tt> will cherry-pick patchset 4 of patch 1234 in project <tt class="docutils literal"><span class="pre">project</span></tt></li> <li><tt class="docutils literal"><span class="pre">["-f</span> <span class="pre">project</span> <span class="pre">1234/4"]</span></tt> will enforce fast-forward on patchset 4 of patch 1234 in project <tt class="docutils literal"><span class="pre">project</span></tt></li> </ul> </div></blockquote> </dd> </dl> <dl class="class"> <dt id="buildbot.steps.source.buildbot.steps.source.repo.RepoDownloadsFromProperties"> <em class="property">class </em><tt class="descclassname">buildbot.steps.source.repo.</tt><tt class="descname">RepoDownloadsFromProperties</tt><a class="headerlink" href="#buildbot.steps.source.buildbot.steps.source.repo.RepoDownloadsFromProperties" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <p><tt class="docutils literal"><span class="pre">RepoDownloadsFromProperties</span></tt> can be used as a renderable of the <tt class="docutils literal"><span class="pre">repoDownload</span></tt> parameter it will look in passed properties for string with following possible format:</p> <blockquote> <div><ul class="simple"> <li><tt class="docutils literal"><span class="pre">repo</span> <span class="pre">download</span> <span class="pre">project</span> <span class="pre">change_number/patchset_number</span></tt>.</li> <li><tt class="docutils literal"><span class="pre">project</span> <span class="pre">change_number/patchset_number</span></tt>.</li> <li><tt class="docutils literal"><span class="pre">project/change_number/patchset_number</span></tt>.</li> </ul> </div></blockquote> <p>All of these properties will be translated into a <strong class="command">repo download</strong>. This feature allows integrators to build with several pending interdependent changes, which at the moment cannot be described properly in Gerrit, and can only be described by humans.</p> <dl class="class"> <dt id="buildbot.steps.source.buildbot.steps.source.repo.RepoDownloadsFromChangeSource"> <em class="property">class </em><tt class="descclassname">buildbot.steps.source.repo.</tt><tt class="descname">RepoDownloadsFromChangeSource</tt><a class="headerlink" href="#buildbot.steps.source.buildbot.steps.source.repo.RepoDownloadsFromChangeSource" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <p><tt class="docutils literal"><span class="pre">RepoDownloadsFromChangeSource</span></tt> can be used as a renderable of the <tt class="docutils literal"><span class="pre">repoDownload</span></tt> parameter</p> <p>This rendereable integrates with <a class="reference internal" href="cfg-changesources.html#chsrc-GerritChangeSource" title="GerritChangeSource"><tt class="xref bb bb-chsrc docutils literal"><span class="pre">GerritChangeSource</span></tt></a>, and will automatically use the <strong class="command">repo download</strong> command of repo to download the additionnal changes introduced by a pending changeset.</p> <div class="admonition note"> <p class="first admonition-title">Note</p> <p class="last">you can use the two above Rendereable in conjuction by using the class <tt class="docutils literal"><span class="pre">buildbot.process.properties.FlattenList</span></tt></p> </div> <p>for example:</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.source.repo</span> <span class="kn">import</span> <span class="n">Repo</span><span class="p">,</span> <span class="n">RepoDownloadsFromChangeSource</span><span class="p">,</span> <span class="kn">from</span> <span class="nn">buildbot.steps.source.repo</span> <span class="kn">import</span> <span class="n">RepoDownloadsFromProperties</span> <span class="kn">from</span> <span class="nn">buildbot.process.properties</span> <span class="kn">import</span> <span class="n">FlattenList</span> <span class="n">factory</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">Repo</span><span class="p">(</span><span class="n">manifestUrl</span><span class="o">=</span><span class="s">'git://mygerrit.org/manifest.git'</span><span class="p">,</span> <span class="n">repoDownloads</span><span class="o">=</span><span class="n">FlattenList</span><span class="p">([</span><span class="n">RepoDownloadsFromChangeSource</span><span class="p">(),</span> <span class="n">RepoDownloadsFromProperties</span><span class="p">(</span><span class="s">"repo_downloads"</span><span class="p">)</span> <span class="p">]</span> <span class="p">)</span> <span class="p">))</span> </pre></div> </div> </div> </div> <div class="section" id="source-checkout-slave-side"> <span id="index-10"></span><span id="id8"></span><h2>Source Checkout (Slave-Side)<a class="headerlink" href="#source-checkout-slave-side" title="Permalink to this headline">¶</a></h2> <p>This section describes the more mature slave-side source steps. Where possible, new users should use the master-side source checkout steps, as the slave-side steps will be removed in a future version. See <a class="reference internal" href="#source-checkout"><em>Source Checkout</em></a>.</p> <p>The first step of any build is typically to acquire the source code from which the build will be performed. There are several classes to handle this, one for each of the different source control system that Buildbot knows about. For a description of how Buildbot treats source control in general, see <a class="reference internal" href="concepts.html#version-control-systems"><em>Version Control Systems</em></a>.</p> <p>All source checkout steps accept some common parameters to control how they get the sources and where they should be placed. The remaining per-VC-system parameters are mostly to specify where exactly the sources are coming from.</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">mode</span></tt></dt> <dd><p class="first">a string describing the kind of VC operation that is desired. Defaults to <tt class="docutils literal"><span class="pre">update</span></tt>.</p> <dl class="last docutils"> <dt><tt class="docutils literal"><span class="pre">update</span></tt></dt> <dd>specifies that the CVS checkout/update should be performed directly into the workdir. Each build is performed in the same directory, allowing for incremental builds. This minimizes disk space, bandwidth, and CPU time. However, it may encounter problems if the build process does not handle dependencies properly (sometimes you must do a <em>clean build</em> to make sure everything gets compiled), or if source files are deleted but generated files can influence test behavior (e.g. Python's .pyc files), or when source directories are deleted but generated files prevent CVS from removing them. Builds ought to be correct regardless of whether they are done <em>from scratch</em> or incrementally, but it is useful to test both kinds: this mode exercises the incremental-build style.</dd> <dt><tt class="docutils literal"><span class="pre">copy</span></tt></dt> <dd>specifies that the CVS workspace should be maintained in a separate directory (called the <tt class="file docutils literal"><span class="pre">copydir</span></tt>), using checkout or update as necessary. For each build, a new workdir is created with a copy of the source tree (<tt class="docutils literal"><span class="pre">rm</span> <span class="pre">-rf</span> <span class="pre">workdir;</span> <span class="pre">cp</span> <span class="pre">-r</span> <span class="pre">copydir</span> <span class="pre">workdir</span></tt>). This doubles the disk space required, but keeps the bandwidth low (update instead of a full checkout). A full 'clean' build is performed each time. This avoids any generated-file build problems, but is still occasionally vulnerable to CVS problems such as a repository being manually rearranged, causing CVS errors on update which are not an issue with a full checkout.</dd> <dt><tt class="docutils literal"><span class="pre">clobber</span></tt></dt> <dd>specifies that the working directory should be deleted each time, necessitating a full checkout for each build. This insures a clean build off a complete checkout, avoiding any of the problems described above. This mode exercises the <em>from-scratch</em> build style.</dd> <dt><tt class="docutils literal"><span class="pre">export</span></tt></dt> <dd>this is like <tt class="docutils literal"><span class="pre">clobber</span></tt>, except that the <tt class="docutils literal"><span class="pre">cvs</span> <span class="pre">export</span></tt> command is used to create the working directory. This command removes all CVS metadata files (the <tt class="file docutils literal"><span class="pre">CVS/</span></tt> directories) from the tree, which is sometimes useful for creating source tarballs (to avoid including the metadata in the tar file).</dd> </dl> </dd> <dt><tt class="docutils literal"><span class="pre">workdir</span></tt></dt> <dd>As for all steps, this indicates the directory where the build will take place. Source Steps are special in that they perform some operations outside of the workdir (like creating the workdir itself).</dd> <dt><tt class="docutils literal"><span class="pre">alwaysUseLatest</span></tt></dt> <dd>if <tt class="docutils literal"><span class="pre">True</span></tt>, bypass the usual <cite>update to the last Change</cite> behavior, and always update to the latest changes instead.</dd> <dt><tt class="docutils literal"><span class="pre">retry</span></tt></dt> <dd>If set, this specifies a tuple of <tt class="docutils literal"><span class="pre">(delay,</span> <span class="pre">repeats)</span></tt> which means that when a full VC checkout fails, it should be retried up to <cite>repeats</cite> times, waiting <cite>delay</cite> seconds between attempts. If you don't provide this, it defaults to <tt class="docutils literal"><span class="pre">None</span></tt>, which means VC operations should not be retried. This is provided to make life easier for buildslaves which are stuck behind poor network connections.</dd> <dt><tt class="docutils literal"><span class="pre">repository</span></tt></dt> <dd><p class="first">The name of this parameter might varies depending on the Source step you are running. The concept explained here is common to all steps and applies to <tt class="docutils literal"><span class="pre">repourl</span></tt> as well as for <tt class="docutils literal"><span class="pre">baseURL</span></tt> (when applicable). Buildbot, now being aware of the repository name via the change source, might in some cases not need the repository url. There are multiple way to pass it through to this step, those correspond to the type of the parameter given to this step:</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">None</span></tt></dt> <dd>In the case where no parameter is specified, the repository url will be taken exactly from the Change attribute. You are looking for that one if your ChangeSource step has all information about how to reach the Change.</dd> <dt>string</dt> <dd>The parameter might be a string, in this case, this string will be taken as the repository url, and nothing more. the value coming from the ChangeSource step will be forgotten.</dd> <dt>format string</dt> <dd>If the parameter is a string containing <tt class="docutils literal"><span class="pre">%s</span></tt>, then this the repository attribute from the <tt class="xref py py-class docutils literal"><span class="pre">Change</span></tt> will be place in place of the <tt class="docutils literal"><span class="pre">%s</span></tt>. This is useful when the change source knows where the repository resides locally, but don't know the scheme used to access it. For instance <tt class="docutils literal"><span class="pre">ssh://server/%s</span></tt> makes sense if the the repository attribute is the local path of the repository.</dd> <dt>dict</dt> <dd>In this case, the repository URL will be the value indexed by the repository attribute in the dict given as parameter.</dd> <dt>callable</dt> <dd>The callable given as parameter will take the repository attribute from the Change and its return value will be used as repository URL.</dd> </dl> <div class="last admonition note"> <p class="first admonition-title">Note</p> <p class="last">this is quite similar to the mechanism used by the WebStatus for the <tt class="docutils literal"><span class="pre">changecommentlink</span></tt>, <tt class="docutils literal"><span class="pre">projects</span></tt> or <tt class="docutils literal"><span class="pre">repositories</span></tt> parameter.</p> </div> </dd> <dt><tt class="docutils literal"><span class="pre">timeout</span></tt></dt> <dd>Specifies the timeout for slave-side operations, in seconds. If your repositories are particularly large, then you may need to increase this value from its default of 1200 (20 minutes).</dd> </dl> <p>My habit as a developer is to do a <tt class="docutils literal"><span class="pre">cvs</span> <span class="pre">update</span></tt> and <strong class="command">make</strong> each morning. Problems can occur, either because of bad code being checked in, or by incomplete dependencies causing a partial rebuild to fail where a complete from-scratch build might succeed. A quick Builder which emulates this incremental-build behavior would use the <tt class="docutils literal"><span class="pre">mode='update'</span></tt> setting.</p> <p>On the other hand, other kinds of dependency problems can cause a clean build to fail where a partial build might succeed. This frequently results from a link step that depends upon an object file that was removed from a later version of the tree: in the partial tree, the object file is still around (even though the Makefiles no longer know how to create it).</p> <p><cite>official</cite> builds (traceable builds performed from a known set of source revisions) are always done as clean builds, to make sure it is not influenced by any uncontrolled factors (like leftover files from a previous build). A <cite>full</cite> <tt class="xref py py-class docutils literal"><span class="pre">Builder</span></tt> which behaves this way would want to use the <tt class="docutils literal"><span class="pre">mode='clobber'</span></tt> setting.</p> <p>Each VC system has a corresponding source checkout class: their arguments are described on the following pages.</p> <div class="section" id="cvs-slave-side"> <span id="step-cvs-slave-side"></span><span id="step-CVS (Slave-Side)"></span><h3>CVS (Slave-Side)<a class="headerlink" href="#cvs-slave-side" title="Permalink to this headline">¶</a></h3> <p>The <tt class="xref py py-class docutils literal"><span class="pre">CVS</span></tt> build step performs a <a class="reference external" href="http://www.nongnu.org/cvs/">CVS</a> checkout or update. It takes the following arguments:</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">cvsroot</span></tt></dt> <dd>(required): specify the CVSROOT value, which points to a CVS repository, probably on a remote machine. For example, the cvsroot value you would use to get a copy of the Buildbot source code is <tt class="docutils literal"><span class="pre">:pserver:anonymous@cvs.sourceforge.net:/cvsroot/buildbot</span></tt></dd> <dt><tt class="docutils literal"><span class="pre">cvsmodule</span></tt></dt> <dd>(required): specify the cvs <tt class="docutils literal"><span class="pre">module</span></tt>, which is generally a subdirectory of the CVSROOT. The <cite>cvsmodule</cite> for the Buildbot source code is <tt class="docutils literal"><span class="pre">buildbot</span></tt>.</dd> <dt><tt class="docutils literal"><span class="pre">branch</span></tt></dt> <dd>a string which will be used in a <em class="xref std std-option">-r</em> argument. This is most useful for specifying a branch to work on. Defaults to <tt class="docutils literal"><span class="pre">HEAD</span></tt>.</dd> <dt><tt class="docutils literal"><span class="pre">global_options</span></tt></dt> <dd>a list of flags to be put before the verb in the CVS command.</dd> </dl> <p><tt class="docutils literal"><span class="pre">checkout_options</span></tt></p> <p><tt class="docutils literal"><span class="pre">export_options</span></tt></p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">extra_options</span></tt></dt> <dd>a list of flags to be put after the verb in the CVS command. <tt class="docutils literal"><span class="pre">checkout_options</span></tt> is only used for checkout operations, <tt class="docutils literal"><span class="pre">export_options</span></tt> is only used for export operations, and <tt class="docutils literal"><span class="pre">extra_options</span></tt> is used for both.</dd> <dt><tt class="docutils literal"><span class="pre">checkoutDelay</span></tt></dt> <dd>if set, the number of seconds to put between the timestamp of the last known Change and the value used for the <em class="xref std std-option">-D</em> option. Defaults to half of the parent <tt class="xref py py-class docutils literal"><span class="pre">Build</span></tt>'s <tt class="docutils literal"><span class="pre">treeStableTimer</span></tt>.</dd> </dl> </div> <div class="section" id="svn-slave-side"> <span id="step-svn-slave-side"></span><span id="step-SVN (Slave-Side)"></span><h3>SVN (Slave-Side)<a class="headerlink" href="#svn-slave-side" title="Permalink to this headline">¶</a></h3> <p>The <a class="reference internal" href="#step-SVN (Slave-Side)" title="SVN (Slave-Side)"><tt class="xref bb bb-step docutils literal"><span class="pre">SVN</span></tt></a> build step performs a <a class="reference external" href="http://subversion.tigris.org">Subversion</a> checkout or update. There are two basic ways of setting up the checkout step, depending upon whether you are using multiple branches or not.</p> <p>The most versatile way to create the <tt class="docutils literal"><span class="pre">SVN</span></tt> step is with the <tt class="docutils literal"><span class="pre">svnurl</span></tt> argument:</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">svnurl</span></tt></dt> <dd>(required): this specifies the <tt class="docutils literal"><span class="pre">URL</span></tt> argument that will be given to the <tt class="docutils literal"><span class="pre">svn</span> <span class="pre">checkout</span></tt> command. It dictates both where the repository is located and which sub-tree should be extracted. In this respect, it is like a combination of the CVS <tt class="docutils literal"><span class="pre">cvsroot</span></tt> and <tt class="docutils literal"><span class="pre">cvsmodule</span></tt> arguments. For example, if you are using a remote Subversion repository which is accessible through HTTP at a URL of <tt class="docutils literal"><span class="pre">http://svn.example.com/repos</span></tt>, and you wanted to check out the <tt class="docutils literal"><span class="pre">trunk/calc</span></tt> sub-tree, you would use <tt class="docutils literal"><span class="pre">svnurl="http://svn.example.com/repos/trunk/calc"</span></tt> as an argument to your <tt class="docutils literal"><span class="pre">SVN</span></tt> step.</dd> </dl> <p>The <tt class="docutils literal"><span class="pre">svnurl</span></tt> argument can be considered as a universal means to create the <tt class="docutils literal"><span class="pre">SVN</span></tt> step as it ignores the branch information in the SourceStamp.</p> <p>Alternatively, if you are building from multiple branches, then you should preferentially create the <tt class="docutils literal"><span class="pre">SVN</span></tt> step with the <tt class="docutils literal"><span class="pre">baseURL</span></tt> and <tt class="docutils literal"><span class="pre">defaultBranch</span></tt> arguments instead:</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">baseURL</span></tt></dt> <dd>(required): this specifies the base repository URL, to which a branch name will be appended. It should probably end in a slash.</dd> <dt><tt class="docutils literal"><span class="pre">defaultBranch</span></tt></dt> <dd><p class="first">(optional): this specifies the name of the branch to use when a Build does not provide one of its own. This will be appended to <tt class="docutils literal"><span class="pre">baseURL</span></tt> to create the string that will be passed to the <tt class="docutils literal"><span class="pre">svn</span> <span class="pre">checkout</span></tt> command.</p> <p class="last">It is possible to mix to have a mix of <tt class="docutils literal"><span class="pre">SVN</span></tt> steps that use either the <tt class="docutils literal"><span class="pre">svnurl</span></tt> or <tt class="docutils literal"><span class="pre">baseURL</span></tt> arguments but not both at the same time.</p> </dd> <dt><tt class="docutils literal"><span class="pre">username</span></tt></dt> <dd>(optional): if specified, this will be passed to the <strong class="command">svn</strong> binary with a <em class="xref std std-option">--username</em> option.</dd> <dt><tt class="docutils literal"><span class="pre">password</span></tt></dt> <dd>(optional): if specified, this will be passed to the <tt class="docutils literal"><span class="pre">svn</span></tt> binary with a <em class="xref std std-option">--password</em> option. The password itself will be suitably obfuscated in the logs.</dd> <dt><tt class="docutils literal"><span class="pre">extra_args</span></tt></dt> <dd>(optional): if specified, an array of strings that will be passed as extra arguments to the <strong class="command">svn</strong> binary.</dd> <dt><tt class="docutils literal"><span class="pre">keep_on_purge</span></tt></dt> <dd>(optional): specific files or directories to keep between purges, like some build outputs that can be reused between builds.</dd> <dt><tt class="docutils literal"><span class="pre">ignore_ignores</span></tt></dt> <dd>(optional): when purging changes, don't use rules defined in <tt class="docutils literal"><span class="pre">svn:ignore</span></tt> properties and global-ignores in subversion/config.</dd> <dt><tt class="docutils literal"><span class="pre">always_purge</span></tt></dt> <dd>(optional): if set to <tt class="docutils literal"><span class="pre">True</span></tt>, always purge local changes before updating. This deletes unversioned files and reverts everything that would appear in a <tt class="docutils literal"><span class="pre">svn</span> <span class="pre">status</span></tt>.</dd> <dt><tt class="docutils literal"><span class="pre">depth</span></tt></dt> <dd><p class="first">(optional): Specify depth argument to achieve sparse checkout. Only available if slave has Subversion 1.5 or higher.</p> <p class="last">If set to "empty" updates will not pull in any files or subdirectories not already present. If set to "files", updates will pull in any files not already present, but not directories. If set to "immediates", updates will pull in any files or subdirectories not already present, the new subdirectories will have depth: empty. If set to "infinity", updates will pull in any files or subdirectories not already present; the new subdirectories will have depth-infinity. Infinity is equivalent to SVN default update behavior, without specifying any depth argument.</p> </dd> </dl> <p>If you are using branches, you must also make sure your <tt class="xref py py-class docutils literal"><span class="pre">ChangeSource</span></tt> will report the correct branch names.</p> </div> <div class="section" id="darcs-slave-side"> <span id="step-Darcs (Slave-Side)"></span><h3>Darcs (Slave-Side)<a class="headerlink" href="#darcs-slave-side" title="Permalink to this headline">¶</a></h3> <p>The <a class="reference internal" href="#step-Darcs (Slave-Side)" title="Darcs (Slave-Side)"><tt class="xref bb bb-step docutils literal"><span class="pre">Darcs</span></tt></a> build step performs a <a class="reference external" href="http://darcs.net/">Darcs</a> checkout or update.</p> <p>Like <a class="reference internal" href="#step-SVN (Slave-Side)" title="SVN (Slave-Side)"><tt class="xref bb bb-step docutils literal"><span class="pre">SVN</span></tt></a>, this step can either be configured to always check out a specific tree, or set up to pull from a particular branch that gets specified separately for each build. Also like SVN, the repository URL given to Darcs is created by concatenating a <tt class="docutils literal"><span class="pre">baseURL</span></tt> with the branch name, and if no particular branch is requested, it uses a <tt class="docutils literal"><span class="pre">defaultBranch</span></tt>. The only difference in usage is that each potential Darcs repository URL must point to a fully-fledged repository, whereas SVN URLs usually point to sub-trees of the main Subversion repository. In other words, doing an SVN checkout of <tt class="docutils literal"><span class="pre">baseURL</span></tt> is legal, but silly, since you'd probably wind up with a copy of every single branch in the whole repository. Doing a Darcs checkout of <tt class="docutils literal"><span class="pre">baseURL</span></tt> is just plain wrong, since the parent directory of a collection of Darcs repositories is not itself a valid repository.</p> <p>The Darcs step takes the following arguments:</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">repourl</span></tt></dt> <dd>(required unless <tt class="docutils literal"><span class="pre">baseURL</span></tt> is provided): the URL at which the Darcs source repository is available.</dd> <dt><tt class="docutils literal"><span class="pre">baseURL</span></tt></dt> <dd>(required unless <tt class="docutils literal"><span class="pre">repourl</span></tt> is provided): the base repository URL, to which a branch name will be appended. It should probably end in a slash.</dd> <dt><tt class="docutils literal"><span class="pre">defaultBranch</span></tt></dt> <dd>(allowed if and only if <tt class="docutils literal"><span class="pre">baseURL</span></tt> is provided): this specifies the name of the branch to use when a Build does not provide one of its own. This will be appended to <tt class="docutils literal"><span class="pre">baseURL</span></tt> to create the string that will be passed to the <tt class="docutils literal"><span class="pre">darcs</span> <span class="pre">get</span></tt> command.</dd> </dl> </div> <div class="section" id="mercurial-slave-side"> <span id="step-Mercurial (Slave-Side)"></span><h3>Mercurial (Slave-Side)<a class="headerlink" href="#mercurial-slave-side" title="Permalink to this headline">¶</a></h3> <p>The <a class="reference internal" href="#step-Mercurial (Slave-Side)" title="Mercurial (Slave-Side)"><tt class="xref bb bb-step docutils literal"><span class="pre">Mercurial</span></tt></a> build step performs a <a class="reference external" href="http://selenic.com/mercurial">Mercurial</a> (aka <cite>hg</cite>) checkout or update.</p> <p>Branches are available in two modes: <cite>dirname</cite> like <a class="reference internal" href="#step-Darcs (Slave-Side)" title="Darcs (Slave-Side)"><tt class="xref bb bb-step docutils literal"><span class="pre">Darcs</span></tt></a>, or <cite>inrepo</cite>, which uses the repository internal branches. Make sure this setting matches your changehook, if you have that installed.</p> <p>The Mercurial step takes the following arguments:</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">repourl</span></tt></dt> <dd>(required unless <tt class="docutils literal"><span class="pre">baseURL</span></tt> is provided): the URL at which the Mercurial source repository is available.</dd> <dt><tt class="docutils literal"><span class="pre">baseURL</span></tt></dt> <dd>(required unless <tt class="docutils literal"><span class="pre">repourl</span></tt> is provided): the base repository URL, to which a branch name will be appended. It should probably end in a slash.</dd> <dt><tt class="docutils literal"><span class="pre">defaultBranch</span></tt></dt> <dd>(allowed if and only if <tt class="docutils literal"><span class="pre">baseURL</span></tt> is provided): this specifies the name of the branch to use when a <tt class="xref py py-class docutils literal"><span class="pre">Build</span></tt> does not provide one of its own. This will be appended to <tt class="docutils literal"><span class="pre">baseURL</span></tt> to create the string that will be passed to the <tt class="docutils literal"><span class="pre">hg</span> <span class="pre">clone</span></tt> command.</dd> <dt><tt class="docutils literal"><span class="pre">branchType</span></tt></dt> <dd>either 'dirname' (default) or 'inrepo' depending on whether the branch name should be appended to the <tt class="docutils literal"><span class="pre">baseURL</span></tt> or the branch is a Mercurial named branch and can be found within the <tt class="docutils literal"><span class="pre">repourl</span></tt>.</dd> <dt><tt class="docutils literal"><span class="pre">clobberOnBranchChange</span></tt></dt> <dd>boolean, defaults to <tt class="docutils literal"><span class="pre">True</span></tt>. If set and using inrepos branches, clobber the tree at each branch change. Otherwise, just update to the branch.</dd> </dl> </div> <div class="section" id="bzr-slave-side"> <span id="step-Bzr (Slave-Side)"></span><h3>Bzr (Slave-Side)<a class="headerlink" href="#bzr-slave-side" title="Permalink to this headline">¶</a></h3> <p>bzr is a descendant of Arch/Baz, and is frequently referred to as simply <cite>Bazaar</cite>. The repository-vs-workspace model is similar to Darcs, but it uses a strictly linear sequence of revisions (one history per branch) like Arch. Branches are put in subdirectories. This makes it look very much like Mercurial. It takes the following arguments:</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">repourl</span></tt></dt> <dd>(required unless <tt class="docutils literal"><span class="pre">baseURL</span></tt> is provided): the URL at which the Bzr source repository is available.</dd> <dt><tt class="docutils literal"><span class="pre">baseURL</span></tt></dt> <dd>(required unless <tt class="docutils literal"><span class="pre">repourl</span></tt> is provided): the base repository URL, to which a branch name will be appended. It should probably end in a slash.</dd> <dt><tt class="docutils literal"><span class="pre">defaultBranch</span></tt></dt> <dd>(allowed if and only if <tt class="docutils literal"><span class="pre">baseURL</span></tt> is provided): this specifies the name of the branch to use when a Build does not provide one of its own. This will be appended to <tt class="docutils literal"><span class="pre">baseURL</span></tt> to create the string that will be passed to the <tt class="docutils literal"><span class="pre">bzr</span> <span class="pre">checkout</span></tt> command.</dd> <dt><tt class="docutils literal"><span class="pre">forceSharedRepo</span></tt></dt> <dd>(boolean, optional, defaults to <tt class="docutils literal"><span class="pre">False</span></tt>): If set to <tt class="docutils literal"><span class="pre">True</span></tt>, the working directory will be made into a bzr shared repository if it is not already. Shared repository greatly reduces the amount of history data that needs to be downloaded if not using update/copy mode, or if using update/copy mode with multiple branches.</dd> </dl> </div> <div class="section" id="p4-slave-side"> <span id="step-P4 (Slave-Side)"></span><h3>P4 (Slave-Side)<a class="headerlink" href="#p4-slave-side" title="Permalink to this headline">¶</a></h3> <p>The <a class="reference internal" href="#step-P4 (Slave-Side)" title="P4 (Slave-Side)"><tt class="xref bb bb-step docutils literal"><span class="pre">P4</span> <span class="pre">(Slave-Side)</span></tt></a> build step creates a <a class="reference external" href="http://www.perforce.com/">Perforce</a> client specification and performs an update.</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">p4base</span></tt></dt> <dd>A view into the Perforce depot without branch name or trailing "...". Typically <tt class="docutils literal"><span class="pre">//depot/proj/</span></tt>.</dd> <dt><tt class="docutils literal"><span class="pre">defaultBranch</span></tt></dt> <dd>A branch name to append on build requests if none is specified. Typically <tt class="docutils literal"><span class="pre">trunk</span></tt>.</dd> <dt><tt class="docutils literal"><span class="pre">p4port</span></tt></dt> <dd>(optional): the <tt class="samp docutils literal"><em><span class="pre">host</span></em><span class="pre">:</span><em><span class="pre">port</span></em></tt> string describing how to get to the P4 Depot (repository), used as the <em class="xref std std-option">-p</em> argument for all p4 commands.</dd> <dt><tt class="docutils literal"><span class="pre">p4user</span></tt></dt> <dd>(optional): the Perforce user, used as the <em class="xref std std-option">-u</em> argument to all p4 commands.</dd> <dt><tt class="docutils literal"><span class="pre">p4passwd</span></tt></dt> <dd>(optional): the Perforce password, used as the <em class="xref std std-option">-p</em> argument to all p4 commands.</dd> <dt><tt class="docutils literal"><span class="pre">p4extra_views</span></tt></dt> <dd>(optional): a list of <tt class="docutils literal"><span class="pre">(depotpath,</span> <span class="pre">clientpath)</span></tt> tuples containing extra views to be mapped into the client specification. Both will have "/..." appended automatically. The client name and source directory will be prepended to the client path.</dd> <dt><tt class="docutils literal"><span class="pre">p4client</span></tt></dt> <dd>(optional): The name of the client to use. In <tt class="docutils literal"><span class="pre">mode='copy'</span></tt> and <tt class="docutils literal"><span class="pre">mode='update'</span></tt>, it's particularly important that a unique name is used for each checkout directory to avoid incorrect synchronization. For this reason, Python percent substitution will be performed on this value to replace %(slave)s with the slave name and %(builder)s with the builder name. The default is <cite>buildbot_%(slave)s_%(build)s</cite>.</dd> <dt><tt class="docutils literal"><span class="pre">p4line_end</span></tt></dt> <dd>(optional): The type of line ending handling P4 should use. This is added directly to the client spec's <tt class="docutils literal"><span class="pre">LineEnd</span></tt> property. The default is <tt class="docutils literal"><span class="pre">local</span></tt>.</dd> </dl> </div> <div class="section" id="git-slave-side"> <span id="step-Git (Slave-Side)"></span><h3>Git (Slave-Side)<a class="headerlink" href="#git-slave-side" title="Permalink to this headline">¶</a></h3> <p>The <a class="reference internal" href="#step-Git (Slave-Side)" title="Git (Slave-Side)"><tt class="xref bb bb-step docutils literal"><span class="pre">Git</span></tt></a> build step clones or updates a <a class="reference external" href="http://git.or.cz/">Git</a> repository and checks out the specified branch or revision. Note that the buildbot supports Git version 1.2.0 and later: earlier versions (such as the one shipped in Ubuntu 'Dapper') do not support the <tt class="docutils literal"><span class="pre">git</span> <span class="pre">init</span></tt> command that the buildbot uses.</p> <p>The <tt class="docutils literal"><span class="pre">Git</span></tt> step takes the following arguments:</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">repourl</span></tt></dt> <dd>(required): the URL of the upstream Git repository.</dd> <dt><tt class="docutils literal"><span class="pre">branch</span></tt></dt> <dd>(optional): this specifies the name of the branch to use when a Build does not provide one of its own. If this this parameter is not specified, and the <tt class="xref py py-class docutils literal"><span class="pre">Build</span></tt> does not provide a branch, the <cite>master</cite> branch will be used.</dd> <dt><tt class="docutils literal"><span class="pre">ignore_ignores</span></tt></dt> <dd>(optional): when purging changes, don't use <tt class="file docutils literal"><span class="pre">.gitignore</span></tt> and <tt class="file docutils literal"><span class="pre">.git/info/exclude</span></tt>.</dd> <dt><tt class="docutils literal"><span class="pre">submodules</span></tt></dt> <dd>(optional): when initializing/updating a Git repository, this decides whether or not buildbot should consider Git submodules. Default: <tt class="docutils literal"><span class="pre">False</span></tt>.</dd> <dt><tt class="docutils literal"><span class="pre">reference</span></tt></dt> <dd>(optional): use the specified string as a path to a reference repository on the local machine. Git will try to grab objects from this path first instead of the main repository, if they exist.</dd> <dt><tt class="docutils literal"><span class="pre">shallow</span></tt></dt> <dd>(optional): instructs Git to attempt shallow clones (<tt class="docutils literal"><span class="pre">--depth</span> <span class="pre">1</span></tt>). If the user/scheduler asks for a specific revision, this parameter is ignored.</dd> <dt><tt class="docutils literal"><span class="pre">progress</span></tt></dt> <dd>(optional): passes the (<tt class="docutils literal"><span class="pre">--progress</span></tt>) flag to (<tt class="docutils literal"><span class="pre">git</span> <span class="pre">fetch</span></tt>). This solves issues of long fetches being killed due to lack of output, but requires Git 1.7.2 or later.</dd> </dl> <p>This Source step integrates with <a class="reference internal" href="cfg-changesources.html#chsrc-GerritChangeSource" title="GerritChangeSource"><tt class="xref bb bb-chsrc docutils literal"><span class="pre">GerritChangeSource</span></tt></a>, and will automatically use Gerrit's "virtual branch" (<tt class="docutils literal"><span class="pre">refs/changes/*</span></tt>) to download the additionnal changes introduced by a pending changeset.</p> <p id="index-11">Gerrit integration can be also triggered using forced build with <tt class="docutils literal"><span class="pre">gerrit_change</span></tt> property with value in format: <tt class="docutils literal"><span class="pre">change_number/patchset_number</span></tt>.</p> </div> <div class="section" id="bitkeeper-slave-side"> <span id="step-BK (Slave-Side)"></span><h3>BitKeeper (Slave-Side)<a class="headerlink" href="#bitkeeper-slave-side" title="Permalink to this headline">¶</a></h3> <p>The <a class="reference internal" href="#step-BK (Slave-Side)" title="BK (Slave-Side)"><tt class="xref bb bb-step docutils literal"><span class="pre">BK</span></tt></a> build step performs a <a class="reference external" href="http://www.bitkeeper.com/">BitKeeper</a> checkout or update.</p> <p>The BitKeeper step takes the following arguments:</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">repourl</span></tt></dt> <dd>(required unless <tt class="docutils literal"><span class="pre">baseURL</span></tt> is provided): the URL at which the BitKeeper source repository is available.</dd> <dt><tt class="docutils literal"><span class="pre">baseURL</span></tt></dt> <dd>(required unless <tt class="docutils literal"><span class="pre">repourl</span></tt> is provided): the base repository URL, to which a branch name will be appended. It should probably end in a slash.</dd> </dl> </div> <div class="section" id="repo-slave-side"> <span id="step-Repo (Slave-Side)"></span><h3>Repo (Slave-Side)<a class="headerlink" href="#repo-slave-side" title="Permalink to this headline">¶</a></h3> <dl class="class"> <dt id="buildbot.steps.source.buildbot.steps.source.Repo"> <em class="property">class </em><tt class="descclassname">buildbot.steps.source.</tt><tt class="descname">Repo</tt><a class="headerlink" href="#buildbot.steps.source.buildbot.steps.source.Repo" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <p>The <a class="reference internal" href="#step-Repo (Slave-Side)" title="Repo (Slave-Side)"><tt class="xref bb bb-step docutils literal"><span class="pre">Repo</span> <span class="pre">(Slave-Side)</span></tt></a> build step performs a <a class="reference external" href="http://lwn.net/Articles/304488/">Repo</a> init and sync.</p> <p>This step is obsolete and should not be used anymore. please use: <cite>Repo</cite> instead</p> <p>The Repo step takes the following arguments:</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">manifest_url</span></tt></dt> <dd>(required): the URL at which the Repo's manifests source repository is available.</dd> <dt><tt class="docutils literal"><span class="pre">manifest_branch</span></tt></dt> <dd>(optional, defaults to <tt class="docutils literal"><span class="pre">master</span></tt>): the manifest repository branch on which repo will take its manifest. Corresponds to the <tt class="docutils literal"><span class="pre">-b</span></tt> argument to the <strong class="command">repo init</strong> command.</dd> <dt><tt class="docutils literal"><span class="pre">manifest_file</span></tt></dt> <dd>(optional, defaults to <tt class="docutils literal"><span class="pre">default.xml</span></tt>): the manifest filename. Corresponds to the <tt class="docutils literal"><span class="pre">-m</span></tt> argument to the <strong class="command">repo init</strong> command.</dd> <dt><tt class="docutils literal"><span class="pre">tarball</span></tt></dt> <dd>(optional, defaults to <tt class="docutils literal"><span class="pre">None</span></tt>): the repo tarball used for fast bootstrap. If not present the tarball will be created automatically after first sync. It is a copy of the <tt class="docutils literal"><span class="pre">.repo</span></tt> directory which contains all the Git objects. This feature helps to minimize network usage on very big projects.</dd> <dt><tt class="docutils literal"><span class="pre">jobs</span></tt></dt> <dd>(optional, defaults to <tt class="docutils literal"><span class="pre">None</span></tt>): Number of projects to fetch simultaneously while syncing. Passed to repo sync subcommand with "-j".</dd> </dl> <p>This Source step integrates with <a class="reference internal" href="cfg-changesources.html#chsrc-GerritChangeSource" title="GerritChangeSource"><tt class="xref bb bb-chsrc docutils literal"><span class="pre">GerritChangeSource</span></tt></a>, and will automatically use the <strong class="command">repo download</strong> command of repo to download the additionnal changes introduced by a pending changeset.</p> <p id="index-12">Gerrit integration can be also triggered using forced build with following properties: <tt class="docutils literal"><span class="pre">repo_d</span></tt>, <tt class="docutils literal"><span class="pre">repo_d[0-9]</span></tt>, <tt class="docutils literal"><span class="pre">repo_download</span></tt>, <tt class="docutils literal"><span class="pre">repo_download[0-9]</span></tt> with values in format: <tt class="docutils literal"><span class="pre">project/change_number/patchset_number</span></tt>. All of these properties will be translated into a <strong class="command">repo download</strong>. This feature allows integrators to build with several pending interdependent changes, which at the moment cannot be described properly in Gerrit, and can only be described by humans.</p> </div> <div class="section" id="monotone-slave-side"> <span id="step-Monotone (Slave-Side)"></span><h3>Monotone (Slave-Side)<a class="headerlink" href="#monotone-slave-side" title="Permalink to this headline">¶</a></h3> <p>The <a class="reference internal" href="#step-Monotone (Slave-Side)" title="Monotone (Slave-Side)"><tt class="xref bb bb-step docutils literal"><span class="pre">Monotone</span></tt></a> build step performs a <a class="reference external" href="http://www.monotone.ca">Monotone</a>, (aka <tt class="docutils literal"><span class="pre">mtn</span></tt>) checkout or update.</p> <p>The Monotone step takes the following arguments:</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">repourl</span></tt></dt> <dd>the URL at which the Monotone source repository is available.</dd> <dt><tt class="docutils literal"><span class="pre">branch</span></tt></dt> <dd>this specifies the name of the branch to use when a Build does not provide one of its own.</dd> <dt><tt class="docutils literal"><span class="pre">progress</span></tt></dt> <dd>this is a boolean that has a pull from the repository use <tt class="docutils literal"><span class="pre">--ticker=dot</span></tt> instead of the default <tt class="docutils literal"><span class="pre">--ticker=none</span></tt>.</dd> </dl> </div> </div> <div class="section" id="shellcommand"> <span id="step-ShellCommand"></span><h2>ShellCommand<a class="headerlink" href="#shellcommand" title="Permalink to this headline">¶</a></h2> <p>Most interesting steps involve executing a process of some sort on the buildslave. The <a class="reference internal" href="#step-ShellCommand" title="ShellCommand"><tt class="xref bb bb-step docutils literal"><span class="pre">ShellCommand</span></tt></a> class handles this activity.</p> <p>Several subclasses of <a class="reference internal" href="#step-ShellCommand" title="ShellCommand"><tt class="xref bb bb-step docutils literal"><span class="pre">ShellCommand</span></tt></a> are provided as starting points for common build steps.</p> <div class="section" id="using-shellcommands"> <h3>Using ShellCommands<a class="headerlink" href="#using-shellcommands" title="Permalink to this headline">¶</a></h3> <dl class="class"> <dt id="buildbot.steps.source.buildbot.steps.shell.ShellCommand"> <em class="property">class </em><tt class="descclassname">buildbot.steps.shell.</tt><tt class="descname">ShellCommand</tt><a class="headerlink" href="#buildbot.steps.source.buildbot.steps.shell.ShellCommand" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <p>This is a useful base class for just about everything you might want to do during a build (except for the initial source checkout). It runs a single command in a child shell on the buildslave. All stdout/stderr is recorded into a <tt class="xref py py-class docutils literal"><span class="pre">LogFile</span></tt>. The step usually finishes with a status of <tt class="docutils literal"><span class="pre">FAILURE</span></tt> if the command's exit code is non-zero, otherwise it has a status of <tt class="docutils literal"><span class="pre">SUCCESS</span></tt>.</p> <p>The preferred way to specify the command is with a list of argv strings, since this allows for spaces in filenames and avoids doing any fragile shell-escaping. You can also specify the command with a single string, in which case the string is given to <tt class="samp docutils literal"><span class="pre">/bin/sh</span> <span class="pre">-c</span> <em><span class="pre">COMMAND</span></em></tt> for parsing.</p> <p>On Windows, commands are run via <tt class="docutils literal"><span class="pre">cmd.exe</span> <span class="pre">/c</span></tt> which works well. However, if you're running a batch file, the error level does not get propagated correctly unless you add 'call' before your batch file's name: <tt class="docutils literal"><span class="pre">cmd=['call',</span> <span class="pre">'myfile.bat',</span> <span class="pre">...]</span></tt>.</p> <p>The <a class="reference internal" href="#step-ShellCommand" title="ShellCommand"><tt class="xref bb bb-step docutils literal"><span class="pre">ShellCommand</span></tt></a> arguments are:</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">command</span></tt></dt> <dd><p class="first">a list of strings (preferred) or single string (discouraged) which specifies the command to be run. A list of strings is preferred because it can be used directly as an argv array. Using a single string (with embedded spaces) requires the buildslave to pass the string to <strong class="command">/bin/sh</strong> for interpretation, which raises all sorts of difficult questions about how to escape or interpret shell metacharacters.</p> <p>If <tt class="docutils literal"><span class="pre">command</span></tt> contains nested lists (for example, from a properties substitution), then that list will be flattened before it is executed.</p> <p class="last">On the topic of shell metacharacters, note that in DOS the pipe character (<tt class="docutils literal"><span class="pre">|</span></tt>) is conditionally escaped (to <tt class="docutils literal"><span class="pre">^|</span></tt>) when it occurs inside a more complex string in a list of strings. It remains unescaped when it occurs as part of a single string or as a lone pipe in a list of strings.</p> </dd> <dt><tt class="docutils literal"><span class="pre">workdir</span></tt></dt> <dd><p class="first">All ShellCommands are run by default in the <tt class="docutils literal"><span class="pre">workdir</span></tt>, which defaults to the <tt class="file docutils literal"><span class="pre">build</span></tt> subdirectory of the slave builder's base directory. The absolute path of the workdir will thus be the slave's basedir (set as an option to <tt class="docutils literal"><span class="pre">buildslave</span> <span class="pre">create-slave</span></tt>, <a class="reference internal" href="installation.html#creating-a-buildslave"><em>Creating a buildslave</em></a>) plus the builder's basedir (set in the builder's <tt class="docutils literal"><span class="pre">builddir</span></tt> key in <tt class="file docutils literal"><span class="pre">master.cfg</span></tt>) plus the workdir itself (a class-level attribute of the BuildFactory, defaults to <tt class="file docutils literal"><span class="pre">build</span></tt>).</p> <p>For example:</p> <div class="last highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.shell</span> <span class="kn">import</span> <span class="n">ShellCommand</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">ShellCommand</span><span class="p">(</span><span class="n">command</span><span class="o">=</span><span class="p">[</span><span class="s">"make"</span><span class="p">,</span> <span class="s">"test"</span><span class="p">],</span> <span class="n">workdir</span><span class="o">=</span><span class="s">"build/tests"</span><span class="p">))</span> </pre></div> </div> </dd> <dt><tt class="docutils literal"><span class="pre">env</span></tt></dt> <dd><p class="first">a dictionary of environment strings which will be added to the child command's environment. For example, to run tests with a different i18n language setting, you might use</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.shell</span> <span class="kn">import</span> <span class="n">ShellCommand</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">ShellCommand</span><span class="p">(</span><span class="n">command</span><span class="o">=</span><span class="p">[</span><span class="s">"make"</span><span class="p">,</span> <span class="s">"test"</span><span class="p">],</span> <span class="n">env</span><span class="o">=</span><span class="p">{</span><span class="s">'LANG'</span><span class="p">:</span> <span class="s">'fr_FR'</span><span class="p">}))</span> </pre></div> </div> <p>These variable settings will override any existing ones in the buildslave's environment or the environment specified in the <tt class="xref py py-class docutils literal"><span class="pre">Builder</span></tt>. The exception is <span class="target" id="index-13"></span><tt class="xref std std-envvar docutils literal"><span class="pre">PYTHONPATH</span></tt>, which is merged with (actually prepended to) any existing <span class="target" id="index-14"></span><tt class="xref std std-envvar docutils literal"><span class="pre">PYTHONPATH</span></tt> setting. The following example will prepend <tt class="file docutils literal"><span class="pre">/home/buildbot/lib/python</span></tt> to any existing <span class="target" id="index-15"></span><tt class="xref std std-envvar docutils literal"><span class="pre">PYTHONPATH</span></tt>:</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.shell</span> <span class="kn">import</span> <span class="n">ShellCommand</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">ShellCommand</span><span class="p">(</span> <span class="n">command</span><span class="o">=</span><span class="p">[</span><span class="s">"make"</span><span class="p">,</span> <span class="s">"test"</span><span class="p">],</span> <span class="n">env</span><span class="o">=</span><span class="p">{</span><span class="s">'PYTHONPATH'</span><span class="p">:</span> <span class="s">"/home/buildbot/lib/python"</span><span class="p">}))</span> </pre></div> </div> <p>To avoid the need of concatenating path together in the master config file, if the value is a list, it will be joined together using the right platform dependant separator.</p> <p>Those variables support expansion so that if you just want to prepend <tt class="file docutils literal"><span class="pre">/home/buildbot/bin</span></tt> to the <span class="target" id="index-16"></span><tt class="xref std std-envvar docutils literal"><span class="pre">PATH</span></tt> environment variable, you can do it by putting the value <tt class="docutils literal"><span class="pre">${PATH}</span></tt> at the end of the value like in the example below. Variables that don't exist on the slave will be replaced by <tt class="docutils literal"><span class="pre">""</span></tt>.</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.shell</span> <span class="kn">import</span> <span class="n">ShellCommand</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">ShellCommand</span><span class="p">(</span> <span class="n">command</span><span class="o">=</span><span class="p">[</span><span class="s">"make"</span><span class="p">,</span> <span class="s">"test"</span><span class="p">],</span> <span class="n">env</span><span class="o">=</span><span class="p">{</span><span class="s">'PATH'</span><span class="p">:</span> <span class="p">[</span><span class="s">"/home/buildbot/bin"</span><span class="p">,</span> <span class="s">"${PATH}"</span><span class="p">]}))</span> </pre></div> </div> <p class="last">Note that environment values must be strings (or lists that are turned into strings). In particular, numeric properties such as <tt class="docutils literal"><span class="pre">buildnumber</span></tt> must be substituted using <a class="reference internal" href="cfg-properties.html#interpolate"><em>Interpolate</em></a>.</p> </dd> <dt><tt class="docutils literal"><span class="pre">want_stdout</span></tt></dt> <dd>if <tt class="docutils literal"><span class="pre">False</span></tt>, stdout from the child process is discarded rather than being sent to the buildmaster for inclusion in the step's <tt class="xref py py-class docutils literal"><span class="pre">LogFile</span></tt>.</dd> <dt><tt class="docutils literal"><span class="pre">want_stderr</span></tt></dt> <dd>like <tt class="docutils literal"><span class="pre">want_stdout</span></tt> but for <tt class="file docutils literal"><span class="pre">stderr</span></tt>. Note that commands run through a PTY do not have separate <tt class="file docutils literal"><span class="pre">stdout</span></tt>/<tt class="file docutils literal"><span class="pre">stderr</span></tt> streams: both are merged into <tt class="file docutils literal"><span class="pre">stdout</span></tt>.</dd> <dt><tt class="docutils literal"><span class="pre">usePTY</span></tt></dt> <dd><p class="first">Should this command be run in a <tt class="docutils literal"><span class="pre">pty</span></tt>? The default is to observe the configuration of the client (<a class="reference internal" href="installation.html#buildslave-options"><em>Buildslave Options</em></a>), but specifying <tt class="docutils literal"><span class="pre">True</span></tt> or <tt class="docutils literal"><span class="pre">False</span></tt> here will override the default. This option is not available on Windows.</p> <p>In general, you do not want to use a pseudo-terminal. This is is <em>only</em> useful for running commands that require a terminal - for example, testing a command-line application that will only accept passwords read from a terminal. Using a pseudo-terminal brings lots of compatibility problems, and prevents Buildbot from distinguishing the standard error (red) and standard output (black) streams.</p> <p>In previous versions, the advantage of using a pseudo-terminal was that <tt class="docutils literal"><span class="pre">grandchild</span></tt> processes were more likely to be cleaned up if the build was interrupted or times out. This occurred because using a pseudo-terminal incidentally puts the command into its own process group.</p> <p class="last">As of Buildbot-0.8.4, all commands are placed in process groups, and thus grandchild processes will be cleaned up properly.</p> </dd> <dt><tt class="docutils literal"><span class="pre">logfiles</span></tt></dt> <dd><p class="first">Sometimes commands will log interesting data to a local file, rather than emitting everything to stdout or stderr. For example, Twisted's <strong class="command">trial</strong> command (which runs unit tests) only presents summary information to stdout, and puts the rest into a file named <tt class="file docutils literal"><span class="pre">_trial_temp/test.log</span></tt>. It is often useful to watch these files as the command runs, rather than using <strong class="command">/bin/cat</strong> to dump their contents afterwards.</p> <p>The <tt class="docutils literal"><span class="pre">logfiles=</span></tt> argument allows you to collect data from these secondary logfiles in near-real-time, as the step is running. It accepts a dictionary which maps from a local Log name (which is how the log data is presented in the build results) to either a remote filename (interpreted relative to the build's working directory), or a dictionary of options. Each named file will be polled on a regular basis (every couple of seconds) as the build runs, and any new text will be sent over to the buildmaster.</p> <p>If you provide a dictionary of options instead of a string, you must specify the <tt class="docutils literal"><span class="pre">filename</span></tt> key. You can optionally provide a <tt class="docutils literal"><span class="pre">follow</span></tt> key which is a boolean controlling whether a logfile is followed or concatenated in its entirety. Following is appropriate for logfiles to which the build step will append, where the pre-existing contents are not interesting. The default value for <tt class="docutils literal"><span class="pre">follow</span></tt> is <tt class="docutils literal"><span class="pre">False</span></tt>, which gives the same behavior as just providing a string filename.</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.shell</span> <span class="kn">import</span> <span class="n">ShellCommand</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">ShellCommand</span><span class="p">(</span> <span class="n">command</span><span class="o">=</span><span class="p">[</span><span class="s">"make"</span><span class="p">,</span> <span class="s">"test"</span><span class="p">],</span> <span class="n">logfiles</span><span class="o">=</span><span class="p">{</span><span class="s">"triallog"</span><span class="p">:</span> <span class="s">"_trial_temp/test.log"</span><span class="p">}))</span> </pre></div> </div> <p>The above example will add a log named 'triallog' on the master, based on <tt class="file docutils literal"><span class="pre">_trial_temp/test.log</span></tt> on the slave.</p> <div class="last highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.shell</span> <span class="kn">import</span> <span class="n">ShellCommand</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">ShellCommand</span><span class="p">(</span> <span class="n">command</span><span class="o">=</span><span class="p">[</span><span class="s">"make"</span><span class="p">,</span> <span class="s">"test"</span><span class="p">],</span> <span class="n">logfiles</span><span class="o">=</span><span class="p">{</span><span class="s">"triallog"</span><span class="p">:</span> <span class="p">{</span><span class="s">"filename"</span><span class="p">:</span> <span class="s">"_trial_temp/test.log"</span><span class="p">,</span> <span class="s">"follow"</span><span class="p">:</span> <span class="bp">True</span><span class="p">,}}))</span> </pre></div> </div> </dd> <dt><tt class="docutils literal"><span class="pre">lazylogfiles</span></tt></dt> <dd>If set to <tt class="docutils literal"><span class="pre">True</span></tt>, logfiles will be tracked lazily, meaning that they will only be added when and if something is written to them. This can be used to suppress the display of empty or missing log files. The default is <tt class="docutils literal"><span class="pre">False</span></tt>.</dd> <dt><tt class="docutils literal"><span class="pre">timeout</span></tt></dt> <dd>if the command fails to produce any output for this many seconds, it is assumed to be locked up and will be killed. This defaults to 1200 seconds. Pass <tt class="docutils literal"><span class="pre">None</span></tt> to disable.</dd> <dt><tt class="docutils literal"><span class="pre">maxTime</span></tt></dt> <dd>if the command takes longer than this many seconds, it will be killed. This is disabled by default.</dd> <dt><tt class="docutils literal"><span class="pre">description</span></tt></dt> <dd>This will be used to describe the command (on the Waterfall display) while the command is still running. It should be a single imperfect-tense verb, like <cite>compiling</cite> or <cite>testing</cite>. The preferred form is a list of short strings, which allows the HTML displays to create narrower columns by emitting a <br> tag between each word. You may also provide a single string.</dd> <dt><tt class="docutils literal"><span class="pre">descriptionDone</span></tt></dt> <dd><p class="first">This will be used to describe the command once it has finished. A simple noun like <cite>compile</cite> or <cite>tests</cite> should be used. Like <tt class="docutils literal"><span class="pre">description</span></tt>, this may either be a list of short strings or a single string.</p> <p>If neither <tt class="docutils literal"><span class="pre">description</span></tt> nor <tt class="docutils literal"><span class="pre">descriptionDone</span></tt> are set, the actual command arguments will be used to construct the description. This may be a bit too wide to fit comfortably on the Waterfall display.</p> <div class="last highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.shell</span> <span class="kn">import</span> <span class="n">ShellCommand</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">ShellCommand</span><span class="p">(</span><span class="n">command</span><span class="o">=</span><span class="p">[</span><span class="s">"make"</span><span class="p">,</span> <span class="s">"test"</span><span class="p">],</span> <span class="n">description</span><span class="o">=</span><span class="p">[</span><span class="s">"testing"</span><span class="p">],</span> <span class="n">descriptionDone</span><span class="o">=</span><span class="p">[</span><span class="s">"tests"</span><span class="p">]))</span> </pre></div> </div> </dd> <dt><tt class="docutils literal"><span class="pre">descriptionSuffix</span></tt></dt> <dd><p class="first">This is an optional suffix appended to the end of the description (ie, after <tt class="docutils literal"><span class="pre">description</span></tt> and <tt class="docutils literal"><span class="pre">descriptionDone</span></tt>). This can be used to distinguish between build steps that would display the same descriptions in the waterfall. This parameter may be set to list of short strings, a single string, or <tt class="docutils literal"><span class="pre">None</span></tt>.</p> <p class="last">For example, a builder might use the <tt class="docutils literal"><span class="pre">Compile</span></tt> step to build two different codebases. The <tt class="docutils literal"><span class="pre">descriptionSuffix</span></tt> could be set to <cite>projectFoo</cite> and <cite>projectBar</cite>, respectively for each step, which will result in the full descriptions <cite>compiling projectFoo</cite> and <cite>compiling projectBar</cite> to be shown in the waterfall.</p> </dd> <dt><tt class="docutils literal"><span class="pre">logEnviron</span></tt></dt> <dd>If this option is <tt class="docutils literal"><span class="pre">True</span></tt> (the default), then the step's logfile will describe the environment variables on the slave. In situations where the environment is not relevant and is long, it may be easier to set <tt class="docutils literal"><span class="pre">logEnviron=False</span></tt>.</dd> <dt><tt class="docutils literal"><span class="pre">interruptSignal</span></tt></dt> <dd>If the command should be interrupted (either by buildmaster or timeout etc.), what signal should be sent to the process, specified by name. By default this is "KILL" (9). Specify "TERM" (15) to give the process a chance to cleanup. This functionality requires a 0.8.6 slave or newer.</dd> <dt><tt class="docutils literal"><span class="pre">initialStdin</span></tt></dt> <dd>If the command expects input on stdin, that can be supplied a a string with this parameter. This value should not be excessively large, as it is handled as a single string throughout Buildbot -- for example, do not pass the contents of a tarball with this parameter.</dd> <dt><tt class="docutils literal"><span class="pre">decodeRC</span></tt></dt> <dd>This is a dictionary that decodes exit codes into results value. e.g: <tt class="docutils literal"><span class="pre">{0:SUCCESS,1:FAILURE,2:WARNINGS}</span></tt>, will treat the exit code <tt class="docutils literal"><span class="pre">2</span></tt> as WARNINGS. The default is to treat just 0 as successful. (<tt class="docutils literal"><span class="pre">{0:SUCCESS}</span></tt>) any exit code not present in the dictionary will be treated as <tt class="docutils literal"><span class="pre">FAILURE</span></tt></dd> </dl> </div> <div class="section" id="configure"> <span id="step-Configure"></span><h3>Configure<a class="headerlink" href="#configure" title="Permalink to this headline">¶</a></h3> <dl class="class"> <dt id="buildbot.steps.source.buildbot.steps.shell.Configure"> <em class="property">class </em><tt class="descclassname">buildbot.steps.shell.</tt><tt class="descname">Configure</tt><a class="headerlink" href="#buildbot.steps.source.buildbot.steps.shell.Configure" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <p>This is intended to handle the <strong class="command">./configure</strong> step from autoconf-style projects, or the <tt class="docutils literal"><span class="pre">perl</span> <span class="pre">Makefile.PL</span></tt> step from perl <tt class="file docutils literal"><span class="pre">MakeMaker.pm</span></tt>-style modules. The default command is <strong class="command">./configure</strong> but you can change this by providing a <tt class="docutils literal"><span class="pre">command=</span></tt> parameter. The arguments are identical to <a class="reference internal" href="#step-ShellCommand" title="ShellCommand"><tt class="xref bb bb-step docutils literal"><span class="pre">ShellCommand</span></tt></a>.</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.shell</span> <span class="kn">import</span> <span class="n">Configure</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">Configure</span><span class="p">())</span> </pre></div> </div> </div> <div class="section" id="compile"> <span id="step-Compile"></span><h3>Compile<a class="headerlink" href="#compile" title="Permalink to this headline">¶</a></h3> <p id="index-17">This is meant to handle compiling or building a project written in C. The default command is <tt class="docutils literal"><span class="pre">make</span> <span class="pre">all</span></tt>. When the compile is finished, the log file is scanned for GCC warning messages, a summary log is created with any problems that were seen, and the step is marked as WARNINGS if any were discovered. Through the <tt class="xref py py-class docutils literal"><span class="pre">WarningCountingShellCommand</span></tt> superclass, the number of warnings is stored in a Build Property named <cite>warnings-count</cite>, which is accumulated over all <a class="reference internal" href="#step-Compile" title="Compile"><tt class="xref bb bb-step docutils literal"><span class="pre">Compile</span></tt></a> steps (so if two warnings are found in one step, and three are found in another step, the overall build will have a <cite>warnings-count</cite> property of 5). Each step can be optionally given a maximum number of warnings via the maxWarnCount parameter. If this limit is exceeded, the step will be marked as a failure.</p> <p>The default regular expression used to detect a warning is <tt class="docutils literal"><span class="pre">'.*warning[:</span> <span class="pre">].*'</span></tt> , which is fairly liberal and may cause false-positives. To use a different regexp, provide a <tt class="docutils literal"><span class="pre">warningPattern=</span></tt> argument, or use a subclass which sets the <tt class="docutils literal"><span class="pre">warningPattern</span></tt> attribute:</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.shell</span> <span class="kn">import</span> <span class="n">Compile</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">Compile</span><span class="p">(</span><span class="n">command</span><span class="o">=</span><span class="p">[</span><span class="s">"make"</span><span class="p">,</span> <span class="s">"test"</span><span class="p">],</span> <span class="n">warningPattern</span><span class="o">=</span><span class="s">"^Warning: "</span><span class="p">))</span> </pre></div> </div> <p>The <tt class="docutils literal"><span class="pre">warningPattern=</span></tt> can also be a pre-compiled Python regexp object: this makes it possible to add flags like <tt class="docutils literal"><span class="pre">re.I</span></tt> (to use case-insensitive matching).</p> <p>Note that the compiled <tt class="docutils literal"><span class="pre">warningPattern</span></tt> will have its <tt class="xref py py-meth docutils literal"><span class="pre">match</span></tt> method called, which is subtly different from a <tt class="xref py py-meth docutils literal"><span class="pre">search</span></tt>. Your regular expression must match the from the beginning of the line. This means that to look for the word "warning" in the middle of a line, you will need to prepend <tt class="docutils literal"><span class="pre">'.*'</span></tt> to your regular expression.</p> <p>The <tt class="docutils literal"><span class="pre">suppressionFile=</span></tt> argument can be specified as the (relative) path of a file inside the workdir defining warnings to be suppressed from the warning counting and log file. The file will be uploaded to the master from the slave before compiling, and any warning matched by a line in the suppression file will be ignored. This is useful to accept certain warnings (eg. in some special module of the source tree or in cases where the compiler is being particularly stupid), yet still be able to easily detect and fix the introduction of new warnings.</p> <p>The file must contain one line per pattern of warnings to ignore. Empty lines and lines beginning with <tt class="docutils literal"><span class="pre">#</span></tt> are ignored. Other lines must consist of a regexp matching the file name, followed by a colon (<tt class="docutils literal"><span class="pre">:</span></tt>), followed by a regexp matching the text of the warning. Optionally this may be followed by another colon and a line number range. For example:</p> <div class="highlight-none"><div class="highlight"><pre># Sample warning suppression file mi_packrec.c : .*result of 32-bit shift implicitly converted to 64 bits.* : 560-600 DictTabInfo.cpp : .*invalid access to non-static.* kernel_types.h : .*only defines private constructors and has no friends.* : 51 </pre></div> </div> <p>If no line number range is specified, the pattern matches the whole file; if only one number is given it matches only on that line.</p> <p>The default warningPattern regexp only matches the warning text, so line numbers and file names are ignored. To enable line number and file name matching, provide a different regexp and provide a function (callable) as the argument of <tt class="docutils literal"><span class="pre">warningExtractor=</span></tt>. The function is called with three arguments: the <tt class="xref py py-class docutils literal"><span class="pre">BuildStep</span></tt> object, the line in the log file with the warning, and the <tt class="docutils literal"><span class="pre">SRE_Match</span></tt> object of the regexp search for <tt class="docutils literal"><span class="pre">warningPattern</span></tt>. It should return a tuple <tt class="docutils literal"><span class="pre">(filename,</span> <span class="pre">linenumber,</span> <span class="pre">warning_test)</span></tt>. For example:</p> <div class="highlight-python"><div class="highlight"><pre><span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">Compile</span><span class="p">(</span><span class="n">command</span><span class="o">=</span><span class="p">[</span><span class="s">"make"</span><span class="p">],</span> <span class="n">warningPattern</span><span class="o">=</span><span class="s">"^(.\*?):([0-9]+): [Ww]arning: (.\*)$"</span><span class="p">,</span> <span class="n">warningExtractor</span><span class="o">=</span><span class="n">Compile</span><span class="o">.</span><span class="n">warnExtractFromRegexpGroups</span><span class="p">,</span> <span class="n">suppressionFile</span><span class="o">=</span><span class="s">"support-files/compiler_warnings.supp"</span><span class="p">))</span> </pre></div> </div> <p>(<tt class="docutils literal"><span class="pre">Compile.warnExtractFromRegexpGroups</span></tt> is a pre-defined function that returns the filename, linenumber, and text from groups (1,2,3) of the regexp match).</p> <p>In projects with source files in multiple directories, it is possible to get full path names for file names matched in the suppression file, as long as the build command outputs the names of directories as they are entered into and left again. For this, specify regexps for the arguments <tt class="docutils literal"><span class="pre">directoryEnterPattern=</span></tt> and <tt class="docutils literal"><span class="pre">directoryLeavePattern=</span></tt>. The <tt class="docutils literal"><span class="pre">directoryEnterPattern=</span></tt> regexp should return the name of the directory entered into in the first matched group. The defaults, which are suitable for .. GNU Make, are these:</p> <div class="highlight-python"><pre>.. directoryEnterPattern = "make.*: Entering directory [\"`'](.*)['`\"]" .. directoryLeavePattern = "make.*: Leaving directory"</pre> </div> <p>(TODO: this step needs to be extended to look for GCC error messages as well, and collect them into a separate logfile, along with the source code filenames involved).</p> <span class="target" id="index-18"></span><span class="target" id="step-VC6"></span><span class="target" id="step-VC7"></span><span class="target" id="step-VC8"></span><span class="target" id="step-VC9"></span><span class="target" id="step-VC10"></span><span class="target" id="step-VC11"></span><span class="target" id="step-VS2003"></span><span class="target" id="step-VS2005"></span><span class="target" id="step-VS2008"></span><span class="target" id="step-VS2010"></span><span class="target" id="step-VS2012"></span><span class="target" id="step-VCExpress9"></span></div> <div class="section" id="visual-c"> <span id="step-MsBuild"></span><h3>Visual C++<a class="headerlink" href="#visual-c" title="Permalink to this headline">¶</a></h3> <p>These steps are meant to handle compilation using Microsoft compilers. VC++ 6-11 (aka Visual Studio 2003-2012 and VCExpress9) are supported via calling <tt class="docutils literal"><span class="pre">devenv</span></tt>. VS2012 as well as Windows Driver Kit 8 are supported via the new <tt class="docutils literal"><span class="pre">MsBuild</span></tt> step. These steps will take care of setting up a clean compilation environment, parsing the generated output in real time and delivering as detailed as possible information about the compilation executed.</p> <p>All of the classes are in <tt class="xref py py-mod docutils literal"><span class="pre">buildbot.steps.vstudio</span></tt>. The available classes are:</p> <blockquote> <div><ul class="simple"> <li><tt class="docutils literal"><span class="pre">VC6</span></tt></li> <li><tt class="docutils literal"><span class="pre">VC7</span></tt></li> <li><tt class="docutils literal"><span class="pre">VC8</span></tt></li> <li><tt class="docutils literal"><span class="pre">VC9</span></tt></li> <li><tt class="docutils literal"><span class="pre">VC10</span></tt></li> <li><tt class="docutils literal"><span class="pre">VC11</span></tt></li> <li><tt class="docutils literal"><span class="pre">VS2003</span></tt></li> <li><tt class="docutils literal"><span class="pre">VS2005</span></tt></li> <li><tt class="docutils literal"><span class="pre">VS2008</span></tt></li> <li><tt class="docutils literal"><span class="pre">VS2010</span></tt></li> <li><tt class="docutils literal"><span class="pre">VS2012</span></tt></li> <li><tt class="docutils literal"><span class="pre">VCExpress9</span></tt></li> <li><tt class="docutils literal"><span class="pre">MsBuild</span></tt></li> </ul> </div></blockquote> <p>The available constructor arguments are</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">mode</span></tt></dt> <dd>The mode default to <tt class="docutils literal"><span class="pre">rebuild</span></tt>, which means that first all the remaining object files will be cleaned by the compiler. The alternate values are <tt class="docutils literal"><span class="pre">build</span></tt>, where only the updated files will be recompiled, and <tt class="docutils literal"><span class="pre">clean</span></tt>, where the current build files are removed and no compilation occurs.</dd> <dt><tt class="docutils literal"><span class="pre">projectfile</span></tt></dt> <dd>This is a mandatory argument which specifies the project file to be used during the compilation.</dd> <dt><tt class="docutils literal"><span class="pre">config</span></tt></dt> <dd>This argument defaults to <tt class="docutils literal"><span class="pre">release</span></tt> an gives to the compiler the configuration to use.</dd> <dt><tt class="docutils literal"><span class="pre">installdir</span></tt></dt> <dd>This is the place where the compiler is installed. The default value is compiler specific and is the default place where the compiler is installed.</dd> <dt><tt class="docutils literal"><span class="pre">useenv</span></tt></dt> <dd>This boolean parameter, defaulting to <tt class="docutils literal"><span class="pre">False</span></tt> instruct the compiler to use its own settings or the one defined through the environment variables <span class="target" id="index-19"></span><tt class="xref std std-envvar docutils literal"><span class="pre">PATH</span></tt>, <span class="target" id="index-20"></span><tt class="xref std std-envvar docutils literal"><span class="pre">INCLUDE</span></tt>, and <span class="target" id="index-21"></span><tt class="xref std std-envvar docutils literal"><span class="pre">LIB</span></tt>. If any of the <tt class="docutils literal"><span class="pre">INCLUDE</span></tt> or <tt class="docutils literal"><span class="pre">LIB</span></tt> parameter is defined, this parameter automatically switches to <tt class="docutils literal"><span class="pre">True</span></tt>.</dd> <dt><tt class="docutils literal"><span class="pre">PATH</span></tt></dt> <dd>This is a list of path to be added to the <span class="target" id="index-22"></span><tt class="xref std std-envvar docutils literal"><span class="pre">PATH</span></tt> environment variable. The default value is the one defined in the compiler options.</dd> <dt><tt class="docutils literal"><span class="pre">INCLUDE</span></tt></dt> <dd>This is a list of path where the compiler will first look for include files. Then comes the default paths defined in the compiler options.</dd> <dt><tt class="docutils literal"><span class="pre">LIB</span></tt></dt> <dd>This is a list of path where the compiler will first look for libraries. Then comes the default path defined in the compiler options.</dd> <dt><tt class="docutils literal"><span class="pre">arch</span></tt></dt> <dd>That one is only available with the class VS2005 (VC8). It gives the target architecture of the built artifact. It defaults to <tt class="docutils literal"><span class="pre">x86</span></tt> and does not apply to <tt class="docutils literal"><span class="pre">MsBuild</span></tt>. Please see <tt class="docutils literal"><span class="pre">platform</span></tt> below.</dd> <dt><tt class="docutils literal"><span class="pre">project</span></tt></dt> <dd>This gives the specific project to build from within a workspace. It defaults to building all projects. This is useful for building cmake generate projects.</dd> <dt><tt class="docutils literal"><span class="pre">platform</span></tt></dt> <dd>This is a mandatory argument for MsBuild specifying the target platform such as 'Win32', 'x64' or 'Vista Debug'. The last one is an example of driver targets that appear once Windows Driver Kit 8 is installed.</dd> </dl> <p>Here is an example on how to drive compilation with Visual Studio 2010:</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.VisualStudio</span> <span class="kn">import</span> <span class="n">VS2010</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span> <span class="n">VS2010</span><span class="p">(</span><span class="n">projectfile</span><span class="o">=</span><span class="s">"project.sln"</span><span class="p">,</span> <span class="n">config</span><span class="o">=</span><span class="s">"release"</span><span class="p">,</span> <span class="n">arch</span><span class="o">=</span><span class="s">"x64"</span><span class="p">,</span> <span class="n">mode</span><span class="o">=</span><span class="s">"build"</span><span class="p">,</span> <span class="n">INCLUDE</span><span class="o">=</span><span class="p">[</span><span class="s">r'C:\3rd-pary\libmagic\include'</span><span class="p">],</span> <span class="n">LIB</span><span class="o">=</span><span class="p">[</span><span class="s">r'C:\3rd-party\libmagic\lib-x64'</span><span class="p">]))</span> </pre></div> </div> <p>Here is a similar example using "msbuild":</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.VisualStudio</span> <span class="kn">import</span> <span class="n">MsBuild</span> <span class="c"># Build one project in Release mode for Win32</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span> <span class="n">MsBuild</span><span class="p">(</span><span class="n">projectfile</span><span class="o">=</span><span class="s">"trunk.sln"</span><span class="p">,</span> <span class="n">config</span><span class="o">=</span><span class="s">"Release"</span><span class="p">,</span> <span class="n">platform</span><span class="o">=</span><span class="s">"Win32"</span><span class="p">,</span> <span class="n">workdir</span><span class="o">=</span><span class="s">"trunk"</span><span class="p">,</span> <span class="n">project</span><span class="o">=</span><span class="s">"tools</span><span class="se">\\</span><span class="s">protoc"</span><span class="p">))</span> <span class="c"># Build the entire solution in Debug mode for x64</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span> <span class="n">MsBuild</span><span class="p">(</span><span class="n">projectfile</span><span class="o">=</span><span class="s">"trunk.sln"</span><span class="p">,</span> <span class="n">config</span><span class="o">=</span><span class="s">'Debug'</span><span class="p">,</span> <span class="n">platform</span><span class="o">=</span><span class="s">'x64'</span><span class="p">,</span> <span class="n">workdir</span><span class="o">=</span><span class="s">"trunk"</span><span class="p">))</span> </pre></div> </div> </div> <div class="section" id="test"> <span id="step-Test"></span><h3>Test<a class="headerlink" href="#test" title="Permalink to this headline">¶</a></h3> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.shell</span> <span class="kn">import</span> <span class="n">Test</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">Test</span><span class="p">())</span> </pre></div> </div> <p>This is meant to handle unit tests. The default command is <strong class="command">make test</strong>, and the <tt class="docutils literal"><span class="pre">warnOnFailure</span></tt> flag is set. The other arguments are identical to <a class="reference internal" href="#step-ShellCommand" title="ShellCommand"><tt class="xref bb bb-step docutils literal"><span class="pre">ShellCommand</span></tt></a>.</p> <span class="target" id="step-TreeSize"></span></div> <div class="section" id="treesize"> <span id="index-23"></span><h3>TreeSize<a class="headerlink" href="#treesize" title="Permalink to this headline">¶</a></h3> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.shell</span> <span class="kn">import</span> <span class="n">TreeSize</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">TreeSize</span><span class="p">())</span> </pre></div> </div> <p>This is a simple command that uses the <strong class="command">du</strong> tool to measure the size of the code tree. It puts the size (as a count of 1024-byte blocks, aka 'KiB' or 'kibibytes') on the step's status text, and sets a build property named <tt class="docutils literal"><span class="pre">tree-size-KiB</span></tt> with the same value. All arguments are identical to <a class="reference internal" href="#step-ShellCommand" title="ShellCommand"><tt class="xref bb bb-step docutils literal"><span class="pre">ShellCommand</span></tt></a>.</p> </div> <div class="section" id="perlmoduletest"> <span id="step-PerlModuleTest"></span><h3>PerlModuleTest<a class="headerlink" href="#perlmoduletest" title="Permalink to this headline">¶</a></h3> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.shell</span> <span class="kn">import</span> <span class="n">PerlModuleTest</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">PerlModuleTest</span><span class="p">())</span> </pre></div> </div> <p>This is a simple command that knows how to run tests of perl modules. It parses the output to determine the number of tests passed and failed and total number executed, saving the results for later query. The command is <tt class="docutils literal"><span class="pre">prove</span> <span class="pre">--lib</span> <span class="pre">lib</span> <span class="pre">-r</span> <span class="pre">t</span></tt>, although this can be overridden with the <tt class="docutils literal"><span class="pre">command</span></tt> argument. All other arguments are identical to those for <a class="reference internal" href="#step-ShellCommand" title="ShellCommand"><tt class="xref bb bb-step docutils literal"><span class="pre">ShellCommand</span></tt></a>.</p> </div> <div class="section" id="mtr-mysql-test-run"> <span id="step-MTR"></span><h3>MTR (mysql-test-run)<a class="headerlink" href="#mtr-mysql-test-run" title="Permalink to this headline">¶</a></h3> <p>The <a class="reference internal" href="#step-MTR" title="MTR"><tt class="xref bb bb-step docutils literal"><span class="pre">MTR</span></tt></a> class is a subclass of <a class="reference internal" href="#step-Test" title="Test"><tt class="xref bb bb-step docutils literal"><span class="pre">Test</span></tt></a>. It is used to run test suites using the mysql-test-run program, as used in MySQL, Drizzle, MariaDB, and MySQL storage engine plugins.</p> <p>The shell command to run the test suite is specified in the same way as for the <a class="reference internal" href="#step-Test" title="Test"><tt class="xref bb bb-step docutils literal"><span class="pre">Test</span></tt></a> class. The <a class="reference internal" href="#step-MTR" title="MTR"><tt class="xref bb bb-step docutils literal"><span class="pre">MTR</span></tt></a> class will parse the output of running the test suite, and use the count of tests executed so far to provide more accurate completion time estimates. Any test failures that occur during the test are summarized on the Waterfall Display.</p> <p>Server error logs are added as additional log files, useful to debug test failures.</p> <p>Optionally, data about the test run and any test failures can be inserted into a database for further analysis and report generation. To use this facility, create an instance of <tt class="xref py py-class docutils literal"><span class="pre">twisted.enterprise.adbapi.ConnectionPool</span></tt> with connections to the database. The necessary tables can be created automatically by setting <tt class="docutils literal"><span class="pre">autoCreateTables</span></tt> to <tt class="docutils literal"><span class="pre">True</span></tt>, or manually using the SQL found in the <tt class="file docutils literal"><span class="pre">mtrlogobserver.py</span></tt> source file.</p> <p>One problem with specifying a database is that each reload of the configuration will get a new instance of <tt class="docutils literal"><span class="pre">ConnectionPool</span></tt> (even if the connection parameters are the same). To avoid that Buildbot thinks the builder configuration has changed because of this, use the <tt class="xref py py-class docutils literal"><span class="pre">process.mtrlogobserver.EqConnectionPool</span></tt> subclass of <tt class="xref py py-class docutils literal"><span class="pre">ConnectionPool</span></tt>, which implements an equiality operation that avoids this problem.</p> <p>Example use:</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.process.mtrlogobserver</span> <span class="kn">import</span> <span class="n">MTR</span><span class="p">,</span> <span class="n">EqConnectionPool</span> <span class="n">myPool</span> <span class="o">=</span> <span class="n">EqConnectionPool</span><span class="p">(</span><span class="s">"MySQLdb"</span><span class="p">,</span> <span class="s">"host"</span><span class="p">,</span> <span class="s">"buildbot"</span><span class="p">,</span> <span class="s">"password"</span><span class="p">,</span> <span class="s">"db"</span><span class="p">)</span> <span class="n">myFactory</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">MTR</span><span class="p">(</span><span class="n">workdir</span><span class="o">=</span><span class="s">"mysql-test"</span><span class="p">,</span> <span class="n">dbpool</span><span class="o">=</span><span class="n">myPool</span><span class="p">,</span> <span class="n">command</span><span class="o">=</span><span class="p">[</span><span class="s">"perl"</span><span class="p">,</span> <span class="s">"mysql-test-run.pl"</span><span class="p">,</span> <span class="s">"--force"</span><span class="p">]))</span> </pre></div> </div> <p>The <a class="reference internal" href="#step-MTR" title="MTR"><tt class="xref bb bb-step docutils literal"><span class="pre">MTR</span></tt></a> step's arguments are:</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">textLimit</span></tt></dt> <dd>Maximum number of test failures to show on the waterfall page (to not flood the page in case of a large number of test failures. Defaults to 5.</dd> <dt><tt class="docutils literal"><span class="pre">testNameLimit</span></tt></dt> <dd>Maximum length of test names to show unabbreviated in the waterfall page, to avoid excessive column width. Defaults to 16.</dd> <dt><tt class="docutils literal"><span class="pre">parallel</span></tt></dt> <dd>Value of <em class="xref std std-option">--parallel</em> option used for <tt class="file docutils literal"><span class="pre">mysql-test-run.pl</span></tt> (number of processes used to run the test suite in parallel). Defaults to 4. This is used to determine the number of server error log files to download from the slave. Specifying a too high value does not hurt (as nonexisting error logs will be ignored), however if using <em class="xref std std-option">--parallel</em> value greater than the default it needs to be specified, or some server error logs will be missing.</dd> <dt><tt class="docutils literal"><span class="pre">dbpool</span></tt></dt> <dd>An instance of <tt class="xref py py-class docutils literal"><span class="pre">twisted.enterprise.adbapi.ConnectionPool</span></tt>, or <tt class="docutils literal"><span class="pre">None</span></tt>. Defaults to <tt class="docutils literal"><span class="pre">None</span></tt>. If specified, results are inserted into the database using the <tt class="xref py py-class docutils literal"><span class="pre">ConnectionPool</span></tt>.</dd> <dt><tt class="docutils literal"><span class="pre">autoCreateTables</span></tt></dt> <dd>Boolean, defaults to <tt class="docutils literal"><span class="pre">False</span></tt>. If <tt class="docutils literal"><span class="pre">True</span></tt> (and <tt class="docutils literal"><span class="pre">dbpool</span></tt> is specified), the necessary database tables will be created automatically if they do not exist already. Alternatively, the tables can be created manually from the SQL statements found in the <tt class="file docutils literal"><span class="pre">mtrlogobserver.py</span></tt> source file.</dd> <dt><tt class="docutils literal"><span class="pre">test_type</span></tt></dt> <dd>Short string that will be inserted into the database in the row for the test run. Defaults to the empty string, but can be specified to identify different types of test runs.</dd> <dt><tt class="docutils literal"><span class="pre">test_info</span></tt></dt> <dd>Descriptive string that will be inserted into the database in the row for the test run. Defaults to the empty string, but can be specified as a user-readable description of this particular test run.</dd> <dt><tt class="docutils literal"><span class="pre">mtr_subdir</span></tt></dt> <dd>The subdirectory in which to look for server error log files. Defaults to <tt class="file docutils literal"><span class="pre">mysql-test</span></tt>, which is usually correct. <a class="reference internal" href="cfg-properties.html#interpolate"><em>Interpolate</em></a> is supported.</dd> </dl> </div> <div class="section" id="subunitshellcommand"> <span id="step-subunitshellcommand"></span><span id="step-SubunitShellCommand"></span><h3>SubunitShellCommand<a class="headerlink" href="#subunitshellcommand" title="Permalink to this headline">¶</a></h3> <dl class="class"> <dt id="buildbot.steps.source.buildbot.steps.subunit.SubunitShellCommand"> <em class="property">class </em><tt class="descclassname">buildbot.steps.subunit.</tt><tt class="descname">SubunitShellCommand</tt><a class="headerlink" href="#buildbot.steps.source.buildbot.steps.subunit.SubunitShellCommand" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <p>This buildstep is similar to <a class="reference internal" href="#step-ShellCommand" title="ShellCommand"><tt class="xref bb bb-step docutils literal"><span class="pre">ShellCommand</span></tt></a>, except that it runs the log content through a subunit filter to extract test and failure counts.</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.subunit</span> <span class="kn">import</span> <span class="n">SubunitShellCommand</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">SubunitShellCommand</span><span class="p">(</span><span class="n">command</span><span class="o">=</span><span class="s">"make test"</span><span class="p">))</span> </pre></div> </div> <p>This runs <tt class="docutils literal"><span class="pre">make</span> <span class="pre">test</span></tt> and filters it through subunit. The 'tests' and 'test failed' progress metrics will now accumulate test data from the test run.</p> <p>If <tt class="docutils literal"><span class="pre">failureOnNoTests</span></tt> is <tt class="docutils literal"><span class="pre">True</span></tt>, this step will fail if no test is run. By default <tt class="docutils literal"><span class="pre">failureOnNoTests</span></tt> is False.</p> </div> </div> <div class="section" id="slave-filesystem-steps"> <span id="id15"></span><h2>Slave Filesystem Steps<a class="headerlink" href="#slave-filesystem-steps" title="Permalink to this headline">¶</a></h2> <p>Here are some buildsteps for manipulating the slave's filesystem.</p> <div class="section" id="fileexists"> <span id="step-FileExists"></span><h3>FileExists<a class="headerlink" href="#fileexists" title="Permalink to this headline">¶</a></h3> <p>This step will assert that a given file exists, failing if it does not. The filename can be specified with a property.</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.slave</span> <span class="kn">import</span> <span class="n">FileExists</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">FileExists</span><span class="p">(</span><span class="nb">file</span><span class="o">=</span><span class="s">'test_data'</span><span class="p">))</span> </pre></div> </div> <p>This step requires slave version 0.8.4 or later.</p> </div> <div class="section" id="copydirectory"> <span id="step-CopyDirectory"></span><h3>CopyDirectory<a class="headerlink" href="#copydirectory" title="Permalink to this headline">¶</a></h3> <p>This command copies a directory on the slave.</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.slave</span> <span class="kn">import</span> <span class="n">CopyDirectory</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">CopyDirectory</span><span class="p">(</span><span class="n">src</span><span class="o">=</span><span class="s">"build/data"</span><span class="p">,</span> <span class="n">dest</span><span class="o">=</span><span class="s">"tmp/data"</span><span class="p">))</span> </pre></div> </div> <p>This step requires slave version 0.8.5 or later.</p> <p>The CopyDirectory step takes the following arguments:</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">timeout</span></tt></dt> <dd>if the copy command fails to produce any output for this many seconds, it is assumed to be locked up and will be killed. This defaults to 120 seconds. Pass <tt class="docutils literal"><span class="pre">None</span></tt> to disable.</dd> <dt><tt class="docutils literal"><span class="pre">maxTime</span></tt></dt> <dd>if the command takes longer than this many seconds, it will be killed. This is disabled by default.</dd> </dl> </div> <div class="section" id="removedirectory"> <span id="step-RemoveDirectory"></span><h3>RemoveDirectory<a class="headerlink" href="#removedirectory" title="Permalink to this headline">¶</a></h3> <p>This command recursively deletes a directory on the slave.</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.slave</span> <span class="kn">import</span> <span class="n">RemoveDirectory</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">RemoveDirectory</span><span class="p">(</span><span class="nb">dir</span><span class="o">=</span><span class="s">"build/build"</span><span class="p">))</span> </pre></div> </div> <p>This step requires slave version 0.8.4 or later.</p> </div> <div class="section" id="makedirectory"> <span id="step-MakeDirectory"></span><h3>MakeDirectory<a class="headerlink" href="#makedirectory" title="Permalink to this headline">¶</a></h3> <p>This command creates a directory on the slave.</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.slave</span> <span class="kn">import</span> <span class="n">MakeDirectory</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">MakeDirectory</span><span class="p">(</span><span class="nb">dir</span><span class="o">=</span><span class="s">"build/build"</span><span class="p">))</span> </pre></div> </div> <p>This step requires slave version 0.8.5 or later.</p> </div> </div> <div class="section" id="python-buildsteps"> <span id="id16"></span><h2>Python BuildSteps<a class="headerlink" href="#python-buildsteps" title="Permalink to this headline">¶</a></h2> <p>Here are some <tt class="xref py py-class docutils literal"><span class="pre">BuildStep</span></tt>s that are specifically useful for projects implemented in Python.</p> <div class="section" id="buildepydoc"> <span id="step-buildepydoc"></span><span id="step-BuildEPYDoc"></span><h3>BuildEPYDoc<a class="headerlink" href="#buildepydoc" title="Permalink to this headline">¶</a></h3> <dl class="class"> <dt id="buildbot.steps.source.buildbot.steps.python.BuildEPYDoc"> <em class="property">class </em><tt class="descclassname">buildbot.steps.python.</tt><tt class="descname">BuildEPYDoc</tt><a class="headerlink" href="#buildbot.steps.source.buildbot.steps.python.BuildEPYDoc" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <p><a class="reference external" href="http://epydoc.sourceforge.net/">epydoc</a> is a tool for generating API documentation for Python modules from their docstrings. It reads all the <tt class="file docutils literal"><span class="pre">.py</span></tt> files from your source tree, processes the docstrings therein, and creates a large tree of <tt class="file docutils literal"><span class="pre">.html</span></tt> files (or a single <tt class="file docutils literal"><span class="pre">.pdf</span></tt> file).</p> <p>The <a class="reference internal" href="#step-BuildEPYDoc" title="BuildEPYDoc"><tt class="xref bb bb-step docutils literal"><span class="pre">BuildEPYDoc</span></tt></a> step will run <strong class="command">epydoc</strong> to produce this API documentation, and will count the errors and warnings from its output.</p> <p>You must supply the command line to be used. The default is <tt class="docutils literal"><span class="pre">make</span> <span class="pre">epydocs</span></tt>, which assumes that your project has a <tt class="file docutils literal"><span class="pre">Makefile</span></tt> with an <cite>epydocs</cite> target. You might wish to use something like <tt class="samp docutils literal"><span class="pre">epydoc</span> <span class="pre">-o</span> <span class="pre">apiref</span> <span class="pre">source/</span><em><span class="pre">PKGNAME</span></em></tt> instead. You might also want to add <em class="xref std std-option">--pdf</em> to generate a PDF file instead of a large tree of HTML files.</p> <p>The API docs are generated in-place in the build tree (under the workdir, in the subdirectory controlled by the <em class="xref std std-option">-o</em> argument). To make them useful, you will probably have to copy them to somewhere they can be read. A command like <tt class="docutils literal"><span class="pre">rsync</span> <span class="pre">-ad</span> <span class="pre">apiref/</span> <span class="pre">dev.example.com:~public_html/current-apiref/</span></tt> might be useful. You might instead want to bundle them into a tarball and publish it in the same place where the generated install tarball is placed.</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.python</span> <span class="kn">import</span> <span class="n">BuildEPYDoc</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">BuildEPYDoc</span><span class="p">(</span><span class="n">command</span><span class="o">=</span><span class="p">[</span><span class="s">"epydoc"</span><span class="p">,</span> <span class="s">"-o"</span><span class="p">,</span> <span class="s">"apiref"</span><span class="p">,</span> <span class="s">"source/mypkg"</span><span class="p">]))</span> </pre></div> </div> </div> <div class="section" id="pyflakes"> <span id="step-pyflake"></span><span id="step-PyFlakes"></span><h3>PyFlakes<a class="headerlink" href="#pyflakes" title="Permalink to this headline">¶</a></h3> <dl class="class"> <dt id="buildbot.steps.source.buildbot.steps.python.PyFlakes"> <em class="property">class </em><tt class="descclassname">buildbot.steps.python.</tt><tt class="descname">PyFlakes</tt><a class="headerlink" href="#buildbot.steps.source.buildbot.steps.python.PyFlakes" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <p><a class="reference external" href="http://divmod.org/trac/wiki/DivmodPyflakes">PyFlakes</a> is a tool to perform basic static analysis of Python code to look for simple errors, like missing imports and references of undefined names. It is like a fast and simple form of the C <strong class="command">lint</strong> program. Other tools (like <a class="reference external" href="http://pychecker.sourceforge.net/">pychecker</a>) provide more detailed results but take longer to run.</p> <p>The <a class="reference internal" href="#step-PyFlakes" title="PyFlakes"><tt class="xref bb bb-step docutils literal"><span class="pre">PyFlakes</span></tt></a> step will run pyflakes and count the various kinds of errors and warnings it detects.</p> <p>You must supply the command line to be used. The default is <tt class="docutils literal"><span class="pre">make</span> <span class="pre">pyflakes</span></tt>, which assumes you have a top-level <tt class="file docutils literal"><span class="pre">Makefile</span></tt> with a <tt class="docutils literal"><span class="pre">pyflakes</span></tt> target. You might want to use something like <tt class="docutils literal"><span class="pre">pyflakes</span> <span class="pre">.</span></tt> or <tt class="docutils literal"><span class="pre">pyflakes</span> <span class="pre">src</span></tt>.</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.python</span> <span class="kn">import</span> <span class="n">PyFlakes</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">PyFlakes</span><span class="p">(</span><span class="n">command</span><span class="o">=</span><span class="p">[</span><span class="s">"pyflakes"</span><span class="p">,</span> <span class="s">"src"</span><span class="p">]))</span> </pre></div> </div> </div> <div class="section" id="sphinx"> <span id="step-sphinx"></span><span id="step-Sphinx"></span><h3>Sphinx<a class="headerlink" href="#sphinx" title="Permalink to this headline">¶</a></h3> <dl class="class"> <dt id="buildbot.steps.source.buildbot.steps.python.Sphinx"> <em class="property">class </em><tt class="descclassname">buildbot.steps.python.</tt><tt class="descname">Sphinx</tt><a class="headerlink" href="#buildbot.steps.source.buildbot.steps.python.Sphinx" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <p><a class="reference external" href="http://sphinx.pocoo.org/">Sphinx</a> is the Python Documentation Generator. It uses <a class="reference external" href="http://docutils.sourceforge.net/rst.html">RestructuredText</a> as input format.</p> <p>The <a class="reference internal" href="#step-Sphinx" title="Sphinx"><tt class="xref bb bb-step docutils literal"><span class="pre">Sphinx</span></tt></a> step will run <strong class="program">sphinx-build</strong> or any other program specified in its <tt class="docutils literal"><span class="pre">sphinx</span></tt> argument and count the various warnings and error it detects.</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.python</span> <span class="kn">import</span> <span class="n">Sphinx</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">Sphinx</span><span class="p">(</span><span class="n">sphinx_builddir</span><span class="o">=</span><span class="s">"_build"</span><span class="p">))</span> </pre></div> </div> <p>This step takes the following arguments:</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">sphinx_builddir</span></tt></dt> <dd>(required) Name of the directory where the documentation will be generated.</dd> <dt><tt class="docutils literal"><span class="pre">sphinx_sourcedir</span></tt></dt> <dd>(optional, defaulting to <tt class="docutils literal"><span class="pre">.</span></tt>), Name the directory where the <tt class="file docutils literal"><span class="pre">conf.py</span></tt> file will be found</dd> <dt><tt class="docutils literal"><span class="pre">sphinx_builder</span></tt></dt> <dd>(optional) Indicates the builder to use.</dd> <dt><tt class="docutils literal"><span class="pre">sphinx</span></tt></dt> <dd>(optional, defaulting to <strong class="program">sphinx-build</strong>) Indicates the executable to run.</dd> <dt><tt class="docutils literal"><span class="pre">tags</span></tt></dt> <dd>(optional) List of <tt class="docutils literal"><span class="pre">tags</span></tt> to pass to <strong class="program">sphinx-build</strong></dd> <dt><tt class="docutils literal"><span class="pre">defines</span></tt></dt> <dd>(optional) Dictionary of defines to overwrite values of the <tt class="file docutils literal"><span class="pre">conf.py</span></tt> file.</dd> <dt><tt class="docutils literal"><span class="pre">mode</span></tt></dt> <dd>(optional) String, one of <tt class="docutils literal"><span class="pre">full</span></tt> or <tt class="docutils literal"><span class="pre">incremental</span></tt> (the default). If set to <tt class="docutils literal"><span class="pre">full</span></tt>, indicates to Sphinx to rebuild everything without re-using the previous build results.</dd> </dl> </div> <div class="section" id="pylint"> <span id="step-pylint"></span><span id="step-PyLint"></span><h3>PyLint<a class="headerlink" href="#pylint" title="Permalink to this headline">¶</a></h3> <p>Similarly, the <a class="reference internal" href="#step-PyLint" title="PyLint"><tt class="xref bb bb-step docutils literal"><span class="pre">PyLint</span></tt></a> step will run <strong class="command">pylint</strong> and analyze the results.</p> <p>You must supply the command line to be used. There is no default.</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.python</span> <span class="kn">import</span> <span class="n">PyLint</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">PyLint</span><span class="p">(</span><span class="n">command</span><span class="o">=</span><span class="p">[</span><span class="s">"pylint"</span><span class="p">,</span> <span class="s">"src"</span><span class="p">]))</span> </pre></div> </div> </div> <div class="section" id="trial"> <span id="step-trial"></span><span id="step-Trial"></span><h3>Trial<a class="headerlink" href="#trial" title="Permalink to this headline">¶</a></h3> <dl class="class"> <dt id="buildbot.steps.source.buildbot.steps.python_twisted.Trial"> <em class="property">class </em><tt class="descclassname">buildbot.steps.python_twisted.</tt><tt class="descname">Trial</tt><a class="headerlink" href="#buildbot.steps.source.buildbot.steps.python_twisted.Trial" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <p>This step runs a unit test suite using <strong class="command">trial</strong>, a unittest-like testing framework that is a component of Twisted Python. Trial is used to implement Twisted's own unit tests, and is the unittest-framework of choice for many projects that use Twisted internally.</p> <p>Projects that use trial typically have all their test cases in a 'test' subdirectory of their top-level library directory. For example, for a package <tt class="docutils literal"><span class="pre">petmail</span></tt>, the tests might be in <tt class="file docutils literal"><span class="pre">petmail/test/test_*.py</span></tt>. More complicated packages (like Twisted itself) may have multiple test directories, like <tt class="file docutils literal"><span class="pre">twisted/test/test_*.py</span></tt> for the core functionality and <tt class="file docutils literal"><span class="pre">twisted/mail/test/test_*.py</span></tt> for the email-specific tests.</p> <p>To run trial tests manually, you run the <strong class="command">trial</strong> executable and tell it where the test cases are located. The most common way of doing this is with a module name. For petmail, this might look like <strong class="command">trial petmail.test</strong>, which would locate all the <tt class="file docutils literal"><span class="pre">test_*.py</span></tt> files under <tt class="file docutils literal"><span class="pre">petmail/test/</span></tt>, running every test case it could find in them. Unlike the <tt class="docutils literal"><span class="pre">unittest.py</span></tt> that comes with Python, it is not necessary to run the <tt class="file docutils literal"><span class="pre">test_foo.py</span></tt> as a script; you always let trial do the importing and running. The step's <tt class="docutils literal"><span class="pre">tests`</span></tt> parameter controls which tests trial will run: it can be a string or a list of strings.</p> <p>To find the test cases, the Python search path must allow something like <tt class="docutils literal"><span class="pre">import</span> <span class="pre">petmail.test</span></tt> to work. For packages that don't use a separate top-level <tt class="file docutils literal"><span class="pre">lib</span></tt> directory, <tt class="docutils literal"><span class="pre">PYTHONPATH=.</span></tt> will work, and will use the test cases (and the code they are testing) in-place. <tt class="docutils literal"><span class="pre">PYTHONPATH=build/lib</span></tt> or <tt class="docutils literal"><span class="pre">PYTHONPATH=build/lib.somearch</span></tt> are also useful when you do a <tt class="docutils literal"><span class="pre">python</span> <span class="pre">setup.py</span> <span class="pre">build</span></tt> step first. The <tt class="docutils literal"><span class="pre">testpath</span></tt> attribute of this class controls what <span class="target" id="index-24"></span><tt class="xref std std-envvar docutils literal"><span class="pre">PYTHONPATH</span></tt> is set to before running <strong class="command">trial</strong>.</p> <p>Trial has the ability, through the <tt class="docutils literal"><span class="pre">--testmodule</span></tt> flag, to run only the set of test cases named by special <tt class="docutils literal"><span class="pre">test-case-name</span></tt> tags in source files. We can get the list of changed source files from our parent Build and provide them to trial, thus running the minimal set of test cases needed to cover the Changes. This is useful for quick builds, especially in trees with a lot of test cases. The <tt class="docutils literal"><span class="pre">testChanges</span></tt> parameter controls this feature: if set, it will override <tt class="docutils literal"><span class="pre">tests</span></tt>.</p> <p>The trial executable itself is typically just <strong class="command">trial</strong>, and is typically found in the shell search path. It can be overridden with the <tt class="docutils literal"><span class="pre">trial</span></tt> parameter. This is useful for Twisted's own unittests, which want to use the copy of bin/trial that comes with the sources.</p> <p>To influence the version of Python being used for the tests, or to add flags to the command, set the <tt class="docutils literal"><span class="pre">python</span></tt> parameter. This can be a string (like <tt class="docutils literal"><span class="pre">python2.2</span></tt>) or a list (like <tt class="docutils literal"><span class="pre">['python2.3',</span> <span class="pre">'-Wall']</span></tt>).</p> <p>Trial creates and switches into a directory named <tt class="file docutils literal"><span class="pre">_trial_temp/</span></tt> before running the tests, and sends the twisted log (which includes all exceptions) to a file named <tt class="file docutils literal"><span class="pre">test.log</span></tt>. This file will be pulled up to the master where it can be seen as part of the status output.</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.python_twisted</span> <span class="kn">import</span> <span class="n">Trial</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">Trial</span><span class="p">(</span><span class="n">tests</span><span class="o">=</span><span class="s">'petmail.test'</span><span class="p">))</span> </pre></div> </div> <p>Trial has the ability to run tests on several workers in parallel (beginning with Twisted 12.3.0). Set <tt class="docutils literal"><span class="pre">jobs</span></tt> to the number of workers you want to run. Note that running <strong class="command">trial</strong> in this way will create multiple log files (named <tt class="file docutils literal"><span class="pre">test.N.log</span></tt>, <tt class="file docutils literal"><span class="pre">err.N.log</span></tt> and <tt class="file docutils literal"><span class="pre">out.N.log</span></tt> starting with <tt class="docutils literal"><span class="pre">N=0</span></tt>) rather than a single <tt class="file docutils literal"><span class="pre">test.log</span></tt>.</p> <p>This step takes the following arguments:</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">jobs</span></tt></dt> <dd>(optional) Number of slave-resident workers to use when running the tests. Defaults to 1 worker. Only works with Twisted>=12.3.0.</dd> </dl> </div> <div class="section" id="removepycs"> <span id="step-RemovePYCs"></span><h3>RemovePYCs<a class="headerlink" href="#removepycs" title="Permalink to this headline">¶</a></h3> <dl class="class"> <dt id="buildbot.steps.source.buildbot.steps.python_twisted.RemovePYCs"> <em class="property">class </em><tt class="descclassname">buildbot.steps.python_twisted.</tt><tt class="descname">RemovePYCs</tt><a class="headerlink" href="#buildbot.steps.source.buildbot.steps.python_twisted.RemovePYCs" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <p>This is a simple built-in step that will remove <tt class="docutils literal"><span class="pre">.pyc</span></tt> files from the workdir. This is useful in builds that update their source (and thus do not automatically delete <tt class="docutils literal"><span class="pre">.pyc</span></tt> files) but where some part of the build process is dynamically searching for Python modules. Notably, trial has a bad habit of finding old test modules.</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.python_twisted</span> <span class="kn">import</span> <span class="n">RemovePYCs</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">RemovePYCs</span><span class="p">())</span> </pre></div> </div> <span class="target" id="index-25"></span><span class="target" id="step-FileUpload"></span></div> </div> <div class="section" id="transferring-files"> <span id="step-FileDownload"></span><h2>Transferring Files<a class="headerlink" href="#transferring-files" title="Permalink to this headline">¶</a></h2> <dl class="class"> <dt id="buildbot.steps.source.buildbot.steps.transfer.FileUpload"> <em class="property">class </em><tt class="descclassname">buildbot.steps.transfer.</tt><tt class="descname">FileUpload</tt><a class="headerlink" href="#buildbot.steps.source.buildbot.steps.transfer.FileUpload" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <dl class="class"> <dt id="buildbot.steps.source.buildbot.steps.transfer.FileDownload"> <em class="property">class </em><tt class="descclassname">buildbot.steps.transfer.</tt><tt class="descname">FileDownload</tt><a class="headerlink" href="#buildbot.steps.source.buildbot.steps.transfer.FileDownload" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <p>Most of the work involved in a build will take place on the buildslave. But occasionally it is useful to do some work on the buildmaster side. The most basic way to involve the buildmaster is simply to move a file from the slave to the master, or vice versa. There are a pair of steps named <a class="reference internal" href="#step-FileUpload" title="FileUpload"><tt class="xref bb bb-step docutils literal"><span class="pre">FileUpload</span></tt></a> and <a class="reference internal" href="#step-FileDownload" title="FileDownload"><tt class="xref bb bb-step docutils literal"><span class="pre">FileDownload</span></tt></a> to provide this functionality. <a class="reference internal" href="#step-FileUpload" title="FileUpload"><tt class="xref bb bb-step docutils literal"><span class="pre">FileUpload</span></tt></a> moves a file <em>up to</em> the master, while <a class="reference internal" href="#step-FileDownload" title="FileDownload"><tt class="xref bb bb-step docutils literal"><span class="pre">FileDownload</span></tt></a> moves a file <em>down from</em> the master.</p> <p>As an example, let's assume that there is a step which produces an HTML file within the source tree that contains some sort of generated project documentation. We want to move this file to the buildmaster, into a <tt class="file docutils literal"><span class="pre">~/public_html</span></tt> directory, so it can be visible to developers. This file will wind up in the slave-side working directory under the name <tt class="file docutils literal"><span class="pre">docs/reference.html</span></tt>. We want to put it into the master-side <tt class="file docutils literal"><span class="pre">~/public_html/ref.html</span></tt>, and add a link to the HTML status to the uploaded file.</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.shell</span> <span class="kn">import</span> <span class="n">ShellCommand</span> <span class="kn">from</span> <span class="nn">buildbot.steps.transfer</span> <span class="kn">import</span> <span class="n">FileUpload</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">ShellCommand</span><span class="p">(</span><span class="n">command</span><span class="o">=</span><span class="p">[</span><span class="s">"make"</span><span class="p">,</span> <span class="s">"docs"</span><span class="p">]))</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">FileUpload</span><span class="p">(</span><span class="n">slavesrc</span><span class="o">=</span><span class="s">"docs/reference.html"</span><span class="p">,</span> <span class="n">masterdest</span><span class="o">=</span><span class="s">"/home/bb/public_html/ref.html"</span><span class="p">,</span> <span class="n">url</span><span class="o">=</span><span class="s">"http://somesite/~buildbot/ref.html"</span><span class="p">))</span> </pre></div> </div> <p>The <tt class="docutils literal"><span class="pre">masterdest=</span></tt> argument will be passed to <tt class="xref py py-meth docutils literal"><span class="pre">os.path.expanduser</span></tt>, so things like <tt class="docutils literal"><span class="pre">~</span></tt> will be expanded properly. Non-absolute paths will be interpreted relative to the buildmaster's base directory. Likewise, the <tt class="docutils literal"><span class="pre">slavesrc=</span></tt> argument will be expanded and interpreted relative to the builder's working directory.</p> <div class="admonition note"> <p class="first admonition-title">Note</p> <p class="last">The copied file will have the same permissions on the master as on the slave, look at the <tt class="docutils literal"><span class="pre">mode=</span></tt> parameter to set it differently.</p> </div> <p>To move a file from the master to the slave, use the <a class="reference internal" href="#step-FileDownload" title="FileDownload"><tt class="xref bb bb-step docutils literal"><span class="pre">FileDownload</span></tt></a> command. For example, let's assume that some step requires a configuration file that, for whatever reason, could not be recorded in the source code repository or generated on the buildslave side:</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.shell</span> <span class="kn">import</span> <span class="n">ShellCommand</span> <span class="kn">from</span> <span class="nn">buildbot.steps.transfer</span> <span class="kn">import</span> <span class="n">FileDownload</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">FileDownload</span><span class="p">(</span><span class="n">mastersrc</span><span class="o">=</span><span class="s">"~/todays_build_config.txt"</span><span class="p">,</span> <span class="n">slavedest</span><span class="o">=</span><span class="s">"build_config.txt"</span><span class="p">))</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">ShellCommand</span><span class="p">(</span><span class="n">command</span><span class="o">=</span><span class="p">[</span><span class="s">"make"</span><span class="p">,</span> <span class="s">"config"</span><span class="p">]))</span> </pre></div> </div> <p>Like <a class="reference internal" href="#step-FileUpload" title="FileUpload"><tt class="xref bb bb-step docutils literal"><span class="pre">FileUpload</span></tt></a>, the <tt class="docutils literal"><span class="pre">mastersrc=</span></tt> argument is interpreted relative to the buildmaster's base directory, and the <tt class="docutils literal"><span class="pre">slavedest=</span></tt> argument is relative to the builder's working directory. If the buildslave is running in <tt class="file docutils literal"><span class="pre">~buildslave</span></tt>, and the builder's <tt class="docutils literal"><span class="pre">builddir</span></tt> is something like <tt class="file docutils literal"><span class="pre">tests-i386</span></tt>, then the workdir is going to be <tt class="file docutils literal"><span class="pre">~buildslave/tests-i386/build</span></tt>, and a <tt class="docutils literal"><span class="pre">slavedest=</span></tt> of <tt class="file docutils literal"><span class="pre">foo/bar.html</span></tt> will get put in <tt class="file docutils literal"><span class="pre">~buildslave/tests-i386/build/foo/bar.html</span></tt>. Both of these commands will create any missing intervening directories.</p> <div class="section" id="other-parameters"> <h3>Other Parameters<a class="headerlink" href="#other-parameters" title="Permalink to this headline">¶</a></h3> <p>The <tt class="docutils literal"><span class="pre">maxsize=</span></tt> argument lets you set a maximum size for the file to be transferred. This may help to avoid surprises: transferring a 100MB coredump when you were expecting to move a 10kB status file might take an awfully long time. The <tt class="docutils literal"><span class="pre">blocksize=</span></tt> argument controls how the file is sent over the network: larger blocksizes are slightly more efficient but also consume more memory on each end, and there is a hard-coded limit of about 640kB.</p> <p>The <tt class="docutils literal"><span class="pre">mode=</span></tt> argument allows you to control the access permissions of the target file, traditionally expressed as an octal integer. The most common value is probably <tt class="docutils literal"><span class="pre">0755</span></tt>, which sets the <cite>x</cite> executable bit on the file (useful for shell scripts and the like). The default value for <tt class="docutils literal"><span class="pre">mode=</span></tt> is None, which means the permission bits will default to whatever the umask of the writing process is. The default umask tends to be fairly restrictive, but at least on the buildslave you can make it less restrictive with a --umask command-line option at creation time (<a class="reference internal" href="installation.html#buildslave-options"><em>Buildslave Options</em></a>).</p> <p>The <tt class="docutils literal"><span class="pre">keepstamp=</span></tt> argument is a boolean that, when <tt class="docutils literal"><span class="pre">True</span></tt>, forces the modified and accessed time of the destination file to match the times of the source file. When <tt class="docutils literal"><span class="pre">False</span></tt> (the default), the modified and accessed times of the destination file are set to the current time on the buildmaster.</p> <p>The <tt class="docutils literal"><span class="pre">url=</span></tt> argument allows you to specify an url that will be displayed in the HTML status. The title of the url will be the name of the item transferred (directory for <tt class="xref py py-class docutils literal"><span class="pre">DirectoryUpload</span></tt> or file for <tt class="xref py py-class docutils literal"><span class="pre">FileUpload</span></tt>). This allows the user to add a link to the uploaded item if that one is uploaded to an accessible place.</p> </div> <div class="section" id="transfering-directories"> <span id="step-DirectoryUpload"></span><h3>Transfering Directories<a class="headerlink" href="#transfering-directories" title="Permalink to this headline">¶</a></h3> <dl class="class"> <dt id="buildbot.steps.source.buildbot.steps.transfer.DirectoryUpload"> <em class="property">class </em><tt class="descclassname">buildbot.steps.transfer.</tt><tt class="descname">DirectoryUpload</tt><a class="headerlink" href="#buildbot.steps.source.buildbot.steps.transfer.DirectoryUpload" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <p>To transfer complete directories from the buildslave to the master, there is a <tt class="xref py py-class docutils literal"><span class="pre">BuildStep</span></tt> named <a class="reference internal" href="#step-DirectoryUpload" title="DirectoryUpload"><tt class="xref bb bb-step docutils literal"><span class="pre">DirectoryUpload</span></tt></a>. It works like <a class="reference internal" href="#step-FileUpload" title="FileUpload"><tt class="xref bb bb-step docutils literal"><span class="pre">FileUpload</span></tt></a>, just for directories. However it does not support the <tt class="docutils literal"><span class="pre">maxsize</span></tt>, <tt class="docutils literal"><span class="pre">blocksize</span></tt> and <tt class="docutils literal"><span class="pre">mode</span></tt> arguments. As an example, let's assume an generated project documentation, which consists of many files (like the output of <strong class="command">doxygen</strong> or <strong class="command">epydoc</strong>). We want to move the entire documentation to the buildmaster, into a <tt class="file docutils literal"><span class="pre">~/public_html/docs</span></tt> directory, and add a link to the uploaded documentation on the HTML status page. On the slave-side the directory can be found under <tt class="file docutils literal"><span class="pre">docs</span></tt>:</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.shell</span> <span class="kn">import</span> <span class="n">ShellCommand</span> <span class="kn">from</span> <span class="nn">buildbot.steps.transfer</span> <span class="kn">import</span> <span class="n">DirectoryUpload</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">ShellCommand</span><span class="p">(</span><span class="n">command</span><span class="o">=</span><span class="p">[</span><span class="s">"make"</span><span class="p">,</span> <span class="s">"docs"</span><span class="p">]))</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">DirectoryUpload</span><span class="p">(</span><span class="n">slavesrc</span><span class="o">=</span><span class="s">"docs"</span><span class="p">,</span> <span class="n">masterdest</span><span class="o">=</span><span class="s">"~/public_html/docs"</span><span class="p">,</span> <span class="n">url</span><span class="o">=</span><span class="s">"~buildbot/docs"</span><span class="p">))</span> </pre></div> </div> <p>The <a class="reference internal" href="#step-DirectoryUpload" title="DirectoryUpload"><tt class="xref bb bb-step docutils literal"><span class="pre">DirectoryUpload</span></tt></a> step will create all necessary directories and transfers empty directories, too.</p> <p>The <tt class="docutils literal"><span class="pre">maxsize</span></tt> and <tt class="docutils literal"><span class="pre">blocksize</span></tt> parameters are the same as for <a class="reference internal" href="#step-FileUpload" title="FileUpload"><tt class="xref bb bb-step docutils literal"><span class="pre">FileUpload</span></tt></a>, although note that the size of the transferred data is implementation-dependent, and probably much larger than you expect due to the encoding used (currently tar).</p> <p>The optional <tt class="docutils literal"><span class="pre">compress</span></tt> argument can be given as <tt class="docutils literal"><span class="pre">'gz'</span></tt> or <tt class="docutils literal"><span class="pre">'bz2'</span></tt> to compress the datastream.</p> <div class="admonition note"> <p class="first admonition-title">Note</p> <p class="last">The permissions on the copied files will be the same on the master as originally on the slave, see <a class="reference internal" href="installation.html#cmdoption-buildslave-create-slave--umask"><em class="xref std std-option">buildslave create-slave --umask</em></a> to change the default one.</p> </div> <span class="target" id="step-StringDownload"></span><span class="target" id="step-JSONStringDownload"></span></div> </div> <div class="section" id="transfering-strings"> <span id="step-JSONPropertiesDownload"></span><h2>Transfering Strings<a class="headerlink" href="#transfering-strings" title="Permalink to this headline">¶</a></h2> <dl class="class"> <dt id="buildbot.steps.source.buildbot.steps.transfer.StringDownload"> <em class="property">class </em><tt class="descclassname">buildbot.steps.transfer.</tt><tt class="descname">StringDownload</tt><a class="headerlink" href="#buildbot.steps.source.buildbot.steps.transfer.StringDownload" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <dl class="class"> <dt id="buildbot.steps.source.buildbot.steps.transfer.JSONStringDownload"> <em class="property">class </em><tt class="descclassname">buildbot.steps.transfer.</tt><tt class="descname">JSONStringDownload</tt><a class="headerlink" href="#buildbot.steps.source.buildbot.steps.transfer.JSONStringDownload" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <dl class="class"> <dt id="buildbot.steps.source.buildbot.steps.transfer.JSONPropertiesDownload"> <em class="property">class </em><tt class="descclassname">buildbot.steps.transfer.</tt><tt class="descname">JSONPropertiesDownload</tt><a class="headerlink" href="#buildbot.steps.source.buildbot.steps.transfer.JSONPropertiesDownload" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <p>Sometimes it is useful to transfer a calculated value from the master to the slave. Instead of having to create a temporary file and then use FileDownload, you can use one of the string download steps.</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.transfer</span> <span class="kn">import</span> <span class="n">StringDownload</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">StringDownload</span><span class="p">(</span><span class="n">Interpolate</span><span class="p">(</span><span class="s">"%(src::branch)s-%(prop:got_revision)s</span><span class="se">\n</span><span class="s">"</span><span class="p">),</span> <span class="n">slavedest</span><span class="o">=</span><span class="s">"buildid.txt"</span><span class="p">))</span> </pre></div> </div> <p><a class="reference internal" href="#step-StringDownload" title="StringDownload"><tt class="xref bb bb-step docutils literal"><span class="pre">StringDownload</span></tt></a> works just like <a class="reference internal" href="#step-FileDownload" title="FileDownload"><tt class="xref bb bb-step docutils literal"><span class="pre">FileDownload</span></tt></a> except it takes a single argument, <tt class="docutils literal"><span class="pre">s</span></tt>, representing the string to download instead of a <tt class="docutils literal"><span class="pre">mastersrc</span></tt> argument.</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.transfer</span> <span class="kn">import</span> <span class="n">JSONStringDownload</span> <span class="n">buildinfo</span> <span class="o">=</span> <span class="p">{</span> <span class="n">branch</span><span class="p">:</span> <span class="n">Property</span><span class="p">(</span><span class="s">'branch'</span><span class="p">),</span> <span class="n">got_revision</span><span class="p">:</span> <span class="n">Property</span><span class="p">(</span><span class="s">'got_revision'</span><span class="p">)</span> <span class="p">}</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">JSONStringDownload</span><span class="p">(</span><span class="n">buildinfo</span><span class="p">,</span> <span class="n">slavedest</span><span class="o">=</span><span class="s">"buildinfo.json"</span><span class="p">))</span> </pre></div> </div> <p><a class="reference internal" href="#step-JSONStringDownload" title="JSONStringDownload"><tt class="xref bb bb-step docutils literal"><span class="pre">JSONStringDownload</span></tt></a> is similar, except it takes an <tt class="docutils literal"><span class="pre">o</span></tt> argument, which must be JSON serializable, and transfers that as a JSON-encoded string to the slave.</p> <div class="highlight-python" id="index-26"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.transfer</span> <span class="kn">import</span> <span class="n">JSONPropertiesDownload</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">JSONPropertiesDownload</span><span class="p">(</span><span class="n">slavedest</span><span class="o">=</span><span class="s">"build-properties.json"</span><span class="p">))</span> </pre></div> </div> <p><a class="reference internal" href="#step-JSONPropertiesDownload" title="JSONPropertiesDownload"><tt class="xref bb bb-step docutils literal"><span class="pre">JSONPropertiesDownload</span></tt></a> transfers a json-encoded string that represents a dictionary where properties maps to a dictionary of build property <tt class="docutils literal"><span class="pre">name</span></tt> to property <tt class="docutils literal"><span class="pre">value</span></tt>; and <tt class="docutils literal"><span class="pre">sourcestamp</span></tt> represents the build's sourcestamp.</p> </div> <div class="section" id="running-commands-on-the-master"> <span id="step-MasterShellCommand"></span><h2>Running Commands on the Master<a class="headerlink" href="#running-commands-on-the-master" title="Permalink to this headline">¶</a></h2> <dl class="class"> <dt id="buildbot.steps.source.buildbot.steps.master.MasterShellCommand"> <em class="property">class </em><tt class="descclassname">buildbot.steps.master.</tt><tt class="descname">MasterShellCommand</tt><a class="headerlink" href="#buildbot.steps.source.buildbot.steps.master.MasterShellCommand" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <p>Occasionally, it is useful to execute some task on the master, for example to create a directory, deploy a build result, or trigger some other centralized processing. This is possible, in a limited fashion, with the <a class="reference internal" href="#step-MasterShellCommand" title="MasterShellCommand"><tt class="xref bb bb-step docutils literal"><span class="pre">MasterShellCommand</span></tt></a> step.</p> <p>This step operates similarly to a regular <a class="reference internal" href="#step-ShellCommand" title="ShellCommand"><tt class="xref bb bb-step docutils literal"><span class="pre">ShellCommand</span></tt></a>, but executes on the master, instead of the slave. To be clear, the enclosing <tt class="xref py py-class docutils literal"><span class="pre">Build</span></tt> object must still have a slave object, just as for any other step -- only, in this step, the slave does not do anything.</p> <p>In this example, the step renames a tarball based on the day of the week.</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.transfer</span> <span class="kn">import</span> <span class="n">FileUpload</span> <span class="kn">from</span> <span class="nn">buildbot.steps.master</span> <span class="kn">import</span> <span class="n">MasterShellCommand</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">FileUpload</span><span class="p">(</span><span class="n">slavesrc</span><span class="o">=</span><span class="s">"widgetsoft.tar.gz"</span><span class="p">,</span> <span class="n">masterdest</span><span class="o">=</span><span class="s">"/var/buildoutputs/widgetsoft-new.tar.gz"</span><span class="p">))</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">MasterShellCommand</span><span class="p">(</span><span class="n">command</span><span class="o">=</span><span class="s">"""</span> <span class="s"> cd /var/buildoutputs;</span> <span class="s"> mv widgetsoft-new.tar.gz widgetsoft-`date +%a`.tar.gz"""</span><span class="p">))</span> </pre></div> </div> <div class="admonition note"> <p class="first admonition-title">Note</p> <p class="last">By default, this step passes a copy of the buildmaster's environment variables to the subprocess. To pass an explicit environment instead, add an <tt class="docutils literal"><span class="pre">env={..}</span></tt> argument.</p> </div> <p>Environment variables constructed using the <tt class="docutils literal"><span class="pre">env</span></tt> argument support expansion so that if you just want to prepend <tt class="file docutils literal"><span class="pre">/home/buildbot/bin</span></tt> to the <span class="target" id="index-27"></span><tt class="xref std std-envvar docutils literal"><span class="pre">PATH</span></tt> environment variable, you can do it by putting the value <tt class="docutils literal"><span class="pre">${PATH}</span></tt> at the end of the value like in the example below. Variables that don't exist on the master will be replaced by <tt class="docutils literal"><span class="pre">""</span></tt>.</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.master</span> <span class="kn">import</span> <span class="n">MasterShellCommand</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">MasterShellCommand</span><span class="p">(</span> <span class="n">command</span><span class="o">=</span><span class="p">[</span><span class="s">"make"</span><span class="p">,</span> <span class="s">"www"</span><span class="p">],</span> <span class="n">env</span><span class="o">=</span><span class="p">{</span><span class="s">'PATH'</span><span class="p">:</span> <span class="p">[</span><span class="s">"/home/buildbot/bin"</span><span class="p">,</span> <span class="s">"${PATH}"</span><span class="p">]}))</span> </pre></div> </div> <p>Note that environment values must be strings (or lists that are turned into strings). In particular, numeric properties such as <tt class="docutils literal"><span class="pre">buildnumber</span></tt> must be substituted using <a class="reference internal" href="cfg-properties.html#interpolate"><em>Interpolate</em></a>.</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">interruptSignal</span></tt></dt> <dd>(optional) Signal to use to end the process, if the step is interrupted.</dd> </dl> <div class="section" id="logrenderable"> <span id="step-LogRenderable"></span><h3>LogRenderable<a class="headerlink" href="#logrenderable" title="Permalink to this headline">¶</a></h3> <dl class="class"> <dt id="buildbot.steps.source.buildbot.steps.master.LogRenderable"> <em class="property">class </em><tt class="descclassname">buildbot.steps.master.</tt><tt class="descname">LogRenderable</tt><a class="headerlink" href="#buildbot.steps.source.buildbot.steps.master.LogRenderable" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <p>This build step takes content which can be renderable and logs it in a pretty-printed format. It can be useful for debugging properties during a build.</p> </div> </div> <div class="section" id="setting-properties"> <span id="index-28"></span><span id="id19"></span><h2>Setting Properties<a class="headerlink" href="#setting-properties" title="Permalink to this headline">¶</a></h2> <p>These steps set properties on the master based on information from the slave.</p> <div class="section" id="setproperty"> <span id="step-setproperty"></span><span id="step-SetProperty"></span><h3>SetProperty<a class="headerlink" href="#setproperty" title="Permalink to this headline">¶</a></h3> <dl class="class"> <dt id="buildbot.steps.source.buildbot.steps.master.SetProperty"> <em class="property">class </em><tt class="descclassname">buildbot.steps.master.</tt><tt class="descname">SetProperty</tt><a class="headerlink" href="#buildbot.steps.source.buildbot.steps.master.SetProperty" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <p>SetProperty takes two arguments of <tt class="docutils literal"><span class="pre">property</span></tt> and <tt class="docutils literal"><span class="pre">value</span></tt> where the <tt class="docutils literal"><span class="pre">value</span></tt> is to be assigned to the <tt class="docutils literal"><span class="pre">property</span></tt> key. It is usually called with the <tt class="docutils literal"><span class="pre">value</span></tt> argument being specifed as a <a class="reference internal" href="cfg-properties.html#interpolate"><em>Interpolate</em></a> object which allows the value to be built from other property values:</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.master</span> <span class="kn">import</span> <span class="n">SetProperty</span> <span class="kn">from</span> <span class="nn">buildbot.process.properties</span> <span class="kn">import</span> <span class="n">Interpolate</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">SetProperty</span><span class="p">(</span><span class="nb">property</span><span class="o">=</span><span class="s">"SomeProperty"</span><span class="p">,</span> <span class="n">value</span><span class="o">=</span><span class="n">Interpolate</span><span class="p">(</span><span class="s">"sch=%(prop:scheduler)s, slave=%(prop:slavename)s"</span><span class="p">)))</span> </pre></div> </div> </div> <div class="section" id="setpropertyfromcommand"> <span id="step-SetPropertyFromCommand"></span><h3>SetPropertyFromCommand<a class="headerlink" href="#setpropertyfromcommand" title="Permalink to this headline">¶</a></h3> <dl class="class"> <dt id="buildbot.steps.source.buildbot.steps.shell.SetPropertyFromCommand"> <em class="property">class </em><tt class="descclassname">buildbot.steps.shell.</tt><tt class="descname">SetPropertyFromCommand</tt><a class="headerlink" href="#buildbot.steps.source.buildbot.steps.shell.SetPropertyFromCommand" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <p>This buildstep is similar to <a class="reference internal" href="#step-ShellCommand" title="ShellCommand"><tt class="xref bb bb-step docutils literal"><span class="pre">ShellCommand</span></tt></a>, except that it captures the output of the command into a property. It is usually used like this:</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps</span> <span class="kn">import</span> <span class="n">shell</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">shell</span><span class="o">.</span><span class="n">SetPropertyFromCommand</span><span class="p">(</span><span class="n">command</span><span class="o">=</span><span class="s">"uname -a"</span><span class="p">,</span> <span class="nb">property</span><span class="o">=</span><span class="s">"uname"</span><span class="p">))</span> </pre></div> </div> <p>This runs <tt class="docutils literal"><span class="pre">uname</span> <span class="pre">-a</span></tt> and captures its stdout, stripped of leading and trailing whitespace, in the property <tt class="docutils literal"><span class="pre">uname</span></tt>. To avoid stripping, add <tt class="docutils literal"><span class="pre">strip=False</span></tt>.</p> <p>The <tt class="docutils literal"><span class="pre">property</span></tt> argument can be specified as a <a class="reference internal" href="cfg-properties.html#interpolate"><em>Interpolate</em></a> object, allowing the property name to be built from other property values.</p> <p>The more advanced usage allows you to specify a function to extract properties from the command output. Here you can use regular expressions, string interpolation, or whatever you would like. In this form, <tt class="xref py py-func docutils literal"><span class="pre">extract_fn</span></tt> should be passed, and not <tt class="xref py py-class docutils literal"><span class="pre">Property</span></tt>. The <tt class="xref py py-func docutils literal"><span class="pre">extract_fn</span></tt> function is called with three arguments: the exit status of the command, its standard output as a string, and its standard error as a string. It should return a dictionary containing all new properties.</p> <div class="highlight-python"><div class="highlight"><pre><span class="k">def</span> <span class="nf">glob2list</span><span class="p">(</span><span class="n">rc</span><span class="p">,</span> <span class="n">stdout</span><span class="p">,</span> <span class="n">stderr</span><span class="p">):</span> <span class="n">jpgs</span> <span class="o">=</span> <span class="p">[</span> <span class="n">l</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span> <span class="k">for</span> <span class="n">l</span> <span class="ow">in</span> <span class="n">stdout</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s">'</span><span class="se">\n</span><span class="s">'</span><span class="p">)</span> <span class="p">]</span> <span class="k">return</span> <span class="p">{</span> <span class="s">'jpgs'</span> <span class="p">:</span> <span class="n">jpgs</span> <span class="p">}</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">SetPropertyFromCommand</span><span class="p">(</span><span class="n">command</span><span class="o">=</span><span class="s">"ls -1 *.jpg"</span><span class="p">,</span> <span class="n">extract_fn</span><span class="o">=</span><span class="n">glob2list</span><span class="p">))</span> </pre></div> </div> <p>Note that any ordering relationship of the contents of stdout and stderr is lost. For example, given</p> <div class="highlight-python"><div class="highlight"><pre><span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">SetPropertyFromCommand</span><span class="p">(</span> <span class="n">command</span><span class="o">=</span><span class="s">"echo output1; echo error >&2; echo output2"</span><span class="p">,</span> <span class="n">extract_fn</span><span class="o">=</span><span class="n">my_extract</span><span class="p">))</span> </pre></div> </div> <p>Then <tt class="docutils literal"><span class="pre">my_extract</span></tt> will see <tt class="docutils literal"><span class="pre">stdout="output1\noutput2\n"</span></tt> and <tt class="docutils literal"><span class="pre">stderr="error\n"</span></tt>.</p> <span class="target" id="step-SetPropertiesFromEnv"></span><dl class="class"> <dt id="buildbot.steps.source.buildbot.steps.slave.SetPropertiesFromEnv"> <em class="property">class </em><tt class="descclassname">buildbot.steps.slave.</tt><tt class="descname">SetPropertiesFromEnv</tt><a class="headerlink" href="#buildbot.steps.source.buildbot.steps.slave.SetPropertiesFromEnv" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> </div> <div class="section" id="setpropertiesfromenv"> <h3>SetPropertiesFromEnv<a class="headerlink" href="#setpropertiesfromenv" title="Permalink to this headline">¶</a></h3> <p>Buildbot slaves (later than version 0.8.3) provide their environment variables to the master on connect. These can be copied into Buildbot properties with the <a class="reference internal" href="#step-SetPropertiesFromEnv" title="SetPropertiesFromEnv"><tt class="xref bb bb-step docutils literal"><span class="pre">SetPropertiesFromEnv</span></tt></a> step. Pass a variable or list of variables in the <tt class="docutils literal"><span class="pre">variables</span></tt> parameter, then simply use the values as properties in a later step.</p> <p>Note that on Windows, environment variables are case-insensitive, but Buildbot property names are case sensitive. The property will have exactly the variable name you specify, even if the underlying environment variable is capitalized differently. If, for example, you use <tt class="docutils literal"><span class="pre">variables=['Tmp']</span></tt>, the result will be a property named <tt class="docutils literal"><span class="pre">Tmp</span></tt>, even though the environment variable is displayed as <span class="target" id="index-29"></span><tt class="xref std std-envvar docutils literal"><span class="pre">TMP</span></tt> in the Windows GUI.</p> <div class="highlight-python"><pre>from buildbot.steps.slave import SetPropertiesFromEnv from buildbot.steps.shell import Compile f.addStep(SetPropertiesFromEnv(variables=["SOME_JAVA_LIB_HOME", "JAVAC"])) f.addStep(Compile(commands=[Interpolate("%(prop:JAVAC)s"), "-cp", Interpolate("%(prop:SOME_JAVA_LIB_HOME)s")))</pre> </div> <p>Note that this step requires that the Buildslave be at least version 0.8.3. For previous versions, no environment variables are available (the slave environment will appear to be empty).</p> <span class="target" id="index-30"></span></div> </div> <div class="section" id="triggering-schedulers"> <span id="step-Trigger"></span><span id="id20"></span><h2>Triggering Schedulers<a class="headerlink" href="#triggering-schedulers" title="Permalink to this headline">¶</a></h2> <p>The counterpart to the Triggerable described in section <a class="reference internal" href="cfg-schedulers.html#sched-Triggerable" title="Triggerable"><tt class="xref bb bb-sched docutils literal"><span class="pre">Triggerable</span></tt></a> is the <a class="reference internal" href="#step-Trigger" title="Trigger"><tt class="xref bb bb-step docutils literal"><span class="pre">Trigger</span></tt></a> build step:</p> <div class="highlight-python"><pre>from buildbot.steps.trigger import Trigger f.addStep(Trigger(schedulerNames=['build-prep'], waitForFinish=True, updateSourceStamp=True, set_properties={ 'quick' : False })</pre> </div> <p>The <tt class="docutils literal"><span class="pre">schedulerNames=</span></tt> argument lists the <a class="reference internal" href="cfg-schedulers.html#sched-Triggerable" title="Triggerable"><tt class="xref bb bb-sched docutils literal"><span class="pre">Triggerable</span></tt></a> schedulers that should be triggered when this step is executed. Note that it is possible, but not advisable, to create a cycle where a build continually triggers itself, because the schedulers are specified by name.</p> <p>If <tt class="docutils literal"><span class="pre">waitForFinish</span></tt> is <tt class="docutils literal"><span class="pre">True</span></tt>, then the step will not finish until all of the builds from the triggered schedulers have finished. Hyperlinks are added to the waterfall and the build detail web pages for each triggered build. If this argument is <tt class="docutils literal"><span class="pre">False</span></tt> (the default) or not given, then the buildstep succeeds immediately after triggering the schedulers.</p> <p>The SourceStamps to use for the triggered build are controlled by the arguments <tt class="docutils literal"><span class="pre">updateSourceStamp</span></tt>, <tt class="docutils literal"><span class="pre">alwaysUseLatest</span></tt>, and <tt class="docutils literal"><span class="pre">sourceStamps</span></tt>. If <tt class="docutils literal"><span class="pre">updateSourceStamp</span></tt> is <tt class="docutils literal"><span class="pre">True</span></tt> (the default), then step updates the <tt class="xref py py-class docutils literal"><span class="pre">SourceStamp`s</span> <span class="pre">given</span> <span class="pre">to</span> <span class="pre">the</span> <span class="pre">:bb:sched:`Triggerable</span></tt> schedulers to include <tt class="docutils literal"><span class="pre">got_revision</span></tt> (the revision actually used in this build) as <tt class="docutils literal"><span class="pre">revision</span></tt> (the revision to use in the triggered builds). This is useful to ensure that all of the builds use exactly the same <tt class="xref py py-class docutils literal"><span class="pre">SourceStamp`s,</span> <span class="pre">even</span> <span class="pre">if</span> <span class="pre">other</span> <span class="pre">:class:`Change</span></tt>s have occurred while the build was running. If <tt class="docutils literal"><span class="pre">updateSourceStamp</span></tt> is False (and neither of the other arguments are specified), then the exact same SourceStamps are used. If <tt class="docutils literal"><span class="pre">alwaysUseLatest</span></tt> is True, then no SourceStamps are given, corresponding to using the latest revisions of the repositories specified in the Source steps. This is useful if the triggered builds use to a different source repository. The argument <tt class="docutils literal"><span class="pre">sourceStamps</span></tt> accepts a list of dictionaries containing the keys <tt class="docutils literal"><span class="pre">branch</span></tt>, <tt class="docutils literal"><span class="pre">revision</span></tt>, <tt class="docutils literal"><span class="pre">repository</span></tt>, <tt class="docutils literal"><span class="pre">project</span></tt>, and optionally <tt class="docutils literal"><span class="pre">patch_level</span></tt>, <tt class="docutils literal"><span class="pre">patch_body</span></tt>, <tt class="docutils literal"><span class="pre">patch_subdir</span></tt>, <tt class="docutils literal"><span class="pre">patch_author</span></tt> and <tt class="docutils literal"><span class="pre">patch_comment</span></tt> and creates the corresponding SourceStamps. If only one sourceStamp has to be specified then the argument <tt class="docutils literal"><span class="pre">sourceStamp</span></tt> can be used for a dictionary containing the keys mentioned above. The arguments <tt class="docutils literal"><span class="pre">updateSourceStamp</span></tt>, <tt class="docutils literal"><span class="pre">alwaysUseLatest</span></tt>, and <tt class="docutils literal"><span class="pre">sourceStamp</span></tt> can be specified using properties.</p> <p>The <tt class="docutils literal"><span class="pre">set_properties</span></tt> parameter allows control of the properties that are passed to the triggered scheduler. The parameter takes a dictionary mapping property names to values. You may use <a class="reference internal" href="cfg-properties.html#interpolate"><em>Interpolate</em></a> here to dynamically construct new property values. For the simple case of copying a property, this might look like</p> <div class="highlight-python"><div class="highlight"><pre><span class="n">set_properties</span><span class="o">=</span><span class="p">{</span><span class="s">"my_prop1"</span> <span class="p">:</span> <span class="n">Property</span><span class="p">(</span><span class="s">"my_prop1"</span><span class="p">)}</span> </pre></div> </div> <p>The <tt class="docutils literal"><span class="pre">copy_properties</span></tt> parameter, given a list of properties to copy into the new build request, has been deprecated in favor of explicit use of <tt class="docutils literal"><span class="pre">set_properties</span></tt>.</p> </div> <div class="section" id="rpm-related-steps"> <h2>RPM-Related Steps<a class="headerlink" href="#rpm-related-steps" title="Permalink to this headline">¶</a></h2> <p>These steps work with RPMs and spec files.</p> <div class="section" id="rpmbuild"> <span id="step-RpmBuild"></span><h3>RpmBuild<a class="headerlink" href="#rpmbuild" title="Permalink to this headline">¶</a></h3> <p>The <a class="reference internal" href="#step-RpmBuild" title="RpmBuild"><tt class="xref bb bb-step docutils literal"><span class="pre">RpmBuild</span></tt></a> step builds RPMs based on a spec file:</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.package.rpm</span> <span class="kn">import</span> <span class="n">RpmBuild</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">RpmBuild</span><span class="p">(</span><span class="n">specfile</span><span class="o">=</span><span class="s">"proj.spec"</span><span class="p">,</span> <span class="n">dist</span><span class="o">=</span><span class="s">'.el5'</span><span class="p">))</span> </pre></div> </div> <p>The step takes the following parameters</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">specfile</span></tt></dt> <dd>The <tt class="docutils literal"><span class="pre">.spec</span></tt> file to build from</dd> <dt><tt class="docutils literal"><span class="pre">topdir</span></tt></dt> <dd>Definition for <tt class="docutils literal"><span class="pre">_topdir</span></tt>, defaulting to the workdir.</dd> <dt><tt class="docutils literal"><span class="pre">builddir</span></tt></dt> <dd>Definition for <tt class="docutils literal"><span class="pre">_builddir</span></tt>, defaulting to the workdir.</dd> <dt><tt class="docutils literal"><span class="pre">rpmdir</span></tt></dt> <dd>Definition for <tt class="docutils literal"><span class="pre">_rpmdir</span></tt>, defaulting to the workdir.</dd> <dt><tt class="docutils literal"><span class="pre">sourcedir</span></tt></dt> <dd>Definition for <tt class="docutils literal"><span class="pre">_sourcedir</span></tt>, defaulting to the workdir.</dd> <dt><tt class="docutils literal"><span class="pre">srcrpmdir</span></tt></dt> <dd>Definition for <tt class="docutils literal"><span class="pre">_srcrpmdir</span></tt>, defaulting to the workdir.</dd> <dt><tt class="docutils literal"><span class="pre">dist</span></tt></dt> <dd>Distribution to build, used as the definition for <tt class="docutils literal"><span class="pre">_dist</span></tt>.</dd> <dt><tt class="docutils literal"><span class="pre">autoRelease</span></tt></dt> <dd>If true, use the auto-release mechanics.</dd> <dt><tt class="docutils literal"><span class="pre">vcsRevision</span></tt></dt> <dd>If true, use the version-control revision mechanics. This uses the <tt class="docutils literal"><span class="pre">got_revision</span></tt> property to determine the revision and define <tt class="docutils literal"><span class="pre">_revision</span></tt>. Note that this will not work with multi-codebase builds.</dd> </dl> </div> <div class="section" id="rpmlint"> <span id="step-RpmLint"></span><h3>RpmLint<a class="headerlink" href="#rpmlint" title="Permalink to this headline">¶</a></h3> <p>The <a class="reference internal" href="#step-RpmLint" title="RpmLint"><tt class="xref bb bb-step docutils literal"><span class="pre">RpmLint</span></tt></a> step checks for common problems in RPM packages or spec files:</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.package.rpm</span> <span class="kn">import</span> <span class="n">RpmLint</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">RpmLint</span><span class="p">())</span> </pre></div> </div> <p>The step takes the following parameters</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">fileloc</span></tt></dt> <dd>The file or directory to check. In case of a directory, it is recursively searched for RPMs and spec files to check.</dd> <dt><tt class="docutils literal"><span class="pre">config</span></tt></dt> <dd>Path to a rpmlint config file. This is passed as the user configuration file if present.</dd> </dl> </div> <div class="section" id="mock-steps"> <h3>Mock Steps<a class="headerlink" href="#mock-steps" title="Permalink to this headline">¶</a></h3> <p>Mock (<a class="reference external" href="http://fedoraproject.org/wiki/Projects/Mock">http://fedoraproject.org/wiki/Projects/Mock</a>) creates chroots and builds packages in them. It populates the changeroot with a basic system and the packages listed as build requirement. The type of chroot to build is specified with the <tt class="docutils literal"><span class="pre">root</span></tt> parameter. To use mock your buildbot user must be added to the <tt class="docutils literal"><span class="pre">mock</span></tt> group.</p> </div> <div class="section" id="mockbuildsrpm-step"> <span id="step-MockBuildSRPM"></span><h3>MockBuildSRPM Step<a class="headerlink" href="#mockbuildsrpm-step" title="Permalink to this headline">¶</a></h3> <p>The <a class="reference internal" href="#step-MockBuildSRPM" title="MockBuildSRPM"><tt class="xref bb bb-step docutils literal"><span class="pre">MockBuildSRPM</span></tt></a> step builds a SourceRPM based on a spec file and optionally a source directory:</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.package.rpm</span> <span class="kn">import</span> <span class="n">MockBuildSRPM</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">MockBuildSRPM</span><span class="p">(</span><span class="n">root</span><span class="o">=</span><span class="s">'default'</span><span class="p">,</span> <span class="n">spec</span><span class="o">=</span><span class="s">'mypkg.spec'</span><span class="p">))</span> </pre></div> </div> <p>The step takes the following parameters</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">root</span></tt></dt> <dd>Use chroot configuration defined in <tt class="docutils literal"><span class="pre">/etc/mock/<root>.cfg</span></tt>.</dd> <dt><tt class="docutils literal"><span class="pre">resultdir</span></tt></dt> <dd>The directory where the logfiles and the SourceRPM are written to.</dd> <dt><tt class="docutils literal"><span class="pre">spec</span></tt></dt> <dd>Build the SourceRPM from this spec file.</dd> <dt><tt class="docutils literal"><span class="pre">sources</span></tt></dt> <dd>Path to the directory containing the sources, defaulting to <tt class="docutils literal"><span class="pre">.</span></tt>.</dd> </dl> </div> <div class="section" id="mockrebuild-step"> <span id="step-MockRebuild"></span><h3>MockRebuild Step<a class="headerlink" href="#mockrebuild-step" title="Permalink to this headline">¶</a></h3> <p>The <a class="reference internal" href="#step-MockRebuild" title="MockRebuild"><tt class="xref bb bb-step docutils literal"><span class="pre">MockRebuild</span></tt></a> step rebuilds a SourceRPM package:</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.package.rpm</span> <span class="kn">import</span> <span class="n">MockRebuild</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">MockRebuild</span><span class="p">(</span><span class="n">root</span><span class="o">=</span><span class="s">'default'</span><span class="p">,</span> <span class="n">spec</span><span class="o">=</span><span class="s">'mypkg-1.0-1.src.rpm'</span><span class="p">))</span> </pre></div> </div> <p>The step takes the following parameters</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">root</span></tt></dt> <dd>Uses chroot configuration defined in <tt class="docutils literal"><span class="pre">/etc/mock/<root>.cfg</span></tt>.</dd> <dt><tt class="docutils literal"><span class="pre">resultdir</span></tt></dt> <dd>The directory where the logfiles and the SourceRPM are written to.</dd> <dt><tt class="docutils literal"><span class="pre">srpm</span></tt></dt> <dd>The path to the SourceRPM to rebuild.</dd> </dl> </div> </div> <div class="section" id="debian-build-steps"> <h2>Debian Build Steps<a class="headerlink" href="#debian-build-steps" title="Permalink to this headline">¶</a></h2> <div class="section" id="debpbuilder"> <span id="step-DebPbuilder"></span><h3>DebPbuilder<a class="headerlink" href="#debpbuilder" title="Permalink to this headline">¶</a></h3> <p>The <a class="reference internal" href="#step-DebPbuilder" title="DebPbuilder"><tt class="xref bb bb-step docutils literal"><span class="pre">DebPbuilder</span></tt></a> step builds Debian packages within a chroot built by pbuilder. It populates the changeroot with a basic system and the packages listed as build requirement. The type of chroot to build is specified with the <tt class="docutils literal"><span class="pre">distribution</span></tt>, <tt class="docutils literal"><span class="pre">distribution</span></tt> and <tt class="docutils literal"><span class="pre">mirror</span></tt> parameter. To use pbuilder your buildbot must have the right to run pbuilder as root through sudo.</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.package.deb.pbuilder</span> <span class="kn">import</span> <span class="n">DebPbuilder</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">DebPbuilder</span><span class="p">())</span> </pre></div> </div> <p>The step takes the following parameters</p> <dl class="docutils"> <dt><tt class="docutils literal"><span class="pre">architecture</span></tt></dt> <dd>Architecture to build chroot for.</dd> <dt><tt class="docutils literal"><span class="pre">distribution</span></tt></dt> <dd>Name, or nickname, of the distribution. Defaults to 'stable'.</dd> <dt><tt class="docutils literal"><span class="pre">basetgz</span></tt></dt> <dd>Path of the basetgz to use for building.</dd> <dt><tt class="docutils literal"><span class="pre">mirror</span></tt></dt> <dd>URL of the mirror used to download the packages from.</dd> <dt><tt class="docutils literal"><span class="pre">extrapackages</span></tt></dt> <dd>List if packages to install in addition to the base system.</dd> <dt><tt class="docutils literal"><span class="pre">keyring</span></tt></dt> <dd>Path to a gpg keyring to verify the downloaded packages. This is necessary if you build for a foreign distribution.</dd> <dt><tt class="docutils literal"><span class="pre">components</span></tt></dt> <dd>Repos to activate for chroot building.</dd> </dl> </div> <div class="section" id="debcowbuilder"> <span id="step-DebCowbuilder"></span><h3>DebCowbuilder<a class="headerlink" href="#debcowbuilder" title="Permalink to this headline">¶</a></h3> <p>The <a class="reference internal" href="#step-DebCowbuilder" title="DebCowbuilder"><tt class="xref bb bb-step docutils literal"><span class="pre">DebCowbuilder</span></tt></a> step is a subclass of <a class="reference internal" href="#step-DebPbuilder" title="DebPbuilder"><tt class="xref bb bb-step docutils literal"><span class="pre">DebPbuilder</span></tt></a>, which use cowbuilder instead of pbuilder.</p> </div> <div class="section" id="deblintian"> <span id="step-DebLintian"></span><h3>DebLintian<a class="headerlink" href="#deblintian" title="Permalink to this headline">¶</a></h3> <p>The <a class="reference internal" href="#step-DebLintian" title="DebLintian"><tt class="xref bb bb-step docutils literal"><span class="pre">DebLintian</span></tt></a> step checks a build .deb for bugs and policy violations. The packages or changes file to test is specified in <tt class="docutils literal"><span class="pre">fileloc</span></tt></p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.package.deb.lintian</span> <span class="kn">import</span> <span class="n">DebLintian</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">DebLintian</span><span class="p">(</span><span class="n">fileloc</span><span class="o">=</span><span class="n">Interpolate</span><span class="p">(</span><span class="s">"%(prop:deb-changes)s"</span><span class="p">)))</span> </pre></div> </div> </div> </div> <div class="section" id="miscellaneous-buildsteps"> <h2>Miscellaneous BuildSteps<a class="headerlink" href="#miscellaneous-buildsteps" title="Permalink to this headline">¶</a></h2> <p>A number of steps do not fall into any particular category.</p> <div class="section" id="hlint"> <span id="step-HLint"></span><h3>HLint<a class="headerlink" href="#hlint" title="Permalink to this headline">¶</a></h3> <p>The <a class="reference internal" href="#step-HLint" title="HLint"><tt class="xref bb bb-step docutils literal"><span class="pre">HLint</span></tt></a> step runs Twisted Lore, a lint-like checker over a set of <tt class="docutils literal"><span class="pre">.xhtml</span></tt> files. Any deviations from recommended style is flagged and put in the output log.</p> <p>The step looks at the list of changes in the build to determine which files to check - it does not check all files. It specifically excludes any <tt class="docutils literal"><span class="pre">.xhtml</span></tt> files in the top-level <tt class="docutils literal"><span class="pre">sandbox/</span></tt> directory.</p> <p>The step takes a single, optional, parameter: <tt class="docutils literal"><span class="pre">python</span></tt>. This specifies the Python executable to use to run Lore.</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.python_twisted</span> <span class="kn">import</span> <span class="n">HLint</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">HLint</span><span class="p">())</span> </pre></div> </div> </div> <div class="section" id="maxq"> <h3>MaxQ<a class="headerlink" href="#maxq" title="Permalink to this headline">¶</a></h3> <p id="step-MaxQ">MaxQ (<a class="reference external" href="http://maxq.tigris.org/">http://maxq.tigris.org/</a>) is a web testing tool that allows you to record HTTP sessions and play them back. The <a class="reference internal" href="#step-MaxQ" title="MaxQ"><tt class="xref bb bb-step docutils literal"><span class="pre">MaxQ</span></tt></a> step runs this framework.</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.steps.maxq</span> <span class="kn">import</span> <span class="n">MaxQ</span> <span class="n">f</span><span class="o">.</span><span class="n">addStep</span><span class="p">(</span><span class="n">MaxQ</span><span class="p">(</span><span class="n">testdir</span><span class="o">=</span><span class="s">'tests/'</span><span class="p">))</span> </pre></div> </div> <p>The single argument, <tt class="docutils literal"><span class="pre">testdir</span></tt>, specifies where the tests should be run. This directory will be passed to the <tt class="docutils literal"><span class="pre">run_maxq.py</span></tt> command, and the results analyzed.</p> </div> </div> </div> </div> </div> </div> </div> <div class="sidebar"> <h3>Table Of Contents</h3> <ul class="current"> <li class="toctree-l1"><a class="reference internal" href="../tutorial/index.html">Buildbot Tutorial</a></li> <li class="toctree-l1 current"><a class="reference internal" href="index.html">Buildbot Manual</a><ul class="current"> <li class="toctree-l2"><a class="reference internal" href="introduction.html">Introduction</a></li> <li class="toctree-l2"><a class="reference internal" href="installation.html">Installation</a></li> <li class="toctree-l2"><a class="reference internal" href="concepts.html">Concepts</a></li> <li class="toctree-l2 current"><a class="reference internal" href="configuration.html">Configuration</a><ul class="current"> <li class="toctree-l3"><a class="reference internal" href="cfg-intro.html">Configuring Buildbot</a></li> <li class="toctree-l3"><a class="reference internal" href="cfg-global.html">Global Configuration</a></li> <li class="toctree-l3"><a class="reference internal" href="cfg-changesources.html">Change Sources</a></li> <li class="toctree-l3"><a class="reference internal" href="cfg-schedulers.html">Schedulers</a></li> <li class="toctree-l3"><a class="reference internal" href="cfg-buildslaves.html">Buildslaves</a></li> <li class="toctree-l3"><a class="reference internal" href="cfg-builders.html">Builder Configuration</a></li> <li class="toctree-l3"><a class="reference internal" href="cfg-buildfactories.html">Build Factories</a></li> <li class="toctree-l3"><a class="reference internal" href="cfg-properties.html">Properties</a></li> <li class="toctree-l3 current"><a class="current reference internal" href="">Build Steps</a><ul class="simple"> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="cfg-interlocks.html">Interlocks</a></li> <li class="toctree-l3"><a class="reference internal" href="cfg-statustargets.html">Status Targets</a></li> </ul> </li> <li class="toctree-l2"><a class="reference internal" href="customization.html">Customization</a></li> <li class="toctree-l2"><a class="reference internal" href="cmdline.html">Command-line Tool</a></li> <li class="toctree-l2"><a class="reference internal" href="resources.html">Resources</a></li> <li class="toctree-l2"><a class="reference internal" href="optimization.html">Optimization</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="../developer/index.html">Buildbot Development</a></li> <li class="toctree-l1"><a class="reference internal" href="../relnotes/index.html">Release Notes for Buildbot v0.8.8</a></li> </ul> <h3 style="margin-top: 1.5em;">Search</h3> <form class="search" action="../search.html" method="get"> <input type="text" name="q" /> <input type="submit" value="Go" /> <input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="area" value="default" /> </form> <p class="searchtip" style="font-size: 90%"> Enter search terms or a module, class or function name. </p> </div> <div class="clearer"></div> </div> </div> <div class="footer-wrapper"> <div class="footer"> <div class="left"> <a href="cfg-properties.html" title="Properties" >previous</a> | <a href="cfg-interlocks.html" title="Interlocks" >next</a> | <a href="../py-modindex.html" title="Python Module Index" >modules</a> | <a href="../genindex.html" title="General Index" >index</a> <br/> <a href="../_sources/manual/cfg-buildsteps.txt" rel="nofollow">Show Source</a> </div> <div class="right"> <div class="footer"> © Copyright Buildbot Team Members. Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 1.1.3. </div> </div> <div class="clearer"></div> </div> </div> </body> </html>