<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Buildbot Coding Style — Buildbot 0.8.8 documentation</title>
<link rel="stylesheet" href="../_static/agogo.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
VERSION: '0.8.8',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="../_static/jquery.js"></script>
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="shortcut icon" href="../_static/buildbot.ico"/>
<link rel="top" title="Buildbot 0.8.8 documentation" href="../index.html" />
<link rel="up" title="Buildbot Development" href="index.html" />
<link rel="next" title="Buildbot's Test Suite" href="tests.html" />
<link rel="prev" title="Definitions" href="definitions.html" />
</head>
<body>
<div class="header-wrapper">
<div class="header">
<p class="logo"><a href="../index.html">
<img class="logo" src="../_static/header-text-transparent.png" alt="Logo"/>
</a></p>
<div class="headertitle"><a
href="../index.html">Buildbot 0.8.8 documentation</a></div>
<div class="rel">
<a href="definitions.html" title="Definitions"
accesskey="P">previous</a> |
<a href="tests.html" title="Buildbot's Test Suite"
accesskey="N">next</a> |
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |
<a href="../genindex.html" title="General Index"
accesskey="I">index</a>
</div>
</div>
</div>
<div class="content-wrapper">
<div class="content">
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body">
<div class="section" id="buildbot-coding-style">
<h1>Buildbot Coding Style<a class="headerlink" href="#buildbot-coding-style" title="Permalink to this headline">¶</a></h1>
<div class="section" id="symbol-names">
<h2>Symbol Names<a class="headerlink" href="#symbol-names" title="Permalink to this headline">¶</a></h2>
<p>Buildbot follows <a class="reference external" href="http://www.python.org/dev/peps/pep-0008/">PEP8</a> regarding
the formatting of symbol names.</p>
<p>The single exception in naming of functions and methods. Because Buildbot uses
Twisted so heavily, and Twisted uses interCaps, Buildbot methods should do the
same. That is, you should spell methods and functions with the first character
in lower-case, and the first letter of subsequent words capitalized, e.g.,
<tt class="docutils literal"><span class="pre">compareToOther</span></tt> or <tt class="docutils literal"><span class="pre">getChangesGreaterThan</span></tt>. This point is not applied very
consistently in Buildbot, but let's try to be consistent in new code.</p>
</div>
<div class="section" id="twisted-idioms">
<h2>Twisted Idioms<a class="headerlink" href="#twisted-idioms" title="Permalink to this headline">¶</a></h2>
<p>Programming with Twisted Python can be daunting. But sticking to a few
well-defined patterns can help avoid surprises.</p>
<div class="section" id="prefer-to-return-deferreds">
<h3>Prefer to Return Deferreds<a class="headerlink" href="#prefer-to-return-deferreds" title="Permalink to this headline">¶</a></h3>
<p>If you're writing a method that doesn't currently block, but could conceivably
block sometime in the future, return a Deferred and document that it does so.
Just about anything might block - even getters and setters!</p>
</div>
<div class="section" id="helpful-twisted-classes">
<h3>Helpful Twisted Classes<a class="headerlink" href="#helpful-twisted-classes" title="Permalink to this headline">¶</a></h3>
<p>Twisted has some useful, but little-known classes.
Brief descriptions follow, but you should consult the API documentation or source code
for the full details.</p>
<dl class="docutils">
<dt><tt class="xref py py-class docutils literal"><span class="pre">twisted.internet.task.LoopingCall</span></tt></dt>
<dd>Calls an asynchronous function repeatedly at set intervals.
Note that this will stop looping if the function fails.
In general, you will want to wrap the function to capture and log errors.</dd>
<dt><tt class="xref py py-class docutils literal"><span class="pre">twisted.application.internet.TimerService</span></tt></dt>
<dd>Similar to <tt class="docutils literal"><span class="pre">t.i.t.LoopingCall</span></tt>, but implemented as a service that will automatically start and stop the function calls when the service starts and stops.
See the warning about failing functions for <tt class="docutils literal"><span class="pre">t.i.t.LoopingCall</span></tt>.</dd>
</dl>
</div>
<div class="section" id="sequences-of-operations">
<h3>Sequences of Operations<a class="headerlink" href="#sequences-of-operations" title="Permalink to this headline">¶</a></h3>
<p>Especially in Buildbot, we're often faced with executing a sequence of
operations, many of which may block.</p>
<p>In all cases where this occurs, there is a danger of pre-emption, so exercise
the same caution you would if writing a threaded application.</p>
<p>For simple cases, you can use nested callback functions. For more complex cases, deferredGenerator is appropriate.</p>
<div class="section" id="nested-callbacks">
<h4>Nested Callbacks<a class="headerlink" href="#nested-callbacks" title="Permalink to this headline">¶</a></h4>
<p>First, an admonition: do not create extra class methods that represent the continuations of the first:</p>
<div class="highlight-python"><pre>def myMethod(self):
d = ...
d.addCallback(self._myMethod_2) # BAD!
def _myMethod_2(self, res): # BAD!
# ...</pre>
</div>
<p>Invariably, this extra method gets separated from its parent as the code
evolves, and the result is completely unreadable. Instead, include all of the
code for a particular function or method within the same indented block, using
nested functions:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="k">def</span> <span class="nf">getRevInfo</span><span class="p">(</span><span class="n">revname</span><span class="p">):</span>
<span class="n">results</span> <span class="o">=</span> <span class="p">{}</span>
<span class="n">d</span> <span class="o">=</span> <span class="n">defer</span><span class="o">.</span><span class="n">succeed</span><span class="p">(</span><span class="bp">None</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">rev_parse</span><span class="p">(</span><span class="n">_</span><span class="p">):</span> <span class="c"># note use of '_' to quietly indicate an ignored parameter</span>
<span class="k">return</span> <span class="n">utils</span><span class="o">.</span><span class="n">getProcessOutput</span><span class="p">(</span><span class="n">git</span><span class="p">,</span> <span class="p">[</span> <span class="s">'rev-parse'</span><span class="p">,</span> <span class="n">revname</span> <span class="p">])</span>
<span class="n">d</span><span class="o">.</span><span class="n">addCallback</span><span class="p">(</span><span class="n">rev_parse</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">parse_rev_parse</span><span class="p">(</span><span class="n">res</span><span class="p">):</span>
<span class="n">results</span><span class="p">[</span><span class="s">'rev'</span><span class="p">]</span> <span class="o">=</span> <span class="n">res</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span>
<span class="k">return</span> <span class="n">utils</span><span class="o">.</span><span class="n">getProcessOutput</span><span class="p">(</span><span class="n">git</span><span class="p">,</span> <span class="p">[</span> <span class="s">'log'</span><span class="p">,</span> <span class="s">'-1'</span><span class="p">,</span> <span class="s">'--format=</span><span class="si">%s</span><span class="s">%n%b'</span><span class="p">,</span> <span class="n">results</span><span class="p">[</span><span class="s">'rev'</span><span class="p">]</span> <span class="p">])</span>
<span class="n">d</span><span class="o">.</span><span class="n">addCallback</span><span class="p">(</span><span class="n">parse_rev_parse</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">parse_log</span><span class="p">(</span><span class="n">res</span><span class="p">):</span>
<span class="n">results</span><span class="p">[</span><span class="s">'comments'</span><span class="p">]</span> <span class="o">=</span> <span class="n">res</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span>
<span class="n">d</span><span class="o">.</span><span class="n">addCallback</span><span class="p">(</span><span class="n">parse_log</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">set_results</span><span class="p">(</span><span class="n">_</span><span class="p">):</span>
<span class="k">return</span> <span class="n">results</span>
<span class="n">d</span><span class="o">.</span><span class="n">addCallback</span><span class="p">(</span><span class="n">set_results</span><span class="p">)</span>
<span class="k">return</span> <span class="n">d</span>
</pre></div>
</div>
<p>it is usually best to make the first operation occur within a callback, as the
deferred machinery will then handle any exceptions as a failure in the outer
Deferred. As a shortcut, <tt class="docutils literal"><span class="pre">d.addCallback</span></tt> works as a decorator:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="n">d</span> <span class="o">=</span> <span class="n">defer</span><span class="o">.</span><span class="n">succeed</span><span class="p">(</span><span class="bp">None</span><span class="p">)</span>
<span class="nd">@d.addCallback</span>
<span class="k">def</span> <span class="nf">rev_parse</span><span class="p">(</span><span class="n">_</span><span class="p">):</span> <span class="c"># note use of '_' to quietly indicate an ignored parameter</span>
<span class="k">return</span> <span class="n">utils</span><span class="o">.</span><span class="n">getProcessOutput</span><span class="p">(</span><span class="n">git</span><span class="p">,</span> <span class="p">[</span> <span class="s">'rev-parse'</span><span class="p">,</span> <span class="n">revname</span> <span class="p">])</span>
</pre></div>
</div>
<p>Be careful with local variables. For example, if <tt class="docutils literal"><span class="pre">parse_rev_parse</span></tt>, above,
merely assigned <tt class="docutils literal"><span class="pre">rev</span> <span class="pre">=</span> <span class="pre">res.strip()</span></tt>, then that variable would be local to
<tt class="docutils literal"><span class="pre">parse_rev_parse</span></tt> and not available in <tt class="docutils literal"><span class="pre">set_results</span></tt>. Mutable variables
(dicts and lists) at the outer function level are appropriate for this purpose.</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">do not try to build a loop in this style by chaining multiple
Deferreds! Unbounded chaining can result in stack overflows, at least on older
versions of Twisted. Use <tt class="docutils literal"><span class="pre">deferredGenerator</span></tt> instead.</p>
</div>
</div>
<div class="section" id="inlinecallbacks">
<h4>inlineCallbacks<a class="headerlink" href="#inlinecallbacks" title="Permalink to this headline">¶</a></h4>
<p><tt class="xref py py-class docutils literal"><span class="pre">twisted.internet.defer.inlineCallbacks</span></tt> is a great help to writing code
that makes a lot of asynchronous calls, particularly if those calls are made in
loop or conditionals. Refer to the Twisted documentation for the details, but
the style within Buildbot is as follows:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">twisted.internet</span> <span class="kn">import</span> <span class="n">defer</span>
<span class="nd">@defer.inlineCallbacks</span>
<span class="k">def</span> <span class="nf">mymethod</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">):</span>
<span class="n">xval</span> <span class="o">=</span> <span class="k">yield</span> <span class="n">getSomething</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
<span class="k">for</span> <span class="n">z</span> <span class="ow">in</span> <span class="p">(</span><span class="k">yield</span> <span class="n">getZValues</span><span class="p">()):</span>
<span class="n">y</span> <span class="o">+=</span> <span class="n">z</span>
<span class="k">if</span> <span class="n">xval</span> <span class="o">></span> <span class="mi">10</span><span class="p">:</span>
<span class="n">defer</span><span class="o">.</span><span class="n">returnValue</span><span class="p">(</span><span class="n">xval</span> <span class="o">+</span> <span class="n">y</span><span class="p">)</span>
<span class="k">return</span>
<span class="bp">self</span><span class="o">.</span><span class="n">someOtherMethod</span><span class="p">()</span>
</pre></div>
</div>
<p>The key points to notice here:</p>
<ul class="simple">
<li>Always import <tt class="docutils literal"><span class="pre">defer</span></tt> as a module, not the names within it.</li>
<li>Use the decorator form of <tt class="docutils literal"><span class="pre">inlineCallbacks</span></tt>.</li>
<li>In most cases, the result of a <tt class="docutils literal"><span class="pre">yield</span></tt> expression should be assigned to a
variable. It can be used in a larger expression, but remember that Python
requires that you enclose the expression in its own set of parentheses.</li>
<li>Python does not permit returning a value from a generator, so statements like
<tt class="docutils literal"><span class="pre">return</span> <span class="pre">xval</span> <span class="pre">+</span> <span class="pre">y</span></tt> are invalid. Instead, yield the result of
<tt class="docutils literal"><span class="pre">defer.returnValue</span></tt>. Although this function does cause an immediate
function exit, for clarity follow it with a bare <tt class="docutils literal"><span class="pre">return</span></tt>, as in
the example, unless it is the last statement in a function.</li>
</ul>
<p>The great advantage of <tt class="docutils literal"><span class="pre">inlineCallbacks</span></tt> is that it allows you to use all
of the usual Pythonic control structures in their natural form. In particular,
it is easy to represent a loop, or even nested loops, in this style without
losing any readability.</p>
<p>Note that code using <tt class="docutils literal"><span class="pre">deferredGenerator</span></tt> is no longer acceptable in Buildbot.</p>
</div>
<div class="section" id="locking">
<h4>Locking<a class="headerlink" href="#locking" title="Permalink to this headline">¶</a></h4>
<p>Remember that asynchronous programming does not free you from the need to worry
about concurrency issues. Particularly if you are executing a sequence of
operations, each time you wait for a Deferred, arbitrary other actions can take
place.</p>
<p>In general, you should try to perform actions atomically, but for the rare
situations that require synchronization, the following might be useful:</p>
<ul class="simple">
<li><tt class="xref py py-class docutils literal"><span class="pre">twisted.internet.defer.DeferredLock</span></tt></li>
<li><a class="reference internal" href="utils.html#buildbot.util.misc.deferredLocked" title="buildbot.util.misc.deferredLocked"><tt class="xref py py-func docutils literal"><span class="pre">buildbot.util.misc.deferredLocked</span></tt></a></li>
<li><a class="reference internal" href="utils.html#buildbot.util.misc.SerializedInvocation" title="buildbot.util.misc.SerializedInvocation"><tt class="xref py py-func docutils literal"><span class="pre">buildbot.util.misc.SerializedInvocation</span></tt></a></li>
</ul>
</div>
</div>
<div class="section" id="joining-sequences">
<h3>Joining Sequences<a class="headerlink" href="#joining-sequences" title="Permalink to this headline">¶</a></h3>
<p>It's often the case that you'll want to perform multiple operations in
parallel, and re-join the results at the end. For this purpose, you'll want to
use a <a class="reference external" href="http://twistedmatrix.com/documents/current/api/twisted.internet.defer.DeferredList.html">DeferredList</a></p>
<div class="highlight-python"><pre>def getRevInfo(revname):
results = {}
finished = dict(rev_parse=False, log=False)
rev_parse_d = utils.getProcessOutput(git, [ 'rev-parse', revname ])
def parse_rev_parse(res):
return res.strip()
rev_parse_d.addCallback(parse_rev_parse)
log_d = utils.getProcessOutput(git, [ 'log', '-1', '--format=%s%n%b', results['rev'] ]))
def parse_log(res):
return res.strip()
log_d.addCallback(parse_log)
d = defer.DeferredList([rev_parse_d, log_d], consumeErrors=1, fireOnFirstErrback=1)
def handle_results(results):
return dict(rev=results[0][1], log=results[1][1])
d.addCallback(handle_results)
return d</pre>
</div>
<p>Here the deferred list will wait for both <tt class="docutils literal"><span class="pre">rev_parse_d</span></tt> and <tt class="docutils literal"><span class="pre">log_d</span></tt> to
fire, or for one of them to fail. You may attach callbacks and errbacks to a
<tt class="docutils literal"><span class="pre">DeferredList</span></tt> just as for a deferred.</p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="sidebar">
<h3>Table Of Contents</h3>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../tutorial/index.html">Buildbot Tutorial</a></li>
<li class="toctree-l1"><a class="reference internal" href="../manual/index.html">Buildbot Manual</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="index.html">Buildbot Development</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="master-overview.html">Master Organization</a></li>
<li class="toctree-l2"><a class="reference internal" href="definitions.html">Definitions</a></li>
<li class="toctree-l2 current"><a class="current reference internal" href="">Buildbot Coding Style</a><ul class="simple">
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="tests.html">Buildbot's Test Suite</a></li>
<li class="toctree-l2"><a class="reference internal" href="config.html">Configuration</a></li>
<li class="toctree-l2"><a class="reference internal" href="config.html#error-handling">Error Handling</a></li>
<li class="toctree-l2"><a class="reference internal" href="config.html#reconfiguration">Reconfiguration</a></li>
<li class="toctree-l2"><a class="reference internal" href="utils.html">Utilities</a></li>
<li class="toctree-l2"><a class="reference internal" href="database.html">Database</a></li>
<li class="toctree-l2"><a class="reference internal" href="results.html">Build Result Codes</a></li>
<li class="toctree-l2"><a class="reference internal" href="formats.html">File Formats</a></li>
<li class="toctree-l2"><a class="reference internal" href="webstatus.html">Web Status</a></li>
<li class="toctree-l2"><a class="reference internal" href="master-slave.html">Master-Slave API</a></li>
<li class="toctree-l2"><a class="reference internal" href="encodings.html">String Encodings</a></li>
<li class="toctree-l2"><a class="reference internal" href="metrics.html">Metrics</a></li>
<li class="toctree-l2"><a class="reference internal" href="classes.html">Classes</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../relnotes/index.html">Release Notes for Buildbot v0.8.8</a></li>
</ul>
<h3 style="margin-top: 1.5em;">Search</h3>
<form class="search" action="../search.html" method="get">
<input type="text" name="q" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div>
<div class="clearer"></div>
</div>
</div>
<div class="footer-wrapper">
<div class="footer">
<div class="left">
<a href="definitions.html" title="Definitions"
>previous</a> |
<a href="tests.html" title="Buildbot's Test Suite"
>next</a> |
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |
<a href="../genindex.html" title="General Index"
>index</a>
<br/>
<a href="../_sources/developer/style.txt"
rel="nofollow">Show Source</a>
</div>
<div class="right">
<div class="footer">
© Copyright Buildbot Team Members.
Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 1.1.3.
</div>
</div>
<div class="clearer"></div>
</div>
</div>
</body>
</html>
