This is buildbot.info, produced by makeinfo version 4.13 from
buildbot.texinfo.
This is the BuildBot manual.
Copyright (C) 2005, 2006, 2009, 2010 Brian Warner
Copying and distribution of this file, with or without
modification, are permitted in any medium without royalty provided
the copyright notice and this notice are preserved.
File: buildbot.info, Node: Top, Next: Introduction, Prev: (dir), Up: (dir)
BuildBot
********
This is the BuildBot manual.
Copyright (C) 2005, 2006, 2009, 2010 Brian Warner
Copying and distribution of this file, with or without
modification, are permitted in any medium without royalty provided
the copyright notice and this notice are preserved.
* Menu:
* Introduction:: What the BuildBot does.
* Installation:: Creating a buildmaster and buildslaves,
running them.
* Concepts:: What goes on in the buildbot's little mind.
* Configuration:: Controlling the buildbot.
* Getting Source Code Changes:: Discovering when to run a build.
* Build Process:: Controlling how each build is run.
* Status Delivery:: Telling the world about the build's results.
* Command-line tool::
* Resources:: Getting help.
* Developer's Appendix::
* Index of Useful Classes::
* Index of master.cfg keys::
* Index:: Complete index.
--- The Detailed Node Listing ---
Introduction
* History and Philosophy::
* System Architecture::
* Control Flow::
System Architecture
* BuildSlave Connections::
* Buildmaster Architecture::
* Status Delivery Architecture::
Installation
* Requirements::
* Installing the code::
* Creating a buildmaster::
* Upgrading an Existing Buildmaster::
* Creating a buildslave::
* Launching the daemons::
* Logfiles::
* Shutdown::
* Maintenance::
* Troubleshooting::
Creating a buildslave
* Buildslave Options::
Troubleshooting
* Starting the buildslave::
* Connecting to the buildmaster::
* Forcing Builds::
Concepts
* Version Control Systems::
* Schedulers::
* BuildSet::
* BuildRequest::
* Builder::
* Users::
* Build Properties::
Version Control Systems
* Generalizing VC Systems::
* Source Tree Specifications::
* How Different VC Systems Specify Sources::
* Attributes of Changes::
Users
* Doing Things With Users::
* Email Addresses::
* IRC Nicknames::
* Live Status Clients::
Configuration
* Config File Format::
* Loading the Config File::
* Testing the Config File::
* Defining the Project::
* Change Sources and Schedulers::
* Setting the slaveport::
* Buildslave Specifiers::
* On-Demand ("Latent") Buildslaves::
* Defining Global Properties::
* Defining Builders::
* Defining Status Targets::
* Debug options::
Change Sources and Schedulers
* Scheduler Scheduler::
* AnyBranchScheduler::
* Dependent Scheduler::
* Periodic Scheduler::
* Nightly Scheduler::
* Try Schedulers::
* Triggerable Scheduler::
Buildslave Specifiers
* When Buildslaves Go Missing::
On-Demand ("Latent") Buildslaves
* Amazon Web Services Elastic Compute Cloud ("AWS EC2")::
* Dangers with Latent Buildslaves::
* Writing New Latent Buildslaves::
Getting Source Code Changes
* Change Sources::
* Choosing ChangeSources::
* CVSToys - PBService::
* Mail-parsing ChangeSources::
* PBChangeSource::
* P4Source::
* BonsaiPoller::
* SVNPoller::
* MercurialHook::
* Bzr Hook::
* Bzr Poller::
Mail-parsing ChangeSources
* Subscribing the Buildmaster::
* Using Maildirs::
* Parsing Email Change Messages::
Parsing Email Change Messages
* FCMaildirSource::
* SyncmailMaildirSource::
* BonsaiMaildirSource::
* SVNCommitEmailMaildirSource::
Build Process
* Build Steps::
* Interlocks::
* Build Factories::
Build Steps
* Common Parameters::
* Using Build Properties::
* Source Checkout::
* ShellCommand::
* Simple ShellCommand Subclasses::
* Python BuildSteps::
* Transferring Files::
* Steps That Run on the Master::
* Triggering Schedulers::
* Writing New BuildSteps::
Source Checkout
* CVS::
* SVN::
* Darcs::
* Mercurial::
* Arch::
* Bazaar::
* Bzr::
* P4::
* Git::
Simple ShellCommand Subclasses
* Configure::
* Compile::
* Test::
* TreeSize::
* PerlModuleTest::
* Testing with mysql-test-run::
* SetProperty::
* SubunitShellCommand::
Python BuildSteps
* BuildEPYDoc::
* PyFlakes::
* PyLint::
Writing New BuildSteps
* BuildStep LogFiles::
* Reading Logfiles::
* Adding LogObservers::
* BuildStep URLs::
Build Factories
* BuildStep Objects::
* BuildFactory::
* Process-Specific build factories::
BuildStep Objects
* BuildFactory Attributes::
* Quick builds::
BuildFactory
* BuildFactory Attributes::
* Quick builds::
Process-Specific build factories
* GNUAutoconf::
* CPAN::
* Python distutils::
* Python/Twisted/trial projects::
Status Delivery
* WebStatus::
* MailNotifier::
* IRC Bot::
* PBListener::
* Writing New Status Plugins::
WebStatus
* WebStatus Configuration Parameters::
* Buildbot Web Resources::
* XMLRPC server::
* HTML Waterfall::
Command-line tool
* Administrator Tools::
* Developer Tools::
* Other Tools::
* .buildbot config directory::
Developer Tools
* statuslog::
* statusgui::
* try::
waiting for results
* try --diff::
Other Tools
* sendchange::
* debugclient::
File: buildbot.info, Node: Introduction, Next: Installation, Prev: Top, Up: Top
1 Introduction
**************
The BuildBot is a system to automate the compile/test cycle required
by most software projects to validate code changes. By automatically
rebuilding and testing the tree each time something has changed,
build problems are pinpointed quickly, before other developers are
inconvenienced by the failure. The guilty developer can be identified
and harassed without human intervention. By running the builds on a
variety of platforms, developers who do not have the facilities to
test their changes everywhere before checkin will at least know
shortly afterwards whether they have broken the build or not. Warning
counts, lint checks, image size, compile time, and other build
parameters can be tracked over time, are more visible, and are
therefore easier to improve.
The overall goal is to reduce tree breakage and provide a platform
to run tests or code-quality checks that are too annoying or pedantic
for any human to waste their time with. Developers get immediate (and
potentially public) feedback about their changes, encouraging them to
be more careful about testing before checkin.
Features:
* run builds on a variety of slave platforms
* arbitrary build process: handles projects using C, Python,
whatever
* minimal host requirements: python and Twisted
* slaves can be behind a firewall if they can still do checkout
* status delivery through web page, email, IRC, other protocols
* track builds in progress, provide estimated completion time
* flexible configuration by subclassing generic build process
classes
* debug tools to force a new build, submit fake Changes, query
slave status
* released under the GPL
* Menu:
* History and Philosophy::
* System Architecture::
* Control Flow::
File: buildbot.info, Node: History and Philosophy, Next: System Architecture, Prev: Introduction, Up: Introduction
1.1 History and Philosophy
==========================
The Buildbot was inspired by a similar project built for a development
team writing a cross-platform embedded system. The various components
of the project were supposed to compile and run on several flavors of
unix (linux, solaris, BSD), but individual developers had their own
preferences and tended to stick to a single platform. From time to
time, incompatibilities would sneak in (some unix platforms want to
use `string.h', some prefer `strings.h'), and then the tree would
compile for some developers but not others. The buildbot was written
to automate the human process of walking into the office, updating a
tree, compiling (and discovering the breakage), finding the developer
at fault, and complaining to them about the problem they had
introduced. With multiple platforms it was difficult for developers to
do the right thing (compile their potential change on all platforms);
the buildbot offered a way to help.
Another problem was when programmers would change the behavior of a
library without warning its users, or change internal aspects that
other code was (unfortunately) depending upon. Adding unit tests to
the codebase helps here: if an application's unit tests pass despite
changes in the libraries it uses, you can have more confidence that
the library changes haven't broken anything. Many developers
complained that the unit tests were inconvenient or took too long to
run: having the buildbot run them reduces the developer's workload to
a minimum.
In general, having more visibility into the project is always good,
and automation makes it easier for developers to do the right thing.
When everyone can see the status of the project, developers are
encouraged to keep the tree in good working order. Unit tests that
aren't run on a regular basis tend to suffer from bitrot just like
code does: exercising them on a regular basis helps to keep them
functioning and useful.
The current version of the Buildbot is additionally targeted at
distributed free-software projects, where resources and platforms are
only available when provided by interested volunteers. The buildslaves
are designed to require an absolute minimum of configuration, reducing
the effort a potential volunteer needs to expend to be able to
contribute a new test environment to the project. The goal is for
anyone who wishes that a given project would run on their favorite
platform should be able to offer that project a buildslave, running on
that platform, where they can verify that their portability code
works, and keeps working.
File: buildbot.info, Node: System Architecture, Next: Control Flow, Prev: History and Philosophy, Up: Introduction
1.2 System Architecture
=======================
The Buildbot consists of a single `buildmaster' and one or more
`buildslaves', connected in a star topology. The buildmaster makes
all decisions about what, when, and how to build. It sends commands
to be run on the build slaves, which simply execute the commands and
return the results. (certain steps involve more local decision
making, where the overhead of sending a lot of commands back and
forth would be inappropriate, but in general the buildmaster is
responsible for everything).
The buildmaster is usually fed `Changes' by some sort of version
control system (*note Change Sources::), which may cause builds to be
run. As the builds are performed, various status messages are
produced, which are then sent to any registered Status Targets (*note
Status Delivery::).
