Buildbot
From charlesreid1
Buildbot creates a framework for automated code testing.
Contents
Installation
Tarballs can be obtained from here: http://trac.buildbot.net/
Download/installation instructions are here: http://trac.buildbot.net/wiki/DownloadInstall
Prerequisites
VirtualEnv
Since I was building Buildbot on a system where I had no administrative permissions, I used virtualenv (see Python#Virtual Python: virtualenv for more information).
I downloaded the master and slave tarballs, extracted them, and ran:
$ ptyhon setyp.py install --prefix=/path/to/virtualenv
After doing this, I can use the python binary in /path/to/virtualenv/bin/python to load the buildbot library through python.
Mock
In order to run BuildBot, you will also need the Mock package, which is also related to testing.
Obtain Mock here: http://pypi.python.org/pypi/mock#downloads
Installing Buildbot
I downloaded Buildbot (both the master and the slave) from http://trac.buildbot.net/wiki/DownloadInstall
Master
I installed the Buildbot master by running
$ python setup.py build
The next thing I had to do was add the future installation directory to my $PYTHONPATH variable (otherwise the python installer throws an error). I put this in one of my Dot files (.bashrc):
export PYTHONPATH="${HOME}/pkg/buildbot/0.8.3_master/lib/python2.7/site-packages:${PYTHONPATH}"
then made the directory:
$ mkdir -p ${HOME}/pkg/buildbot/0.8.3_master/lib/python2.7/site-packages
and then ran setup's installer:
$ python setup.py install --prefix=${HOME}/pkg/buildbot/0.8.3_master
Sheep
I basically repeated the master build process. First, I downloaded the tarball, then untarred it.
Then I ran:
$ python setup.py build
Then I added the path to the Python package to my $PYTHONPATH variable:
export PYTHONPATH="${HOME}/pkg/buildbot/0.8.3_slave/lib/python2.7/site-packages:${PYTHONPATH}"
then made the directory:
$ mkdir -p ${HOME}/pkg/buildbot/0.8.3_slave/lib/python2.7/site-packages
and then ran setup's installer:
$ python setup.py install --prefix=${HOME}/pkg/buildbot/0.8.3_slave
Confirming Installation
Once both installs finished, I added both to my path:
export PATH="${HOME}/pkg/buildbot/0.8.3_master/bin:${PATH}"
export PATH="${HOME}/pkg/buildbot/0.8.3_slave/bin:${PATH}"
and then ran:
$ buildbot --version Buildbot version: 0.8.3p1 Twisted version: 11.0.0 $ buildslave --version Buildslave version: 0.8.3 Twisted version: 11.0.0
Creating Buildbots
Creating a Master
First, create the build master with the create-master command:
$ cd /path/to/buildbot/stuff/ $ buildbot create-master master
This makes a directory called "master".
Second, create the build master config file by creating /path/to/buildbot/stuff/master/master.cfg (see http://buildbot.net/buildbot/docs/latest)
If you're looking for instant gratification, just do this:
$ mv master/master.cfg.sample master/master.cfg
This will use an example git repository for the pyflakes project. However, since I was running on a system that already had a Buildbot running on it, and I just wanted to leave it alone and use my own, I had to change the default HTTP port (8010) to something else, and change the build slave port (9989) to something else.
Third, start the build master:
buildbot start master
(or, if you are not in the same directory as your buildbot master directory,)
buildbot start /path/to/master
You should see something like this:
$ buildbot start master
Following twistd.log until startup finished..
2011-06-03 16:20:09-0600 [-] Log opened.
2011-06-03 16:20:09-0600 [-] twistd 11.0.0 (/opt/local/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python 2.7.1) starting up.
2011-06-03 16:20:09-0600 [-] reactor class: twisted.internet.selectreactor.SelectReactor.
2011-06-03 16:20:09-0600 [-] Creating BuildMaster -- buildbot.version: 0.8.3p1
2011-06-03 16:20:09-0600 [-] loading configuration from /uufs/chpc.utah.edu/common/home/u0552682/codes/buildbot/master/master.cfg
2011-06-03 16:20:09-0600 [-] unable to import dnotify, so Maildir will use polling instead
2011-06-03 16:20:09-0600 [-] creating adbapi pool: sqlite3 ('/uufs/chpc.utah.edu/common/home/u0552682/codes/buildbot/master/state.sqlite',) {'check_same_thread': False, 'cp_noisy': True, 'cp_reconnect': True}
2011-06-03 16:20:09-0600 [-] twisted.spread.pb.PBServerFactory starting on 9999
2011-06-03 16:20:09-0600 [-] Starting factory <twisted.spread.pb.PBServerFactory instance at 0x10231c290>
2011-06-03 16:20:09-0600 [-] adding new builder runtests for category None
2011-06-03 16:20:09-0600 [-] trying to load status pickle from /uufs/chpc.utah.edu/common/home/u0552682/codes/buildbot/master/runtests/builder
2011-06-03 16:20:09-0600 [-] added builder runtests in category None
2011-06-03 16:20:09-0600 [-] setBuilders._add: [<buildbot.util.loop.DelegateLoop instance at 0x101e11ea8>, <BuildSlave 'example-slave', current builders: >] [<Builder ''runtests'' at 4331784312>]
2011-06-03 16:20:09-0600 [-] adding IStatusReceiver <WebStatus on port tcp:8888 at 0x101f755a8>
2011-06-03 16:20:09-0600 [-] buildbot.status.web.baseweb.RotateLogSite starting on 8888
2011-06-03 16:20:09-0600 [-] Starting factory <buildbot.status.web.baseweb.RotateLogSite instance at 0x10231cbd8>
2011-06-03 16:20:09-0600 [-] Setting up http.log rotating 10 files of 10000000 bytes each
2011-06-03 16:20:09-0600 [-] WebStatus using (/uufs/chpc.utah.edu/common/home/u0552682/codes/buildbot/master/public_html)
2011-06-03 16:20:09-0600 [-] removing 0 old schedulers, adding 1 new ones
2011-06-03 16:20:09-0600 [-] configuration update started
2011-06-03 16:20:09-0600 [-] adbapi connecting: sqlite3 ('/uufs/chpc.utah.edu/common/home/u0552682/codes/buildbot/master/state.sqlite',){'check_same_thread': False}
2011-06-03 16:20:09-0600 [-] scheduler 'all' got id 1
2011-06-03 16:20:09-0600 [-] adding 1 new changesources, removing 0
2011-06-03 16:20:09-0600 [-] gitpoller: polling git repo at git://github.com/buildbot/pyflakes.git
2011-06-03 16:20:09-0600 [-] configuration update complete
The buildmaster appears to have (re)started correctly.
See #Errors below for error messages and resolution.
Creating a Sheep
To create the build slave, you'll need some information from master.cfg:
c['slavePortnum'] = PPPPc['slaves'] = [BuildSlave("example-slave", "pass")]
You can create the build slave by running:
$ buildslave create-slave slave localhost:PPPP example-slave pass
which should give the output:
$ buildslave create-slave slave localhost:PPPP example-slave pass mkdir /uufs/chpc.utah.edu/common/home/u0552682/codes/buildbot/slave chdir /uufs/chpc.utah.edu/common/home/u0552682/codes/buildbot/slave mkdir /uufs/chpc.utah.edu/common/home/u0552682/codes/buildbot/slave/info Creating info/admin, you need to edit it appropriately Creating info/host, you need to edit it appropriately Not creating info/access_uri - add it if you wish Please edit the files in /uufs/chpc.utah.edu/common/home/u0552682/codes/buildbot/slave/info appropriately. buildslave configured in /uufs/chpc.utah.edu/common/home/u0552682/codes/buildbot/slave
Second, start the build slave:
buildslave start slave
(or, if you are not in the same directory as your buildbot slave,)
buildslave start /path/to/slave
which should return the output:
$ buildslave start slave Following twistd.log until startup finished.. 2011-06-03 22:39:28-0600 [-] Log opened. 2011-06-03 22:39:28-0600 [-] twistd 11.0.0 (/opt/local/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python 2.7.1) starting up. 2011-06-03 22:39:28-0600 [-] reactor class: twisted.internet.selectreactor.SelectReactor. 2011-06-03 22:39:28-0600 [-] Starting factory <buildslave.bot.BotFactory instance at 0x101b007a0> 2011-06-03 22:39:28-0600 [-] Connecting to localhost:9999 2011-06-03 22:39:28-0600 [Broker,client] message from master: attached The buildslave appears to have (re)started correctly.
Webpage View
You should be able to visit hostname:8010 (or whatever port you used) and see a message that indicates success with setting up the project:
Running Buildbots
Starting Buildbot Daemons
You can start the buildmaster daemon by running:
buildbot start /path/to/master
and likewise, you can start the buildslave daemon by running:
buildslave start /path/to/slave
(Note: if you experience errors, see the #Errors section. If you don't see the error you are experiencing listed there, please email me at root(at)charlesmartinreid.com, and I will add your problem/resolution to the page.)
Restarting/Reconfiguring Buildbot
If you edit the configuration file (e.g. master.cfg), then you can restart the buildbot and it will re-load the configuration file:
buildbot reconfig /path/to/master
Checking Buildbot Config File
You can check the buildbot config file by running
$ buildbot checkconfig master.cfg
Buildbot Configuration
Conceptualization of Configure Files
Documentation on configuration files: http://buildbot.net/buildbot/docs/current/Configuration.html#Configuration
digraph G {
"Master Configuration File" -> "Scheduler"
"Master Configuration File" -> "Periodic Scheduler"
"Scheduler" -> "Changes"
"Periodic Scheduler" -> "Changes"
"Periodic Scheduler" -> "Periodic Builds"
"Periodic Builds" -> "BuildSet"
"Scheduler" -> "BuildSet"
"BuildSet" -> "BuildMaster"
"BuildMaster" -> "BuildRequest1" -> "Associated BuildSlaves"
"BuildMaster" -> "BuildRequest2" -> "Associated BuildSlaves"
"BuildMaster" -> "BuildRequest3" -> "Associated BuildSlaves"
"BuildMaster" -> "BuildRequest4" -> "Associated BuildSlaves"
"Associated BuildSlaves" -> "Builder1" -> "BuildFactory1" -> "Build1"
"Associated BuildSlaves" -> "Builder2" -> "BuildFactory2" -> "Build2"
"Associated BuildSlaves" -> "Builder3" -> "BuildFactory3" -> "Build3"
"Associated BuildSlaves" -> "Builder4" -> "BuildFactory4" -> "Build4"
}
In words:
The BuildMaster has Scheduler objects. These scheduler objects create BuildSet objects and give them to the BuildMaster. The BuildMaster then determines how to construct BuildRequests out of the BuildSet objects, and passes the BuildRequests to individual Builders (or BuildSlaves).
Each Builder shares a common operating system, libraries, compilers, headers, etc. If any of these are different, there should be a different builder.
A Builder can be part of multiple BuildSlaves. This is for redundancy; e.g. if there is a single Builder that is part of two BuildSlaves, and one of them goes offline, the Builder still receives instructions from the BuildSlave that has not gone offline.
A Build is a single compilation or test run of a particular version of code. (This is typically something like a checkout, configure, make, make check sequence).
The Builder uses a BuildFactory to specify the steps involved in the Build.
Project Source Code
Specifying Project Source Code
You can specify a "poller" that will check the version control logs for any changes, and test out a build if there are changes detected.
I use the SVNPoller object, so the following goes in my master.cfg configuration file:
from buildbot.changes.svnpoller import SVNPoller
svn_poller = SVNPoller(
project='Project Name',
svnurl='http://project_url/svn/trunk'
pollinterval=600,
svnbin='/bin/svn'
)
c['change_source'] = [ svn_poller ]
Information about detecting changes in a project source code are here: http://buildbot.net/buildbot/docs/0.8.3/Change-Sources.html#Change-Sources
Scheduler
Buildbot documentation on different kinds of schedulers: http://buildbot.net/buildbot/docs/0.8.3/Schedulers.html#Schedulers
I'm just using a regular old Scheduler:
from buildbot.scheduler import Scheduler
c['schedulers'] = []
c['schedulers'].append(Scheduler(name="default-test",
branch=None,
treeStableTimer=None,
builderNames=["run-default-test"]))
List of Buildslaves
Buildbot documentation: http://buildbot.net/buildbot/docs/0.8.3/Buildslaves.html#Buildslaves
(Top level configuration: [1])
Again, I'm not doing anything unusual:
from buildbot.buildslave import BuildSlave
c['slaves'] = [BuildSlave("slave-name", "******")]
Errors
Running Buildbot
Couldn't listen on any XXXX: Address already in use
If you are seeing error messages about an address already in use:
$ buildbot start master
Following twistd.log until startup finished..
2011-06-03 16:08:17-0600 [-] Log opened.
2011-06-03 16:08:17-0600 [-] twistd 11.0.0 (/opt/local/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python 2.7.1) starting up.
2011-06-03 16:08:17-0600 [-] reactor class: twisted.internet.selectreactor.SelectReactor.
2011-06-03 16:08:17-0600 [-] Creating BuildMaster -- buildbot.version: 0.8.3p1
2011-06-03 16:08:17-0600 [-] loading configuration from /uufs/chpc.utah.edu/common/home/u0552682/codes/buildbot/master/master.cfg
2011-06-03 16:08:18-0600 [-] unable to import dnotify, so Maildir will use polling instead
2011-06-03 16:08:18-0600 [-] creating adbapi pool: sqlite3 ('/uufs/chpc.utah.edu/common/home/u0552682/codes/buildbot/master/state.sqlite',) {'check_same_thread': False, 'cp_noisy': True, 'cp_reconnect': True}
2011-06-03 16:08:18-0600 [-] twisted.spread.pb.PBServerFactory starting on 9989
2011-06-03 16:08:18-0600 [-] Starting factory <twisted.spread.pb.PBServerFactory instance at 0x10231c290>
2011-06-03 16:08:18-0600 [-] adding new builder runtests for category None
2011-06-03 16:08:18-0600 [-] trying to load status pickle from /uufs/chpc.utah.edu/common/home/u0552682/codes/buildbot/master/runtests/builder
2011-06-03 16:08:18-0600 [-] no saved status pickle, creating a new one
2011-06-03 16:08:18-0600 [-] added builder runtests in category None
2011-06-03 16:08:18-0600 [-] setBuilders._add: [<buildbot.util.loop.DelegateLoop instance at 0x101e11ea8>, <BuildSlave 'example-slave', current builders: >] [<Builder ''runtests'' at 4331782584>]
2011-06-03 16:08:18-0600 [-] adding IStatusReceiver <WebStatus on port tcp:8010 at 0x101f745a8>
2011-06-03 16:08:18-0600 [-] configuration update started
2011-06-03 16:08:18-0600 [-] Unhandled Error
Traceback (most recent call last):
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-macosx-10.6-x86_64.egg/twisted/application/service.py", line 184, in setServiceParent
self.parent.addService(self)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-macosx-10.6-x86_64.egg/twisted/application/service.py", line 303, in addService
service.privilegedStartService()
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-macosx-10.6-x86_64.egg/twisted/application/internet.py", line 357, in privilegedStartService
self._waitingForPort = self.endpoint.listen(self.factory)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-macosx-10.6-x86_64.egg/twisted/internet/endpoints.py", line 187, in listen
interface=self._interface)
--- <exception caught here> ---
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-macosx-10.6-x86_64.egg/twisted/internet/defer.py", line 104, in execute
result = callable(*args, **kw)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-macosx-10.6-x86_64.egg/twisted/internet/posixbase.py", line 419, in listenTCP
p.startListening()
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-macosx-10.6-x86_64.egg/twisted/internet/tcp.py", line 867, in startListening
raise CannotListenError, (self.interface, self.port, le)
twisted.internet.error.CannotListenError: Couldn't listen on any:8010: [Errno 48] Address already in use.
then it is likely there is already a Buildbot server running (or some other service that is using that port). In this case, you can edit your master.cfg configuration file and change the port 8010 to a different and non-standard port.
It is a good idea to make sure the Buildbot is stopped if you are changing the configuration file. Stop the buildbot by running:
buildbot stop master
and edit master.cfg to use non-standard ports (not 8010 and not 9989).
Next, start the buildbot again:
buildbot start master
Reconfiguring
Reconfiguration Failed
If you make a syntax error in your buildbot and try to reconfigure the buildbot, you will see an error like this:
2011-06-01 1:52:22 [-] loading configuration from /path/to/buildbot/master/master.cfg
2011-06-01 1:52:22 [-] error while parsing config file
2011-06-01 1:52:22 [-] error during loadConfig
2011-06-01 1:52:22 [-] Unhandled Error
Traceback (most recent call last):
File "path/to/buildbot/lib/python2.6/site-packages/Twisted-8.2.0-py2.6-linux-x86_64.egg/twisted/application/app.py", line 348, in runReactorWithLogging
reactor.run()
File "/path/to/buildbot/lib/python2.6/site-packages/Twisted-8.2.0-py2.6-linux-x86_64.egg/twisted/internet/base.py", line 1128, in run
self.mainLoop()
File "/path/to/buildbot/lib/python2.6/site-packages/Twisted-8.2.0-py2.6-linux-x86_64.egg/twisted/internet/base.py", line 1137, in mainLoop
self.runUntilCurrent()
File "/path/to/buildbot/lib/python2.6/site-packages/Twisted-8.2.0-py2.6-linux-x86_64.egg/twisted/internet/base.py", line 757, in runUntilCurrent
call.func(*call.args, **call.kw)
--- <exception caught here> ---
File "/path/to/buildbot/src/buildbot/master.py", line 511, in loadTheConfigFile
self.loadConfig(f)
File "/path/to/buildbot/src/buildbot/master.py", line 529, in loadConfig
exec f in localDict
exceptions.SyntaxError: EOL while scanning string literal (master.cfg, line 191)
2011-06-01 1:52:22 [-] The new config file is unusable, so I'll ignore it.
2011-06-01 1:52:22 [-] I will keep using the previous config file instead.
Reconfiguration failed. Please inspect the master.cfg file for errors,
correct them, then try 'buildbot reconfig' again.