Automated Testing (Buildbot)

Automated Testing (Buildbot)

Our automated testing it currently run though Buildbot. Buildbot is a master/slave system. This means there is a master process running somewhere which manages all the build slaves, testers, schedulers, etc. Then there are individual slave processes, potentially running on completely different machines (as is the case in LU's set up), which connect to the master and perform actions as and when the master process tells them to.

In the case of LU's testing setup, the master process is at diogenes.learningu.org:9580. We currently have the following slaves set up:

• hippo-slave -- Running on hippo.learningu.org, an Ubuntu VM.

(Updated 11/14/11 by Gurtej)

Pretty much all the documentation you need regarding Buildbot can be found in Buildbot's documentation on their site: http://buildbot.net/buildbot/docs/current/index.html.

Buildbot has a very user-friendly web interface, which give useful features like the Waterfall view for building, and the ability to force builds. LU's buildbot web interface is located at http://buildbot.learningu.org.

Setting up a new slave

Configure the master

First, you need to tell the master to look for the buildslave that we're going to create, and create a username and password for it. You do this by editing /lu/buildbot/master/master.cfg. Find the lines that say:

# the 'slaves' list defines the set of allowable buildslaves. Each element is
# a BuildSlave object, which is created with bot-name, bot-password.  These
# correspond to values given to the buildslave's mktap invocation.
from buildbot.buildslave import BuildSlave
c['slaves'] = [BuildSlave("someslave", "somepass", max_builds=1),
               BuildSlave("anotherslave", "somepass", max_builds=1),]

You need to add another BuildSlave object to the list of BuildSlaves. To do this, add to this list the following (setting your own name and password): BuildSlave("slave-username", "slave-pass", max_builds=1).

You have changed the master configuration file, but the master doesn't know this until you tell it so by going up a directory (cd ..), and calling sudo buildbot reconfig master, where master is the name of the directory containing your master process (in the case of LU it is just 'master').

Create the slave

Now you need to create the slave and connect it to the master process.

It may be useful to have the buildslave running in a python virtual environment. If you wish to do this, make sure virtualenv is installed, then activate the buildbot directory. To use the example of Hippo, you would cd to /mnt/ebs/buildbot then run virtualenv --no-site-packages sandbox and source sandbox/bin/activate.

Now create your buildslave: buildslave create-slave [slave-name] [master-location] [slave-username] [slave-password] where slave-username and slave-password are the same as the ones you added to master.cfg earlier. In the case of hippo: buildslave create-slave hippo-slave diogenes.learningu.org:9580 hippo-slave [hippo's password]. Then run the buildbot: buildslave start [slave-name] to start the buildslave process. In the case of hippo: buildslave start hippo-slave. This should connect with no errors and print a message saying that the slave attached. Congratulations, you've successfully set up your buildslave!

Add your slave to some builders

You have a slave now, but it's not useful until you add it to one or more builders. The builders are what actually get told to build. They then choose the first available buildslave that they contain and run the build on it. In the case of LU, we have the builders set up to have one builder per buildslave (to be able to force builds on specific slaves) and one builder ('test-all') that contains all the slaves. Test-all is set up with a scheduler to run tests whenever changes to the code happen. To change which builders your buildslave is included in, you need to once again edit master.cfg (located at /lu/buildbot/master/master.cfg on diogenes). Find the following section:

b1 = {'name': "test-all",
      'slavenames': ["someslave", "anotherslave"],
      'builddir': "test-all",
      'factory': f1, # This refers to the set of instructions on how to build (also in master.cfg)
      }
b2 = {'name': "test-someslave",
      'slavenames': ["someslave"],
      'builddir': "test-someslave",
      'factory': f1,
     }
b3 = {'name': "test-anotherslave",
      'slavenames': ["anotherslave"],
      'builddir': "test-anotherslave",
      'factory': f1,
     }
c['builders'] = [b1, b2, b3]

This creates and then adds all of the builders. If you want to add another builder, you should create and add it:

# Create your builder
b4 = {'name': "test-yourslave",
      'slavenames': ["yourslave"],
      'builddir': "test-yourslave",
      'factory': f1, # You may want to change this if you have want a different set of build instructions for your slave
     }
# Add your builder:
c['builders'] = [b1, b2, b3, b4]

In the case of LU, you should also add your buildslave to the test-all builder, by adding it to the list of slave names. Once again, don't forget to reconfig your master (sudo buildbot reconfig master). When you go to the buildbot site your builder should now show up.