Test engineering FAQ

General questions

How do I get started with edX development?

See Developing on the edX Developer Stack for details as to how to develop and debug the edX platform.

Where is the general documentation on testing the edx_platform code?

Here is a link to doc included in the repo itself on Writing and Running Tests. Many questions are answered here.

I'm working with devstack and want to debug the Jasmine or Acceptance tests in the browser on my host system. How do I do that?

• First off, for Mac OS you will need XQuartz installed to support X Windows. We have tested with version 2.7.5.
• Make sure this stanza is in your Vagrantfile (in the devstack directory, from which you usually vagrant up). It should be, as this was merged in on 2/28/2014.

  # Enable X11 forwarding so we can interact with GUI applications
  if ENV['VAGRANT_X11']
      config.ssh.forward_x11 = true
  end

• Set the VAGRANT_X11 environment variable on your host machine, then reload the image. Note that the reload will not reprovision your vagrant image if it has already been provisioned. It will redo the port forwarding and setting up of the file shares. See the vagrant docs for more info on the vagrant commands.

export VAGRANT_X11=1
vagrant reload

• ssh into the vagrant image. Note you will be the 'vagrant' user. Start up firefox, then quit it. (It will be passed through to your host machine's display).

vagrant ssh

firefox

• Now try it as the edxapp user.

sudo su edxapp

firefox

• Once this has been set up, if you want to run the acceptance tests without browser windows popping up, redirect the DISPLAY environment variable.

export DISPLAY=:1

• If you want to switch back to using your host system's display:

export DISPLAY=localhost:10.0

Everything was working at first, but now my tests hang and fail without launching the host browser. What happened?

If you are seeing messages like this on a Mac:

WebDriverException: Message: 'The browser appears to have exited before we could connect. The output was: Error: cannot open display: localhost:10.0\n'

then you may be experiencing X11 forwarding timeouts. The problem and solution are discussed in detail here, and the quick fix is to increase the value for ForwardX11Timeout in /etc/ssh_config to a nice high value (596h is the apparent maximum).

I've changed some code and in the diff-cover report, those lines are coming up as uncovered. But I know I'm testing them!

• Changes to .coffee files will always show up as uncovered by diff-cover. This is because JsCover is the coverage reporter and it doesn't know about .coffee files, just .js files.
• Changes to /common/lib files that are covered with tests under /foo/djangoapps (where foo = common, cms, lms) are reported as uncovered. That's not a unit test. Your test code should also be somewhere under /common/lib, close to what you are testing. Common library functionality should not presume a Django implementation.