phantomjs for local tests

[edsu]

take from @csillag's email on dev

SauceLabs is great, because

• Its browser is very real
• It can provide real cross-browser (or even cross-platform) testing,
  alleviating the need for the developer to keep all those browsers in
  working order on his own machine

PhantomJS is great, because

• It's easy to set up
• It can be run anywhere, independently, even without network,

Ideally, we could have a test suite which

• can be run locally with PhantomJS (which could also be used when
  developing test cases)
• Upon commit, is automatically executed with SauceLabs, using all the
  supported browsers.

Is that possible?

[edsu]

Unfortunately test_annotation doesn't seem to work using PhantomJS instead of Firefox as the webdriver.

E       NoSuchElementException: Message: u'Error Message => \'Unable to find element with class name \'annotation\'\'\n caused by Request => {"headers":{"Accept":"application/json","Accept-Encoding":"identity","Connection":"close","Content-Length":"99","Content-Type":"application/json;charset=UTF-8","Host":"127.0.0.1:55777","User-Agent":"Python-urllib/2.7"},"httpVersion":"1.1","method":"POST","post":"{\\"using\\": \\"class name\\", \\"sessionId\\": \\"9252b7c0-0766-11e3-87a1-f96ba2cdae2c\\", \\"value\\": \\"annotation\\"}","url":"/element","urlParsed":{"anchor":"","query":"","file":"element","directory":"/","path":"/element","relative":"/element","port":"","host":"","password":"","user":"","userInfo":"","authority":"","protocol":"","source":"/element","queryKey":{},"chunks":["element"]},"urlOriginal":"/session/9252b7c0-0766-11e3-87a1-f96ba2cdae2c/element"}' ; Screenshot: available via screen

If you want to try for yourself just change this line in tests/functional/__init__.py:

self.driver = webdriver.Firefox()

to:

self.driver = webdriver.PhantomJS()

[edsu]

I saved a screenshot before the find element failure:

driver.save_screenshot("screen.png")
annotation = driver.find_element_by_class_name('annotation')

and this is what I got.

Kind of strange because it looks like it never logged in, but test_login.py works fine ...

[csillag]

Since PhantomJS is based on old version of WebKit (see ariya/phantomjs#10031), so it should be closer to Chrome than to Firefox. Therefore, when encountering FF vs Phantom differences, it might make sense to check with Chrome, as a middle ground.

[csillag]

It would be interesting to see a screenshot after test_login.py has succeeded.

[csillag]

One more note: it looks like Sauce Labs has some support for capturing the console log. It would be great if we could incorporate this on our testing system.

[edsu]

@csillag tests pass with chromedriver

[edsu]

Since you thought it would be interesting, here's a screenshot of what test_login.py looks like at the end.

[csillag]

If the tests work with chrome, but not with PhantomJs, then we can do
four things:

• Give up on using PhantomJS
• Wait for PhantomJs to be updated, so that the incompatibilities go away
• Find out where our tests fail, and and workarounds for the
  incompatibilities
• Help with updating PhantomJS, see Bleeding-edge WebKit ariya/phantomjs#10031 and Switch to Qt 5 ariya/phantomjs#10448

[csillag]

So it looks like login really does work with the test case written
specifically for that. But then why does it fail in the other test case?
Is the code not exactly the same?

[tilgovi]

We're now running unit tests of the front-end code through PhantomJS and the selenium tests have been disabled.