Why is the test suite so slow?

[bryanwweber]

On the recent commits (e.g., https://travis-ci.org/pr-omethe-us/PyKED/builds/285888781) the Travis tests take almost 10 minutes, and I'm seeing a similar problem on my local machine. This shouldn't be the case (or at least, I don't think it used to be the case), so what's going on?

[bryanwweber]

We're also starting to get 500 errors from the DOI lookup... I wonder if there's either a problem with Habanero, or if its rate-limited.

[kyleniemeyer]

Agreed, I've also noticed these are slow... not sure why.

[bryanwweber]

Based on the jobs here: https://travis-ci.org/pr-omethe-us/PyKED/builds/287938661 it seems like a significant fraction of the time is spent in the SSL library, presumably trying to connect to the CrossRef servers. Those jobs only took ~3 minutes, not quite as long as some other recent jobs. Might just be an intermittent connection issue with CrossRef.

[bryanwweber]

We might be able to create a single Request object from Habanero and reuse it. This might help with some of the rate-limiting or the repeated handshakes that are required with the API website.

Should probably also consider the CrossRef API etiquette: https://github.com/CrossRef/rest-api-doc#etiquette

And, they made a recent change to their TOS for "polite" users: https://github.com/CrossRef/rest-api-doc#good-manners--more-reliable-service

Basically, we need to add a mailto in the HTTP headers we send to Crossref. This should be possible to do from within Habanero, so let's look into that.