Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Group tests by language on Travis-CI #1029

Closed
methane opened this issue Aug 18, 2014 · 5 comments
Closed

Group tests by language on Travis-CI #1029

methane opened this issue Aug 18, 2014 · 5 comments

Comments

@methane
Copy link
Contributor

methane commented Aug 18, 2014

After #995 , tests using same language can re-use many things.
(Language toolchain, download cache for libraries, etc...)

Grouping tests by language may make CI very faster.

@hamiltont
Copy link
Contributor

Awesome idea. I'm worried we may need to setup a cache first--I think
building all three versions of python and then running all the python tests
would take more than 50 minutes

@methane
Copy link
Contributor Author

methane commented Aug 19, 2014

@hamiltont
Copy link
Contributor

Linking in #423 as it's critical to making this a success Issue closed

Download cache is already used for Python

Sorry, I should have been more clear. I meant we should setup a cache for building the dependencies on travis. I've got an initial example working. The idea is that we only run a build of python, or perl, etc, one time on travis. We then tar the resulting installs/ folder and upload it somewhere, and later travis runs can just download and untar, and they will not have to actually run the compilation themselves.

@hamiltont hamiltont added this to the Round 11 milestone Dec 19, 2014
@hamiltont
Copy link
Contributor

FYI to all - travis-ci has asked us to implement this because we occasionally generate quite a load on their system. It’s my intention to do this once R10 is done, and frankly I think it will reduce the frequency of seeing unexpected travis problems (tricky-to-debug network failure issues, mainly) if we’re not creating thousands of jobs across 10s of queues

@hamiltont
Copy link
Contributor

Ok, so I fully implemented this in #1446 . However, you cannot put all frameworks for a language into one travis job because of the 50 minute limit. I split it as much as necessary, but the end result is terrible (many mixed logs, hard to know where a framework is, hard to maintain).

Travis-CI contacted us and offered to increase our limits to better support maintaining the existing system with one job per framework, so for now this idea is shelved. To avoid the massive re-compilations, my current idea is to start using #1479 , which I have implemented here and appears to be workin ok for fixing the mono installation (which now completes in ~5 minutes)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants