Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Caching bigger downloads on the host machine would speed up rebuild times for the vm #21

Closed
lo5an opened this issue Mar 19, 2015 · 8 comments

Comments

@lo5an
Copy link
Contributor

lo5an commented Mar 19, 2015

Saving bigger downloads in a cache in the mounted /vagrant folder would speed up subsequent rebuild times.

@ruebot
Copy link
Contributor

ruebot commented Mar 23, 2015

Thoughts on adding the downloads folder to the repo? This would close the ticket, but it would also make the repo bigger, and we'd be distributing a bunch of binaries.

@lo5an
Copy link
Contributor Author

lo5an commented Mar 23, 2015

I like the idea of the script downloading stuff, but then caching it between machine rebuilds. I don't think we want to keep our own copy of the binaries.

@ruebot
Copy link
Contributor

ruebot commented Mar 23, 2015

Welp, it looks like we can't even do it!

$ git push origin issue-21 
Counting objects: 14, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (12/12), done.
Writing objects: 100% (12/12), 345.71 MiB | 1.20 MiB/s, done.
Total 12 (delta 1), reused 0 (delta 0)
remote: warning: File downloads/fits-0.8.4_0.zip is 70.63 MB; this is larger than GitHub's recommended maximum file size of 50 MB
remote: error: GH001: Large files detected.
remote: error: Trace: 9cd5e4b75ac0b6a2810a79d8c84faadd
remote: error: See http://git.io/iEPt8g for more information.
remote: error: File downloads/fcrepo-installer-3.8.0.jar is 145.32 MB; this exceeds GitHub's file size limit of 100 MB
remote: error: File downloads/solr-4.2.0.tgz is 111.14 MB; this exceeds GitHub's file size limit of 100 MB
To git@github.com:Islandora-Labs/islandora_vagrant.git
 ! [remote rejected] issue-21 -> issue-21 (pre-receive hook declined)
error: failed to push some refs to 'git@github.com:Islandora-Labs/islandora_vagrant.git'

So, I guess we're good to go with b3ec9b4 closing the ticket. If that's the case, I'll let @lo5an decide since he created the issue.

@whikloj
Copy link
Contributor

whikloj commented Mar 23, 2015

There is also the possibility of:

  1. Replace apt-get with apt-get -d to download source into the downloads directory.
  2. Use dpkg -i to install the packages from the downloads directory.
    This would extend the download once, install many idea. Just not sure how it would handle dependencies.

@ruebot
Copy link
Contributor

ruebot commented Mar 23, 2015

...and we should probably add downloads to .gitignore?

@ruebot
Copy link
Contributor

ruebot commented Mar 23, 2015

@whikloj++ #28

@lo5an
Copy link
Contributor Author

lo5an commented Mar 23, 2015

Maybe a vagrant plugin like https://github.com/fgrehm/vagrant-cachier would be better for caching system packages?

Updated to expand:

I've only been using vagrant-cachier for a week or so, but it's been working well so far, and I like that it handles more than just .debs. It seems to be a reasonably active project, and it's nice and modular, so we could recommend that people use it without making it a dependency. I've got the vagrant-cachier config in a Vagrantfile in my lookup path so that it applies to all of my vagrant vms.

@lo5an
Copy link
Contributor Author

lo5an commented Mar 23, 2015

That said, caching OS packages is definitely beyond the scope of what I was originally thinking, so I'm happy to mark this as closed and create a new issue for package caching :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants