Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase download speed #28

Closed
analog-cbarber opened this issue Jun 22, 2021 · 3 comments
Closed

Increase download speed #28

analog-cbarber opened this issue Jun 22, 2021 · 3 comments

Comments

@analog-cbarber
Copy link

The code currently uses a chunk size of 1024 bytes. Consider using a much larger size (1MB)

Also consider using a session.

See https://stackoverflow.com/questions/23369625/ideal-chunk-size-for-python-requests

@analog-cbarber
Copy link
Author

Playing with this a bit I really don't see any real performance difference when I change the chunk size on my machine and network, but using a common requests.Session object can make a big difference. I saw something like a 30% to 400% speedup when using a session. I think it may be a bigger win when packages are small.

analog-cbarber added a commit to analog-garage/conda-mirror that referenced this issue Jun 24, 2021
@analog-cbarber
Copy link
Author

@xhochy
Copy link
Member

xhochy commented Jun 24, 2021

This sounds very useful as this should get the TLS connection alive. This is probably the major cost for very small packages.

xhochy added a commit that referenced this issue Jun 25, 2021
Use requests.Session to improve download speed (issue #28)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants