Skip to content
This repository has been archived by the owner on Aug 8, 2023. It is now read-only.

OnlineFileSource should throttle the number of concurrent requests #3858

Closed
jfirebaugh opened this issue Feb 8, 2016 · 2 comments
Closed

Comments

@jfirebaugh
Copy link
Contributor

Offline downloads can produce a large number of concurrent requests, bloating memory, overloading the network connection, and causing many of the requests to time out. In order to avoid these problems, OnlineFileSource should throttle concurrent requests at a reasonable number, placing throttled requests in a queue for execution once existing requests have completed.

This could be implemented in the offline code itself, but I think it makes more sense in OnlineFileSource, because there could be non-offline corner cases that also generate a large number of simultaneous requests, for example with huge screens. These should also be throttled.

@jfirebaugh
Copy link
Contributor Author

Included in #3715.

@tmpsantos
Copy link
Contributor

👍

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants