Implement limited per-second hedging

[dannykopping]

This PR is an upstreaming of an implementation we have within Grafana Labs.

We need to restrict the number of hedged requests on a per-process level, in order to prevent large bursts of hedged requests when we hit the upstream service's limits and all requests start to become slow.

I've imported a dependency, which I believe violates one of the features, but I believe in this case it is warranted and the dependency can be trusted (it's an experimental package from the Go team, and may be upstreamed into the stdlib later).

I still need to write docs for usage, but wanted to get your thoughts on this first.

Credit to @cyriltovena for the initial implementation.

[cristaloleg]

Hey, thanks for the PR.

I understand your intention to get this merged but this reminds me our old discussion #18 (and the code example https://github.com/cristalhq/hedgedhttp/blob/main/examples_test.go#L89).

Looks like everything can be done via custom http.RoundTripper implementation, which can rate-limit requests. I'm not sure what will be solved by adding it to the library. Can you clarify please? (Forgive me my intentions to keep lib as simple as possible, but not simpler).

My 2c: keeping rate-limited http.RoundTripper outside of cristalhq/hedgedhttp somewhere in grafana/dskit/httputil make more sense, 'cause reusing it across all Grafana projects will be very simple and intuitive.

[dannykopping]

@cristaloleg haha I completely forgot about that discussion.

One benefit I see of having it included in the lib is the addition of the RateLimitedRoundTrips stat, but that's probably not a strong enough argument for something that could be passed in instead.

Thanks for the fast review! I'll close this.

[dannykopping]

My 2c: keeping rate-limited http.RoundTripper outside of cristalhq/hedgedhttp somewhere in grafana/dskit/httputil make more sense, 'cause reusing it across all Grafana projects will be very simple and intuitive.

Yes, that's a good idea actually

[cristaloleg]

Glad to hear that you find my idea useful, have a good day 👋