[internal] add network-level gRPC timeouts #16196
Conversation
|
This PR currently vendors the Tower |
e2edebc to
775bc67
Compare
|
(Forgot to mark this for review so I could ask the question in earlier comment. Doing so now. Doh.) |
|
Thanks!
I'm not following why we need to vendor, rather than importing the code? Either way, should probably add comments that this is vendored, and what source it comes from. |
Just look at the Pants-specific code in the file that accesses workunit store ... |
|
... which relates to my comment at #16196 (comment) about finding a different way to be able to increment that counter. |
|
ping |
775bc67 to
ffc9296
Compare
|
Latest commit ditches the vendored timeout layer and just uses a layer to count errors (which will be timeouts) coming from the Tower |
ffc9296 to
d1d5782
Compare
d1d5782 to
46e7668
Compare
[ci skip-build-wheels]
[ci skip-build-wheels]
[ci skip-build-wheels]
[ci skip-build-wheels]
[ci skip-build-wheels]
[ci skip-build-wheels]
46e7668 to
f6b19a7
Compare
|
Awesome, thank you! |
…18695) #16196 moved the cache read timeout down to the network layer, which made it much more accurate. But cache lookups also involve a number of calls to a remote `Store`, which did not have their own timeout. This change adds an RPC timeout for `Store` accesses to allow for retries of tar-pitted remote store RPCs, and adjusts the naming of the `--remote-cache-rpc-timeout-millis` option to make it clear that it applies to all cache operations (including writes).
…antsbuild#18695) pantsbuild#16196 moved the cache read timeout down to the network layer, which made it much more accurate. But cache lookups also involve a number of calls to a remote `Store`, which did not have their own timeout. This change adds an RPC timeout for `Store` accesses to allow for retries of tar-pitted remote store RPCs, and adjusts the naming of the `--remote-cache-rpc-timeout-millis` option to make it clear that it applies to all cache operations (including writes).
…(Cherry-pick of #18695) (#18697) #16196 moved the cache read timeout down to the network layer, which made it much more accurate. But cache lookups also involve a number of calls to a remote `Store`, which did not have their own timeout. This change adds an RPC timeout for `Store` accesses to allow for retries of tar-pitted remote store RPCs, and adjusts the naming of the `--remote-cache-rpc-timeout-millis` option to make it clear that it applies to all cache operations (including writes).
Add network level gRPC timeouts starting with the
check_action_cachefunction (which maps to the REAPI GetActionResult API).Fixes #16188.