[consul] Make max number of catalog services configurable #2383
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This change adds a mechanism to increase the number of consul services that can be queries by the agent: Currently, without this change, the culling mechanism in consul.py would remove any services (even whitelisted ones) in excess of the first 50. With this change, you can set `max_services` in consul.yaml, per consul agent. Further, to optionally remove any limit on the number of services, this makes it possible to set `max_services` to a negative number disable service culling entirely. This change also adds tests for the new negative-maximum behavior and straightens out the docs in consul.yaml.example.
|
Hi @antifuchs and thanks for opening this PR! This seems like a legit use case. |
|
It appears that this has been addressed in DataDog/integrations-core#140 (didn't realize that the present PR existed when I merged the other one), so I'm going to close this PR. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently, the consul check cuts off the list of catalogued services that it interrogates at 50, even when using a whitelist to pick out the most important ones. (Which is slightly frustrating if you have more than 50 important services!)
This PR adds a new consul.yaml configuration key which allows setting that limit to be higher, lower, or entirely non-existent (by setting it to be <1).
The configuration key is documented in consul.yaml.example (and I've taken the liberty of adjusting the config key docs for the whitelist to not imply that the whitelist can be used query for more than 50 services).
Testing
I have added a test and a new configuration key; tests pass locally, hope that Travis is happy too. We tried this out on a consul cluster running ~160 services, and it picks up the configured number of services there.