-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement probe_failure_info
Metric
#1334
Draft
slrtbtfs
wants to merge
25
commits into
prometheus:master
Choose a base branch
from
slrtbtfs:probe_failure_info
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
4df06d6
to
73630e8
Compare
4630786
to
4f0dddb
Compare
b79e00e
to
0d443ed
Compare
Signed-off-by: Tobias Guggenmos <guggenmos@dfn-cert.de>
Signed-off-by: Tobias Guggenmos <guggenmos@dfn-cert.de>
Signed-off-by: Tobias Guggenmos <guggenmos@dfn-cert.de>
Signed-off-by: Tobias Guggenmos <guggenmos@dfn-cert.de>
Signed-off-by: Tobias Guggenmos <guggenmos@dfn-cert.de>
Signed-off-by: Tobias Guggenmos <guggenmos@dfn-cert.de>
Signed-off-by: Tobias Guggenmos <guggenmos@dfn-cert.de>
Signed-off-by: Tobias Guggenmos <guggenmos@dfn-cert.de>
Signed-off-by: Tobias Guggenmos <guggenmos@dfn-cert.de>
Signed-off-by: Tobias Guggenmos <guggenmos@dfn-cert.de>
Signed-off-by: Tobias Guggenmos <guggenmos@dfn-cert.de>
Signed-off-by: Tobias Guggenmos <guggenmos@dfn-cert.de>
Signed-off-by: Tobias Guggenmos <guggenmos@dfn-cert.de>
Signed-off-by: Tobias Guggenmos <guggenmos@dfn-cert.de>
Signed-off-by: Tobias Guggenmos <guggenmos@dfn-cert.de>
Signed-off-by: Tobias Guggenmos <guggenmos@dfn-cert.de>
Signed-off-by: Tobias Guggenmos <guggenmos@dfn-cert.de>
Signed-off-by: Tobias Guggenmos <guggenmos@dfn-cert.de>
Signed-off-by: Tobias Guggenmos <guggenmos@dfn-cert.de>
Signed-off-by: Tobias Guggenmos <guggenmos@dfn-cert.de>
Signed-off-by: Tobias Guggenmos <guggenmos@dfn-cert.de>
0d443ed
to
274a2f2
Compare
Signed-off-by: Tobias Guggenmos <guggenmos@dfn-cert.de>
Signed-off-by: Tobias Guggenmos <guggenmos@dfn-cert.de>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a first draft forimplementing #1077.This make information about Probe failures that is already logged is additionally available via the
probe_failure_info
metric.The underlying mechanisms are mainly implemented in
prober/prober.go
andprober/handler.go
, the rest of the diff is mostly refactoring.The main change is adding a new type
ProbeResult
, which replaces the boolean return type of Probes. For failed Probes this type includes diagnostic information which is then published in theprobe_failure_reason
metric. If error messages contain information that is quite large or may change very frequently, leading to a high cardinality, its not included in this metric and instead just logged. .The underlying mechanisms are mainly implemented in
prober/prober.go
andprober/handler.go
, the rest is mostly refactoring. Since this new type is used in almost every Probe function and their tests, the diff got quite big. I hope that doesn't make the review to cumbersomeAs a side effect, now most tests not only check whether a probe succeeded, but also whether failed tests failed with the correct error message. Having the error message in the return type also allowed to remove some more complex testing code that used to extract this information from the prober logs.
Before merging, this should still get some improved tests and polishing, however I'd like to get some early feedback about whether you agree with the implementation approach in general.There is still space for improvement as some error messages provide only limited detail (e.g. the generic
HTTP request failed
one). IMO this makes most sense as a follow up once the basic implementation of this metric is merged.