-
Notifications
You must be signed in to change notification settings - Fork 3.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve documentation based on what I learned when I did loki setup. #461
Conversation
garo
commented
Apr 9, 2019
- Explain that api needs the start parameter set if you want to see logs older than one hour
- Explain that S3 and DynamoDB can also use instance roles in EC2
- Add couple of hints how to debug Promtail
- Add documentation how the Promtail prometheus scrape_configs work
* Explain that api needs the start parameter set if you want to see logs older than one hour * Explain that S3 and DynamoDB can also use instance roles in EC2 * Add couple of hints how to debug Promtail * Add documentation how the Promtail prometheus scrape_configs work
34e6a56
to
b23c6ff
Compare
please look at #447 to ensure no duplicate contents. |
Sorry for the typos, I did write that before the morning coffee. I don't think that the promtail.md duplicates any of the information what is already in the #447 as that document is about extracting labels from the log content by parsing. When that feature gets implemented additions to the promtail.md document are justified. |
d02a512
to
375ec5e
Compare
@@ -12,12 +12,14 @@ This can have several reasons: | |||
- Restarting promtail will not necessarily resend log messages that have been read. To force sending all messages again, delete the positions file (default location `/tmp/positions.yaml`) or make sure new log messages are written after both promtail and Loki have started. | |||
- Promtail is ignoring targets because of a configuration rule | |||
- Detect this by turning on debug logging and then look for `dropping target, no labels` or `ignoring target` messages. | |||
- Promtail cannot find the location of your log files. Check that the scrape_configs contains valid path setting for finding the logs in your worker nodes. | |||
- Your pods are running but not with the labels Promtail is expecting. Check the Promtail scape_configs. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
those part is a little duplicate with https://github.com/grafana/loki/blob/master/docs/troubleshooting.md#connecting-to-a-promtail-pod-to-troubleshoot
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The first (cannot find the location of your log files) reason was added based on the discussion on the slack where one user had this problem.
I added the second bullet based how I was personally hit with the fact that not all my streams were visible because I wasn't using compatible labels.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The "Connecting to a promtail pod to troubleshoot" is a good chapter with further explanation how to debug the subject. I feel that mentioning these two cases specifically adds valuable information.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a very welcome effort to document promtail.
Thanks for your contribution @garo 🎉