Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Syslog Improvements (2) #459

Merged

Conversation

maxmoehl
Copy link
Member

Summary

This PR corrects and improves the direct-to-syslog logging of access logs for Gorouter.

  • does not log an empty new line when writing to syslog
  • allows the operator to configure truncation on syslog, as syslog servers usually have a message limit
  • significant performance improvement to the syslog implementation

The syslog implementation in Go has not been maintained by the Go authors since around 2015. While there are forks of the syslog code, all of them are also dormant. This PR inlines and significantly rewrites the syslog implementation, focusing on the following performance improvements:

  • No use of the expensive fmt.Sprintf
  • Determine all "static" fields (e.g. the PID) on initialization, not per message
  • Correct handling of UTF-8 BOM according to RFC5424

Limitations:

  • Only TCP and UDP are supported, as described in the routing-release property for the syslog network. All other means (unix sockets, etc.) are not supported.

Backward Compatibility

Breaking Change? Yes

  • This PR moves the syslog implementation from the BSD-style syslog format, which is not standardised, to the RFC5424 format.

maxmoehl and others added 6 commits December 15, 2024 20:37
* do not log an empty new line when writing to syslog
* allow operator to configure truncation on syslog
The go standard library implementation of log/syslog has been abandoned
and is no longer maintained. In our tests with syslog logging in
gorouter we noticed a significant performance penalty over file based
logging. However, with file based logging we frequently observe broken
logs because logrotate interferes with log reading / writing.

This commit inlines the relevant file from log/syslog and refactors the
implementation to be on-par with the file based logging.

Co-authored-by: Alexander Lais <alexander.lais@sap.com>
@ameowlia ameowlia merged commit b610356 into cloudfoundry:main Dec 16, 2024
1 check passed
Dariquest pushed a commit to sap-contributions/gorouter that referenced this pull request Dec 17, 2024
* feat: make syslog logging useable

* do not log an empty new line when writing to syslog
* allow operator to configure truncation on syslog

* fix: allow syslog-only logging

* feat: replace log/syslog with internal version

The go standard library implementation of log/syslog has been abandoned
and is no longer maintained. In our tests with syslog logging in
gorouter we noticed a significant performance penalty over file based
logging. However, with file based logging we frequently observe broken
logs because logrotate interferes with log reading / writing.

This commit inlines the relevant file from log/syslog and refactors the
implementation to be on-par with the file based logging.

Co-authored-by: Alexander Lais <alexander.lais@sap.com>

* fix: rename testState.newRequest -> testState.newGetRequest

* test: add integration test for syslog access logs

* test: check errors

---------

Co-authored-by: Alexander Lais <alexander.lais@sap.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

Successfully merging this pull request may close these issues.

2 participants