Skip to content

Commit

Permalink
add syslog receiver support (#2482)
Browse files Browse the repository at this point in the history
Add Syslog receiver
**Link to tracking Issue:** #2331 

**Testing:** Unit tests are included

**Documentation:** Syslog receiver README.md is included
  • Loading branch information
wph95 authored Mar 16, 2021
1 parent 14b484e commit 030675b
Show file tree
Hide file tree
Showing 14 changed files with 1,879 additions and 3 deletions.
3 changes: 3 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ require (
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/collectdreceiver v0.0.0-00010101000000-000000000000
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/dockerstatsreceiver v0.0.0-00010101000000-000000000000
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/filelogreceiver v0.0.0-00010101000000-000000000000
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/syslogreceiver v0.0.0-00010101000000-000000000000
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/jmxreceiver v0.0.0-00010101000000-000000000000
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/k8sclusterreceiver v0.0.0-00010101000000-000000000000
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/kubeletstatsreceiver v0.0.0-00010101000000-000000000000
Expand Down Expand Up @@ -178,6 +179,8 @@ replace github.com/open-telemetry/opentelemetry-collector-contrib/receiver/zooke

replace github.com/open-telemetry/opentelemetry-collector-contrib/receiver/filelogreceiver => ./receiver/filelogreceiver

replace github.com/open-telemetry/opentelemetry-collector-contrib/receiver/syslogreceiver => ./receiver/syslogreceiver

replace github.com/open-telemetry/opentelemetry-collector-contrib/receiver/memcachedreceiver => ./receiver/memcachedreceiver

replace github.com/open-telemetry/opentelemetry-collector-contrib/processor/groupbyattrsprocessor => ./processor/groupbyattrsprocessor
Expand Down
4 changes: 4 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -1110,6 +1110,8 @@ github.com/onsi/gomega v1.10.4/go.mod h1:g/HbgYopi++010VEqkFgJHKC09uJiW9UkXvMUuK
github.com/op/go-logging v0.0.0-20160315200505-970db520ece7/go.mod h1:HzydrMdWErDVzsI23lYNej1Htcns9BCg93Dk0bBINWk=
github.com/open-telemetry/opentelemetry-log-collection v0.16.0 h1:5DCDDrx95Hg4GPl2trng/CtYoGD0W0fjKQ3v6NQWmL4=
github.com/open-telemetry/opentelemetry-log-collection v0.16.0/go.mod h1:J1tOFFS2olDMZyiU1mtnCLnAv+uElKeYDbZ2ZAgjMU4=
github.com/open-telemetry/opentelemetry-log-collection v0.16.1-0.20210315170618-aa8d82509c31 h1:FruNezloXnyrkbsQhprOfk0i1M5ftEOZ3sG0+lpRJP4=
github.com/open-telemetry/opentelemetry-log-collection v0.16.1-0.20210315170618-aa8d82509c31/go.mod h1:9E7qy5/82Mr4FysOjfbxcHxk2+AXyOukDhdmezEjNdo=
github.com/opencontainers/go-digest v1.0.0-rc1/go.mod h1:cMLVZDEM3+U2I4VmLI6N8jQYUd2OVphdqWwCJHrFt2s=
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
Expand All @@ -1133,6 +1135,7 @@ github.com/openzipkin/zipkin-go v0.2.1/go.mod h1:NaW6tEwdmWMaCDZzg8sh+IBNOxHMPnh
github.com/openzipkin/zipkin-go v0.2.2/go.mod h1:NaW6tEwdmWMaCDZzg8sh+IBNOxHMPnhQw8ySjnjRyN4=
github.com/openzipkin/zipkin-go v0.2.5 h1:UwtQQx2pyPIgWYHRg+epgdx1/HnBQTgN3/oIYEJTQzU=
github.com/openzipkin/zipkin-go v0.2.5/go.mod h1:KpXfKdgRDnnhsxw4pNIH9Md5lyFqKUa4YDFlwRYAMyE=
github.com/orijtech/prometheus-go-metrics-exporter v0.0.6/go.mod h1:BiTx/ugZex8LheBk3j53tktWaRdFjV5FCfT2o0P7msE=
github.com/ory/go-acc v0.2.6/go.mod h1:4Kb/UnPcT8qRAk3IAxta+hvVapdxTLWtrr7bFLlEgpw=
github.com/ory/viper v1.7.5/go.mod h1:ypOuyJmEUb3oENywQZRgeAMwqgOyDqwboO1tj3DjTaM=
github.com/pact-foundation/pact-go v1.0.4/go.mod h1:uExwJY4kCzNPcHRj+hCR/HBbOOIwwtUjcrb0b5/5kLM=
Expand Down Expand Up @@ -1189,6 +1192,7 @@ github.com/prometheus/client_golang v0.9.3/go.mod h1:/TN21ttK/J9q6uSwhBd54HahCDf
github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo=
github.com/prometheus/client_golang v1.3.0/go.mod h1:hJaj2vgQTGQmVCsAACORcieXFeDPbaTKGT+JTgUa3og=
github.com/prometheus/client_golang v1.4.0/go.mod h1:e9GMxYsXl05ICDXkRhurwBS4Q3OK1iX/F2sw+iXX5zU=
github.com/prometheus/client_golang v1.4.1/go.mod h1:e9GMxYsXl05ICDXkRhurwBS4Q3OK1iX/F2sw+iXX5zU=
github.com/prometheus/client_golang v1.5.1/go.mod h1:e9GMxYsXl05ICDXkRhurwBS4Q3OK1iX/F2sw+iXX5zU=
github.com/prometheus/client_golang v1.6.0/go.mod h1:ZLOG9ck3JLRdB5MgO8f+lLTe83AXG6ro35rLTxvnIl4=
github.com/prometheus/client_golang v1.7.1/go.mod h1:PY5Wy2awLA44sXw4AOSfFBetzPP4j5+D6mVACh+pe2M=
Expand Down
3 changes: 1 addition & 2 deletions internal/stanza/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,7 @@ module github.com/open-telemetry/opentelemetry-collector-contrib/internal/stanza
go 1.14

require (
github.com/open-telemetry/opentelemetry-log-collection v0.16.0
github.com/pelletier/go-toml v1.8.0 // indirect
github.com/open-telemetry/opentelemetry-log-collection v0.16.1-0.20210315170618-aa8d82509c31
github.com/stretchr/testify v1.7.0
go.opentelemetry.io/collector v0.22.1-0.20210313012550-03904de3dd61
go.uber.org/zap v1.16.0
Expand Down
2 changes: 2 additions & 0 deletions internal/stanza/go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -692,6 +692,8 @@ github.com/onsi/gomega v1.9.0/go.mod h1:Ho0h+IUsWyvy1OpqCwxlQ/21gkhVunqlU8fDGcoT
github.com/op/go-logging v0.0.0-20160315200505-970db520ece7/go.mod h1:HzydrMdWErDVzsI23lYNej1Htcns9BCg93Dk0bBINWk=
github.com/open-telemetry/opentelemetry-log-collection v0.16.0 h1:5DCDDrx95Hg4GPl2trng/CtYoGD0W0fjKQ3v6NQWmL4=
github.com/open-telemetry/opentelemetry-log-collection v0.16.0/go.mod h1:J1tOFFS2olDMZyiU1mtnCLnAv+uElKeYDbZ2ZAgjMU4=
github.com/open-telemetry/opentelemetry-log-collection v0.16.1-0.20210315170618-aa8d82509c31 h1:FruNezloXnyrkbsQhprOfk0i1M5ftEOZ3sG0+lpRJP4=
github.com/open-telemetry/opentelemetry-log-collection v0.16.1-0.20210315170618-aa8d82509c31/go.mod h1:9E7qy5/82Mr4FysOjfbxcHxk2+AXyOukDhdmezEjNdo=
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
github.com/opencontainers/image-spec v1.0.1/go.mod h1:BtxoFyWECRxE4U/7sNtV5W15zMzWCbyJoFRP3s7yZA0=
github.com/opentracing-contrib/go-grpc v0.0.0-20191001143057-db30781987df/go.mod h1:DYR5Eij8rJl8h7gblRrOZ8g0kW1umSpKqYIBTgeDtLo=
Expand Down
2 changes: 1 addition & 1 deletion receiver/filelogreceiver/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ go 1.14
require (
github.com/observiq/nanojack v0.0.0-20201106172433-343928847ebc
github.com/open-telemetry/opentelemetry-collector-contrib/internal/stanza v0.0.0-00010101000000-000000000000
github.com/open-telemetry/opentelemetry-log-collection v0.16.0
github.com/open-telemetry/opentelemetry-log-collection v0.16.1-0.20210315170618-aa8d82509c31
github.com/stretchr/testify v1.7.0
go.opentelemetry.io/collector v0.22.1-0.20210313012550-03904de3dd61
go.uber.org/zap v1.16.0
Expand Down
2 changes: 2 additions & 0 deletions receiver/filelogreceiver/go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -799,6 +799,8 @@ github.com/onsi/gomega v1.9.0/go.mod h1:Ho0h+IUsWyvy1OpqCwxlQ/21gkhVunqlU8fDGcoT
github.com/op/go-logging v0.0.0-20160315200505-970db520ece7/go.mod h1:HzydrMdWErDVzsI23lYNej1Htcns9BCg93Dk0bBINWk=
github.com/open-telemetry/opentelemetry-log-collection v0.16.0 h1:5DCDDrx95Hg4GPl2trng/CtYoGD0W0fjKQ3v6NQWmL4=
github.com/open-telemetry/opentelemetry-log-collection v0.16.0/go.mod h1:J1tOFFS2olDMZyiU1mtnCLnAv+uElKeYDbZ2ZAgjMU4=
github.com/open-telemetry/opentelemetry-log-collection v0.16.1-0.20210315170618-aa8d82509c31 h1:FruNezloXnyrkbsQhprOfk0i1M5ftEOZ3sG0+lpRJP4=
github.com/open-telemetry/opentelemetry-log-collection v0.16.1-0.20210315170618-aa8d82509c31/go.mod h1:9E7qy5/82Mr4FysOjfbxcHxk2+AXyOukDhdmezEjNdo=
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
github.com/opencontainers/image-spec v1.0.1 h1:JMemWkRwHx4Zj+fVxWoMCFm/8sYGGrUVojFA6h/TRcI=
Expand Down
1 change: 1 addition & 0 deletions receiver/syslogreceiver/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
include ../../Makefile.Common
69 changes: 69 additions & 0 deletions receiver/syslogreceiver/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# Syslog Receiver

Parses Syslogs from tcp/udp using
the [opentelemetry-log-collection](https://github.com/open-telemetry/opentelemetry-log-collection) library.

Supported pipeline types: logs

> :construction: This receiver is in alpha and configuration fields are subject to change.
## Configuration

| Field | Default | Description |
| ---------- | ---------------- | ------------------------------------------------------------ |
| `id` | `syslog_input` | A unique identifier for the operator |
| `udp` |`nil` | Defined udp_input operator. (see the UDP configuration section) |
| `tcp` | `nil` | Defined tcp_input operator. (see the TCP configuration section) |
| `protocol` | required | The protocol to parse the syslog messages as. Options are `rfc3164` and `rfc5424` |
| `location` | `UTC` | The geographic location (timezone) to use when parsing the timestamp (Syslog RFC 3164 only). The available locations depend on the local IANA Time Zone database. [This page](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) contains many examples, such as `America/New_York`. |
| `timestamp` | `nil` | An optional [timestamp](https://github.com/open-telemetry/opentelemetry-log-collection/blob/main/docs/types/timestamp.md) block which will parse a timestamp field before passing the entry to the output operator |
| `severity` | `nil` | An optional [severity](https://github.com/open-telemetry/opentelemetry-log-collection/blob/main/docs/docs/types/severity.md) block which will parse a severity field before passing the entry to the output operator
| `attributes` | {} | A map of `key: value` labels to add to the entry's attributes |
| `resource` | {} | A map of `key: value` labels to add to the entry's resource |

### UDP Configuration

| Field | Default | Description |
| --- | --- | --- |
| `listen_address` | required | A listen address of the form `<ip>:<port>` |

### TCP Configuration

| Field | Default | Description |
| --- | --- | --- |
| `max_buffer_size` | `1024kib` | Maximum size of buffer that may be allocated while reading TCP input |
| `listen_address` | required | A listen address of the form `<ip>:<port>` |
| `tls` | | An optional `TLS` configuration (see the TLS configuration section) |

#### TLS Configuration

The `tcp_input` operator supports TLS, disabled by default.

| Field | Default | Description |
| --- | --- | --- |
| `cert_file` | | Path to the TLS cert to use for TLS required connections. |
| `key_file` | | Path to the TLS key to use for TLS required connections.|
| `ca_file` | | Path to the CA cert. For a client this verifies the server certificate. For a server this verifies client certificates. If empty uses system root CA. |
| `client_ca_file` | | (optional) Path to the TLS cert to use by the server to verify a client certificate. This sets the ClientCAs and ClientAuth to RequireAndVerifyClientCert in the TLSConfig. Please refer to godoc.org/crypto/tls#Config for more information.
|

## Example Configurations

TCP Configuration:

```yaml
- type: syslog_input
protocol: rfc5424
tcp:
listen_address: "0.0.0.0:54526"
```
UDP Configuration:
```yaml
- type: syslog_input
udp:
listen_address: "0.0.0.0:54526"
protocol: rfc3164
location: UTC
```
15 changes: 15 additions & 0 deletions receiver/syslogreceiver/doc.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
// Copyright 2021 OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package syslogreceiver
14 changes: 14 additions & 0 deletions receiver/syslogreceiver/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
module github.com/open-telemetry/opentelemetry-collector-contrib/receiver/syslogreceiver

go 1.14

require (
github.com/open-telemetry/opentelemetry-collector-contrib/internal/stanza v0.0.0-00010101000000-000000000000
github.com/open-telemetry/opentelemetry-log-collection v0.16.1-0.20210315170618-aa8d82509c31
github.com/stretchr/testify v1.7.0
go.opentelemetry.io/collector v0.22.1-0.20210313012550-03904de3dd61
go.uber.org/zap v1.16.0
gopkg.in/yaml.v2 v2.4.0
)

replace github.com/open-telemetry/opentelemetry-collector-contrib/internal/stanza => ../../internal/stanza
Loading

0 comments on commit 030675b

Please sign in to comment.