Skip to content

Commit

Permalink
[Encoding Extension] Added module outline (#23000)
Browse files Browse the repository at this point in the history
**Description:**

Adding the required steps to help manage a new component and reduce the
amount of noise generated in one PR.

**Link to tracking Issue:** 

https://github.com/open-telemetry/opentelemetry-collector/issues/6272

**Testing:** 
NA

**Documentation:**

Added the rough outline of the component development to provide more of
a public idea of the project outline.
Not strictly tied to it since it is more a user facing version of the
linked issue.
  • Loading branch information
MovieStoreGuy authored Jul 10, 2023
1 parent cd5c49b commit 398c974
Show file tree
Hide file tree
Showing 13 changed files with 561 additions and 5 deletions.
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ extension/asapauthextension/ @open-telemetry/collect
extension/awsproxy/ @open-telemetry/collector-contrib-approvers @Aneurysm9 @mxiamxia
extension/basicauthextension/ @open-telemetry/collector-contrib-approvers @jpkrohling @svrakitin
extension/bearertokenauthextension/ @open-telemetry/collector-contrib-approvers @jpkrohling @pavankrish123 @frzifus
extension/encodingextension/ @open-telemetry/collector-contrib-approvers @MovieStoreGuy
extension/headerssetterextension/ @open-telemetry/collector-contrib-approvers @jpkrohling @kovrus
extension/healthcheckextension/ @open-telemetry/collector-contrib-approvers @jpkrohling
extension/httpforwarder/ @open-telemetry/collector-contrib-approvers @atoulme @rmfitzpatrick
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/bug_report.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,7 @@ body:
- extension/awsproxy
- extension/basicauth
- extension/bearertokenauth
- extension/encoding
- extension/headerssetter
- extension/healthcheck
- extension/httpforwarder
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/feature_request.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ body:
- extension/awsproxy
- extension/basicauth
- extension/bearertokenauth
- extension/encoding
- extension/headerssetter
- extension/healthcheck
- extension/httpforwarder
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/other.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ body:
- extension/awsproxy
- extension/basicauth
- extension/bearertokenauth
- extension/encoding
- extension/headerssetter
- extension/healthcheck
- extension/httpforwarder
Expand Down
10 changes: 5 additions & 5 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -322,6 +322,11 @@ updates:
schedule:
interval: "weekly"
day: "wednesday"
- package-ecosystem: "gomod"
directory: "/extension/encodingextension"
schedule:
interval: "weekly"
day: "wednesday"
- package-ecosystem: "gomod"
directory: "/extension/headerssetterextension"
schedule:
Expand Down Expand Up @@ -1097,8 +1102,3 @@ updates:
schedule:
interval: "weekly"
day: "wednesday"
- package-ecosystem: "gomod"
directory: "/receiver/statsdreceiver"
schedule:
interval: "weekly"
day: "wednesday"
1 change: 1 addition & 0 deletions extension/encodingextension/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
include ../../Makefile.Common
72 changes: 72 additions & 0 deletions extension/encodingextension/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# Encoding extension

<!-- status autogenerated section -->
| Status | |
| ------------- |-----------|
| Stability | [development] |
| Distributions | [] |

[development]: https://github.com/open-telemetry/opentelemetry-collector#development
<!-- end autogenerated section -->

The encoding extension allows for other components to reference ingress/egress data formats
to allow for simplified operability.

_🚧 Under active development 🚧_

## Component Milestones

To help track what work needs to be done with this component, these are the currently active goals being
worked towards.

### Development

- CodecFactory: Allows other components to reference code by name and version
- Codec: Defining the expected interface for a codec
- Length Delimited Codec: Used fixed length payloads to represent a collection of data that can be merged together
- Add existing formats to the codec's libraries.
- Add support to the following components
- `file receiver`
- `file exporter`
- `kafka receiver`
- `kafka exporter`
- `kinesis exporter`
- Add codecs for open source formats, ie: `otlp`, `zipkin`, `jaeger`

Component support will be behind a feature gate to avoid change in behavior

### Alpha

- Adopt vendor formats as codecs
- Transition feature gate to default to using encoding extension
- Add dynamic mapping support for `avro`, `parquet`, and `gotemplate`
- Not fixed on these but I am aware that users are interested in these

### Beta

- Add support for remote schema management
- Using `AWS Glue`, `kafka schema registry`
- Performance break down of difference codecs (static, dynamic, remote)
- This is more to help provide information to users to expected performance impact.

### Stable

- All features complete
- Absent of bugs that impact core behaviours
- Dependencies can also be marked as stable

## Example configuration

```yaml
extensions:
encoding:

receivers:
kafka:
encoding:
name: zipkin/v1
format: proto
# ... other configuration values
```

[translators]: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/pkg/translator
5 changes: 5 additions & 0 deletions extension/encodingextension/doc.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

//go:generate mdatagen metadata.yml
package encodingextension // import "github.com/open-telemetry/opentelemetry-collector-contrib/extension/encodingextension"
31 changes: 31 additions & 0 deletions extension/encodingextension/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
module github.com/open-telemetry/opentelemetry-collector-contrib/extension/encodingextension

go 1.19

require go.opentelemetry.io/collector/component v0.81.0

require (
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/protobuf v1.5.3 // indirect
github.com/knadh/koanf v1.5.0 // indirect
github.com/knadh/koanf/v2 v2.0.1 // indirect
github.com/mitchellh/copystructure v1.2.0 // indirect
github.com/mitchellh/mapstructure v1.5.1-0.20220423185008-bf980b35cac4 // indirect
github.com/mitchellh/reflectwalk v1.0.2 // indirect
go.opentelemetry.io/collector/config/configtelemetry v0.81.0 // indirect
go.opentelemetry.io/collector/confmap v0.81.0 // indirect
go.opentelemetry.io/collector/featuregate v1.0.0-rcv0013 // indirect
go.opentelemetry.io/collector/pdata v1.0.0-rcv0013 // indirect
go.opentelemetry.io/otel v1.16.0 // indirect
go.opentelemetry.io/otel/metric v1.16.0 // indirect
go.opentelemetry.io/otel/trace v1.16.0 // indirect
go.uber.org/atomic v1.10.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.uber.org/zap v1.24.0 // indirect
golang.org/x/net v0.11.0 // indirect
golang.org/x/sys v0.9.0 // indirect
golang.org/x/text v0.10.0 // indirect
google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 // indirect
google.golang.org/grpc v1.56.1 // indirect
google.golang.org/protobuf v1.31.0 // indirect
)
Loading

0 comments on commit 398c974

Please sign in to comment.