Skip to content

Commit

Permalink
Update Readme
Browse files Browse the repository at this point in the history
  • Loading branch information
jriguera committed Jul 25, 2024
1 parent 1be3041 commit 542b2fe
Showing 1 changed file with 27 additions and 59 deletions.
86 changes: 27 additions & 59 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,66 +1,53 @@
# Context processor
# Context OTEL processor

The purpose of this code is quickly integrate a new component "Context processor" in OpenTelemetry collector. This is a temporary solution while this functionality is not available in upstream.
A processor to manipulate the Context Medatada in OTEL Collector. This processor allows to move resource attributes of Metrics, Logs and Traces to the context, which makes possible to use extensions like `headers_setter` dynamically.

Typical use cases:

* Be able to dynamically define tenants for Mimir/Cortex, Loki and Tempo
* Dynamically define metadata attributes in the context, to offer a link to pass resource attribute to extensions
* Change metadata generated from the receivers

The purpose of this code is quickly integrate a new component "Context processor" in our OpenTelemetry collectors. This is a temporary solution while this functionality is not available in upstream.

You can read about what Context Processor does in [otelcol-dev/contextprocessor/README.md](./otelcol-dev/contextprocessor/README.md). And you can see an example configuration in [otelcol-dev/otelcol.yaml](./otelcol-dev/otelcol.yaml)

# About

This is a quick hands-on guide, describing a basic and minimum setup of a custom metrics processor. There are no tests, information about styles, best practices ... You can see this repo as a short version of https://opentelemetry.io/docs/collector/custom-collector/

Also, to get starting contributing in OpenTelemetry, please have a look at these resources:
To get starting contributing in OpenTelemetry, please have a look at these resources:

* OpenTelemetry Collector: https://github.com/open-telemetry/opentelemetry-collector/blob/main/CONTRIBUTING.md
* OpenTelemetry Collector-Contrib: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CONTRIBUTING.md

# Starting ...
# Building the collector

1. Get golang installed: https://go.dev/doc/install
2. Install the [ocb](https://github.com/open-telemetry/opentelemetry-collector/tree/main/cmd/builder) tool: https://github.com/open-telemetry/opentelemetry-collector/releases

## ... from zero

If you want to get to your own custom collector:
If you want to get to your own custom collector compiled or updated:


1. Generate a `builder-config.yaml`. Adjust `name`, `output_path`, `otelcol_version` and the gomod versions, if needed.
1. Update the file `builder-config.yaml`, specially the new custom version in `version`. To completely update all components and add new ones as in the [Contrib distribution](https://github.com/open-telemetry/opentelemetry-collector-contrib), you can copy it from upstream https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/cmd/otelcontribcol/builder-config.yaml and adjust `name`, `output_path`, `otelcol_version` and all the gomod versions and make sure you remove the section `replaces`.
```
cat <<EOF > builder-config.yaml
dist:
name: otelcol-dev
description: Basic OTel Collector distribution for Developers
description: Local OpenTelemetry Collector Contrib binary, testing only.
output_path: ./otelcol-dev
otelcol_version: 0.88.0
extensions:
- gomod: go.opentelemetry.io/collector/extension/zpagesextension v0.88.0
- gomod: go.opentelemetry.io/collector/extension/ballastextension v0.88.0
- gomod: github.com/open-telemetry/opentelemetry-collector-contrib/extension/pprofextension v0.88.0
version: 0.105.0-sn3
otelcol_version: 0.105.0
```

exporters:
- gomod: go.opentelemetry.io/collector/exporter/debugexporter v0.88.0
- gomod: go.opentelemetry.io/collector/exporter/otlpexporter v0.88.0
- gomod: go.opentelemetry.io/collector/exporter/otlphttpexporter v0.88.0
- gomod: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/prometheusremotewriteexporter v0.88.0
processors:
- gomod: go.opentelemetry.io/collector/processor/batchprocessor v0.88.0
- gomod: go.opentelemetry.io/collector/processor/memorylimiterprocessor v0.88.0
- gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/attributesprocessor v0.88.0
- gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/filterprocessor v0.88.0
- gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/groupbyattrsprocessor v0.88.0
- gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourceprocessor v0.88.0
- gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/routingprocessor v0.88.0
- gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/transformprocessor v0.88.0
4. Add the Context processor component to the processors section of `builder-config.yaml` (with the same reference you used in the `go mod init ...`). Create another section to replace this module name with the local directory where the code is (`./contextprocessor` in this case). Put the same version as the rest of the modules (this only to avoid a build error with `ocb`, is not important for the compiler!)
```
processors:
- gomod: github.com/springernature/o11y-otel-contextprocessor/processor/contextprocessor v0.105.0
receivers:
- gomod: go.opentelemetry.io/collector/receiver/otlpreceiver v0.88.0
- gomod: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/prometheusreceiver v0.88.0
replaces:
- github.com/springernature/o11y-otel-contextprocessor/processor/contextprocessor => ./contextprocessor
```

connectors:
- gomod: go.opentelemetry.io/collector/connector/forwardconnector v0.88.0
EOF
```
2. Run `ocb --config builder-config.yaml`. A new directory `otelcol-dev` (or what you have defined in `output_path`) is created. The code generated should not be touched, all changes should be done via the `builder-config.yaml` file. You can see the builder configuration of the OpenTelemetry contrib here: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/cmd/otelcontribcol/builder-config.yaml
2. Run `ocb --config builder-config.yaml`. A new directory `otelcol-dev` (or what you have defined in `output_path`) is created. The code generated should not be touched, all changes should be done via the `builder-config.yaml` file.
3. Go to new `otelcold-dev` and you will see a new executable `otelcol-dev` (or the value defined in `name`). You can execute it by providing a basic configuration file `otelcold-dev --config otelcol.yaml`:
```
cat <<EOF > otelcol.yaml
Expand Down Expand Up @@ -100,23 +87,4 @@ If you want to get to your own custom collector:
```
With this configuration the collector will scrape its own metrics.
## ... from the current repo
You can have a look to the folder `otelcol-dev/helloworldmetricsprocessor` and start modifying the code. Or
1. Create a new directory like `helloworldmetricsprocessor`.
2. Create a new golang module `go mod init github.com/jriguera/otel-helloworldprocessor/otelcol-dev/helloworldmetricsprocessor` (change according to your choice)
3. Create new golang files. Use same package with same name as the directory `helloworldmetricsprocessor`:
1. `config.go` to define the component parameters and the function to validate them.
2. `factory.go` to define the facture which create instances of the new component. Use the name `NewFactory` and the factory provided by the component package (eg `processor.NewFactory`)
3. Other files to run the task (eg `processor.go`).
4. Add the component to the section of `builder-config.yaml` with the same reference you used in the `go mod init ...`. Create another section to replace this module name with the local directory created in point 1.
```
processors:
- gomod: github.com/jriguera/otel-helloworldprocessor/otelcol-dev/helloworldmetricsprocessor v0.88.0

replaces:
- github.com/jriguera/otel-helloworldprocessor/otelcol-dev/helloworldmetricsprocessor => ./helloworldmetricsprocessor
```
5. Finally run `ocb --config builder-config.yaml` to generate the binary with the new component

0 comments on commit 542b2fe

Please sign in to comment.