From 542b2feba23cb23a687eed82200ea94f08bc8210 Mon Sep 17 00:00:00 2001 From: Jose Riguera Date: Fri, 26 Jul 2024 01:05:18 +0200 Subject: [PATCH] Update Readme --- README.md | 86 +++++++++++++++++-------------------------------------- 1 file changed, 27 insertions(+), 59 deletions(-) diff --git a/README.md b/README.md index 6e2adba..b6f2e38 100644 --- a/README.md +++ b/README.md @@ -1,66 +1,53 @@ -# Context processor +# Context OTEL processor -The purpose of this code is quickly integrate a new component "Context processor" in OpenTelemetry collector. This is a temporary solution while this functionality is not available in upstream. +A processor to manipulate the Context Medatada in OTEL Collector. This processor allows to move resource attributes of Metrics, Logs and Traces to the context, which makes possible to use extensions like `headers_setter` dynamically. + +Typical use cases: + +* Be able to dynamically define tenants for Mimir/Cortex, Loki and Tempo +* Dynamically define metadata attributes in the context, to offer a link to pass resource attribute to extensions +* Change metadata generated from the receivers + +The purpose of this code is quickly integrate a new component "Context processor" in our OpenTelemetry collectors. This is a temporary solution while this functionality is not available in upstream. You can read about what Context Processor does in [otelcol-dev/contextprocessor/README.md](./otelcol-dev/contextprocessor/README.md). And you can see an example configuration in [otelcol-dev/otelcol.yaml](./otelcol-dev/otelcol.yaml) # About -This is a quick hands-on guide, describing a basic and minimum setup of a custom metrics processor. There are no tests, information about styles, best practices ... You can see this repo as a short version of https://opentelemetry.io/docs/collector/custom-collector/ - -Also, to get starting contributing in OpenTelemetry, please have a look at these resources: +To get starting contributing in OpenTelemetry, please have a look at these resources: * OpenTelemetry Collector: https://github.com/open-telemetry/opentelemetry-collector/blob/main/CONTRIBUTING.md * OpenTelemetry Collector-Contrib: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CONTRIBUTING.md -# Starting ... +# Building the collector 1. Get golang installed: https://go.dev/doc/install 2. Install the [ocb](https://github.com/open-telemetry/opentelemetry-collector/tree/main/cmd/builder) tool: https://github.com/open-telemetry/opentelemetry-collector/releases -## ... from zero -If you want to get to your own custom collector: +If you want to get to your own custom collector compiled or updated: + -1. Generate a `builder-config.yaml`. Adjust `name`, `output_path`, `otelcol_version` and the gomod versions, if needed. +1. Update the file `builder-config.yaml`, specially the new custom version in `version`. To completely update all components and add new ones as in the [Contrib distribution](https://github.com/open-telemetry/opentelemetry-collector-contrib), you can copy it from upstream https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/cmd/otelcontribcol/builder-config.yaml and adjust `name`, `output_path`, `otelcol_version` and all the gomod versions and make sure you remove the section `replaces`. ``` - cat < builder-config.yaml dist: name: otelcol-dev - description: Basic OTel Collector distribution for Developers + description: Local OpenTelemetry Collector Contrib binary, testing only. output_path: ./otelcol-dev - otelcol_version: 0.88.0 - - extensions: - - gomod: go.opentelemetry.io/collector/extension/zpagesextension v0.88.0 - - gomod: go.opentelemetry.io/collector/extension/ballastextension v0.88.0 - - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/extension/pprofextension v0.88.0 + version: 0.105.0-sn3 + otelcol_version: 0.105.0 + ``` - exporters: - - gomod: go.opentelemetry.io/collector/exporter/debugexporter v0.88.0 - - gomod: go.opentelemetry.io/collector/exporter/otlpexporter v0.88.0 - - gomod: go.opentelemetry.io/collector/exporter/otlphttpexporter v0.88.0 - - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/prometheusremotewriteexporter v0.88.0 - - processors: - - gomod: go.opentelemetry.io/collector/processor/batchprocessor v0.88.0 - - gomod: go.opentelemetry.io/collector/processor/memorylimiterprocessor v0.88.0 - - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/attributesprocessor v0.88.0 - - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/filterprocessor v0.88.0 - - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/groupbyattrsprocessor v0.88.0 - - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourceprocessor v0.88.0 - - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/routingprocessor v0.88.0 - - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/transformprocessor v0.88.0 +4. Add the Context processor component to the processors section of `builder-config.yaml` (with the same reference you used in the `go mod init ...`). Create another section to replace this module name with the local directory where the code is (`./contextprocessor` in this case). Put the same version as the rest of the modules (this only to avoid a build error with `ocb`, is not important for the compiler!) + ``` + processors: + - gomod: github.com/springernature/o11y-otel-contextprocessor/processor/contextprocessor v0.105.0 - receivers: - - gomod: go.opentelemetry.io/collector/receiver/otlpreceiver v0.88.0 - - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/prometheusreceiver v0.88.0 + replaces: + - github.com/springernature/o11y-otel-contextprocessor/processor/contextprocessor => ./contextprocessor + ``` - connectors: - - gomod: go.opentelemetry.io/collector/connector/forwardconnector v0.88.0 - EOF - ``` -2. Run `ocb --config builder-config.yaml`. A new directory `otelcol-dev` (or what you have defined in `output_path`) is created. The code generated should not be touched, all changes should be done via the `builder-config.yaml` file. You can see the builder configuration of the OpenTelemetry contrib here: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/cmd/otelcontribcol/builder-config.yaml +2. Run `ocb --config builder-config.yaml`. A new directory `otelcol-dev` (or what you have defined in `output_path`) is created. The code generated should not be touched, all changes should be done via the `builder-config.yaml` file. 3. Go to new `otelcold-dev` and you will see a new executable `otelcol-dev` (or the value defined in `name`). You can execute it by providing a basic configuration file `otelcold-dev --config otelcol.yaml`: ``` cat < otelcol.yaml @@ -100,23 +87,4 @@ If you want to get to your own custom collector: ``` With this configuration the collector will scrape its own metrics. -## ... from the current repo - -You can have a look to the folder `otelcol-dev/helloworldmetricsprocessor` and start modifying the code. Or - -1. Create a new directory like `helloworldmetricsprocessor`. -2. Create a new golang module `go mod init github.com/jriguera/otel-helloworldprocessor/otelcol-dev/helloworldmetricsprocessor` (change according to your choice) -3. Create new golang files. Use same package with same name as the directory `helloworldmetricsprocessor`: - 1. `config.go` to define the component parameters and the function to validate them. - 2. `factory.go` to define the facture which create instances of the new component. Use the name `NewFactory` and the factory provided by the component package (eg `processor.NewFactory`) - 3. Other files to run the task (eg `processor.go`). -4. Add the component to the section of `builder-config.yaml` with the same reference you used in the `go mod init ...`. Create another section to replace this module name with the local directory created in point 1. - ``` - processors: - - gomod: github.com/jriguera/otel-helloworldprocessor/otelcol-dev/helloworldmetricsprocessor v0.88.0 - - replaces: - - github.com/jriguera/otel-helloworldprocessor/otelcol-dev/helloworldmetricsprocessor => ./helloworldmetricsprocessor - ``` -5. Finally run `ocb --config builder-config.yaml` to generate the binary with the new component