From de868b19de68a13b164f9db03b4a60283b22a530 Mon Sep 17 00:00:00 2001 From: Jack Berg Date: Fri, 22 Dec 2023 11:28:32 -0600 Subject: [PATCH 1/8] Define sdk extension component support in file configuration --- .../configuration/file-configuration.md | 127 ++++++++++++++++-- 1 file changed, 118 insertions(+), 9 deletions(-) diff --git a/specification/configuration/file-configuration.md b/specification/configuration/file-configuration.md index 8b2bcec2564..6aa2542a4e8 100644 --- a/specification/configuration/file-configuration.md +++ b/specification/configuration/file-configuration.md @@ -16,9 +16,13 @@ linkTitle: File * [Environment variable substitution](#environment-variable-substitution) - [SDK Configuration](#sdk-configuration) * [In-Memory Configuration Model](#in-memory-configuration-model) + * [SDK Extension Components](#sdk-extension-components) + + [Component Provider](#component-provider) + + [Create Plugin](#create-plugin) * [Operations](#operations) + [Parse](#parse) + [Create](#create) + + [Register Component Provider](#register-component-provider) - [References](#references) @@ -142,6 +146,85 @@ to provide this in-memory representation in a manner that is idiomatic for their language. If an SDK needs to expose a class or interface, the name `Configuration` is RECOMMENDED. +### SDK Extension Components + +The SDK supports a variety +extension [plugin interfaces](../glossary.md#sdk-plugins), allowing users and +libraries to customize behaviors including the sampling, processing, and +exporting of data. In general, the [configuration model](#configuration-model) +defines specific types for built-in implementations of these plugin interfaces. +For example, +the [BatchSpanProcessor](https://github.com/open-telemetry/opentelemetry-configuration/blob/f38ac7c3a499ae5f81924ef9c455c27a56130562/schema/tracer_provider.json#L22) +type refers to the +built-in [Batching span processor](../trace/sdk.md#batching-processor). The +schema SHOULD also support the ability to specify custom implementations of +plugin interfaces defined by libraries or users. + +For example, a custom [span exporter](../trace/sdk.md#span-exporter) might be configured as follows: + +```yaml +tracer_provider: + processors: + - batch: + exporter: + my-exporter: + config-parameter: value +``` + +Here we specify that the tracer provider has a batch span processor +paired with a custom span exporter named `my-exporter`, which is configured +with `config-parameter: value`. For this configuration to succeed, +a [component provider](#component-provider) must +be [registered](#register-component-provider) with `type: SpanExporter`, +and `name: my-exporter`. When [create](#create) is called, the implementation +will encounter `my-exporter` and invoke [create plugin](#create-plugin) +with `properties: {config-parameter: value}` on the registered component +provider. + +#### Component Provider + +A component provider is responsible for interpreting configuration and returning +an implementation of a particular type of SDK extension plugin interface. + +Component providers are registered with an SDK implementation of configuration +via [register](#register-component-provider). This MAY be done automatically or +require manual intervention by the user based on what is possible and idiomatic +in the language ecosystem. For example in Java, component providers might be +registered automatically using +the [service provider interface (SPI)](https://docs.oracle.com/javase/tutorial/sound/SPI-intro.html) +mechanism. + +See [create](#create), which details component provider usage in file +configuration interpretation. + +#### Create Plugin + +Interpret configuration to create a instance of a SDK extension plugin +interface. + +**Parameters:** + +* `properties` - The configuration properties. Properties MUST fully represent + the configuration as specified in + the [configuration file](#configuration-file), including the ability to access + scalars, mappings, and sequences (of scalars and other structures). It MUST be + possible to determine if a particular property is present. It SHOULD be + possible to access properties in a type safe manor, based on what is idiomatic + in the language. + +**Returns:** A configured SDK extension plugin interface implementation. + +The plugin interface MAY have properties which are optional or required, and +have specific requirements around type or format. The set of properties a +component provider accepts, along with their requirement level and expected +type, comprise a configuration schema. A component provider SHOULD document its +configuration schema. + +When Create Plugin is invoked, the component provider interprets `properties` +and attempts to extract data according to its configuration schema. If this +fails (e.g. a required property is not present, a type is mismatches, etc.), +Create Plugin SHOULD return an error. + ### Operations SDK implementations of configuration MUST provide the following operations. @@ -183,15 +266,6 @@ This SHOULD return an error if: Interpret [configuration model](#in-memory-configuration-model) and return SDK components. -If a field is null or unset and a default value is defined, Create MUST ensure -the SDK component is configured with the default value. If a field is null or -unset and no default value is defined, Create SHOULD return an error. For -example, if configuring -the [span batching processor](../trace/sdk.md#batching-processor) and -the `scheduleDelayMillis` field is null or unset, the component is configured -with the default value of `5000`. However, if the `exporter` field is null or -unset, Create fails fast since there is no default value for `exporter`. - **Parameters:** * `configuration` - The configuration model. @@ -206,12 +280,47 @@ unset, Create fails fast since there is no default value for `exporter`. The multiple responses MAY be returned using a tuple, or some other data structure encapsulating the components. +If a field is null or unset and a default value is defined, Create MUST ensure +the SDK component is configured with the default value. If a field is null or +unset and no default value is defined, Create SHOULD return an error. For +example, if configuring +the [span batching processor](../trace/sdk.md#batching-processor) and +the `scheduleDelayMillis` field is null or unset, the component is configured +with the default value of `5000`. However, if the `exporter` field is null or +unset, Create fails fast since there is no default value for `exporter`. + +When encountering a reference to +a [SDK extension component](#sdk-extension-components) which is not built in to +the SDK, Create MUST resolve the component using [Create Plugin](#create-plugin) +of the [component provider](#component-provider) of the corresponding `type` +and `name` used to [register](#register-component-provider). If no component +provider is registered with the `type` and `name`, Create SHOULD return an +error. If [Create Plugin](#create-plugin) returns an error, Create SHOULD +propagate the error. + This SHOULD return an error if it encounters an error in `configuration` (i.e. fail fast) in accordance with initialization [error handling principles](../error-handling.md#basic-error-handling-principles). TODO: define behavior if some portion of configuration model is not supported +#### Register Component Provider + +The SDK MUST provide a mechanism to +register [component providers](#component-provider). + +**Parameters:** + +* `component_provider` - The [component provider](#component-provider). +* `type` - The type of plugin interface it provides (e.g. SpanExporter, Sampler, + etc). +* `name` - The name used to identify the type of component. This is used + in [configuration files](#configuration-file) to specify that the + corresponding `component_provider` is to provide the component. + +The `type` and `name` comprise a unique key. Register MUST return an error if it +is called multiple times with the same `type` and `name` combination. + ## References * Configuration From 7303deaa7b984d831efcdf28e97842975303caab Mon Sep 17 00:00:00 2001 From: Jack Berg Date: Tue, 2 Jan 2024 16:12:11 -0600 Subject: [PATCH 2/8] Fix typo --- specification/configuration/file-configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/configuration/file-configuration.md b/specification/configuration/file-configuration.md index 6aa2542a4e8..24a93edd8ea 100644 --- a/specification/configuration/file-configuration.md +++ b/specification/configuration/file-configuration.md @@ -209,7 +209,7 @@ interface. the [configuration file](#configuration-file), including the ability to access scalars, mappings, and sequences (of scalars and other structures). It MUST be possible to determine if a particular property is present. It SHOULD be - possible to access properties in a type safe manor, based on what is idiomatic + possible to access properties in a type safe manner, based on what is idiomatic in the language. **Returns:** A configured SDK extension plugin interface implementation. From 78e0ba67598671bd0eb21a639fc4535289f29b66 Mon Sep 17 00:00:00 2001 From: Jack Berg Date: Wed, 10 Jan 2024 17:30:53 -0600 Subject: [PATCH 3/8] SDK -> file configuration implementation --- specification/configuration/file-configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/configuration/file-configuration.md b/specification/configuration/file-configuration.md index 24a93edd8ea..4e532ecf397 100644 --- a/specification/configuration/file-configuration.md +++ b/specification/configuration/file-configuration.md @@ -306,7 +306,7 @@ TODO: define behavior if some portion of configuration model is not supported #### Register Component Provider -The SDK MUST provide a mechanism to +The file configuration implementation MUST provide a mechanism to register [component providers](#component-provider). **Parameters:** From c4efc4e40d83180da525f0c9cec10818e746fc1d Mon Sep 17 00:00:00 2001 From: Jack Berg Date: Fri, 19 Jan 2024 11:32:19 -0600 Subject: [PATCH 4/8] Clarify parse expectations for custom components --- .../configuration/file-configuration.md | 36 ++++++++++++------- 1 file changed, 23 insertions(+), 13 deletions(-) diff --git a/specification/configuration/file-configuration.md b/specification/configuration/file-configuration.md index 4e532ecf397..7ed6357a821 100644 --- a/specification/configuration/file-configuration.md +++ b/specification/configuration/file-configuration.md @@ -176,10 +176,13 @@ paired with a custom span exporter named `my-exporter`, which is configured with `config-parameter: value`. For this configuration to succeed, a [component provider](#component-provider) must be [registered](#register-component-provider) with `type: SpanExporter`, -and `name: my-exporter`. When [create](#create) is called, the implementation -will encounter `my-exporter` and invoke [create plugin](#create-plugin) -with `properties: {config-parameter: value}` on the registered component -provider. +and `name: my-exporter`. When [parse](#parse) is called, the implementation will +encounter `my-exporter` and translate the corresponding configuration to an +equivalent generic `properties` representation ( +i.e. `properties: {config-parameter: value}`). When [create](#create) is called, +the implementation will encounter `my-exporter` and +invoke [create plugin](#create-plugin) on the registered component provider +with the configuration `properties` determined during `parse`. #### Component Provider @@ -240,10 +243,6 @@ with OpAmp Parse and validate a [configuration file](#configuration-file). -Parse MUST perform [environment variable substitution](#environment-variable-substitution). - -Parse MUST interpret null as equivalent to unset. - **Parameters:** * `file`: The [configuration file](#configuration-file) to parse. This MAY be a @@ -256,7 +255,17 @@ Parse MUST interpret null as equivalent to unset. **Returns:** [configuration model](#in-memory-configuration-model) -This SHOULD return an error if: +Parse MUST perform [environment variable substitution](#environment-variable-substitution). + +Parse MUST interpret null as equivalent to unset. + +When encountering a reference to +a [SDK extension component](#sdk-extension-components) which is not built in to +the SDK, Parse MUST resolve corresponding configuration to a +generic `properties` representation as described +in [Create Plugin](#create-plugin). + +Parse SHOULD return an error if: * The `file` doesn't exist or is invalid * The parsed `file` content does not conform to @@ -293,10 +302,11 @@ When encountering a reference to a [SDK extension component](#sdk-extension-components) which is not built in to the SDK, Create MUST resolve the component using [Create Plugin](#create-plugin) of the [component provider](#component-provider) of the corresponding `type` -and `name` used to [register](#register-component-provider). If no component -provider is registered with the `type` and `name`, Create SHOULD return an -error. If [Create Plugin](#create-plugin) returns an error, Create SHOULD -propagate the error. +and `name` used to [register](#register-component-provider), including the +configuration `properties` as an argument. If no component provider is +registered with the `type` and `name`, Create SHOULD return an error. +If [Create Plugin](#create-plugin) returns an error, Create SHOULD propagate the +error. This SHOULD return an error if it encounters an error in `configuration` (i.e. fail fast) in accordance with From a31b3e8492c4c6af9dad6a70467a779a3a7704de Mon Sep 17 00:00:00 2001 From: Jack Berg Date: Thu, 25 Jan 2024 10:36:08 -0600 Subject: [PATCH 5/8] Fix typo --- specification/configuration/file-configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/configuration/file-configuration.md b/specification/configuration/file-configuration.md index 7ed6357a821..674e955d860 100644 --- a/specification/configuration/file-configuration.md +++ b/specification/configuration/file-configuration.md @@ -148,7 +148,7 @@ name `Configuration` is RECOMMENDED. ### SDK Extension Components -The SDK supports a variety +The SDK supports a variety of extension [plugin interfaces](../glossary.md#sdk-plugins), allowing users and libraries to customize behaviors including the sampling, processing, and exporting of data. In general, the [configuration model](#configuration-model) From f0b88d86fd4f7f7d4f529f91ee4e64bf989ccf6c Mon Sep 17 00:00:00 2001 From: Jack Berg Date: Fri, 26 Jan 2024 14:56:36 -0600 Subject: [PATCH 6/8] Note expectation of differences between languages --- specification/configuration/file-configuration.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/specification/configuration/file-configuration.md b/specification/configuration/file-configuration.md index 674e955d860..b530ee73ab1 100644 --- a/specification/configuration/file-configuration.md +++ b/specification/configuration/file-configuration.md @@ -184,6 +184,11 @@ the implementation will encounter `my-exporter` and invoke [create plugin](#create-plugin) on the registered component provider with the configuration `properties` determined during `parse`. +Given the inherent differences across languages, the details of extension +component mechanisms are likely to vary to a greater degree than is the case +with other APIs defined by OpenTelemetry. This is to be expected and is +acceptable so long as the implementation results in the defined behaviors. + #### Component Provider A component provider is responsible for interpreting configuration and returning From b38325807fc81d1c7fd556c3356a80d177693248 Mon Sep 17 00:00:00 2001 From: Jack Berg Date: Fri, 26 Jan 2024 14:57:43 -0600 Subject: [PATCH 7/8] Add changelog entry --- CHANGELOG.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 73cdb9ca2bf..c78bff6ac42 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -28,6 +28,8 @@ release. - Add file configuration section to spec compliance matrix. ([#3804](https://github.com/open-telemetry/opentelemetry-specification/pull/3804)) +- Define mechanism for SDK extension components. + ([#3802](https://github.com/open-telemetry/opentelemetry-specification/pull/3802)) ### Common From 8c0b521edf92d8350a77c68dfa42e888dbb7cadd Mon Sep 17 00:00:00 2001 From: Jack Berg Date: Fri, 26 Jan 2024 15:02:26 -0600 Subject: [PATCH 8/8] Add entries to spec compliance matrix --- spec-compliance-matrix.md | 27 +++++++++++++++------------ 1 file changed, 15 insertions(+), 12 deletions(-) diff --git a/spec-compliance-matrix.md b/spec-compliance-matrix.md index 8614d69ecf4..c35173bd309 100644 --- a/spec-compliance-matrix.md +++ b/spec-compliance-matrix.md @@ -313,18 +313,21 @@ Note: Support for environment variables is optional. See [File Configuration](./specification/configuration/file-configuration.md) for details. -| Feature | Go | Java | JS | Python | Ruby | Erlang | PHP | Rust | C++ | .NET | Swift | -|------------------------------------------------------------------|----|------|----|--------|------|--------|-----|------|-----|------|-------| -| `Parse` a configuration file | | | | | | | | | | | | -| The `Parse` operation accepts the configuration YAML file format | | | | | | | | | | | | -| The `Parse` operation performs environment variable substitution | | | | | | | | | | | | -| The `Parse` operation returns configuration model | | | | | | | | | | | | -| `Create` SDK components | | | | | | | | | | | | -| The `Create` operation accepts configuration model | | | | | | | | | | | | -| The `Create` operation returns `TracerProvider` | | | | | | | | | | | | -| The `Create` operation returns `MeterProvider` | | | | | | | | | | | | -| The `Create` operation returns `LoggerProvider` | | | | | | | | | | | | -| The `Create` operation returns `Propagators` | | | | | | | | | | | | +| Feature | Go | Java | JS | Python | Ruby | Erlang | PHP | Rust | C++ | .NET | Swift | +|-------------------------------------------------------------------------------------------------------------------------|----|------|----|--------|------|--------|-----|------|-----|------|-------| +| `Parse` a configuration file | | | | | | | | | | | | +| The `Parse` operation accepts the configuration YAML file format | | | | | | | | | | | | +| The `Parse` operation performs environment variable substitution | | | | | | | | | | | | +| The `Parse` operation returns configuration model | | | | | | | | | | | | +| The `Parse` operation resolves extension component configuration to `properties` | | | | | | | | | | | | +| `Create` SDK components | | | | | | | | | | | | +| The `Create` operation accepts configuration model | | | | | | | | | | | | +| The `Create` operation returns `TracerProvider` | | | | | | | | | | | | +| The `Create` operation returns `MeterProvider` | | | | | | | | | | | | +| The `Create` operation returns `LoggerProvider` | | | | | | | | | | | | +| The `Create` operation returns `Propagators` | | | | | | | | | | | | +| The `Create` operation calls `CreatePlugin` of corresponding `ComponentProvider` when encountering extension components | | | | | | | | | | | | +| Register a `ComponentProvider` | | | | | | | | | | | | ## Exporters