From 57c61d6a08d1242255de7785407a455f8ee99edb Mon Sep 17 00:00:00 2001 From: gechetspr Date: Fri, 6 Dec 2024 15:53:52 +0200 Subject: [PATCH 1/2] Added overview of OTEL --- .../opentelemetry/overview.md | 216 ++++++++++++++++++ 1 file changed, 216 insertions(+) create mode 100644 docs/dg/dev/backend-development/opentelemetry/overview.md diff --git a/docs/dg/dev/backend-development/opentelemetry/overview.md b/docs/dg/dev/backend-development/opentelemetry/overview.md new file mode 100644 index 0000000000..86c7f85b0c --- /dev/null +++ b/docs/dg/dev/backend-development/opentelemetry/overview.md @@ -0,0 +1,216 @@ +--- +title: Opentelemetry overview +description: Describing what is Opentelemetry and how Spryker integrates it in the SCCOS +template: howto-guide-template +last_updated: Dec 5, 2024 +related: +--- + +Spryker allows to integrate different monitoring tools like NewRelic into the SCCOS. But usually those tools are vendor locked and allows to work only with limited amount of backends or have not so much customizability options. +In order to give more customisable options, Spryker introduces a OpenTelemetry integration that allows to make your monitoring system be custom tailored for you and in the same time uses open standards that are widely accepted in community. + +## Overview + +You may want to get familiar with [OpenTelemetry Documentation first](https://opentelemetry.io/docs/concepts/signals/traces/), but we are going to give you a brief overview here too. + +### Things that you should know + +#### Trace +Trace represents a single transaction. It has an unique ID and all spans are related to it. Trace have a name that is defined automatically or can be changed with `\Spryker\Service\Monitoring\MonitoringServiceInterface::setTransactionName` method if you wire a plugin for it. + +#### Span +A span represents a unit of work or operation. Spans are the building blocks of Traces. In OpenTelemetry, they include the following information: +- Name +- Parent span ID (empty for root spans) +- Start and End Timestamps +- Span Context +- Attributes +- Span Events +- Span Links +- Span Status + +You can say that Span in our case represents an execution of single method of the class. + +#### Hook +OpenTelemetry provide a way to instrument your code without modifying it directly. Hook is a function that execute a function before and after method execution. + +Example of hook you can see below. Hooks should be registered before the class is executed. +```php +tracer() + ->spanBuilder('ModuleName-ClassName::methodName') + ->setParent($context) + ->setAttribute(\OpenTelemetry\SemConv\TraceAttributes::CODE_FUNCTION, $function) + ->setAttribute(\OpenTelemetry\SemConv\TraceAttributes::CODE_NAMESPACE, $class) + ->setAttribute(\OpenTelemetry\SemConv\TraceAttributes::CODE_FILEPATH, $filename) + ->setAttribute(\OpenTelemetry\SemConv\TraceAttributes::CODE_LINENO, $lineno) + ->startSpan(); + + \OpenTelemetry\Context\Context::storage()->attach($span->storeInContext($context)); + }, + + post: static function ($instance, array $params, $returnValue, ?\Throwable $exception) { + $scope = \OpenTelemetry\Context\Context::storage()->scope(); + + if (null === $scope) { + return; + } + + $error = error_get_last(); + + if (is_array($error) && in_array($error['type'], [E_ERROR, E_CORE_ERROR, E_COMPILE_ERROR, E_PARSE], true)) { + $exception = new \Exception( + 'Error: ' . $error['message'] . ' in ' . $error['file'] . ' on line ' . $error['line'] + ); + } + + $scope->detach(); + $span = \Spryker\Service\Opentelemetry\Instrumentation\Span\Span::fromContext($scope->context()); + + if ($exception !== null) { + $span->recordException($exception); + $span->setAttribute('error_message', isset($exception) ? $exception->getMessage() : ''); + $span->setAttribute('error_code', isset($exception) ? $exception->getCode() : ''); + } + + $span->setStatus($exception !== null ? \OpenTelemetry\API\Trace\StatusCode::STATUS_ERROR : \OpenTelemetry\API\Trace\StatusCode::STATUS_OK); + + $span->end(); + } + ); +``` + +### Collector +Collector does what you think it is. It collects traces and send it to your backend after. Sending traces to collector is done after that request is sent, so it will make a client wait just for that. + +## Integration +For the integration you can use a script from a [Installer repo](https://github.com/spryker/opentelemetry-installer/). Use the latest version and run it in your system. +Also all steps can be done manually and with more control from your side. + +### Install required packages +Spryker provides a few packages in order to instrument as much of code and services as we can. +- [The main package spryker/opentelemetry](https://packagist.org/packages/spryker/opentelemetry). It includes the entry point for instrumentation, plugin that you can wire in your monitoring service and a console command that allows you to generate [hooks](https://opentelemetry.io/docs/zero-code/php/#how-it-works) that will generate spans automatically. +- [Propel instrumentation package](https://packagist.org/packages/spryker/otel-propel-instrumentation). It instruments all database queries that were made via Propel. +- [RabbitMq instrumentation package](https://packagist.org/packages/spryker/otel-rabbit-mq-instrumentation). Instruments how Spryker works with a Queue via RabbitMQ. +- [Elastic Search instrumentation package](https://packagist.org/packages/spryker/otel-elastica-instrumentation). Instruments calls to the Elasticsearch and adds those to a trace. + +Last 3 are optional and you should install them if you are interested that those thins are part of your monitoring system. The spryker/opentelemetry one is a mandatory one to install. + +In addition you can install packages from 3rd party vendors to instrument other parts. E.g. we tested `mismatch/opentelemetry-auto-redis` for instrumenting Redis and `open-telemetry/opentelemetry-auto-guzzle` for Guzzle (that is used in Gateway calls in SCCOS). +You also feel free to install packages that you are interested in. Usually such instrumentation packages require only installation and will be auto-wired after. + +### (Optional) Install Monitoring module +Opentelemetry integration doesn't require to use Monitoring service. But it allows you to add custom attributes or change your tracaes (transaction) name during the request execution. +You can get a module [here](https://packagist.org/packages/spryker/monitoring). +After an install you can wire a plugin from a spryker/opentelemtry module. +```php + + */ + protected function getMonitoringExtensions(): array + { + return [ + new OpentelemetryMonitoringExtensionPlugin(), + ]; + } +} + +``` +After that you can call method from Monitoring service and they will be transalted to Opentelemtry action. Be advised that some of the methods are not covered due to the fact that those things have no direct implementation. E.g. `\Spryker\Service\Opentelemetry\Plugin\OpentelemetryMonitoringExtensionPlugin::markAsConsoleCommand` does nothing and transaction is marked as console command automatically. + +### Wire a console command + +```php + Date: Tue, 7 Jan 2025 14:16:48 +0200 Subject: [PATCH 2/2] Added configuration references --- .../opentelemetry/overview.md | 195 +++++++++++++++--- 1 file changed, 162 insertions(+), 33 deletions(-) diff --git a/docs/dg/dev/backend-development/opentelemetry/overview.md b/docs/dg/dev/backend-development/opentelemetry/overview.md index 86c7f85b0c..df244992dd 100644 --- a/docs/dg/dev/backend-development/opentelemetry/overview.md +++ b/docs/dg/dev/backend-development/opentelemetry/overview.md @@ -38,14 +38,14 @@ Example of hook you can see below. Hooks should be registered before the class i ```php tracer() - ->spanBuilder('ModuleName-ClassName::methodName') + ->spanBuilder('ModuleName-MyClass::methodName') ->setParent($context) ->setAttribute(\OpenTelemetry\SemConv\TraceAttributes::CODE_FUNCTION, $function) ->setAttribute(\OpenTelemetry\SemConv\TraceAttributes::CODE_NAMESPACE, $class) @@ -88,28 +88,21 @@ Example of hook you can see below. Hooks should be registered before the class i ``` ### Collector -Collector does what you think it is. It collects traces and send it to your backend after. Sending traces to collector is done after that request is sent, so it will make a client wait just for that. +Collector does what you think it is. It collects traces and send it to your backend after. Sending traces to collector is done after that request is sent, so it will not make a client wait just for that. ## Integration For the integration you can use a script from a [Installer repo](https://github.com/spryker/opentelemetry-installer/). Use the latest version and run it in your system. -Also all steps can be done manually and with more control from your side. +Below you can see what steps this script executes if you want to do everything manually. ### Install required packages -Spryker provides a few packages in order to instrument as much of code and services as we can. -- [The main package spryker/opentelemetry](https://packagist.org/packages/spryker/opentelemetry). It includes the entry point for instrumentation, plugin that you can wire in your monitoring service and a console command that allows you to generate [hooks](https://opentelemetry.io/docs/zero-code/php/#how-it-works) that will generate spans automatically. -- [Propel instrumentation package](https://packagist.org/packages/spryker/otel-propel-instrumentation). It instruments all database queries that were made via Propel. -- [RabbitMq instrumentation package](https://packagist.org/packages/spryker/otel-rabbit-mq-instrumentation). Instruments how Spryker works with a Queue via RabbitMQ. -- [Elastic Search instrumentation package](https://packagist.org/packages/spryker/otel-elastica-instrumentation). Instruments calls to the Elasticsearch and adds those to a trace. - -Last 3 are optional and you should install them if you are interested that those thins are part of your monitoring system. The spryker/opentelemetry one is a mandatory one to install. - -In addition you can install packages from 3rd party vendors to instrument other parts. E.g. we tested `mismatch/opentelemetry-auto-redis` for instrumenting Redis and `open-telemetry/opentelemetry-auto-guzzle` for Guzzle (that is used in Gateway calls in SCCOS). -You also feel free to install packages that you are interested in. Usually such instrumentation packages require only installation and will be auto-wired after. +OpenTelementry provides instrumentation via packages that can be installed and register hooks automatically. If you want to instrument something that is not covered in our code base, but is required for you, you can try to install packages that are listed [here](https://opentelemetry.io/ecosystem/registry/?language=php) or something that you consider useful for you from other sources. +Spryker provides [spryker/opentelemetry](https://packagist.org/packages/spryker/opentelemetry) package that covers essential parts. It includes the entry point for instrumentation, plugin that you can wire in your monitoring service and a console command that allows you to generate [hooks](https://opentelemetry.io/docs/zero-code/php/#how-it-works) that will generate spans automatically. Also it includes instrumentation of Propel, Redis, ElasticSeach and RabbitMQ calls. +In addition, you may want to install a `open-telemetry/opentelemetry-auto-guzzle` package to cover Guzzle calls to 3rd party services or to the services in your application. ### (Optional) Install Monitoring module -Opentelemetry integration doesn't require to use Monitoring service. But it allows you to add custom attributes or change your tracaes (transaction) name during the request execution. +Opentelemetry integration doesn't require to use Monitoring service, but this is highly recommended as it allows you to add custom attributes, change your traces (transaction) name during the request execution and have exceptions added to the root span for visibility. You can get a module [here](https://packagist.org/packages/spryker/monitoring). -After an install you can wire a plugin from a spryker/opentelemtry module. +After install you can wire a Monitoring plugin from a `spryker/opentelemtry` module to get all listed features. ```php