Add semantic conventions for instrumenting AWS Lambda.

CHANGELOG.md

@@ -26,6 +26,7 @@ release.
 
 ### Semantic Conventions
 
+- Add details for filling semantic conventiosn for AWS Lambda ([#1442](https://github.com/open-telemetry/opentelemetry-specification/pull/1442))
 - Update semantic conventions to distinguish between int and double ([#1550](https://github.com/open-telemetry/opentelemetry-specification/pull/1550))
 
 ### Compatibility

semantic_conventions/trace/messaging.yaml

@@ -9,7 +9,7 @@ groups:
           type: string
           required: always
           brief: 'A string identifying the messaging system.'
-          examples: ['kafka', 'rabbitmq', 'activemq']
+          examples: ['kafka', 'rabbitmq', 'activemq', 'AmazonSQS']
         - id: destination
           type: string
           required: always

specification/trace/semantic_conventions/README.md

@@ -21,7 +21,8 @@ The following semantic conventions for spans are defined:
 * [RPC/RMI](rpc.md): Spans for remote procedure calls (e.g., gRPC).
 * [Messaging](messaging.md): Spans for interaction with messaging systems (queues, publish/subscribe, etc.).
 * [FaaS](faas.md): Spans for Function as a Service (e.g., AWS Lambda).
-* [Exceptions](exceptions.md): Attributes for recording exceptions associated with a span
+  * [AWS Lambda](instrumentation/aws-lambda.md): Details on populating attributes for AWS Lambda.
+* [Exceptions](exceptions.md): Attributes for recording exceptions associated with a span.
 
 Apart from semantic conventions for traces and [metrics](../../metrics/semantic_conventions/README.md),
 OpenTelemetry also defines the concept of overarching [Resources](../../resource/sdk.md) with their own

specification/trace/semantic_conventions/instrumentation/aws-lambda.md

@@ -15,10 +15,10 @@ For all events, a span with kind `SERVER` MUST be created corresponding to the f
 otherwise below. Unless stated otherwise below, the name of the span MUST be set to the function name from the
 Lambda `Context`.
 
-The following attributes SHOULD be set.
+The following attributes SHOULD be set:
 
 - [`faas.execution`](../faas.md) - The value of the AWS Request ID, which is always available through an accessor on the Lambda `Context`
-- [`faas.id`](../../../resource/semantic_conventions/faas.md) - The value of the invocation arn for the function, which is always available through an accessor on the Lambda `Context`
+- [`faas.id`](../../../resource/semantic_conventions/faas.md) - The value of the invocation ARN for the function, which is always available through an accessor on the Lambda `Context`
 - [`cloud.account.id`](../../../resource/semantic_conventions/cloud.md) - In some languages, this is available as an accessor on the Lambda `Context`. Otherwise, it can be parsed from the value of `faas.id` as the fifth item when splitting on `:`
 
 ### Determining the parent of a span
@@ -43,20 +43,26 @@ configuration for a REST API, in which case only a deserialized body payload is
 gateway is configured to proxy to the Lambda function, the instrumented request handler will have access to all
 the information about the HTTP request in the form of an API Gateway Proxy Request Event.
 
-The Lambda span name and the [`http.route` span attribute](../http.md) SHOULD be set to the `Resource` from the
-proxy request event, which corresponds to the user configured HTTP route instead of the function name.
+The Lambda span name and the [`http.route` span attribute](../http.md#http-server-semantic-conventions) SHOULD
+be set to the [resource property][] from the proxy request event, which corresponds to the user configured HTTP
+route instead of the function name.
 
 [`faas.trigger`](../faas.md) MUST be set to `http`. [HTTP attributes](../http.md) SHOULD be set based on the
-available information in the proxy request event.
+available information in the proxy request event. `http.scheme` is available as the `x-forwarded-proto` header
+in the proxy request. Refer to the [input format][] for more details.
+
+[resource property]: https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-lambda-proxy-integrations.html#api-gateway-simple-proxy-for-lambda-input-format
+[input format]: https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-lambda-proxy-integrations.html#api-gateway-simple-proxy-for-lambda-input-format
 
 ## SQS
 
-SQS is a message queue that triggers a Lambda function with a batch of messages. So we consider processing both
-of a batch and of each individual message. The function invocation span MUST correspond to the SQS event, which
-is the batch of messages. For each message, an additional span SHOULD be created to correspond with the handling
-of the SQS message. Because handling of a message will be inside user business logic, not the Lambda framework,
-automatic instrumentation mechanisms without code change will often not be able to instrument the processing of
-the individual messages.
+Amazon Simple Queue Service (SQS) is a message queue that triggers a Lambda function with a batch of messages.
+So we consider processing both of a batch and of each individual message. The function invocation span MUST
+correspond to the SQS event, which is the batch of messages. For each message, an additional span SHOULD be
+created to correspond with the handling of the SQS message. Because handling of a message will be inside user
+business logic, not the Lambda framework, automatic instrumentation mechanisms without code change will often
+not be able to instrument the processing of the individual messages. Instrumentation SHOULD provide utilities
+for creating message processing spans within user code.
 
 The span kind for both types of SQS spans MUST be `CONSUMER`.
 
@@ -66,31 +72,117 @@ For the SQS event span, if all the messages in the event have the same event sou
 be `<event source> process`. If there are multiple sources in the batch, the name MUST be
 `multiple_sources process`. The parent MUST be the `SERVER` span corresponding to the function invocation.
 
-For every message in the event, the message's system attributes (not message attributes, which are provided by
+For every message in the event, the [message system attributes][] (not message attributes, which are provided by
 the user) SHOULD be checked for the key `AWSTraceHeader`. If it is present, an OpenTelemetry `Context` SHOULD be
 parsed from the value of the attribute using the [AWS X-Ray Propagator](../../../context/api-propagators.md) and
 added as a link to the span. This means the span may have as many links as messages in the batch.
 
-[`faas.trigger`](../faas.md) MUST be set to `pubsub`.
-[`messaging.operation`](../messaging.md) MUST be set to `process`.
-[`messaging.system`](../messaging.md) MUST be set to `AmazonSQS`.
+- [`faas.trigger`](../faas.md) MUST be set to `pubsub`.
+- [`messaging.operation`](../messaging.md) MUST be set to `process`.
+- [`messaging.system`](../messaging.md) MUST be set to `AmazonSQS`.
+- [`messaging.destination_kind`](../messaging.md#messaging-attributes) MUST be set to `queue`.
 
 ### SQS Message
 
 For the SQS message span, the name MUST be `<event source> process`.  The parent MUST be the `CONSUMER` span
-corresponding to the SQS event. The message's system attributes (not message attributes, which are provided by
+corresponding to the SQS event. The [message system attributes][] (not message attributes, which are provided by
 the user) SHOULD be checked for the key `AWSTraceHeader`. If it is present, an OpenTelemetry `Context` SHOULD be
 parsed from the value of the attribute using the [AWS X-Ray Propagator](../../../context/api-propagators.md) and
 added as a link to the span.
 
-[`faas.trigger`](../faas.md) MUST be set to `pubsub`.
-[`messaging.operation`](../messaging.md) MUST be set to `process`.
-[`messaging.system`](../messaging.md) MUST be set to `AmazonSQS`.
+- [`faas.trigger`](../faas.md) MUST be set to `pubsub`.
+- [`messaging.operation`](../messaging.md#messaging-attributes) MUST be set to `process`.
+- [`messaging.system`](../messaging.md#messaging-attributes) MUST be set to `AmazonSQS`.
+- [`messaging.destination_kind`](../messaging.md#messaging-attributes) MUST be set to `queue`.
 
-Other [Messaging attributes](../messaging.md) SHOULD be set based on the available information in the SQS message
+Other [Messaging attributes](../messaging.md#messaging-attributes) SHOULD be set based on the available information in the SQS message
 event.
 
-Note that `AWSTraceHeader` is the only supported mechanism for propagating `Context` for SQS to prevent conflicts
-with other sources. Notably, message attributes (user-provided, not system) are not supported - the linked contexts
-are always expected to have been sent as HTTP headers of the `SQS.SendMessage` request that the message originated
-from. This is a function of AWS SDK instrumentation, not Lambda instrumentation.
+Note that `AWSTraceHeader` is the only supported mechanism for propagating `Context` in instrumentation for SQS
+to prevent conflicts with other sources. Notably, message attributes (user-provided, not system) are not supported -
+the linked contexts are always expected to have been sent as HTTP headers of the `SQS.SendMessage` request that
+the message originated from. This is a function of AWS SDK instrumentation, not Lambda instrumentation.
+
+Using the `AWSTraceHeader` ensures that propagation will work across AWS services that may be integrated to
+Lambda via SQS, for example a flow that goes through S3 -> SNS -> SQS -> Lambda. `AWSTraceHeader` is only a means
+of propagating context and not tied to any particular observability backend. Notably, using it does not imply
+using AWS X-Ray - any observability backend will fully function using this propagation mechanism.
+
+[message system attributes]: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-message-metadata.html#sqs-message-system-attributes
+
+## Examples
+
+### API Gateway Request Proxy (Lambda tracing passive)
+
+Given a process C that sends an HTTP request to an API Gateway endpoint with path `/pets/{petId}` configured for
+a Lambda function F:
+
+```
+Process C: | Span Client        |
+--
+Function F:    | Span Function |
+```
+
+| Field or Attribute | `Span Client` | `Span Function` |
+|-|-|-|
+| Span name | `HTTP GET` | `/pets/{petId}` |
+| Parent |  | Span Client |
+| SpanKind | `CLIENT` | `SERVER` |
+| Status | `Ok` | `Ok` |
+| `faas.execution` | | `79104EXAMPLEB723` |
+| `faas.id` | | `arn:aws:lambda:us-west-2:123456789012:function:my-function` |
+| `faas.trigger` | | `http` |
+| `cloud.account.id` | | `12345678912` |
+| `net.peer.name` | `foo.execute-api.us-east-1.amazonaws.com` |  |
+| `net.peer.port` | `413` |  |
+| `http.method` | `GET` | `GET` |
+| `http.user_agent` | `okhttp 3.0` | `okhttp 3.0` |
+| `http.url` | `https://foo.execute-api.us-east-1.amazonaws.com/pets/10` |  |
+| `http.scheme` | | `https` |
+| `http.host` | | `foo.execute-api.us-east-1.amazonaws.com` |
+| `http.target` | | `/pets/10` |
+| `http.route` | | `/pets/{petId}` |
+| `http.status_code` | `200` | `200` |
+
+### API Gateway Request Proxy (Lambda tracing active)
+
+Active tracing in Lambda means an API Gateway span `Span APIGW` and a Lambda runtime invocation span `Span Lambda`
+will be exported to AWS X-Ray. All attributes above are the same except that in this case, the parent of `APIGW`
+is `Span Client` and the parent of `Span Function` is `Span Lambda`.
+
+### SQS (Lambda tracing passive)
+
+Given a process P, that sends two messages to a queue Q on SQS, and a Lambda function F, which processes both of them in one batch (Span ProcBatch) and
+generates a processing span for each message separately (Spans Proc1 and Proc2).
+
+```
+Process P: | Span Prod1 | Span Prod2 |
+--
+Function F:                      | Span ProcBatch |
+                                        | Span Proc1 |
+                                               | Span Proc2 |
+```
+
+| Field or Attribute | Span Prod1 | Span Prod2 | Span ProcBatch | Span Proc1 | Span Proc2 |
+|-|-|-|-|-|-|
+| Span name | `Q send` | `Q send` | `Q process` | `Q process` | `Q process` |
+| Parent |  |  |  | Span ProcBatch | Span ProcBatch |
+| Links |  |  |  | Span Prod1 | Span Prod2 |
+| SpanKind | `PRODUCER` | `PRODUCER` | `CONSUMER` | `CONSUMER` | `CONSUMER` |
+| Status | `Ok` | `Ok` | `Ok` | `Ok` | `Ok` |
+| `messaging.system` | `AmazonSQS` | `AmazonSQS` | `AmazonSQS` | `AmazonSQS` | `AmazonSQS` |
+| `messaging.destination` | `Q` | `Q` | | `Q` | `Q` |
+| `messaging.destination_kind` | `queue` | `queue` | `queue` | `queue` | `queue` |
+| `messaging.operation` |  |  | `process` | `process"` | `process` |
+| `messaging.message_id` | | | | `"a1"` | `"a2"` |
+
+The above requires user code change to create `Span Proc1` and `Span Proc2` - in Java, the user would inherit from
+[TracingSqsMessageHandler][] instead of Lambda's standard `RequestHandler` to enable them. Otherwise the spans would
+not exist.
+
+[TracingSqsMessageHandler]: https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/instrumentation/aws-lambda-1.0/library/src/main/java/io/opentelemetry/instrumentation/awslambda/v1_0/TracingSqsMessageHandler.java
+
+### SQS (Lambda tracing active)
+
+Active tracing in Lambda means a Lambda runtime invocation span `Span Lambda` will be exported to X-Ray. In this
+case, all of the above is the same except `Span ProcBatch` will have a parent of `Span Lambda`.

specification/trace/semantic_conventions/messaging.md

@@ -127,7 +127,7 @@ The following operations related to messages are defined for these semantic conv
 <!-- semconv messaging -->
 | Attribute  | Type | Description  | Examples  | Required |
 |---|---|---|---|---|
-| `messaging.system` | string | A string identifying the messaging system. | `kafka`; `rabbitmq`; `activemq` | Yes |
+| `messaging.system` | string | A string identifying the messaging system. | `kafka`; `rabbitmq`; `activemq`; `AmazonSQS` | Yes |
 | `messaging.destination` | string | The message destination name. This might be equal to the span name but is required nevertheless. | `MyQueue`; `MyTopic` | Yes |
 | `messaging.destination_kind` | string | The kind of message destination | `queue` | Conditional [1] |
 | `messaging.temp_destination` | boolean | A boolean that is true if the message destination is temporary. |  | If missing, it is assumed to be false. |