diff --git a/CHANGELOG.md b/CHANGELOG.md index 87b719942ec..298635bba94 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,9 @@ release. ### Traces +- Rename `http.retry_count` to `http.resend_count` and clarify its meaning. + ([#2743](https://github.com/open-telemetry/opentelemetry-specification/pull/2743)) + ### Metrics ### Logs @@ -19,10 +22,21 @@ release. ### Semantic Conventions +- Add gRPC request and response metadata semantic conventions + ([#2874](https://github.com/open-telemetry/opentelemetry-specification/pull/2874)) +- Add `process.paging.faults` metric to semantic conventions + ([#2827](https://github.com/open-telemetry/opentelemetry-specification/pull/2827)) + ### Compatibility +- Specify how Prometheus exporters and receivers handle instrumentation scope. + ([#2703](https://github.com/open-telemetry/opentelemetry-specification/pull/2703)). + ### OpenTelemetry Protocol +- Clarify that lowerCamelCase field names MUST be used for OTLP/JSON + ([#2829](https://github.com/open-telemetry/opentelemetry-specification/pull/2829)) + ### SDK Configuration ### Telemetry Schemas diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 92561c0270f..182abb69a5a 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -266,3 +266,41 @@ owner should bring it to the [OpenTelemetry Specification SIG meeting](https://github.com/open-telemetry/community#cross-language-specification). [nvm]: https://github.com/nvm-sh/nvm/blob/master/README.md#installing-and-updating + +## Releasing + +Release Procedure: + +1. Prepare a [draft release here](https://github.com/open-telemetry/opentelemetry-specification/releases). + Don't publish it yet. +2. Create a PR with updated [CHANGELOG.md](CHANGELOG.md). The CHANGELOG.md must have a + heading with the new version number. Ensure that no CHANGELOG entries are missing or + ended up in the wrong section (e.g., in the last released version rather than Unreleased). + The PR will fail the `schemas-check` Github action (all other actions must pass). + This is expected and will be fixed in the next steps. Have this PR reviewed and approved + and ready to be merged. While it is being reviewed you can work on step 3-4 in parallel. +3. Prepare the schema file for the upcoming release. The schema file should be placed + in the `schemas` directory. If no changes to semantic conventions happened + since the last release which require a corresponding section in the schema file then + simply copy the previous schema file, rename it to the new version and add a section + with the new version number to the file. See for example the schema file for [1.9.0]( + https://github.com/open-telemetry/opentelemetry-specification/blob/main/schemas/1.9.0) + that has no changes from 1.8.0. + The schema file may already exist if there were changes done to semantic conventions + and the schema file was created with corresponding changes. +4. Create and merge the PR with the new schema file. Note the commit hash after merging. + If the schema file for the new release version previously existed then no new PR is + necessary, just note that latest commit hash of this repository. +5. Once CHANGELOG.md PR is approved and ready to be merged we are ready to make the release. + Update the [opentelemetry.io](https://github.com/open-telemetry/opentelemetry.io) + repository: the [opentelemetry-specification]( + https://github.com/open-telemetry/opentelemetry.io/tree/main/content-modules) + submodule points to this repository. Create a PR and update the submodule to point to + the commit hash from step 4. Merge this PR. This should update the + [https://opentelemetry.io/](https://opentelemetry.io/) website and the new schema file + should be downloadable at `https://opentelemetry.io/schemas/`. +6. Re-trigger the `schema-checks` Github action on the PR that updates the CHANGELOG.md. + The action should pass now. Merge the PR. +7. Add the changelog entries from `CHANGELOG.md` to the description of the previously + created [draft release here]( + https://github.com/open-telemetry/opentelemetry-specification/releases) and publish it. diff --git a/experimental/serialization/json.md b/experimental/serialization/json.md index e4084bdd6e3..6b60258b861 100644 --- a/experimental/serialization/json.md +++ b/experimental/serialization/json.md @@ -73,12 +73,3 @@ This is an example showing logs: {"resourceLogs":[{"resource":{"attributes":[{"key":"resource-attr","value":{"stringValue":"resource-attr-val-1"}}]},"instrumentationLibraryLogs":[{"instrumentationLibrary":{},"logs":[{"timeUnixNano":"1581452773000005443","severityNumber":"SEVERITY_NUMBER_INFO","severityText":"Info","name":"logA","body":{"stringValue":"This is a log message"},"attributes":[{"key":"app","value":{"stringValue":"server"}},{"key":"instance_num","value":{"intValue":"1"}}],"droppedAttributesCount":1,"traceId":"08040201000000000000000000000000","spanId":"0102040800000000"},{"timeUnixNano":"1581452773000000789","severityNumber":"SEVERITY_NUMBER_INFO","severityText":"Info","name":"logB","body":{"stringValue":"something happened"},"attributes":[{"key":"customer","value":{"stringValue":"acme"}},{"key":"env","value":{"stringValue":"dev"}}],"droppedAttributesCount":1,"traceId":"","spanId":""}]}]}]} {"resourceLogs":[{"resource":{"attributes":[{"key":"resource-attr","value":{"stringValue":"resource-attr-val-1"}}]},"instrumentationLibraryLogs":[{"instrumentationLibrary":{},"logs":[{"timeUnixNano":"1581452773000009875","severityNumber":"SEVERITY_NUMBER_INFO","severityText":"Info","name":"logA","body":{"stringValue":"This is a log message"},"attributes":[{"key":"app","value":{"stringValue":"server"}},{"key":"instance_num","value":{"intValue":"1"}}],"droppedAttributesCount":1,"traceId":"08040201000000000000000000000000","spanId":"0102040800000000"},{"timeUnixNano":"1581452773000000789","severityNumber":"SEVERITY_NUMBER_INFO","severityText":"Info","name":"logB","body":{"stringValue":"something happened"},"attributes":[{"key":"customer","value":{"stringValue":"acme"}},{"key":"env","value":{"stringValue":"dev"}}],"droppedAttributesCount":1,"traceId":"","spanId":""}]}]}]} ``` - -This is an example showing traces, metrics and logs together: - -```json lines -{"resourceSpans":[{"resource":{"attributes":[{"key":"resource-attr","value":{"stringValue":"resource-attr-val-1"}}]},"instrumentationLibrarySpans":[{"instrumentationLibrary":{},"spans":[{"traceId":"","spanId":"","parentSpanId":"","name":"operationA","startTimeUnixNano":"1581452772000000321","endTimeUnixNano":"1581452773000000789","droppedAttributesCount":1,"events":[{"timeUnixNano":"1581452773000000123","name":"event-with-attr","attributes":[{"key":"span-event-attr","value":{"stringValue":"span-event-attr-val"}}],"droppedAttributesCount":2},{"timeUnixNano":"1581452773000000123","name":"event","droppedAttributesCount":2}],"droppedEventsCount":1,"status":{"deprecatedCode":"DEPRECATED_STATUS_CODE_UNKNOWN_ERROR","message":"status-cancelled","code":"STATUS_CODE_ERROR"}},{"traceId":"","spanId":"","parentSpanId":"","name":"operationB","startTimeUnixNano":"1581452772000000321","endTimeUnixNano":"1581452773000000789","links":[{"traceId":"","spanId":"","attributes":[{"key":"span-link-attr","value":{"stringValue":"span-link-attr-val"}}],"droppedAttributesCount":4},{"traceId":"","spanId":"","droppedAttributesCount":1}],"droppedLinksCount":3,"status":{}}]}]}]} -{"resourceMetrics":[{"resource":{"attributes":[{"key":"resource-attr","value":{"stringValue":"resource-attr-val-1"}}]},"instrumentationLibraryMetrics":[{"instrumentationLibrary":{},"metrics":[{"name":"counter-int","unit":"1","sum":{"dataPoints":[{"attributes":[{"key":"label-1","value":{"stringValue":"label-value-1"}}],"startTimeUnixNano":"1581452773000001459","timeUnixNano":"1581452773000001459","asInt":"120"},{"attributes":[{"key":"label-2","value":{"stringValue":"label-value-2"}}],"startTimeUnixNano":"1581452773000001459","timeUnixNano":"1581452773000001459","asInt":"456"}],"aggregationTemporality":"AGGREGATION_TEMPORALITY_CUMULATIVE","isMonotonic":true}},{"name":"counter-int","unit":"1","sum":{"dataPoints":[{"attributes":[{"key":"label-1","value":{"stringValue":"label-value-1"}}],"startTimeUnixNano":"1581452773000001459","timeUnixNano":"1581452773000001459","asInt":"123"},{"attributes":[{"key":"label-2","value":{"stringValue":"label-value-2"}}],"startTimeUnixNano":"1581452773000001459","timeUnixNano":"1581452773000001459","asInt":"456"}],"aggregationTemporality":"AGGREGATION_TEMPORALITY_CUMULATIVE","isMonotonic":true}}]}]}]} -{"resourceSpans":[{"resource":{"attributes":[{"key":"resource-attr","value":{"stringValue":"resource-attr-val-1"}}]},"instrumentationLibrarySpans":[{"instrumentationLibrary":{},"spans":[{"traceId":"","spanId":"","parentSpanId":"","name":"operationA","startTimeUnixNano":"1581452772000000321","endTimeUnixNano":"1581452773000000789","droppedAttributesCount":1,"events":[{"timeUnixNano":"1581452773000000826","name":"event-with-attr","attributes":[{"key":"span-event-attr","value":{"stringValue":"span-event-attr-val"}}],"droppedAttributesCount":2},{"timeUnixNano":"1581452773000000826","name":"event","droppedAttributesCount":2}],"droppedEventsCount":1,"status":{"deprecatedCode":"DEPRECATED_STATUS_CODE_UNKNOWN_ERROR","message":"status-cancelled","code":"STATUS_CODE_ERROR"}},{"traceId":"","spanId":"","parentSpanId":"","name":"operationB","startTimeUnixNano":"1581452772000200521","endTimeUnixNano":"1581452773000004789","links":[{"traceId":"","spanId":"","attributes":[{"key":"span-link-attr","value":{"stringValue":"span-link-attr-val"}}],"droppedAttributesCount":5},{"traceId":"","spanId":"","droppedAttributesCount":2}],"droppedLinksCount":3,"status":{}}]}]}]} -{"resourceLogs":[{"resource":{"attributes":[{"key":"resource-attr","value":{"stringValue":"resource-attr-val-1"}}]},"instrumentationLibraryLogs":[{"instrumentationLibrary":{},"logs":[{"timeUnixNano":"1581452773000005443","severityNumber":"SEVERITY_NUMBER_INFO","severityText":"Info","name":"logA","body":{"stringValue":"This is a log message"},"attributes":[{"key":"app","value":{"stringValue":"server"}},{"key":"instance_num","value":{"intValue":"1"}}],"droppedAttributesCount":1,"traceId":"08040201000000000000000000000000","spanId":"0102040800000000"},{"timeUnixNano":"1581452773000000789","severityNumber":"SEVERITY_NUMBER_INFO","severityText":"Info","name":"logB","body":{"stringValue":"something happened"},"attributes":[{"key":"customer","value":{"stringValue":"acme"}},{"key":"env","value":{"stringValue":"dev"}}],"droppedAttributesCount":1,"traceId":"","spanId":""}]}]}]} -``` diff --git a/internal/tools/schema_check.sh b/internal/tools/schema_check.sh index bbe0219770e..52063d3dd59 100755 --- a/internal/tools/schema_check.sh +++ b/internal/tools/schema_check.sh @@ -31,6 +31,13 @@ grep -o -e '## v[1-9].*\s' $root_dir/CHANGELOG.md | grep -o '[1-9].*' | while re echo "FAILED: $file does not exist. The schema file must exist because the version is declared in CHANGELOG.md." exit 3 fi + + curl --no-progress-meter https://opentelemetry.io/schemas/$ver > verify$ver + + diff verify$ver $file && echo "Published schema at https://opentelemetry.io/schemas/$ver is correct" \ + || (echo "Published schema at https://opentelemetry.io/schemas/$ver is incorrect!" && exit 3) + + rm verify$ver done # Now check the content of all schema files in the ../shemas directory. diff --git a/schemas/1.15.0 b/schemas/1.15.0 new file mode 100644 index 00000000000..dc850a30dab --- /dev/null +++ b/schemas/1.15.0 @@ -0,0 +1,34 @@ +file_format: 1.1.0 +schema_url: https://opentelemetry.io/schemas/1.15.0 +versions: + 1.15.0: + spans: + changes: + # https://github.com/open-telemetry/opentelemetry-specification/pull/2743 + - rename_attributes: + attribute_map: + http.retry_count: http.resend_count + 1.14.0: + 1.13.0: + spans: + changes: + # https://github.com/open-telemetry/opentelemetry-specification/pull/2614 + - rename_attributes: + attribute_map: + net.peer.ip: net.sock.peer.addr + net.host.ip: net.sock.host.addr + 1.12.0: + 1.11.0: + 1.10.0: + 1.9.0: + 1.8.0: + spans: + changes: + - rename_attributes: + attribute_map: + db.cassandra.keyspace: db.name + db.hbase.namespace: db.name + 1.7.0: + 1.6.1: + 1.5.0: + 1.4.0: diff --git a/semantic_conventions/trace/http.yaml b/semantic_conventions/trace/http.yaml index 0f38daaf174..d29672e4d3f 100644 --- a/semantic_conventions/trace/http.yaml +++ b/semantic_conventions/trace/http.yaml @@ -113,10 +113,14 @@ groups: note: > When [request target](https://www.rfc-editor.org/rfc/rfc9110.html#target.resource) is absolute URI, `net.peer.name` MUST match URI port identifier, otherwise it MUST match `Host` header port identifier. - - id: retry_count + - id: resend_count type: int brief: > - The ordinal number of request re-sending attempt. + The ordinal number of request resending attempt (for any reason, including redirects). + note: > + The resend count SHOULD be updated each time an HTTP request gets resent by the client, regardless of what + was the cause of the resending (e.g. redirection, authorization failure, 503 Server Unavailable, network issues, + or any other). requirement_level: recommended: if and only if request was retried. examples: 3 diff --git a/specification/README.md b/specification/README.md index 0cb2f9e5aa5..b63492ebe3d 100644 --- a/specification/README.md +++ b/specification/README.md @@ -53,11 +53,17 @@ implementation of the [specification][] is compliant if it satisfies all the "MUST", "MUST NOT", "REQUIRED", "SHALL", and "SHALL NOT" requirements defined in the [specification][]. -## Acronym +## Project Naming -- The official acronym used by the OpenTelemetry project is "OTel". -- Refrain from using "OT" in order to avoid confusion with the now deprecated - "OpenTracing" project. +- The official project name is "OpenTelemetry" (with no space between "Open" and + "Telemetry"). +- The official acronym used by the OpenTelemetry project is "OTel". Refrain from + using "OT" in order to avoid confusion with the now deprecated "OpenTracing" + project. +- The official names for sub-projects, like language specific implementations, + follow the pattern of "OpenTelemetry {the name of the programming language, + runtime or component}", for example, "OpenTelemetry Python", "OpenTelemetry + .NET" or "OpenTelemetry Collector". ## About the project diff --git a/specification/glossary.md b/specification/glossary.md index 6de6aaf419d..c8e0e5affd4 100644 --- a/specification/glossary.md +++ b/specification/glossary.md @@ -106,7 +106,8 @@ Coding against the OpenTelemetry API such as the [Tracing API](trace/api.md), [M ### Automatic Instrumentation Refers to telemetry collection methods that do not require the end-user to modify application's source code. -Methods vary by programming language, and examples include bytecode injection or monkey patching. +Methods vary by programming language, and examples include code manipulation (during compilation or at runtime), +monkey patching, or running eBPF programs. Synonym: *Auto-instrumentation*. diff --git a/specification/logs/data-model.md b/specification/logs/data-model.md index 1b20c0c77ac..9faab0e1b89 100644 --- a/specification/logs/data-model.md +++ b/specification/logs/data-model.md @@ -474,70 +474,8 @@ If included, they MUST follow the OpenTelemetry ## Example Log Records -Below are examples that show one possible representation of log records in JSON. -These are just examples to help understand the data model. Don’t treat the -examples as _the_ way to represent this data model in JSON. - -This document does not define the actual encoding and format of the log record -representation. Format definitions will be done in separate OTEPs (e.g. the log -records may be represented as msgpack, JSON, Protocol Buffer messages, etc). - -Example 1 - -```javascript -{ - "Timestamp": "1586960586000000000", - "Attributes": { - "http.status_code": 500, - "http.url": "http://example.com", - "my.custom.application.tag": "hello", - }, - "Resource": { - "service.name": "donut_shop", - "service.version": "2.0.0", - "k8s.pod.uid": "1138528c-c36e-11e9-a1a7-42010a800198", - }, - "TraceId": "f4dbb3edd765f620", // this is a byte sequence - // (hex-encoded in JSON) - "SpanId": "43222c2d51a7abe3", - "SeverityText": "INFO", - "SeverityNumber": 9, - "Body": "20200415T072306-0700 INFO I like donuts" -} -``` - -Example 2 - -```javascript -{ - "Timestamp": "1586960586000000000", - ... - "Body": { - "i": "am", - "an": "event", - "of": { - "some": "complexity" - } - } -} -``` - -Example 3 - -```javascript -{ - "Timestamp": "1586960586000000000", - "Attributes":{ - "http.scheme":"https", - "http.host":"donut.mycie.com", - "http.target":"/order", - "http.method":"post", - "http.status_code":500, - "http.flavor":"1.1", - "http.user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36", - } -} -``` +For example log records see +[JSON File serialization](../../experimental/serialization/json.md). ## Appendix A. Example Mappings diff --git a/specification/metrics/data-model.md b/specification/metrics/data-model.md index 24e162c40c5..f74e2b72f2f 100644 --- a/specification/metrics/data-model.md +++ b/specification/metrics/data-model.md @@ -1,5 +1,6 @@ # Metrics Data Model @@ -66,9 +67,11 @@ linkTitle: Data Model + [Dropped Types](#dropped-types) + [Start Time](#start-time) + [Exemplars](#exemplars-1) + + [Instrumentation Scope](#instrumentation-scope) + [Resource Attributes](#resource-attributes) * [OTLP Metric points to Prometheus](#otlp-metric-points-to-prometheus) + [Metric Metadata](#metric-metadata-1) + + [Instrumentation Scope](#instrumentation-scope-1) + [Gauges](#gauges-1) + [Sums](#sums-1) + [Histograms](#histograms-1) @@ -731,8 +734,8 @@ double-width floating point values have indices in the range func MapToIndexScale0(value float64) int32 { rawBits := math.Float64bits(value) - // rawExponent is an 11-bit biased representation of the base-2 - // exponent: + // rawExponent is an 11-bit biased representation of the base-2 + // exponent: // - value 0 indicates a subnormal representation or a zero value // - value 2047 indicates an Inf or NaN value // - value [1, 2046] are offset by ExponentBias (1023) @@ -752,13 +755,13 @@ func MapToIndexScale0(value float64) int32 { rawExponent -= int64(bits.LeadingZeros64(rawFragment - 1) - 12) // In the example with 0x1p-1023, the preceding expression subtracts - // (13-12)=1, leaving the rawExponent equal to -1. The next statement - // below subtracts `ExponentBias` (1023), leaving `ieeeExponent` equal + // (13-12)=1, leaving the rawExponent equal to -1. The next statement + // below subtracts `ExponentBias` (1023), leaving `ieeeExponent` equal // to -1024, which is the correct upper-inclusive bucket index for // the value 0x1p-1023. } ieeeExponent := int32(rawExponent - ExponentBias) - // Note that rawFragment and rawExponent cannot both be zero, + // Note that rawFragment and rawExponent cannot both be zero, // or else the value is exactly zero, in which case the the ZeroCount // bucket is used. if rawFragment == 0 { @@ -777,7 +780,7 @@ smallest normal value, which may permit the use of a built-in function: func MapToIndexScale0(value float64) int { // Note: Frexp() rounds submnormal values to the smallest normal // value and returns an exponent corresponding to fractions in the - // range [0.5, 1), whereas an exponent for the range [1, 2), so + // range [0.5, 1), whereas an exponent for the range [1, 2), so // subtract 1 from the exponent immediately. frac, exp := math.Frexp(value) exp-- @@ -876,7 +879,7 @@ func MapToIndex(value float64) int { return ((exp - 1) << scale) - 1 } scaleFactor := math.Ldexp(math.Log2E, scale) - // Note: math.Floor(value) equals math.Ceil(value)-1 when value + // Note: math.Floor(value) equals math.Ceil(value)-1 when value // is not a power of two, which is checked above. return math.Floor(math.Log(value) * scaleFactor) } @@ -1385,6 +1388,47 @@ retrieved from the `trace_id` and `span_id` label keys, respectively. All labels not used for the trace and span ids MUST be added to the OpenTelemetry exemplar as attributes. +#### Instrumentation Scope + +Each `otel_scope_info` metric point present in a batch of metrics +SHOULD be dropped from the incoming scrape, and converted to an instrumentation +scope. The `otel_scope_name` and `otel_scope_version` labels, if present, MUST +be converted to the Name and Version of the Instrumentation Scope. Additional +labels MUST be added as scope attributes, with keys and values unaltered. Other +metrics in the batch which have `otel_scope_name` and `otel_scope_version` +labels that match an instrumentation scope MUST be placed within the matching +instrumentation scope, and MUST remove those labels. For example, the +OpenMetrics metrics: + +``` +# TYPE otel_scope_info info +otel_scope_info{otel_scope_name="go.opentelemetry.io.contrib.instrumentation.net.http.otelhttp",otel_scope_version="v0.24.0",library_mascot="bear"} 1 +# TYPE http_server_duration counter +http_server_duration{otel_scope_name="go.opentelemetry.io.contrib.instrumentation.net.http.otelhttp",otel_scope_version="v0.24.0"...} 1 +``` + +becomes: + +```yaml +# within a resource_metrics +scope_metrics: + scope: + name: go.opentelemetry.io.contrib.instrumentation.net.http.otelhttp + version: v0.24.0 + attributes: + library_mascot: bear + metrics: + - name: http_server_duration + data: + sum: + data_points: + - value: 1 +``` + +Metrics which are not found to be associated with an instrumentation scope MUST +all be placed within an empty instrumentation scope, and MUST not have any labels +removed. + #### Resource Attributes When scraping a Prometheus endpoint, resource attributes MUST be added to the @@ -1445,6 +1489,22 @@ The data point type of an OTLP metric MUST be added as [OpenMetrics TYPE metadata](https://github.com/OpenObservability/OpenMetrics/blob/main/specification/OpenMetrics.md#metricfamily). It also dictates type-specific conversion rules listed below. +#### Instrumentation Scope + +Prometheus exporters SHOULD generate an [Info](https://github.com/OpenObservability/OpenMetrics/blob/main/specification/OpenMetrics.md#info)-typed +metric named `otel_scope_info`. If present, Instrumentation Scope +`name` and `version` MUST be added as `otel_scope_name` and +`otel_scope_version` labels. Scope attributes MUST also be added as labels +following the rules described in the [`Metric Attributes`](#metric-attributes) +section below. + +Prometheus exporters MUST add the scope name as the `otel_scope_name` label and +the scope version as the `otel_scope_version` label on all metric points by +default. + +Prometheus exporters SHOULD provide a configuration option to disable the +`otel_scope_info` metric and `otel_scope_` labels. + #### Gauges An [OpenTelemetry Gauge](#gauge) MUST be converted to a Prometheus Gauge. diff --git a/specification/metrics/semantic_conventions/process-metrics.md b/specification/metrics/semantic_conventions/process-metrics.md index 84c83e91af6..0f44943eb01 100644 --- a/specification/metrics/semantic_conventions/process-metrics.md +++ b/specification/metrics/semantic_conventions/process-metrics.md @@ -31,17 +31,18 @@ metrics](runtime-environment-metrics.md). Below is a table of Process metric instruments. -| Name | Instrument Type ([\*](README.md#instrument-types)) | Unit | Description | Labels | -| -------------------------------------- | -------------------------------------------------- | --------- | ----------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `process.cpu.time` | Counter | s | Total CPU seconds broken down by different states. | `state`, if specified, SHOULD be one of: `system`, `user`, `wait`. A process SHOULD be characterized _either_ by data points with no `state` labels, _or only_ data points with `state` labels. | -| `process.cpu.utilization` | Gauge | 1 | Difference in process.cpu.time since the last measurement, divided by the elapsed time and number of CPUs available to the process. | `state`, if specified, SHOULD be one of: `system`, `user`, `wait`. A process SHOULD be characterized _either_ by data points with no `state` labels, _or only_ data points with `state` labels. | -| `process.memory.usage` | UpDownCounter | By | The amount of physical memory in use. | | -| `process.memory.virtual` | UpDownCounter | By | The amount of committed virtual memory. | | -| `process.disk.io` | Counter | By | Disk bytes transferred. | `direction` SHOULD be one of: `read`, `write` | -| `process.network.io` | Counter | By | Network bytes transferred. | `direction` SHOULD be one of: `receive`, `transmit` | -| `process.threads` | UpDownCounter | {threads} | Process threads count. | | -| `process.open_file_descriptors` | UpDownCounter | {count} | Number of file descriptors in use by the process. | | -| `process.context_switches` | Counter | {count} | Number of times the process has been context switched. | `type` SHOULD be one of: `involuntary`, `voluntary` | +| Name | Instrument Type ([\*](README.md#instrument-types)) | Unit | Description | Labels | +|---------------------------------|----------------------------------------------------|-----------|-------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `process.cpu.time` | Counter | s | Total CPU seconds broken down by different states. | `state`, if specified, SHOULD be one of: `system`, `user`, `wait`. A process SHOULD be characterized _either_ by data points with no `state` labels, _or only_ data points with `state` labels. | +| `process.cpu.utilization` | Gauge | 1 | Difference in process.cpu.time since the last measurement, divided by the elapsed time and number of CPUs available to the process. | `state`, if specified, SHOULD be one of: `system`, `user`, `wait`. A process SHOULD be characterized _either_ by data points with no `state` labels, _or only_ data points with `state` labels. | +| `process.memory.usage` | UpDownCounter | By | The amount of physical memory in use. | | +| `process.memory.virtual` | UpDownCounter | By | The amount of committed virtual memory. | | +| `process.disk.io` | Counter | By | Disk bytes transferred. | `direction` SHOULD be one of: `read`, `write` | +| `process.network.io` | Counter | By | Network bytes transferred. | `direction` SHOULD be one of: `receive`, `transmit` | +| `process.threads` | UpDownCounter | {threads} | Process threads count. | | +| `process.open_file_descriptors` | UpDownCounter | {count} | Number of file descriptors in use by the process. | | +| `process.context_switches` | Counter | {count} | Number of times the process has been context switched. | `type` SHOULD be one of: `involuntary`, `voluntary` | +| `process.paging.faults` | Counter | {faults} | Number of page faults the process has made. | `type`, if specified, SHOULD be one of: `major` (for major, or hard, page faults), `minor` (for minor, or soft, page faults). | ## Attributes diff --git a/specification/protocol/otlp.md b/specification/protocol/otlp.md index 75f6fd6f754..158e54c26fe 100644 --- a/specification/protocol/otlp.md +++ b/specification/protocol/otlp.md @@ -424,6 +424,13 @@ for mapping between Protobuf and JSON, with the following deviations from that m This aligns with the behavior of the Binary Protobuf unmarshaler and ensures that adding new fields to OTLP messages does not break existing receivers. +- The keys of JSON objects are field names converted to lowerCamelCase. Original + field names are not valid to use a keys of JSON objects. + For example this is a valid JSON representation of a Resource: + `{ "attributes": {...}, "droppedAttributesCount": 123 }`, and this is NOT a valid + representation: + `{ "attributes": {...}, "dropped_attributes_count": 123 }`. + Note that according to [Protobuf specs]( https://developers.google.com/protocol-buffers/docs/proto3#json) 64-bit integer numbers in JSON-encoded payloads are encoded as decimal strings, and either diff --git a/specification/trace/semantic_conventions/http.md b/specification/trace/semantic_conventions/http.md index a9b42408349..a35f9a53cc6 100644 --- a/specification/trace/semantic_conventions/http.md +++ b/specification/trace/semantic_conventions/http.md @@ -19,9 +19,11 @@ and various HTTP versions like 1.1, 2 and SPDY. - [HTTP server](#http-server) * [HTTP server definitions](#http-server-definitions) * [HTTP Server semantic conventions](#http-server-semantic-conventions) -- [HTTP client-server example](#http-client-server-example) -- [HTTP retries examples](#http-retries-examples) -- [HTTP redirects examples](#http-redirects-examples) +- [Examples](#examples) + * [HTTP client-server example](#http-client-server-example) + * [HTTP client retries examples](#http-client-retries-examples) + * [HTTP client authorization retry examples](#http-client-authorization-retry-examples) + * [HTTP client redirects examples](#http-client-redirects-examples) @@ -128,13 +130,15 @@ before any HTTP-redirects that may happen when executing the request. | Attribute | Type | Description | Examples | Requirement Level | |---|---|---|---|---| | `http.url` | string | Full HTTP request URL in the form `scheme://host[:port]/path?query[#fragment]`. Usually the fragment is not transmitted over HTTP, but if it is known, it should be included nevertheless. [1] | `https://www.foo.bar/search?q=OpenTelemetry#SemConv` | Required | -| `http.retry_count` | int | The ordinal number of request re-sending attempt. | `3` | Recommended: if and only if request was retried. | -| [`net.peer.name`](span-general.md) | string | Host identifier of the ["URI origin"](https://www.rfc-editor.org/rfc/rfc9110.html#name-uri-origin) HTTP request is sent to. [2] | `example.com` | Required | -| [`net.peer.port`](span-general.md) | int | Port identifier of the ["URI origin"](https://www.rfc-editor.org/rfc/rfc9110.html#name-uri-origin) HTTP request is sent to. [3] | `80`; `8080`; `443` | Conditionally Required: [4] | +| `http.resend_count` | int | The ordinal number of request resending attempt (for any reason, including redirects). [2] | `3` | Recommended: if and only if request was retried. | +| [`net.peer.name`](span-general.md) | string | Host identifier of the ["URI origin"](https://www.rfc-editor.org/rfc/rfc9110.html#name-uri-origin) HTTP request is sent to. [3] | `example.com` | Required | +| [`net.peer.port`](span-general.md) | int | Port identifier of the ["URI origin"](https://www.rfc-editor.org/rfc/rfc9110.html#name-uri-origin) HTTP request is sent to. [4] | `80`; `8080`; `443` | Conditionally Required: [5] | **[1]:** `http.url` MUST NOT contain credentials passed via URL in form of `https://username:password@www.example.com/`. In such case the attribute's value should be `https://www.example.com/`. -**[2]:** Determined by using the first of the following that applies +**[2]:** The resend count SHOULD be updated each time an HTTP request gets resent by the client, regardless of what was the cause of the resending (e.g. redirection, authorization failure, 503 Server Unavailable, network issues, or any other). + +**[3]:** Determined by using the first of the following that applies - Host identifier of the [request target](https://www.rfc-editor.org/rfc/rfc9110.html#target.resource) if it's sent in absolute-form @@ -142,9 +146,9 @@ before any HTTP-redirects that may happen when executing the request. SHOULD NOT be set if capturing it would require an extra DNS lookup. -**[3]:** When [request target](https://www.rfc-editor.org/rfc/rfc9110.html#target.resource) is absolute URI, `net.peer.name` MUST match URI port identifier, otherwise it MUST match `Host` header port identifier. +**[4]:** When [request target](https://www.rfc-editor.org/rfc/rfc9110.html#target.resource) is absolute URI, `net.peer.name` MUST match URI port identifier, otherwise it MUST match `Host` header port identifier. -**[4]:** If not default (`80` for `http` scheme, `443` for `https`). +**[5]:** If not default (`80` for `http` scheme, `443` for `https`). Following attributes MUST be provided **at span creation time** (when provided at all), so they can be considered for sampling decisions: @@ -158,13 +162,15 @@ Note that in some cases host and port identifiers in the `Host` header might be ### HTTP request retries and redirects Retries and redirects cause more than one physical HTTP request to be sent. +A request is resent when an HTTP client library sends more than one HTTP request to satisfy the same API call. +This may happen due to following redirects, authorization challenges, 503 Server Unavailable, network issues, or any other. A CLIENT span SHOULD be created for each one of these physical requests. No span is created corresponding to the "logical" (encompassing) request. -For retries, `http.retry_count` attribute SHOULD be added to each retry span -with the value that reflects the ordinal number of request retry attempt. +Each time an HTTP request is resent, the `http.resend_count` attribute SHOULD be added to each repeated span +and set to the ordinal number of the request resend attempt. -See [examples](#http-retries-examples) for more details. +See [examples](#http-client-retries-examples) for more details. ## HTTP server @@ -285,7 +291,9 @@ Following attributes MUST be provided **at span creation time** (when provided a Note that in some cases host and port identifiers in the `Host` header might be different from the `net.host.name` and `net.host.port`, in this case instrumentation MAY populate `Host` header on `http.request.header.host` attribute even if it's not enabled by user. -## HTTP client-server example +## Examples + +### HTTP client-server example As an example, if a browser request for `https://example.com:8080/webshop/articles/4?s=1` is invoked from a host with IP 192.0.2.4, we may have the following Span on the client side: @@ -319,7 +327,7 @@ Span name: `/webshop/articles/:article_id`. | `net.sock.peer.addr` | `"192.0.2.5"` (the client goes through a proxy) | | `http.user_agent` | `"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:72.0) Gecko/20100101 Firefox/72.0"` | -## HTTP retries examples +### HTTP client retries examples Example of retries in the presence of a trace started by an inbound request: @@ -330,11 +338,11 @@ request (SERVER, trace=t1, span=s1) | | | --- server (SERVER, trace=t1, span=s3) | - -- GET / - 500 (CLIENT, trace=t1, span=s4, http.retry_count=1) + -- GET / - 500 (CLIENT, trace=t1, span=s4, http.resend_count=1) | | | --- server (SERVER, trace=t1, span=s5) | - -- GET / - 200 (CLIENT, trace=t1, span=s6, http.retry_count=2) + -- GET / - 200 (CLIENT, trace=t1, span=s6, http.resend_count=2) | --- server (SERVER, trace=t1, span=s7) ``` @@ -346,16 +354,44 @@ GET / - 500 (CLIENT, trace=t1, span=s1) | --- server (SERVER, trace=t1, span=s2) -GET / - 500 (CLIENT, trace=t2, span=s1, http.retry_count=1) +GET / - 500 (CLIENT, trace=t2, span=s1, http.resend_count=1) | --- server (SERVER, trace=t2, span=s2) -GET / - 200 (CLIENT, trace=t3, span=s1, http.retry_count=2) +GET / - 200 (CLIENT, trace=t3, span=s1, http.resend_count=2) | --- server (SERVER, trace=t3, span=s1) ``` -## HTTP redirects examples +### HTTP client authorization retry examples + +Example of retries in the presence of a trace started by an inbound request: + +``` +request (SERVER, trace=t1, span=s1) + | + -- GET /hello - 401 (CLIENT, trace=t1, span=s2) + | | + | --- server (SERVER, trace=t1, span=s3) + | + -- GET /hello - 200 (CLIENT, trace=t1, span=s4, http.resend_count=1) + | + --- server (SERVER, trace=t1, span=s5) +``` + +Example of retries with no trace started upfront: + +``` +GET /hello - 401 (CLIENT, trace=t1, span=s1) + | + --- server (SERVER, trace=t1, span=s2) + +GET /hello - 200 (CLIENT, trace=t2, span=s1, http.resend_count=1) + | + --- server (SERVER, trace=t2, span=s2) +``` + +### HTTP client redirects examples Example of redirects in the presence of a trace started by an inbound request: @@ -366,7 +402,7 @@ request (SERVER, trace=t1, span=s1) | | | --- server (SERVER, trace=t1, span=s3) | - -- GET /hello - 200 (CLIENT, trace=t1, span=s4) + -- GET /hello - 200 (CLIENT, trace=t1, span=s4, http.resend_count=1) | --- server (SERVER, trace=t1, span=s5) ``` @@ -378,7 +414,7 @@ GET / - 302 (CLIENT, trace=t1, span=s1) | --- server (SERVER, trace=t1, span=s2) -GET /hello - 200 (CLIENT, trace=t2, span=s1) +GET /hello - 200 (CLIENT, trace=t2, span=s1, http.resend_count=1) | --- server (SERVER, trace=t2, span=s2) ``` diff --git a/specification/trace/semantic_conventions/rpc.md b/specification/trace/semantic_conventions/rpc.md index d800f0c5b59..6d066c3ea2d 100644 --- a/specification/trace/semantic_conventions/rpc.md +++ b/specification/trace/semantic_conventions/rpc.md @@ -19,6 +19,7 @@ This document defines how to describe remote procedure calls - [gRPC](#grpc) * [gRPC Attributes](#grpc-attributes) * [gRPC Status](#grpc-status) + * [gRPC Request and Response Metadata](#grpc-request-and-response-metadata) - [JSON RPC](#json-rpc) * [JSON RPC Attributes](#json-rpc-attributes) @@ -202,6 +203,16 @@ For remote procedure calls via [gRPC][], additional conventions are described in The [Span Status](../api.md#set-status) MUST be left unset for an `OK` gRPC status code, and set to `Error` for all others. +### gRPC Request and Response Metadata + +| Attribute | Type | Description | Examples | Requirement Level | +|-------------------------------|----------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------|-------------------| +| `rpc.request.metadata.` | string[] | gRPC request metadata, `` being the normalized gRPC Metadata key (lowercase, with `-` characters replaced by `_`), the value being the metadata values. [1] | `rpc.request.metadata.my_custom_metadata_attribute=["1.2.3.4", "1.2.3.5"]` | Optional | +| `rpc.response.metadata.` | string[] | gRPC response metadata, `` being the normalized gRPC Metadata key (lowercase, with `-` characters replaced by `_`), the value being the metadata values. [1] | `rpc.response.metadata.my_custom_metadata_attribute=["attribute_value"]` | Optional | + +**[1]:** Instrumentations SHOULD require an explicit configuration of which metadata values are to be captured. +Including all request/response metadata values can be a security risk - explicit configuration helps avoid leaking sensitive information. + ## JSON RPC Conventions specific to [JSON RPC](https://www.jsonrpc.org/). diff --git a/specification/trace/tracestate-handling.md b/specification/trace/tracestate-handling.md index cd310fb9216..8d91dc30b73 100644 --- a/specification/trace/tracestate-handling.md +++ b/specification/trace/tracestate-handling.md @@ -66,7 +66,7 @@ in order to preserve existing values belonging to other OTel concerns. For examp if a given concern K wants to set `k1:13`: * `ot=p:8;r:62` will become `ot=p:8;r:62;k1:13`. -* `ot=p:8;k1:7;r:62` will become `ot=p8;r:62;k1:13`. Preserving the order is not required. +* `ot=p:8;k1:7;r:62` will become `ot=p:8;r:62;k1:13`. Preserving the order is not required. If setting a value ends up making the entire `ot` entry exceed the 256 characters limit, SDKs are advised to abort the operation and signal the user about the error, e.g.