diff --git a/receiver/hostmetricsreceiver/README.md b/receiver/hostmetricsreceiver/README.md index 8a999ffb061c..f463881aea19 100644 --- a/receiver/hostmetricsreceiver/README.md +++ b/receiver/hostmetricsreceiver/README.md @@ -132,111 +132,101 @@ service: ### Feature gate configurations -- `receiver.hostmetricsreceiver.removeDirectionAttributeNetworkMetrics` - - Description - - Some network metrics reported by the hostmetricsreceiver are transitioning from being reported - with a direction attribute to being reported with the direction included in the metric name to adhere to the - OpenTelemetry specification. Please update any monitoring this might affect. - - Affected Metrics - - `system.network.dropped` will become: - - `system.network.dropped.receive` - - `system.network.dropped.transmit` - - `system.network.errors` will become: - - `system.network.errors.receive` - - `system.network.errors.transmit` - - `system.network.io` will become: - - `system.network.io.receive` - - `system.network.io.transmit` - - `system.network.packets` will become: - - `system.network.packets.receive` - - `system.network.packets.transmit` - - Stages and Timeline - - Alpha (current stage) - - In this stage the feature gate is disabled by default and must be enabled by the user. This allows users to preemptively opt in and start using the bug fix by enabling the feature gate. - - Collector version: v0.55.0 - - Release Date: Early July 2022 - - Beta - - In this stage the feature gate is enabled by default and can be disabled by the user. - - Collector version: v0.57.0 - - Release Date: Early August 2022 - - Generally Available - - In this stage the feature gate is permanently enabled and the feature gate is no longer available for anyone. - - Users could experience some friction in this stage, they may have to update monitoring for the affected metrics or be blocked from upgrading the collector to versions v0.59.0 and newer. - - Collector version: v0.59.0 - - Release Date: Early September 2022 - - Usage - - Feature gate identifiers prefixed with - will disable the gate and prefixing with + or with no prefix will enable the gate. - - Start the otelcol with the feature gate enabled: - - otelcol {other_arguments} --feature-gates=receiver.hostmetricsreceiver.removeDirectionAttributeNetworkMetrics - - Start the otelcol with the feature gate disabled: - - otelcol {other_arguments} --feature-gates=-receiver.hostmetricsreceiver.removeDirectionAttributeNetworkMetrics - - More information: - - https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/11815 - - https://github.com/open-telemetry/opentelemetry-specification/pull/2617 -- `receiver.hostmetricsreceiver.removeDirectionAttributePagingMetrics` - - Description - - Some paging metrics reported by the hostmetricsreceiver are transitioning from being reported - with a direction attribute to being reported with the direction included in the metric name to adhere to the - OpenTelemetry specification. Please update any monitoring this might affect. - - Affected Metrics - - `system.paging.operations` will become: - - `system.paging.operations.page_in` - - `system.paging.operations.page_out` - - Stages and Timeline - - Alpha (current stage) - - In this stage the feature gate is disabled by default and must be enabled by the user. This allows users to preemptively opt in and start using the bug fix by enabling the feature gate. - - Collector version: v0.55.0 - - Release Date: Early July 2022 - - Beta - - In this stage the feature gate is enabled by default and can be disabled by the user. - - Collector version: v0.57.0 - - Release Date: Early August 2022 - - Generally Available - - In this stage the feature gate is permanently enabled and the feature gate is no longer available for anyone. - - Users could experience some friction in this stage, they may have to update monitoring for the affected metrics or be blocked from upgrading the collector to versions v0.59.0 and newer. - - Collector version: v0.59.0 - - Release Date: Early September 2022 - - Usage - - Feature gate identifiers prefixed with - will disable the gate and prefixing with + or with no prefix will enable the gate. - - Start the otelcol with the feature gate enabled: - - otelcol {other_arguments} --feature-gates=receiver.hostmetricsreceiver.removeDirectionAttributePagingMetrics - - Start the otelcol with the feature gate disabled: - - otelcol {other_arguments} --feature-gates=-receiver.hostmetricsreceiver.removeDirectionAttributePagingMetrics - - More information: - - https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/11815 - - https://github.com/open-telemetry/opentelemetry-specification/pull/2617 -- `receiver.hostmetricsreceiver.removeDirectionAttributeProcessMetrics` - - Description - - Some process metrics reported by the hostmetricsreceiver are transitioning from being reported - with a direction attribute to being reported with the direction included in the metric name to adhere to the - OpenTelemetry specification. Please update any monitoring this might affect. - - Affected Metrics - - `process.disk.io` will become: - - `process.disk.io.read` - - `process.disk.io.write` - - Stages and Timeline - - Alpha (current stage) - - In this stage the feature gate is disabled by default and must be enabled by the user. This allows users to preemptively opt in and start using the bug fix by enabling the feature gate. - - Collector version: v0.55.0 - - Release Date: Early July 2022 - - Beta - - In this stage the feature gate is enabled by default and can be disabled by the user. - - Collector version: v0.57.0 - - Release Date: Early August 2022 - - Generally Available - - In this stage the feature gate is permanently enabled and the feature gate is no longer available for anyone. - - Users could experience some friction in this stage, they may have to update monitoring for the affected metrics or be blocked from upgrading the collector to versions v0.59.0 and newer. - - Collector version: v0.59.0 - - Release Date: Early September 2022 - - Usage - - Feature gate identifiers prefixed with - will disable the gate and prefixing with + or with no prefix will enable the gate. - - Start the otelcol with the feature gate enabled: - - otelcol {other_arguments} --feature-gates=receiver.hostmetricsreceiver.removeDirectionAttributeProcessMetrics - - Start the otelcol with the feature gate disabled: - - otelcol {other_arguments} --feature-gates=-receiver.hostmetricsreceiver.removeDirectionAttributeProcessMetrics - - More information: - - https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/11815 - - https://github.com/open-telemetry/opentelemetry-specification/pull/2617 +#### Transition from metrics with "direction" attribute + +Some host metrics reported are transitioning from being reported with a `direction` attribute to being reported with the +direction included in the metric name to adhere to the OpenTelemetry specification +(https://github.com/open-telemetry/opentelemetry-specification/pull/2617): + +- `network` scraper metrics: + - `system.network.dropped` will become: + - `system.network.dropped.receive` + - `system.network.dropped.transmit` + - `system.network.errors` will become: + - `system.network.errors.receive` + - `system.network.errors.transmit` + - `system.network.io` will become: + - `system.network.io.receive` + - `system.network.io.transmit` + - `system.network.packets` will become: + - `system.network.packets.receive` + - `system.network.packets.transmit` +- `paging` scraper metrics: + - `system.paging.operations` will become: + - `system.paging.operations.page_in` + - `system.paging.operations.page_out` +- `process` scraper metrics: + - `process.disk.io` will become: + - `process.disk.io.read` + - `process.disk.io.write` + +The following feature gates control the transition process: + +- **receiver.hostmetricsreceiver.emitMetricsWithoutDirectionAttribute**: controls if the new metrics without + `direction` attribute are emitted by the receiver. +- **receiver.hostmetricsreceiver.emitMetricsWithDirectionAttribute**: controls if the deprecated metrics with + `direction` + attribute are emitted by the receiver. + +##### Transition schedule: + +1. v0.55.0, July 2022: + +- Most of the scrapers except for `disk` scraper can emit the new metrics without the `direction` attribute if + feature gates enabled. +- `receiver.hostmetricsreceiver.emitMetricsWithDirectionAttribute` is enabled by default. +- `receiver.hostmetricsreceiver.emitMetricsWithoutDirectionAttribute` is disabled by default. + +2. v0.56.0, July 2022: + +- The new metrics are available for all scrapers, but disabled by default, they can be enabled with the feature gates. +- The old metrics with `direction` attribute are deprecated with a warning. +- `receiver.hostmetricsreceiver.emitMetricsWithDirectionAttribute` is enabled by default. +- `receiver.hostmetricsreceiver.emitMetricsWithoutDirectionAttribute` is disabled by default. + +3. v0.58.0, August 2022: + +- The new metrics are enabled by default, deprecated metrics disabled, they can be enabled with the feature gates. +- `receiver.hostmetricsreceiver.emitMetricsWithDirectionAttribute` is disabled by default. +- `receiver.hostmetricsreceiver.emitMetricsWithoutDirectionAttribute` is enabled by default. + +4. v0.60.0, September 2022: + +- The feature gates are removed. +- The new metrics without `direction` attribute are always emitted. +- The deprecated metrics with `direction` attribute are no longer available. + +##### Usage: + +To enable the new metrics without `direction` attribute and disable the deprecated metrics, run OTel Collector with the +following arguments: + +```sh +otelcol --feature-gates=-receiver.hostmetricsreceiver.emitMetricsWithDirectionAttribute,+receiver.hostmetricsreceiver.emitMetricsWithoutDirectionAttribute +``` + +It's also possible to emit both the deprecated and the new metrics: + +```sh +otelcol --feature-gates=+receiver.hostmetricsreceiver.emitMetricsWithDirectionAttribute,+receiver.hostmetricsreceiver.emitMetricsWithoutDirectionAttribute +``` + +If both feature gates are enabled, each particular metric can be disabled with the user settings, for example: + +```yaml +receivers: + hostmetrics: + scrapers: + paging: + metrics: + system.paging.operations: + enabled: false +``` + +##### More information: + +- https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/11815 +- https://github.com/open-telemetry/opentelemetry-specification/pull/2617 [beta]: https://github.com/open-telemetry/opentelemetry-collector#beta [contrib]: https://github.com/open-telemetry/opentelemetry-collector-releases/tree/main/distributions/otelcol-contrib diff --git a/receiver/hostmetricsreceiver/internal/scraper.go b/receiver/hostmetricsreceiver/internal/scraper.go index 5ef990f50545..6f802afa8444 100644 --- a/receiver/hostmetricsreceiver/internal/scraper.go +++ b/receiver/hostmetricsreceiver/internal/scraper.go @@ -19,8 +19,41 @@ import ( "go.opentelemetry.io/collector/component" "go.opentelemetry.io/collector/receiver/scraperhelper" + "go.opentelemetry.io/collector/service/featuregate" ) +const ( + EmitMetricsWithDirectionAttributeFeatureGateID = "receiver.hostmetricsreceiver.emitMetricsWithDirectionAttribute" + EmitMetricsWithoutDirectionAttributeFeatureGateID = "receiver.hostmetricsreceiver.emitMetricsWithoutDirectionAttribute" +) + +var ( + emitMetricsWithDirectionAttributeFeatureGate = featuregate.Gate{ + ID: EmitMetricsWithDirectionAttributeFeatureGateID, + Enabled: true, + Description: "Some process host metrics reported are transitioning from being reported with a direction " + + "attribute to being reported with the direction included in the metric name to adhere to the " + + "OpenTelemetry specification. This feature gate controls emitting the old metrics with the direction " + + "attribute. For more details, see: " + + "https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/hostmetricsreceiver/README.md#feature-gate-configurations", + } + + emitMetricsWithoutDirectionAttributeFeatureGate = featuregate.Gate{ + ID: EmitMetricsWithoutDirectionAttributeFeatureGateID, + Enabled: false, + Description: "Some process host metrics reported are transitioning from being reported with a direction " + + "attribute to being reported with the direction included in the metric name to adhere to the " + + "OpenTelemetry specification. This feature gate controls emitting the new metrics without the direction " + + "attribute. For more details, see: " + + "https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/hostmetricsreceiver/README.md#feature-gate-configurations", + } +) + +func init() { + featuregate.GetRegistry().MustRegister(emitMetricsWithDirectionAttributeFeatureGate) + featuregate.GetRegistry().MustRegister(emitMetricsWithoutDirectionAttributeFeatureGate) +} + // ScraperFactory can create a MetricScraper. type ScraperFactory interface { // CreateDefaultConfig creates the default configuration for the Scraper. diff --git a/receiver/hostmetricsreceiver/internal/scraper/networkscraper/metadata.yaml b/receiver/hostmetricsreceiver/internal/scraper/networkscraper/metadata.yaml index 8b5060dbffc5..a6bbeabb91a5 100644 --- a/receiver/hostmetricsreceiver/internal/scraper/networkscraper/metadata.yaml +++ b/receiver/hostmetricsreceiver/internal/scraper/networkscraper/metadata.yaml @@ -18,6 +18,7 @@ attributes: description: State of the network connection. metrics: + # produced when receiver.hostmetricsreceiver.emitMetricsWithDirectionAttribute feature gate is enabled system.network.packets: enabled: true description: The number of packets transferred. (Deprecated) @@ -28,7 +29,7 @@ metrics: monotonic: true attributes: [device, direction] -# produced when receiver.hostmetricsreceiver.removeDirectionAttributeNetworkMetrics feature gate is enabled + # produced when receiver.hostmetricsreceiver.emitMetricsWithoutDirectionAttribute feature gate is enabled system.network.packets.transmit: enabled: true description: The number of packets transmitted. @@ -39,7 +40,7 @@ metrics: monotonic: true attributes: [device] -# produced when receiver.hostmetricsreceiver.removeDirectionAttributeNetworkMetrics feature gate is enabled + # produced when receiver.hostmetricsreceiver.emitMetricsWithoutDirectionAttribute feature gate is enabled system.network.packets.receive: enabled: true description: The number of packets received. @@ -50,6 +51,7 @@ metrics: monotonic: true attributes: [device] + # produced when receiver.hostmetricsreceiver.emitMetricsWithDirectionAttribute feature gate is enabled system.network.dropped: enabled: true description: The number of packets dropped. (Deprecated) @@ -60,7 +62,7 @@ metrics: monotonic: true attributes: [device, direction] -# produced when receiver.hostmetricsreceiver.removeDirectionAttributeNetworkMetrics feature gate is enabled + # produced when receiver.hostmetricsreceiver.emitMetricsWithoutDirectionAttribute feature gate is enabled system.network.dropped.transmit: enabled: true description: The number of packets dropped on transmit. @@ -71,7 +73,7 @@ metrics: monotonic: true attributes: [device] -# produced when receiver.hostmetricsreceiver.removeDirectionAttributeNetworkMetrics feature gate is enabled + # produced when receiver.hostmetricsreceiver.emitMetricsWithoutDirectionAttribute feature gate is enabled system.network.dropped.receive: enabled: true description: The number of packets dropped on receive. @@ -82,6 +84,7 @@ metrics: monotonic: true attributes: [device] + # produced when receiver.hostmetricsreceiver.emitMetricsWithDirectionAttribute feature gate is enabled system.network.errors: enabled: true description: The number of errors encountered. (Deprecated) @@ -92,7 +95,7 @@ metrics: monotonic: true attributes: [device, direction] -# produced when receiver.hostmetricsreceiver.removeDirectionAttributeNetworkMetrics feature gate is enabled + # produced when receiver.hostmetricsreceiver.emitMetricsWithoutDirectionAttribute feature gate is enabled system.network.errors.transmit: enabled: true description: The number of errors encountered on transmit. @@ -103,7 +106,7 @@ metrics: monotonic: true attributes: [device] -# produced when receiver.hostmetricsreceiver.removeDirectionAttributeNetworkMetrics feature gate is enabled + # produced when receiver.hostmetricsreceiver.emitMetricsWithoutDirectionAttribute feature gate is enabled system.network.errors.receive: enabled: true description: The number of errors encountered on receive. @@ -114,6 +117,7 @@ metrics: monotonic: true attributes: [device] + # produced when receiver.hostmetricsreceiver.emitMetricsWithDirectionAttribute feature gate is enabled system.network.io: enabled: true description: The number of bytes transmitted and received. (Deprecated) @@ -124,7 +128,7 @@ metrics: monotonic: true attributes: [device, direction] -# produced when receiver.hostmetricsreceiver.removeDirectionAttributeNetworkMetrics feature gate is enabled + # produced when receiver.hostmetricsreceiver.emitMetricsWithoutDirectionAttribute feature gate is enabled system.network.io.transmit: enabled: true description: The number of bytes transmitted. @@ -135,7 +139,7 @@ metrics: monotonic: true attributes: [device] -# produced when receiver.hostmetricsreceiver.removeDirectionAttributeNetworkMetrics feature gate is enabled + # produced when receiver.hostmetricsreceiver.emitMetricsWithoutDirectionAttribute feature gate is enabled system.network.io.receive: enabled: true description: The number of bytes received. diff --git a/receiver/hostmetricsreceiver/internal/scraper/networkscraper/network_scraper.go b/receiver/hostmetricsreceiver/internal/scraper/networkscraper/network_scraper.go index 1fa47582c9eb..9c5e8ecfe783 100644 --- a/receiver/hostmetricsreceiver/internal/scraper/networkscraper/network_scraper.go +++ b/receiver/hostmetricsreceiver/internal/scraper/networkscraper/network_scraper.go @@ -28,14 +28,13 @@ import ( "go.opentelemetry.io/collector/service/featuregate" "github.com/open-telemetry/opentelemetry-collector-contrib/internal/coreinternal/processor/filterset" + "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal" "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/networkscraper/internal/metadata" ) const ( networkMetricsLen = 4 connectionsMetricsLen = 1 - // ID for a temporary feature gate" - removeDirectionAttributeFeatureGateID = "receiver.hostmetricsreceiver.removeDirectionAttributeNetworkMetrics" ) // scraper for Network Metrics @@ -48,40 +47,21 @@ type scraper struct { excludeFS filterset.FilterSet // for mocking - bootTime func() (uint64, error) - ioCounters func(bool) ([]net.IOCountersStat, error) - connections func(string) ([]net.ConnectionStat, error) -} - -var removeDirectionAttributeFeatureGate = featuregate.Gate{ - ID: removeDirectionAttributeFeatureGateID, - Enabled: false, - Description: "Some network metrics reported by the hostmetricsreceiver are transitioning from being reported " + - "with a direction attribute to being reported with the direction included in the metric name to adhere to the " + - "OpenTelemetry specification. You can control whether the hostmetricsreceiver reports metrics with a direction " + - "attribute using the " + removeDirectionAttributeFeatureGateID + " feature gate. For more details, see: " + - "https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/hostmetricsreceiver/README.md#feature-gate-configurations", -} - -func init() { - featuregate.GetRegistry().MustRegister(removeDirectionAttributeFeatureGate) + bootTime func() (uint64, error) + ioCounters func(bool) ([]net.IOCountersStat, error) + connections func(string) ([]net.ConnectionStat, error) + emitMetricsWithDirectionAttribute bool + emitMetricsWithoutDirectionAttribute bool } // newNetworkScraper creates a set of Network related metrics func newNetworkScraper(_ context.Context, settings component.ReceiverCreateSettings, cfg *Config) (*scraper, error) { scraper := &scraper{settings: settings, config: cfg, bootTime: host.BootTime, ioCounters: net.IOCounters, connections: net.Connections} + scraper.emitMetricsWithDirectionAttribute = featuregate.GetRegistry().IsEnabled(internal.EmitMetricsWithDirectionAttributeFeatureGateID) + scraper.emitMetricsWithoutDirectionAttribute = featuregate.GetRegistry().IsEnabled(internal.EmitMetricsWithoutDirectionAttributeFeatureGateID) var err error - if featuregate.GetRegistry().IsEnabled(removeDirectionAttributeFeatureGateID) { - settings.Logger.Info("The " + removeDirectionAttributeFeatureGateID + " featre gate is enabled. This " + - "otel collector will report metrics without a direction attribute, which is good for future support") - } else { - settings.Logger.Info("WARNING - Breaking Change: " + removeDirectionAttributeFeatureGate.Description) - settings.Logger.Info("The feature gate " + removeDirectionAttributeFeatureGateID + " is disabled. This " + - "otel collector will report metrics with a direction attribute, be aware this will not be supported in the future") - } - if len(cfg.Include.Interfaces) > 0 { scraper.includeFS, err = filterset.CreateFilterSet(cfg.Include.Interfaces, &cfg.Include.Config) if err != nil { @@ -150,10 +130,11 @@ func (s *scraper) recordNetworkCounterMetrics() error { func (s *scraper) recordNetworkPacketsMetric(now pcommon.Timestamp, ioCountersSlice []net.IOCountersStat) { for _, ioCounters := range ioCountersSlice { - if featuregate.GetRegistry().IsEnabled(removeDirectionAttributeFeatureGateID) { + if s.emitMetricsWithoutDirectionAttribute { s.mb.RecordSystemNetworkPacketsTransmitDataPoint(now, int64(ioCounters.PacketsSent), ioCounters.Name) s.mb.RecordSystemNetworkPacketsReceiveDataPoint(now, int64(ioCounters.PacketsRecv), ioCounters.Name) - } else { + } + if s.emitMetricsWithDirectionAttribute { s.mb.RecordSystemNetworkPacketsDataPoint(now, int64(ioCounters.PacketsSent), ioCounters.Name, metadata.AttributeDirectionTransmit) s.mb.RecordSystemNetworkPacketsDataPoint(now, int64(ioCounters.PacketsRecv), ioCounters.Name, metadata.AttributeDirectionReceive) } @@ -162,10 +143,11 @@ func (s *scraper) recordNetworkPacketsMetric(now pcommon.Timestamp, ioCountersSl func (s *scraper) recordNetworkDroppedPacketsMetric(now pcommon.Timestamp, ioCountersSlice []net.IOCountersStat) { for _, ioCounters := range ioCountersSlice { - if featuregate.GetRegistry().IsEnabled(removeDirectionAttributeFeatureGateID) { + if s.emitMetricsWithoutDirectionAttribute { s.mb.RecordSystemNetworkDroppedTransmitDataPoint(now, int64(ioCounters.Dropout), ioCounters.Name) s.mb.RecordSystemNetworkDroppedReceiveDataPoint(now, int64(ioCounters.Dropin), ioCounters.Name) - } else { + } + if s.emitMetricsWithDirectionAttribute { s.mb.RecordSystemNetworkDroppedDataPoint(now, int64(ioCounters.Dropout), ioCounters.Name, metadata.AttributeDirectionTransmit) s.mb.RecordSystemNetworkDroppedDataPoint(now, int64(ioCounters.Dropin), ioCounters.Name, metadata.AttributeDirectionReceive) } @@ -174,10 +156,11 @@ func (s *scraper) recordNetworkDroppedPacketsMetric(now pcommon.Timestamp, ioCou func (s *scraper) recordNetworkErrorPacketsMetric(now pcommon.Timestamp, ioCountersSlice []net.IOCountersStat) { for _, ioCounters := range ioCountersSlice { - if featuregate.GetRegistry().IsEnabled(removeDirectionAttributeFeatureGateID) { + if s.emitMetricsWithoutDirectionAttribute { s.mb.RecordSystemNetworkErrorsTransmitDataPoint(now, int64(ioCounters.Errout), ioCounters.Name) s.mb.RecordSystemNetworkErrorsReceiveDataPoint(now, int64(ioCounters.Errin), ioCounters.Name) - } else { + } + if s.emitMetricsWithDirectionAttribute { s.mb.RecordSystemNetworkErrorsDataPoint(now, int64(ioCounters.Errout), ioCounters.Name, metadata.AttributeDirectionTransmit) s.mb.RecordSystemNetworkErrorsDataPoint(now, int64(ioCounters.Errin), ioCounters.Name, metadata.AttributeDirectionReceive) } @@ -186,10 +169,11 @@ func (s *scraper) recordNetworkErrorPacketsMetric(now pcommon.Timestamp, ioCount func (s *scraper) recordNetworkIOMetric(now pcommon.Timestamp, ioCountersSlice []net.IOCountersStat) { for _, ioCounters := range ioCountersSlice { - if featuregate.GetRegistry().IsEnabled(removeDirectionAttributeFeatureGateID) { + if s.emitMetricsWithoutDirectionAttribute { s.mb.RecordSystemNetworkIoTransmitDataPoint(now, int64(ioCounters.BytesSent), ioCounters.Name) s.mb.RecordSystemNetworkIoReceiveDataPoint(now, int64(ioCounters.BytesRecv), ioCounters.Name) - } else { + } + if s.emitMetricsWithDirectionAttribute { s.mb.RecordSystemNetworkIoDataPoint(now, int64(ioCounters.BytesSent), ioCounters.Name, metadata.AttributeDirectionTransmit) s.mb.RecordSystemNetworkIoDataPoint(now, int64(ioCounters.BytesRecv), ioCounters.Name, metadata.AttributeDirectionReceive) } diff --git a/receiver/hostmetricsreceiver/internal/scraper/networkscraper/network_scraper_test.go b/receiver/hostmetricsreceiver/internal/scraper/networkscraper/network_scraper_test.go index 9b22d77b937b..fd0d96a18973 100644 --- a/receiver/hostmetricsreceiver/internal/scraper/networkscraper/network_scraper_test.go +++ b/receiver/hostmetricsreceiver/internal/scraper/networkscraper/network_scraper_test.go @@ -26,7 +26,6 @@ import ( "go.opentelemetry.io/collector/pdata/pcommon" "go.opentelemetry.io/collector/pdata/pmetric" "go.opentelemetry.io/collector/receiver/scrapererror" - "go.opentelemetry.io/collector/service/featuregate" "github.com/open-telemetry/opentelemetry-collector-contrib/internal/coreinternal/processor/filterset" "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal" @@ -35,18 +34,20 @@ import ( func TestScrape(t *testing.T) { type testCase struct { - name string - config Config - bootTimeFunc func() (uint64, error) - ioCountersFunc func(bool) ([]net.IOCountersStat, error) - connectionsFunc func(string) ([]net.ConnectionStat, error) - expectNetworkMetrics bool - expectedStartTime pcommon.Timestamp - newErrRegex string - initializationErr string - expectedErr string - expectedErrCount int - removeDirectionAttributeFeatureGateEnabled bool + name string + config Config + bootTimeFunc func() (uint64, error) + ioCountersFunc func(bool) ([]net.IOCountersStat, error) + connectionsFunc func(string) ([]net.ConnectionStat, error) + expectNetworkMetrics bool + expectedStartTime pcommon.Timestamp + newErrRegex string + initializationErr string + expectedErr string + expectedErrCount int + expectMetricsWithDirectionAttribute bool + expectMetricsWithoutDirectionAttribute bool + mutateScraper func(*scraper) } testCases := []testCase{ @@ -55,24 +56,31 @@ func TestScrape(t *testing.T) { config: Config{ Metrics: metadata.DefaultMetricsSettings(), }, - expectNetworkMetrics: true, + expectNetworkMetrics: true, + expectMetricsWithDirectionAttribute: true, }, { name: "Standard with direction removed", config: Config{ Metrics: metadata.DefaultMetricsSettings(), }, - expectNetworkMetrics: true, - removeDirectionAttributeFeatureGateEnabled: true, + expectNetworkMetrics: true, + expectMetricsWithDirectionAttribute: false, + expectMetricsWithoutDirectionAttribute: true, + mutateScraper: func(s *scraper) { + s.emitMetricsWithDirectionAttribute = false + s.emitMetricsWithoutDirectionAttribute = true + }, }, { name: "Validate Start Time", config: Config{ Metrics: metadata.DefaultMetricsSettings(), }, - bootTimeFunc: func() (uint64, error) { return 100, nil }, - expectNetworkMetrics: true, - expectedStartTime: 100 * 1e9, + bootTimeFunc: func() (uint64, error) { return 100, nil }, + expectNetworkMetrics: true, + expectMetricsWithDirectionAttribute: true, + expectedStartTime: 100 * 1e9, }, { name: "Include Filter that matches nothing", @@ -119,8 +127,10 @@ func TestScrape(t *testing.T) { for _, test := range testCases { t.Run(test.name, func(t *testing.T) { - featuregate.GetRegistry().Apply(map[string]bool{removeDirectionAttributeFeatureGateID: test.removeDirectionAttributeFeatureGateEnabled}) scraper, err := newNetworkScraper(context.Background(), componenttest.NewNopReceiverCreateSettings(), &test.config) + if test.mutateScraper != nil { + test.mutateScraper(scraper) + } if test.newErrRegex != "" { require.Error(t, err) require.Regexp(t, test.newErrRegex, err) @@ -163,9 +173,10 @@ func TestScrape(t *testing.T) { expectedMetricCount := 1 if test.expectNetworkMetrics { - if test.removeDirectionAttributeFeatureGateEnabled { + if test.expectMetricsWithoutDirectionAttribute { expectedMetricCount += 8 - } else { + } + if test.expectMetricsWithDirectionAttribute { expectedMetricCount += 4 } } @@ -175,20 +186,32 @@ func TestScrape(t *testing.T) { idx := 0 assertNetworkConnectionsMetricValid(t, metrics.At(idx)) if test.expectNetworkMetrics { - if test.removeDirectionAttributeFeatureGateEnabled { - assertNetworkIOMetricValid(t, metrics.At(idx+1), "system.network.dropped.receive", test.expectedStartTime, test.removeDirectionAttributeFeatureGateEnabled) - assertNetworkIOMetricValid(t, metrics.At(idx+2), "system.network.dropped.transmit", test.expectedStartTime, test.removeDirectionAttributeFeatureGateEnabled) - assertNetworkIOMetricValid(t, metrics.At(idx+3), "system.network.errors.receive", test.expectedStartTime, test.removeDirectionAttributeFeatureGateEnabled) - assertNetworkIOMetricValid(t, metrics.At(idx+4), "system.network.errors.transmit", test.expectedStartTime, test.removeDirectionAttributeFeatureGateEnabled) - assertNetworkIOMetricValid(t, metrics.At(idx+5), "system.network.io.receive", test.expectedStartTime, test.removeDirectionAttributeFeatureGateEnabled) - assertNetworkIOMetricValid(t, metrics.At(idx+6), "system.network.io.transmit", test.expectedStartTime, test.removeDirectionAttributeFeatureGateEnabled) - assertNetworkIOMetricValid(t, metrics.At(idx+7), "system.network.packets.receive", test.expectedStartTime, test.removeDirectionAttributeFeatureGateEnabled) - assertNetworkIOMetricValid(t, metrics.At(idx+8), "system.network.packets.transmit", test.expectedStartTime, test.removeDirectionAttributeFeatureGateEnabled) - } else { - assertNetworkIOMetricValid(t, metrics.At(idx+1), "system.network.dropped", test.expectedStartTime, test.removeDirectionAttributeFeatureGateEnabled) - assertNetworkIOMetricValid(t, metrics.At(idx+2), "system.network.errors", test.expectedStartTime, test.removeDirectionAttributeFeatureGateEnabled) - assertNetworkIOMetricValid(t, metrics.At(idx+3), "system.network.io", test.expectedStartTime, test.removeDirectionAttributeFeatureGateEnabled) - assertNetworkIOMetricValid(t, metrics.At(idx+4), "system.network.packets", test.expectedStartTime, test.removeDirectionAttributeFeatureGateEnabled) + if test.expectMetricsWithoutDirectionAttribute { + assertNetworkIOMetricValid(t, metrics.At(idx+1), "system.network.dropped.receive", + test.expectedStartTime, true) + assertNetworkIOMetricValid(t, metrics.At(idx+2), "system.network.dropped.transmit", + test.expectedStartTime, true) + assertNetworkIOMetricValid(t, metrics.At(idx+3), "system.network.errors.receive", + test.expectedStartTime, true) + assertNetworkIOMetricValid(t, metrics.At(idx+4), "system.network.errors.transmit", + test.expectedStartTime, true) + assertNetworkIOMetricValid(t, metrics.At(idx+5), "system.network.io.receive", + test.expectedStartTime, true) + assertNetworkIOMetricValid(t, metrics.At(idx+6), "system.network.io.transmit", + test.expectedStartTime, true) + assertNetworkIOMetricValid(t, metrics.At(idx+7), "system.network.packets.receive", + test.expectedStartTime, true) + assertNetworkIOMetricValid(t, metrics.At(idx+8), "system.network.packets.transmit", + test.expectedStartTime, true) + } + if test.expectMetricsWithDirectionAttribute { + assertNetworkIOMetricValid(t, metrics.At(idx+1), "system.network.dropped", + test.expectedStartTime, false) + assertNetworkIOMetricValid(t, metrics.At(idx+2), "system.network.errors", test.expectedStartTime, + false) + assertNetworkIOMetricValid(t, metrics.At(idx+3), "system.network.io", test.expectedStartTime, false) + assertNetworkIOMetricValid(t, metrics.At(idx+4), "system.network.packets", + test.expectedStartTime, false) } internal.AssertSameTimeStampForMetrics(t, metrics, 1, 5) idx += 4 diff --git a/receiver/hostmetricsreceiver/internal/scraper/pagingscraper/factory.go b/receiver/hostmetricsreceiver/internal/scraper/pagingscraper/factory.go index 245deabc9495..94307d985972 100644 --- a/receiver/hostmetricsreceiver/internal/scraper/pagingscraper/factory.go +++ b/receiver/hostmetricsreceiver/internal/scraper/pagingscraper/factory.go @@ -19,7 +19,6 @@ import ( "go.opentelemetry.io/collector/component" "go.opentelemetry.io/collector/receiver/scraperhelper" - "go.opentelemetry.io/collector/service/featuregate" "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal" "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/pagingscraper/internal/metadata" @@ -30,24 +29,8 @@ import ( const ( // TypeStr the value of "type" key in configuration. TypeStr = "paging" - // ID for a temporary feature gate" - removeDirectionAttributeFeatureGateID = "receiver.hostmetricsreceiver.removeDirectionAttributePagingMetrics" ) -var removeDirectionAttributeFeatureGate = featuregate.Gate{ - ID: removeDirectionAttributeFeatureGateID, - Enabled: false, - Description: "Some paging metrics reported by the hostmetricsreceiver are transitioning from being reported " + - "with a direction attribute to being reported with the direction included in the metric name to adhere to the " + - "OpenTelemetry specification. You can control whether the hostmetricsreceiver reports metrics with a direction " + - "attribute using the " + removeDirectionAttributeFeatureGateID + " feature gate. For more details, see: " + - "https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/hostmetricsreceiver/README.md#feature-gate-configurations", -} - -func init() { - featuregate.GetRegistry().MustRegister(removeDirectionAttributeFeatureGate) -} - // Factory is the Factory for scraper. type Factory struct { } @@ -67,7 +50,6 @@ func (f *Factory) CreateMetricsScraper( ) (scraperhelper.Scraper, error) { cfg := config.(*Config) s := newPagingScraper(ctx, settings, cfg) - logFeatureGateMessage(settings) return scraperhelper.NewScraper( TypeStr, @@ -75,14 +57,3 @@ func (f *Factory) CreateMetricsScraper( scraperhelper.WithStart(s.start), ) } - -func logFeatureGateMessage(settings component.ReceiverCreateSettings) { - if featuregate.GetRegistry().IsEnabled(removeDirectionAttributeFeatureGateID) { - settings.Logger.Info("The " + removeDirectionAttributeFeatureGateID + " featre gate is enabled. This " + - "otel collector will report metrics without a direction attribute, which is good for future support") - } else { - settings.Logger.Info("WARNING - Breaking Change: " + removeDirectionAttributeFeatureGate.Description) - settings.Logger.Info("The feature gate " + removeDirectionAttributeFeatureGateID + " is disabled. This " + - "otel collector will report metrics with a direction attribute, be aware this will not be supported in the future") - } -} diff --git a/receiver/hostmetricsreceiver/internal/scraper/pagingscraper/metadata.yaml b/receiver/hostmetricsreceiver/internal/scraper/pagingscraper/metadata.yaml index cf3a2f06c54f..b78d7b6624f7 100644 --- a/receiver/hostmetricsreceiver/internal/scraper/pagingscraper/metadata.yaml +++ b/receiver/hostmetricsreceiver/internal/scraper/pagingscraper/metadata.yaml @@ -29,6 +29,7 @@ metrics: monotonic: false attributes: [device, state] + # produced when receiver.hostmetricsreceiver.emitMetricsWithDirectionAttribute feature gate is enabled system.paging.operations: enabled: true description: The number of paging operations. @@ -39,7 +40,7 @@ metrics: monotonic: true attributes: [direction, type] -# produced when receiver.hostmetricsreceiver.removeDirectionAttributePagingMetrics feature gate is enabled + # produced when receiver.hostmetricsreceiver.emitMetricsWithoutDirectionAttribute feature gate is enabled system.paging.operations.page_in: enabled: true description: The number of page_in operations. @@ -50,7 +51,7 @@ metrics: monotonic: true attributes: [type] -# produced when receiver.hostmetricsreceiver.removeDirectionAttributePagingMetrics feature gate is enabled + # produced when receiver.hostmetricsreceiver.emitMetricsWithoutDirectionAttribute feature gate is enabled system.paging.operations.page_out: enabled: true description: The number of page_out operations. diff --git a/receiver/hostmetricsreceiver/internal/scraper/pagingscraper/paging_scraper_others.go b/receiver/hostmetricsreceiver/internal/scraper/pagingscraper/paging_scraper_others.go index 38549f282d7e..21f0216597c7 100644 --- a/receiver/hostmetricsreceiver/internal/scraper/pagingscraper/paging_scraper_others.go +++ b/receiver/hostmetricsreceiver/internal/scraper/pagingscraper/paging_scraper_others.go @@ -30,6 +30,7 @@ import ( "go.opentelemetry.io/collector/receiver/scrapererror" "go.opentelemetry.io/collector/service/featuregate" + "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal" "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/pagingscraper/internal/metadata" ) @@ -45,14 +46,24 @@ type scraper struct { mb *metadata.MetricsBuilder // for mocking - bootTime func() (uint64, error) - getPageFileStats func() ([]*pageFileStats, error) - swapMemory func() (*mem.SwapMemoryStat, error) + bootTime func() (uint64, error) + getPageFileStats func() ([]*pageFileStats, error) + swapMemory func() (*mem.SwapMemoryStat, error) + emitMetricsWithDirectionAttribute bool + emitMetricsWithoutDirectionAttribute bool } // newPagingScraper creates a Paging Scraper func newPagingScraper(_ context.Context, settings component.ReceiverCreateSettings, cfg *Config) *scraper { - return &scraper{settings: settings, config: cfg, bootTime: host.BootTime, getPageFileStats: getPageFileStats, swapMemory: mem.SwapMemory} + return &scraper{ + settings: settings, + config: cfg, + bootTime: host.BootTime, + getPageFileStats: getPageFileStats, + swapMemory: mem.SwapMemory, + emitMetricsWithDirectionAttribute: featuregate.GetRegistry().IsEnabled(internal.EmitMetricsWithDirectionAttributeFeatureGateID), + emitMetricsWithoutDirectionAttribute: featuregate.GetRegistry().IsEnabled(internal.EmitMetricsWithoutDirectionAttributeFeatureGateID), + } } func (s *scraper) start(context.Context, component.Host) error { @@ -126,13 +137,14 @@ func (s *scraper) scrapePagingMetrics() error { } func (s *scraper) recordPagingOperationsDataPoints(now pcommon.Timestamp, swap *mem.SwapMemoryStat) { - if featuregate.GetRegistry().IsEnabled(removeDirectionAttributeFeatureGateID) { + if s.emitMetricsWithoutDirectionAttribute { s.mb.RecordSystemPagingOperationsPageInDataPoint(now, int64(swap.Sin), metadata.AttributeTypeMajor) s.mb.RecordSystemPagingOperationsPageOutDataPoint(now, int64(swap.Sout), metadata.AttributeTypeMajor) s.mb.RecordSystemPagingOperationsPageInDataPoint(now, int64(swap.PgIn), metadata.AttributeTypeMinor) s.mb.RecordSystemPagingOperationsPageOutDataPoint(now, int64(swap.PgOut), metadata.AttributeTypeMinor) - } else { + } + if s.emitMetricsWithDirectionAttribute { s.mb.RecordSystemPagingOperationsDataPoint(now, int64(swap.Sin), metadata.AttributeDirectionPageIn, metadata.AttributeTypeMajor) s.mb.RecordSystemPagingOperationsDataPoint(now, int64(swap.Sout), metadata.AttributeDirectionPageOut, metadata.AttributeTypeMajor) s.mb.RecordSystemPagingOperationsDataPoint(now, int64(swap.PgIn), metadata.AttributeDirectionPageIn, metadata.AttributeTypeMinor) diff --git a/receiver/hostmetricsreceiver/internal/scraper/pagingscraper/paging_scraper_test.go b/receiver/hostmetricsreceiver/internal/scraper/pagingscraper/paging_scraper_test.go index 664e17ba699f..69dd7a518417 100644 --- a/receiver/hostmetricsreceiver/internal/scraper/pagingscraper/paging_scraper_test.go +++ b/receiver/hostmetricsreceiver/internal/scraper/pagingscraper/paging_scraper_test.go @@ -25,7 +25,6 @@ import ( "go.opentelemetry.io/collector/component/componenttest" "go.opentelemetry.io/collector/pdata/pcommon" "go.opentelemetry.io/collector/pdata/pmetric" - "go.opentelemetry.io/collector/service/featuregate" "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal" "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/pagingscraper/internal/metadata" @@ -33,12 +32,13 @@ import ( func TestScrape(t *testing.T) { type testCase struct { - name string - config Config - bootTimeFunc func() (uint64, error) - expectedStartTime pcommon.Timestamp - initializationErr string - removeDirectionAttributeFeatureGateEnabled bool + name string + config Config + expectedStartTime pcommon.Timestamp + initializationErr string + expectMetricsWithDirectionAttribute bool + expectMetricsWithoutDirectionAttribute bool + mutateScraper func(*scraper) } config := metadata.DefaultMetricsSettings() @@ -46,34 +46,43 @@ func TestScrape(t *testing.T) { testCases := []testCase{ { - name: "Standard", - config: Config{Metrics: config}, + name: "Standard", + config: Config{Metrics: config}, + expectMetricsWithDirectionAttribute: true, }, { - name: "Standard with direction removed", - config: Config{Metrics: config}, - removeDirectionAttributeFeatureGateEnabled: true, + name: "Standard with direction removed", + config: Config{Metrics: config}, + expectMetricsWithDirectionAttribute: false, + expectMetricsWithoutDirectionAttribute: true, + mutateScraper: func(s *scraper) { + s.emitMetricsWithDirectionAttribute = false + s.emitMetricsWithoutDirectionAttribute = true + }, }, { - name: "Validate Start Time", - config: Config{Metrics: config}, - bootTimeFunc: func() (uint64, error) { return 100, nil }, + name: "Validate Start Time", + config: Config{Metrics: config}, + mutateScraper: func(s *scraper) { + s.bootTime = func() (uint64, error) { return 100, nil } + }, expectedStartTime: 100 * 1e9, }, { - name: "Boot Time Error", - config: Config{Metrics: config}, - bootTimeFunc: func() (uint64, error) { return 0, errors.New("err1") }, + name: "Boot Time Error", + config: Config{Metrics: config}, + mutateScraper: func(s *scraper) { + s.bootTime = func() (uint64, error) { return 0, errors.New("err1") } + }, initializationErr: "err1", }, } for _, test := range testCases { t.Run(test.name, func(t *testing.T) { - featuregate.GetRegistry().Apply(map[string]bool{removeDirectionAttributeFeatureGateID: test.removeDirectionAttributeFeatureGateEnabled}) scraper := newPagingScraper(context.Background(), componenttest.NewNopReceiverCreateSettings(), &test.config) - if test.bootTimeFunc != nil { - scraper.bootTime = test.bootTimeFunc + if test.mutateScraper != nil { + test.mutateScraper(scraper) } err := scraper.start(context.Background(), componenttest.NewNopHost()) @@ -92,7 +101,7 @@ func TestScrape(t *testing.T) { if runtime.GOOS == "windows" { expectedMetrics = 3 } - if test.removeDirectionAttributeFeatureGateEnabled { + if test.expectMetricsWithoutDirectionAttribute { // in/out are separated into an additional metric expectedMetrics++ } @@ -105,11 +114,14 @@ func TestScrape(t *testing.T) { startIndex++ } - if test.removeDirectionAttributeFeatureGateEnabled { - assertPagingOperationsMetricValid(t, []pmetric.Metric{metrics.At(startIndex), metrics.At(startIndex + 1)}, test.expectedStartTime, test.removeDirectionAttributeFeatureGateEnabled) + if test.expectMetricsWithoutDirectionAttribute { + assertPagingOperationsMetricValid(t, []pmetric.Metric{metrics.At(startIndex), + metrics.At(startIndex + 1)}, test.expectedStartTime, true) startIndex++ - } else { - assertPagingOperationsMetricValid(t, []pmetric.Metric{metrics.At(startIndex)}, test.expectedStartTime, test.removeDirectionAttributeFeatureGateEnabled) + } + if test.expectMetricsWithDirectionAttribute { + assertPagingOperationsMetricValid(t, []pmetric.Metric{metrics.At(startIndex)}, + test.expectedStartTime, false) } internal.AssertSameTimeStampForMetrics(t, metrics, 0, metrics.Len()-2) diff --git a/receiver/hostmetricsreceiver/internal/scraper/pagingscraper/paging_scraper_windows.go b/receiver/hostmetricsreceiver/internal/scraper/pagingscraper/paging_scraper_windows.go index eeadde1758c7..04a00edcf751 100644 --- a/receiver/hostmetricsreceiver/internal/scraper/pagingscraper/paging_scraper_windows.go +++ b/receiver/hostmetricsreceiver/internal/scraper/pagingscraper/paging_scraper_windows.go @@ -29,6 +29,7 @@ import ( "go.opentelemetry.io/collector/receiver/scrapererror" "go.opentelemetry.io/collector/service/featuregate" + "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal" "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/perfcounters" "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/pagingscraper/internal/metadata" ) @@ -52,13 +53,23 @@ type scraper struct { perfCounterScraper perfcounters.PerfCounterScraper // for mocking - bootTime func() (uint64, error) - pageFileStats func() ([]*pageFileStats, error) + bootTime func() (uint64, error) + pageFileStats func() ([]*pageFileStats, error) + emitMetricsWithDirectionAttribute bool + emitMetricsWithoutDirectionAttribute bool } // newPagingScraper creates a Paging Scraper func newPagingScraper(_ context.Context, settings component.ReceiverCreateSettings, cfg *Config) *scraper { - return &scraper{settings: settings, config: cfg, perfCounterScraper: &perfcounters.PerfLibScraper{}, bootTime: host.BootTime, pageFileStats: getPageFileStats} + return &scraper{ + settings: settings, + config: cfg, + perfCounterScraper: &perfcounters.PerfLibScraper{}, + bootTime: host.BootTime, + pageFileStats: getPageFileStats, + emitMetricsWithDirectionAttribute: featuregate.GetRegistry().IsEnabled(internal.EmitMetricsWithDirectionAttributeFeatureGateID), + emitMetricsWithoutDirectionAttribute: featuregate.GetRegistry().IsEnabled(internal.EmitMetricsWithoutDirectionAttributeFeatureGateID), + } } func (s *scraper) start(context.Context, component.Host) error { @@ -140,10 +151,11 @@ func (s *scraper) scrapePagingOperationsMetric() error { } func (s *scraper) recordPagingOperationsDataPoints(now pcommon.Timestamp, memoryCounterValues *perfcounters.CounterValues) { - if featuregate.GetRegistry().IsEnabled(removeDirectionAttributeFeatureGateID) { + if s.emitMetricsWithoutDirectionAttribute { s.mb.RecordSystemPagingOperationsPageInDataPoint(now, memoryCounterValues.Values[pageReadsPerSec], metadata.AttributeTypeMajor) s.mb.RecordSystemPagingOperationsPageOutDataPoint(now, memoryCounterValues.Values[pageWritesPerSec], metadata.AttributeTypeMajor) - } else { + } + if s.emitMetricsWithDirectionAttribute { s.mb.RecordSystemPagingOperationsDataPoint(now, memoryCounterValues.Values[pageReadsPerSec], metadata.AttributeDirectionPageIn, metadata.AttributeTypeMajor) s.mb.RecordSystemPagingOperationsDataPoint(now, memoryCounterValues.Values[pageWritesPerSec], metadata.AttributeDirectionPageOut, metadata.AttributeTypeMajor) } diff --git a/receiver/hostmetricsreceiver/internal/scraper/processscraper/factory.go b/receiver/hostmetricsreceiver/internal/scraper/processscraper/factory.go index 084e3982efd4..ad2174ed39e8 100644 --- a/receiver/hostmetricsreceiver/internal/scraper/processscraper/factory.go +++ b/receiver/hostmetricsreceiver/internal/scraper/processscraper/factory.go @@ -21,7 +21,6 @@ import ( "go.opentelemetry.io/collector/component" "go.opentelemetry.io/collector/receiver/scraperhelper" - "go.opentelemetry.io/collector/service/featuregate" "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal" "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/processscraper/internal/metadata" @@ -32,24 +31,8 @@ import ( const ( // TypeStr the value of "type" key in configuration. TypeStr = "process" - // ID for a temporary feature gate" - removeDirectionAttributeFeatureGateID = "receiver.hostmetricsreceiver.removeDirectionAttributeProcessMetrics" ) -var removeDirectionAttributeFeatureGate = featuregate.Gate{ - ID: removeDirectionAttributeFeatureGateID, - Enabled: false, - Description: "Some process metrics reported by the hostmetricsreceiver are transitioning from being reported " + - "with a direction attribute to being reported with the direction included in the metric name to adhere to the " + - "OpenTelemetry specification. You can control whether the hostmetricsreceiver reports metrics with a direction " + - "attribute using the " + removeDirectionAttributeFeatureGateID + " feature gate. For more details, see: " + - "https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/hostmetricsreceiver/README.md#feature-gate-configurations", -} - -func init() { - featuregate.GetRegistry().MustRegister(removeDirectionAttributeFeatureGate) -} - // Factory is the Factory for scraper. type Factory struct { } @@ -75,7 +58,6 @@ func (f *Factory) CreateMetricsScraper( if err != nil { return nil, err } - logFeatureGateMessage(settings) return scraperhelper.NewScraper( TypeStr, @@ -83,14 +65,3 @@ func (f *Factory) CreateMetricsScraper( scraperhelper.WithStart(s.start), ) } - -func logFeatureGateMessage(settings component.ReceiverCreateSettings) { - if featuregate.GetRegistry().IsEnabled(removeDirectionAttributeFeatureGateID) { - settings.Logger.Info("The " + removeDirectionAttributeFeatureGateID + " featre gate is enabled. This " + - "otel collector will report metrics without a direction attribute, which is good for future support") - } else { - settings.Logger.Info("WARNING - Breaking Change: " + removeDirectionAttributeFeatureGate.Description) - settings.Logger.Info("The feature gate " + removeDirectionAttributeFeatureGateID + " is disabled. This " + - "otel collector will report metrics with a direction attribute, be aware this will not be supported in the future") - } -} diff --git a/receiver/hostmetricsreceiver/internal/scraper/processscraper/metadata.yaml b/receiver/hostmetricsreceiver/internal/scraper/processscraper/metadata.yaml index ee4015c962a8..d12a2414a3ca 100644 --- a/receiver/hostmetricsreceiver/internal/scraper/processscraper/metadata.yaml +++ b/receiver/hostmetricsreceiver/internal/scraper/processscraper/metadata.yaml @@ -73,6 +73,7 @@ metrics: aggregation: cumulative monotonic: false + # produced when receiver.hostmetricsreceiver.emitMetricsWithDirectionAttribute feature gate is enabled process.disk.io: enabled: true description: Disk bytes transferred. @@ -83,7 +84,7 @@ metrics: monotonic: true attributes: [direction] -# produced when receiver.hostmetricsreceiver.removeDirectionAttributeProcessMetrics feature gate is enabled + # produced when receiver.hostmetricsreceiver.emitMetricsWithoutDirectionAttribute feature gate is enabled process.disk.io.read: enabled: true description: Disk bytes read. @@ -93,7 +94,7 @@ metrics: aggregation: cumulative monotonic: true -# produced when receiver.hostmetricsreceiver.removeDirectionAttributeProcessMetrics feature gate is enabled + # produced when receiver.hostmetricsreceiver.emitMetricsWithoutDirectionAttribute feature gate is enabled process.disk.io.write: enabled: true description: Disk bytes written. diff --git a/receiver/hostmetricsreceiver/internal/scraper/processscraper/process_scraper.go b/receiver/hostmetricsreceiver/internal/scraper/processscraper/process_scraper.go index a6e39a6118df..4428ffa609b7 100644 --- a/receiver/hostmetricsreceiver/internal/scraper/processscraper/process_scraper.go +++ b/receiver/hostmetricsreceiver/internal/scraper/processscraper/process_scraper.go @@ -28,6 +28,7 @@ import ( "go.opentelemetry.io/collector/service/featuregate" "github.com/open-telemetry/opentelemetry-collector-contrib/internal/coreinternal/processor/filterset" + "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal" "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/processscraper/internal/metadata" ) @@ -48,13 +49,22 @@ type scraper struct { excludeFS filterset.FilterSet // for mocking - bootTime func() (uint64, error) - getProcessHandles func() (processHandles, error) + bootTime func() (uint64, error) + getProcessHandles func() (processHandles, error) + emitMetricsWithDirectionAttribute bool + emitMetricsWithoutDirectionAttribute bool } // newProcessScraper creates a Process Scraper func newProcessScraper(settings component.ReceiverCreateSettings, cfg *Config) (*scraper, error) { - scraper := &scraper{settings: settings, config: cfg, bootTime: host.BootTime, getProcessHandles: getProcessHandlesInternal} + scraper := &scraper{ + settings: settings, + config: cfg, + bootTime: host.BootTime, + getProcessHandles: getProcessHandlesInternal, + emitMetricsWithDirectionAttribute: featuregate.GetRegistry().IsEnabled(internal.EmitMetricsWithDirectionAttributeFeatureGateID), + emitMetricsWithoutDirectionAttribute: featuregate.GetRegistry().IsEnabled(internal.EmitMetricsWithoutDirectionAttributeFeatureGateID), + } var err error @@ -201,12 +211,14 @@ func (s *scraper) scrapeAndAppendDiskIOMetric(now pcommon.Timestamp, handle proc return err } - if featuregate.GetRegistry().IsEnabled(removeDirectionAttributeFeatureGateID) { + if s.emitMetricsWithoutDirectionAttribute { s.mb.RecordProcessDiskIoReadDataPoint(now, int64(io.ReadBytes)) s.mb.RecordProcessDiskIoWriteDataPoint(now, int64(io.WriteBytes)) - } else { + } + if s.emitMetricsWithDirectionAttribute { s.mb.RecordProcessDiskIoDataPoint(now, int64(io.ReadBytes), metadata.AttributeDirectionRead) s.mb.RecordProcessDiskIoDataPoint(now, int64(io.WriteBytes), metadata.AttributeDirectionWrite) } + return nil } diff --git a/receiver/hostmetricsreceiver/internal/scraper/processscraper/process_scraper_test.go b/receiver/hostmetricsreceiver/internal/scraper/processscraper/process_scraper_test.go index 31a676b31924..e10d0b8d82a5 100644 --- a/receiver/hostmetricsreceiver/internal/scraper/processscraper/process_scraper_test.go +++ b/receiver/hostmetricsreceiver/internal/scraper/processscraper/process_scraper_test.go @@ -31,7 +31,6 @@ import ( "go.opentelemetry.io/collector/pdata/pmetric" "go.opentelemetry.io/collector/receiver/scrapererror" conventions "go.opentelemetry.io/collector/semconv/v1.6.1" - "go.opentelemetry.io/collector/service/featuregate" "github.com/open-telemetry/opentelemetry-collector-contrib/internal/coreinternal/processor/filterset" "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal" @@ -47,28 +46,46 @@ func skipTestOnUnsupportedOS(t *testing.T) { func TestScrape(t *testing.T) { skipTestOnUnsupportedOS(t) type testCase struct { - name string - removeDirectionAttributeFeatureGateEnabled bool + name string + expectMetricsWithDirectionAttribute bool + expectMetricsWithoutDirectionAttribute bool + mutateScraper func(*scraper) } testCases := []testCase{ { - name: "Standard", + name: "Standard", + expectMetricsWithDirectionAttribute: true, + expectMetricsWithoutDirectionAttribute: false, }, { - name: "Standard with direction removed", - removeDirectionAttributeFeatureGateEnabled: true, + name: "Standard with direction removed", + expectMetricsWithDirectionAttribute: false, + expectMetricsWithoutDirectionAttribute: true, + mutateScraper: func(s *scraper) { + s.emitMetricsWithDirectionAttribute = false + s.emitMetricsWithoutDirectionAttribute = true + }, + }, + { + name: "Emit both old and new metrics", + expectMetricsWithDirectionAttribute: true, + expectMetricsWithoutDirectionAttribute: true, + mutateScraper: func(s *scraper) { + s.emitMetricsWithDirectionAttribute = true + s.emitMetricsWithoutDirectionAttribute = true + }, }, } const bootTime = 100 const expectedStartTime = 100 * 1e9 - originalVal := featuregate.GetRegistry().IsEnabled(removeDirectionAttributeFeatureGateID) for _, test := range testCases { t.Run(test.name, func(t *testing.T) { - featuregate.GetRegistry().Apply(map[string]bool{removeDirectionAttributeFeatureGateID: test.removeDirectionAttributeFeatureGateEnabled}) - scraper, err := newProcessScraper(componenttest.NewNopReceiverCreateSettings(), &Config{Metrics: metadata.DefaultMetricsSettings()}) + if test.mutateScraper != nil { + test.mutateScraper(scraper) + } scraper.bootTime = func() (uint64, error) { return bootTime, nil } require.NoError(t, err, "Failed to create process scraper: %v", err) err = scraper.start(context.Background(), componenttest.NewNopHost()) @@ -95,11 +112,15 @@ func TestScrape(t *testing.T) { assertProcessResourceAttributesExist(t, md.ResourceMetrics()) assertCPUTimeMetricValid(t, md.ResourceMetrics(), expectedStartTime) assertMemoryUsageMetricValid(t, md.ResourceMetrics(), expectedStartTime) - assertDiskIOMetricValid(t, md.ResourceMetrics(), expectedStartTime, test.removeDirectionAttributeFeatureGateEnabled) + if test.expectMetricsWithDirectionAttribute { + assertOldDiskIOMetricValid(t, md.ResourceMetrics(), expectedStartTime) + } + if test.expectMetricsWithoutDirectionAttribute { + assertNewDiskIOMetricValid(t, md.ResourceMetrics(), expectedStartTime) + } assertSameTimeStampForAllMetricsWithinResource(t, md.ResourceMetrics()) }) } - featuregate.GetRegistry().Apply(map[string]bool{removeDirectionAttributeFeatureGateID: originalVal}) } func assertProcessResourceAttributesExist(t *testing.T, resourceMetrics pmetric.ResourceMetricsSlice) { @@ -142,28 +163,28 @@ func assertMemoryUsageMetricValid(t *testing.T, resourceMetrics pmetric.Resource } } -func assertDiskIOMetricValid(t *testing.T, resourceMetrics pmetric.ResourceMetricsSlice, startTime pcommon.Timestamp, removeDirection bool) { - if removeDirection { - for _, metricName := range []string{"process.disk.io.read", "process.disk.io.write"} { - diskIOMetric := getMetric(t, metricName, resourceMetrics) - assert.Equal(t, metricName, diskIOMetric.Name()) - if startTime != 0 { - internal.AssertSumMetricStartTimeEquals(t, diskIOMetric, startTime) - } - } - } else { - diskIOMetric := getMetric(t, "process.disk.io", resourceMetrics) - assert.Equal(t, "process.disk.io", diskIOMetric.Name()) +func assertNewDiskIOMetricValid(t *testing.T, resourceMetrics pmetric.ResourceMetricsSlice, + startTime pcommon.Timestamp) { + for _, metricName := range []string{"process.disk.io.read", "process.disk.io.write"} { + diskIOMetric := getMetric(t, metricName, resourceMetrics) if startTime != 0 { internal.AssertSumMetricStartTimeEquals(t, diskIOMetric, startTime) } - internal.AssertSumMetricHasAttributeValue(t, diskIOMetric, 0, "direction", - pcommon.NewValueString(metadata.AttributeDirectionRead.String())) - internal.AssertSumMetricHasAttributeValue(t, diskIOMetric, 1, "direction", - pcommon.NewValueString(metadata.AttributeDirectionWrite.String())) } } +func assertOldDiskIOMetricValid(t *testing.T, resourceMetrics pmetric.ResourceMetricsSlice, + startTime pcommon.Timestamp) { + diskIOMetric := getMetric(t, "process.disk.io", resourceMetrics) + if startTime != 0 { + internal.AssertSumMetricStartTimeEquals(t, diskIOMetric, startTime) + } + internal.AssertSumMetricHasAttributeValue(t, diskIOMetric, 0, "direction", + pcommon.NewValueString(metadata.AttributeDirectionRead.String())) + internal.AssertSumMetricHasAttributeValue(t, diskIOMetric, 1, "direction", + pcommon.NewValueString(metadata.AttributeDirectionWrite.String())) +} + func assertSameTimeStampForAllMetricsWithinResource(t *testing.T, resourceMetrics pmetric.ResourceMetricsSlice) { for i := 0; i < resourceMetrics.Len(); i++ { ilms := resourceMetrics.At(i).ScopeMetrics() diff --git a/unreleased/hostmetrics-remove-direction-paging.yaml b/unreleased/hostmetrics-remove-direction-paging.yaml deleted file mode 100644 index 5f525f09deee..000000000000 --- a/unreleased/hostmetrics-remove-direction-paging.yaml +++ /dev/null @@ -1,12 +0,0 @@ -change_type: deprecation - -component: hostmetricsreceiver - -note: "Remove direction for paging metrics. The feature gate: `receiver.hostmetricsreceiver.removeDirectionAttributePagingMetrics` can be set to apply the following" - -issues: [11849] - -subtext: |2 - - `system.paging.operations` will become: - - `system.paging.operations.page_in` - - `system.paging.operations.page_out` \ No newline at end of file diff --git a/unreleased/hostmetrics-rm-direction-process.yaml b/unreleased/hostmetrics-rm-direction-process.yaml deleted file mode 100755 index f34f8f352204..000000000000 --- a/unreleased/hostmetrics-rm-direction-process.yaml +++ /dev/null @@ -1,12 +0,0 @@ -change_type: deprecation - -component: hostmetricsreceiver - -note: "Remove direction for paging metrics. The feature gate: `receiver.hostmetricsreceiver.removeDirectionAttributeProcessMetrics` can be set to apply the following" - -issues: [11819] - -subtext: | - - `process.disk.io` will become: - - `process.disk.io.read` - - `process.disk.io.write` diff --git a/unreleased/hostmetrics-rm-direction.yaml b/unreleased/hostmetrics-rm-direction.yaml new file mode 100755 index 000000000000..49953f88e6ef --- /dev/null +++ b/unreleased/hostmetrics-rm-direction.yaml @@ -0,0 +1,30 @@ +change_type: deprecation + +component: hostmetricsreceiver + +note: >- + Remove direction for network, paging and process metrics. The feature gates + `receiver.hostmetricsreceiver.emitMetricsWithoutDirectionAttribute` and + `receiver.hostmetricsreceiver.emitMetricsWithDirectionAttribute` can be used to control the transition. + +issues: [11815] + +subtext: | + - `system.network.dropped` will become: + - `system.network.dropped.receive` + - `system.network.dropped.transmit` + - `system.network.errors` will become: + - `system.network.errors.receive` + - `system.network.errors.transmit` + - `system.network.io` will become: + - `system.network.io.receive` + - `system.network.io.transmit` + - `system.network.packets` will become: + - `system.network.packets.receive` + - `system.network.packets.transmit` + - `system.paging.operations` will become: + - `system.paging.operations.page_in` + - `system.paging.operations.page_out` + - `process.disk.io` will become: + - `process.disk.io.read` + - `process.disk.io.write` diff --git a/unreleased/hostmetricsrec-rm-direction.yaml b/unreleased/hostmetricsrec-rm-direction.yaml deleted file mode 100755 index 218f68ec8e55..000000000000 --- a/unreleased/hostmetricsrec-rm-direction.yaml +++ /dev/null @@ -1,28 +0,0 @@ -# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix' -change_type: deprecation - -# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver) -component: hostmetricsreceiver - -# A brief description of the change -note: "Remove direction attribute. The feature gate: `receiver.hostmetricsreceiver.removeDirectionAttribute` can be set to apply the following" - -# One or more tracking issues related to the change -issues: [11820] - -# (Optional) One or more lines of additional information to render under the primary note. -# These lines will be inserted directly into the document, so must be formatted accordingly. -# Use "subtext: |2" to preserve two spaces of indentation relative to the current level -subtext: |2 - - `system.network.dropped` will become: - - `system.network.dropped.receive` - - `system.network.dropped.transmit` - - `system.network.errors` will become: - - `system.network.errors.receive` - - `system.network.errors.transmit` - - `system.network.io` will become: - - `system.network.io.receive` - - `system.network.io.transmit` - - `system.network.packets` will become: - - `system.network.packets.receive` - - `system.network.packets.transmit`