Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[podmanreceiver] Add metrics and resource metadata #30232

Merged
merged 24 commits into from
Apr 9, 2024
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
cbeaf84
feat: add Podman receiver metadata
rogercoll Dec 28, 2023
b9a3406
docs: add change log file
rogercoll Dec 28, 2023
78901ea
fix: gci linter
rogercoll Dec 28, 2023
64c0efc
fix: tidy module file
rogercoll Dec 28, 2023
5455ff6
fix: spelling mistake
rogercoll Dec 28, 2023
b9935f8
fix: allow container duplicated metrics
rogercoll Dec 28, 2023
df281be
fix: use "By" unit type for memory metrics
rogercoll Dec 29, 2023
d528755
Merge branch 'main' into add_podman_metadata
rogercoll Dec 29, 2023
5f3528c
docs: fix blkio metric kernel documentation url
rogercoll Dec 31, 2023
befc88c
Update cmd/mdatagen/validate_test.go
rogercoll Jan 3, 2024
6c37699
Merge branch 'main' into add_podman_metadata
rogercoll Jan 4, 2024
0cbdc4e
Merge branch 'main' into add_podman_metadata
rogercoll Jan 9, 2024
1bd427e
Merge branch 'main' into add_podman_metadata
rogercoll Jan 10, 2024
0032f70
Merge branch 'main' into add_podman_metadata
rogercoll Jan 10, 2024
afce91c
Merge branch 'main' into add_podman_metadata
rogercoll Jan 14, 2024
a5a0099
feat: move cpu usage metrics to seconds
rogercoll Jan 14, 2024
65cf771
Merge branch 'main' into add_podman_metadata
rogercoll Jan 27, 2024
91e24ff
rollback fix for networking tx/rx metrics
rogercoll Jan 27, 2024
b2ed9a8
Merge branch 'main' into add_podman_metadata
rogercoll Mar 27, 2024
fa12570
Merge branch 'main' into add_podman_metadata
rogercoll Mar 27, 2024
6d89c48
Merge branch 'main' into add_podman_metadata
rogercoll Mar 28, 2024
70d142c
Merge branch 'main' into add_podman_metadata
rogercoll Apr 9, 2024
384cb7d
update changelog with breaking changes
rogercoll Apr 9, 2024
0dfb234
Merge branch 'main' into add_podman_metadata
rogercoll Apr 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions .chloggen/add_podman_metadata.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Use this changelog template to create an entry for release notes.

# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: enhancement

# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
component: podmanreceiver

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: "Adds metrics and resources metadata"

# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
issues: [28640]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext:

# If your change doesn't affect end users or the exported elements of any package,
# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
# Optional: The change log or logs in which this entry should be included.
# e.g. '[user]' or '[user, api]'
# Include 'user' if the change is relevant to end users.
# Include 'api' if there is a change to a library API.
# Default: '[user]'
change_logs: []
14 changes: 11 additions & 3 deletions cmd/mdatagen/validate_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -108,9 +108,17 @@ func TestValidate(t *testing.T) {

func TestValidateMetricDuplicates(t *testing.T) {
allowedMetrics := map[string][]string{
"container.cpu.utilization": {"docker_stats", "kubeletstats"},
"container.memory.rss": {"docker_stats", "kubeletstats"},
"container.uptime": {"docker_stats", "kubeletstats"},
"container.cpu.utilization": {"docker_stats", "kubeletstats"},
"container.cpu.usage.system": {"docker_stats", "podman_stats"},
"container.cpu.usage.percpu": {"docker_stats", "podman_stats"},
"container.cpu.usage.total": {"docker_stats", "podman_stats"},
Copy link
Member

@mx-psi mx-psi Jan 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it okay to report the same metric with different units?

As discussed above, on the Docker stats receiver we have nanoseconds

### container.cpu.usage.total
Total CPU time consumed.
| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic |
| ---- | ----------- | ---------- | ----------------------- | --------- |
| ns | Sum | Int | Cumulative | true |
while here we use seconds
### container.cpu.usage.total
Total CPU time consumed.
| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic |
| ---- | ----------- | ---------- | ----------------------- | --------- |
| s | Sum | Int | Cumulative | true |

Is that okay? If seconds is the right unit, shouldn't we use it on the Docker stats receiver as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so, I would not use second's precision for the sake of convenience at the expense of precision. Instead, we should endeavor to establish distinct conventions specifically tailored to containers. Should we wait for the container's semantic convention open-telemetry/semantic-conventions#282 (nanoseconds)?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so, I would not use second's precision for the sake of convenience at the expense of precision. Instead, we should endeavor to establish distinct conventions specifically tailored to containers.

My objection here is with the use of different units on each metric, I would expect them to have the same unit (whether it is nanoseconds or seconds I agree is something we can leave open-telemetry/semantic-conventions#282 to decide on)

"container.memory.usage.limit": {"docker_stats", "podman_stats"},
"container.memory.percent": {"docker_stats", "podman_stats"},
"container.memory.usage.total": {"docker_stats", "podman_stats"},
"container.network.io.usage.tx_bytes": {"docker_stats", "podman_stats"},
"container.network.io.usage.rx_bytes": {"docker_stats", "podman_stats"},
"container.memory.rss": {"docker_stats", "kubeletstats"},
"container.uptime": {"docker_stats", "kubeletstats"},
}
allMetrics := map[string][]string{}
err := filepath.Walk("../../receiver", func(path string, info fs.FileInfo, err error) error {
Expand Down
6 changes: 6 additions & 0 deletions receiver/podmanreceiver/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ The following settings are optional:
- `collection_interval` (default = `10s`): The interval at which to gather container stats.
- `initial_delay` (default = `1s`): defines how long this receiver waits before starting.
- `timeout` (default = `5s`): The maximum amount of time to wait for Podman API responses.
- `metrics` (defaults at [./documentation.md](./documentation.md)): Enables/disables individual metrics. See [./documentation.md](./documentation.md) for full detail.

Example:

Expand All @@ -43,6 +44,9 @@ receivers:
timeout: 10s
collection_interval: 10s
initial_delay: 1s
metrics:
container.cpu.usage.system:
enabled: false
```

The full list of settings exposed for this receiver are documented [here](./config.go)
Expand Down Expand Up @@ -85,6 +89,8 @@ The receiver emits the following metrics:
container.cpu.percent
container.cpu.usage.percpu

See [./documentation.md](./documentation.md) for full detail.

## Building

This receiver uses the official libpod Go bindings for Podman. In order to include
Expand Down
5 changes: 5 additions & 0 deletions receiver/podmanreceiver/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ import (
"go.opentelemetry.io/collector/component"
"go.opentelemetry.io/collector/config/configopaque"
"go.opentelemetry.io/collector/receiver/scraperhelper"

"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/podmanreceiver/internal/metadata"
)

var _ component.Config = (*Config)(nil)
Expand All @@ -22,6 +24,9 @@ type Config struct {
APIVersion string `mapstructure:"api_version"`
SSHKey string `mapstructure:"ssh_key"`
SSHPassphrase configopaque.String `mapstructure:"ssh_passphrase"`

// MetricsBuilderConfig config. Enable or disable stats by name.
metadata.MetricsBuilderConfig `mapstructure:",squash"`
}

func (config Config) Validate() error {
Expand Down
10 changes: 6 additions & 4 deletions receiver/podmanreceiver/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,9 @@ func TestLoadConfig(t *testing.T) {
InitialDelay: time.Second,
Timeout: 5 * time.Second,
},
APIVersion: defaultAPIVersion,
Endpoint: "unix:///run/podman/podman.sock",
APIVersion: defaultAPIVersion,
Endpoint: "unix:///run/podman/podman.sock",
MetricsBuilderConfig: metadata.DefaultMetricsBuilderConfig(),
},
},
{
Expand All @@ -48,8 +49,9 @@ func TestLoadConfig(t *testing.T) {
InitialDelay: time.Second,
Timeout: 20 * time.Second,
},
APIVersion: defaultAPIVersion,
Endpoint: "http://example.com/",
APIVersion: defaultAPIVersion,
Endpoint: "http://example.com/",
MetricsBuilderConfig: metadata.DefaultMetricsBuilderConfig(),
},
},
}
Expand Down
120 changes: 120 additions & 0 deletions receiver/podmanreceiver/documentation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
[comment]: <> (Code generated by mdatagen. DO NOT EDIT.)

# podman_stats

## Default Metrics

The following metrics are emitted by default. Each of them can be disabled by applying the following configuration:

```yaml
metrics:
<metric_name>:
enabled: false
```

### container.blockio.io_service_bytes_recursive.read

Number of bytes transferred from the disk by the container

[More docs](https://www.kernel.org/doc/Documentation/cgroup-v1/blkio-controller.txt).

| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic |
| ---- | ----------- | ---------- | ----------------------- | --------- |
| {operations} | Sum | Int | Cumulative | true |

### container.blockio.io_service_bytes_recursive.write

Number of bytes transferred to the disk by the container

[More docs](https://www.kernel.org/doc/Documentation/cgroup-v1/blkio-controller.txt).

| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic |
| ---- | ----------- | ---------- | ----------------------- | --------- |
| {operations} | Sum | Int | Cumulative | true |

### container.cpu.percent

Percent of CPU used by the container.

| Unit | Metric Type | Value Type |
| ---- | ----------- | ---------- |
| 1 | Gauge | Double |

### container.cpu.usage.percpu

Total CPU time consumed per CPU-core.

| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic |
| ---- | ----------- | ---------- | ----------------------- | --------- |
| s | Sum | Int | Cumulative | true |

#### Attributes

| Name | Description | Values |
| ---- | ----------- | ------ |
| core | The CPU core number when utilising per-CPU metrics. | Any Str |

### container.cpu.usage.system

System CPU usage.

| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic |
| ---- | ----------- | ---------- | ----------------------- | --------- |
| s | Sum | Int | Cumulative | true |

### container.cpu.usage.total

Total CPU time consumed.

| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic |
| ---- | ----------- | ---------- | ----------------------- | --------- |
| s | Sum | Int | Cumulative | true |

### container.memory.percent

Percentage of memory used.

| Unit | Metric Type | Value Type |
| ---- | ----------- | ---------- |
| 1 | Gauge | Double |

### container.memory.usage.limit

Memory limit of the container.

| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic |
| ---- | ----------- | ---------- | ----------------------- | --------- |
| By | Sum | Int | Cumulative | false |

### container.memory.usage.total

Memory usage of the container.

| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic |
| ---- | ----------- | ---------- | ----------------------- | --------- |
| By | Sum | Int | Cumulative | false |

### container.network.io.usage.rx_bytes

Bytes received by the container.

| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic |
| ---- | ----------- | ---------- | ----------------------- | --------- |
| By | Sum | Int | Cumulative | true |

### container.network.io.usage.tx_bytes

Bytes sent by the container.

| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic |
| ---- | ----------- | ---------- | ----------------------- | --------- |
| By | Sum | Int | Cumulative | true |

## Resource Attributes

| Name | Description | Values | Enabled |
| ---- | ----------- | ------ | ------- |
| container.id | The ID of the container. | Any Str | true |
| container.image.name | The name of the image in use by the container. | Any Str | true |
| container.name | The name of the container. | Any Str | true |
| container.runtime | The runtime of the container. For this receiver, it will always be 'podman'. | Any Str | true |
1 change: 1 addition & 0 deletions receiver/podmanreceiver/factory.go
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ func createDefaultConfig() *Config {
ScraperControllerSettings: cfg,
Endpoint: "unix:///run/podman/podman.sock",
APIVersion: defaultAPIVersion,
MetricsBuilderConfig: metadata.DefaultMetricsBuilderConfig(),
}
}

Expand Down
2 changes: 1 addition & 1 deletion receiver/podmanreceiver/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,14 @@ module github.com/open-telemetry/opentelemetry-collector-contrib/receiver/podman
go 1.20

require (
github.com/google/go-cmp v0.6.0
github.com/stretchr/testify v1.8.4
go.opentelemetry.io/collector/component v0.93.1-0.20240125183026-3cacd40b27e8
go.opentelemetry.io/collector/config/configopaque v0.93.1-0.20240125183026-3cacd40b27e8
go.opentelemetry.io/collector/confmap v0.93.1-0.20240125183026-3cacd40b27e8
go.opentelemetry.io/collector/consumer v0.93.1-0.20240125183026-3cacd40b27e8
go.opentelemetry.io/collector/pdata v1.0.2-0.20240125183026-3cacd40b27e8
go.opentelemetry.io/collector/receiver v0.93.1-0.20240125183026-3cacd40b27e8
go.opentelemetry.io/collector/semconv v0.93.1-0.20240125183026-3cacd40b27e8
go.opentelemetry.io/otel/metric v1.22.0
go.opentelemetry.io/otel/trace v1.22.0
go.uber.org/multierr v1.11.0
Expand Down
3 changes: 1 addition & 2 deletions receiver/podmanreceiver/go.sum

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading