forked from open-telemetry/opentelemetry-collector-contrib
-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for group process monitoring #38
Merged
Merged
Changes from 14 commits
Commits
Show all changes
15 commits
Select commit
Hold shift + click to select a range
24f8a80
initial config changes for process monitoring
mithunbelur 1963eab
separated group process scrapper
225dc9f
makefile
mithunbelur c44dd54
mdatagen changes
5627ce1
added group process scrapper new factory
59eddda
removed unused functions
2919368
changed cpu and memory usage into percentage
9988f9f
fixed cpu percentage
476a3b5
added search support for comm, exe and cmdline
4a52368
code refactor and readme
fee8075
review comments for doc
5bcd3f3
reverted deploy yaml
Pratyush8083 50b60b5
code refactor
17dc68a
added validation for some regex patterns
9bc3eb8
removed ucal package
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
67 changes: 67 additions & 0 deletions
67
receiver/hostmetricsreceiver/internal/scraper/groupprocessscraper/config.go
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,67 @@ | ||
| // Copyright The OpenTelemetry Authors | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| package groupprocessscraper // import "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/groupprocessscraper" | ||
|
|
||
| import ( | ||
| "time" | ||
|
|
||
| "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal" | ||
| "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/groupprocessscraper/internal/metadata" | ||
| ) | ||
|
|
||
| // Config relating to Process Metric Scraper. | ||
| type Config struct { | ||
| // MetricsBuilderConfig allows to customize scraped metrics/attributes representation. | ||
| metadata.MetricsBuilderConfig `mapstructure:",squash"` | ||
| internal.ScraperConfig | ||
|
|
||
| GroupConfig []GroupMatchConfig `mapstructure:"process_configs"` | ||
|
|
||
| // MuteProcessAllErrors is a flag that will mute all the errors encountered when trying to read metrics of a process. | ||
| // When this flag is enabled, there is no need to activate any other error suppression flags. | ||
| MuteProcessAllErrors bool `mapstructure:"mute_process_all_errors,omitempty"` | ||
|
|
||
| // MuteProcessNameError is a flag that will mute the error encountered when trying to read a process name the | ||
| // collector does not have permission to read. | ||
| // See https://github.com/open-telemetry/opentelemetry-collector/issues/3004 for more information. | ||
| // This flag is ignored when MuteProcessAllErrors is set to true as all errors are muted. | ||
| MuteProcessNameError bool `mapstructure:"mute_process_name_error,omitempty"` | ||
|
|
||
| // MuteProcessIOError is a flag that will mute the error encountered when trying to read IO metrics of a process | ||
| // the collector does not have permission to read. | ||
| // This flag is ignored when MuteProcessAllErrors is set to true as all errors are muted. | ||
| MuteProcessIOError bool `mapstructure:"mute_process_io_error,omitempty"` | ||
|
|
||
| // MuteProcessCgroupError is a flag that will mute the error encountered when trying to read the cgroup of a process | ||
| // the collector does not have permission to read. | ||
| // This flag is ignored when MuteProcessAllErrors is set to true as all errors are muted. | ||
| MuteProcessCgroupError bool `mapstructure:"mute_process_cgroup_error,omitempty"` | ||
|
|
||
| // MuteProcessExeError is a flag that will mute the error encountered when trying to read the executable path of a process | ||
| // the collector does not have permission to read (Linux). | ||
| // This flag is ignored when MuteProcessAllErrors is set to true as all errors are muted. | ||
| MuteProcessExeError bool `mapstructure:"mute_process_exe_error,omitempty"` | ||
|
|
||
| // MuteProcessUserError is a flag that will mute the error encountered when trying to read uid which | ||
| // doesn't exist on the system, eg. is owned by user existing in container only. | ||
| // This flag is ignored when MuteProcessAllErrors is set to true as all errors are muted. | ||
| MuteProcessUserError bool `mapstructure:"mute_process_user_error,omitempty"` | ||
|
|
||
| // ScrapeProcessDelay is used to indicate the minimum amount of time a process must be running | ||
| // before metrics are scraped for it. The default value is 0 seconds (0s). | ||
| ScrapeProcessDelay time.Duration `mapstructure:"scrape_process_delay"` | ||
| } | ||
|
|
||
| type MatchConfig struct { | ||
| Names []string `mapstructure:"names"` | ||
| MatchType string `mapstructure:"match_type"` | ||
| } | ||
|
|
||
| type GroupMatchConfig struct { | ||
| Names []string `mapstructure:"names"` | ||
| ProcessName string `mapstructure:"group_name"` | ||
| Comm MatchConfig `mapstructure:"comm"` | ||
| Exe MatchConfig `mapstructure:"exe"` | ||
| Cmdline MatchConfig `mapstructure:"cmdline"` | ||
| } |
6 changes: 6 additions & 0 deletions
6
receiver/hostmetricsreceiver/internal/scraper/groupprocessscraper/doc.go
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| // Copyright The OpenTelemetry Authors | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| //go:generate mdatagen metadata.yaml | ||
|
|
||
| package groupprocessscraper // import "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/groupprocessscraper" |
67 changes: 67 additions & 0 deletions
67
receiver/hostmetricsreceiver/internal/scraper/groupprocessscraper/documentation.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,67 @@ | ||
| [comment]: <> (Code generated by mdatagen. DO NOT EDIT.) | ||
|
|
||
| # hostmetricsreceiver/groupprocess | ||
|
|
||
| **Parent Component:** hostmetrics | ||
|
|
||
| ## Default Metrics | ||
|
|
||
| The following metrics are emitted by default. Each of them can be disabled by applying the following configuration: | ||
|
|
||
| ```yaml | ||
| metrics: | ||
| <metric_name>: | ||
| enabled: false | ||
| ``` | ||
|
|
||
| ### process.count | ||
|
|
||
| Total number of processes | ||
|
|
||
| | Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic | | ||
| | ---- | ----------- | ---------- | ----------------------- | --------- | | ||
| | count | Sum | Int | Cumulative | false | | ||
|
|
||
| ### process.cpu.percent | ||
|
|
||
| Total CPU percent used by the process | ||
|
|
||
| | Unit | Metric Type | Value Type | | ||
| | ---- | ----------- | ---------- | | ||
| | % | Gauge | Double | | ||
|
|
||
| #### Attributes | ||
|
|
||
| | Name | Description | Values | | ||
| | ---- | ----------- | ------ | | ||
| | state | Breakdown of CPU usage by type. | Str: ``system``, ``user``, ``wait``, ``total`` | | ||
|
|
||
| ### process.memory.percent | ||
|
|
||
| Total memory percent used by the process | ||
|
|
||
| | Unit | Metric Type | Value Type | | ||
| | ---- | ----------- | ---------- | | ||
| | % | Gauge | Double | | ||
|
|
||
| ### process.open_file_descriptors | ||
|
|
||
| Total number of open file descriptors | ||
|
|
||
| | Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic | | ||
| | ---- | ----------- | ---------- | ----------------------- | --------- | | ||
| | count | Sum | Int | Cumulative | false | | ||
|
|
||
| ### process.threads | ||
|
|
||
| Total number of threads | ||
|
|
||
| | Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic | | ||
| | ---- | ----------- | ---------- | ----------------------- | --------- | | ||
| | count | Sum | Int | Cumulative | false | | ||
|
|
||
| ## Resource Attributes | ||
|
|
||
| | Name | Description | Values | Enabled | | ||
| | ---- | ----------- | ------ | ------- | | ||
| | process.name | Name of the process | Any Str | true | |
74 changes: 74 additions & 0 deletions
74
receiver/hostmetricsreceiver/internal/scraper/groupprocessscraper/factory.go
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,74 @@ | ||
| // Copyright The OpenTelemetry Authors | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| package groupprocessscraper // import "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/groupprocessscraper" | ||
|
|
||
| import ( | ||
| "context" | ||
| "errors" | ||
| "runtime" | ||
|
|
||
| "go.opentelemetry.io/collector/component" | ||
| "go.opentelemetry.io/collector/featuregate" | ||
| "go.opentelemetry.io/collector/receiver" | ||
| "go.opentelemetry.io/collector/receiver/scraperhelper" | ||
|
|
||
| "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal" | ||
| "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/groupprocessscraper/internal/metadata" | ||
| ) | ||
|
|
||
| // This file implements Factory for Group Process scraper. | ||
|
|
||
| const ( | ||
| // TypeStr the value of "type" key in configuration. | ||
| TypeStr = "groupprocess" | ||
| ) | ||
|
|
||
| var ( | ||
| // scraperType is the component type used for the built scraper. | ||
| scraperType component.Type = component.MustNewType(TypeStr) | ||
| ) | ||
|
|
||
| var ( | ||
| bootTimeCacheFeaturegateID = "hostmetrics.groupprocess.bootTimeCache" | ||
| bootTimeCacheFeaturegate = featuregate.GlobalRegistry().MustRegister( | ||
| bootTimeCacheFeaturegateID, | ||
| featuregate.StageBeta, | ||
| featuregate.WithRegisterDescription("When enabled, all groupprocess scrapes will use the boot time value that is cached at the start of the process."), | ||
| featuregate.WithRegisterReferenceURL("https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/28849"), | ||
| featuregate.WithRegisterFromVersion("v0.98.0"), | ||
| ) | ||
| ) | ||
|
|
||
| // Factory is the Factory for scraper. | ||
| type Factory struct { | ||
| } | ||
|
|
||
| // CreateDefaultConfig creates the default configuration for the Scraper. | ||
| func (f *Factory) CreateDefaultConfig() internal.Config { | ||
| return &Config{ | ||
| MetricsBuilderConfig: metadata.DefaultMetricsBuilderConfig(), | ||
| } | ||
| } | ||
|
|
||
| // CreateMetricsScraper creates a resource scraper based on provided config. | ||
| func (f *Factory) CreateMetricsScraper( | ||
| _ context.Context, | ||
| settings receiver.Settings, | ||
| cfg internal.Config, | ||
| ) (scraperhelper.Scraper, error) { | ||
| if runtime.GOOS != "linux" && runtime.GOOS != "windows" && runtime.GOOS != "darwin" { | ||
| return nil, errors.New("groupprocess scraper only available on Linux, Windows, or MacOS") | ||
| } | ||
|
|
||
| s, err := newGroupProcessScraper(settings, cfg.(*Config)) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
|
|
||
| return scraperhelper.NewScraper( | ||
| scraperType, | ||
| s.scrape, | ||
| scraperhelper.WithStart(s.start), | ||
| ) | ||
| } |
35 changes: 35 additions & 0 deletions
35
receiver/hostmetricsreceiver/internal/scraper/groupprocessscraper/factory_test.go
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| // Copyright The OpenTelemetry Authors | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| package groupprocessscraper | ||
|
|
||
| import ( | ||
| "context" | ||
| "runtime" | ||
| "testing" | ||
|
|
||
| "github.com/stretchr/testify/assert" | ||
| "go.opentelemetry.io/collector/receiver/receivertest" | ||
| ) | ||
|
|
||
| func TestCreateDefaultConfig(t *testing.T) { | ||
| factory := &Factory{} | ||
| cfg := factory.CreateDefaultConfig() | ||
| assert.IsType(t, &Config{}, cfg) | ||
| } | ||
|
|
||
| func TestCreateResourceMetricsScraper(t *testing.T) { | ||
| factory := &Factory{} | ||
| cfg := &Config{} | ||
|
|
||
| scraper, err := factory.CreateMetricsScraper(context.Background(), receivertest.NewNopSettings(), cfg) | ||
|
|
||
| if runtime.GOOS == "linux" || runtime.GOOS == "windows" || runtime.GOOS == "darwin" { | ||
| assert.NoError(t, err) | ||
| assert.NotNil(t, scraper) | ||
| assert.Equal(t, scraperType.String(), scraper.ID().String()) | ||
| } else { | ||
| assert.Error(t, err) | ||
| assert.Nil(t, scraper) | ||
| } | ||
| } |
9 changes: 9 additions & 0 deletions
9
.../hostmetricsreceiver/internal/scraper/groupprocessscraper/internal/handlecount/handles.go
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| // Copyright The OpenTelemetry Authors | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| package handlecount // import "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/groupprocessscraper/internal/handlecount" | ||
|
|
||
| type Manager interface { | ||
| Refresh() error | ||
| GetProcessHandleCount(pid int64) (uint32, error) | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, add a line.
If more than one out of(comm, exe and cmdline) is given all of them has to match.
if a process matches to one group, the same process will not be part of other groups.