Merge from main #2
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
**Description:** Increase the timeout of the "Cache Go" step in the `build-and-test-windows` workflow. I had a few failures with that today and glancing at the errors for the workflow I can see a few others. Few instances below: * https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/6722644168/job/18271035294#step:5:22 * https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/6725656509/job/18280490403#step:5:23 * https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/6726302253/job/18282301386#step:5:21
… names and attempt to fix flaky tests (#28903) **Description:** <Describe what has changed.> Using unique container networks and container names and attempt to fix flaky tests **Link to tracking Issue:** #26293 **Testing:** **Preparation:** DIR = receiver/kafkametricsreceiver CMD = go test -v -count=1 -race -timeout 360s -parallel 4 -tags=integration,"" -run=Integration ./... **Tests:** 1. If we manually modify the code(as shown below) and use invalid kafka broker, such as `localhost:invalid-port`, the same error as shown in the issue may occur. ``` // receiver/kafkametricsreceiver/integration_test.go scraperinttest.WithCustomConfig( func(t *testing.T, cfg component.Config, ci *scraperinttest.ContainerInfo) { rCfg := cfg.(*Config) rCfg.CollectionInterval = 5 * time.Second rCfg.Brokers = []string{"localhost:invalid-port"} rCfg.Scrapers = []string{"brokers", "consumers", "topics"} }), ``` 2. If we execute the test commands **sequentially** , it seems that the execution results are all correct. ``` # all result are correct for i in {1..100}; do echo "Run $i"; ./${CMD} ; done ``` 3. If we execute the commands in **parallel** end with **`&`**, sometimes the error shown in the issue may occur. ``` # sometimes result occur error for i in {1..20}; do echo "Run $i"; ./${CMD} &; done ``` **Inference:** I have found that duplicate container networks and container names can cause container creation to fail or result in successfully created containers with unavailable ports, which may lead to issues similar to the one shown. **Additional information:** Since Kafka's startup relies on ZooKeeper (which waits for the default `zookeeper.connection.timeout.ms=18000`), if Kafka starts first and ZooKeeper fails to start properly after the timeout duration, it will cause the Kafka container to fail to start correctly. I found the issue testcontainers/testcontainers-go#1791 wants to support that. **Documentation:** --------- Signed-off-by: sakulali <sakulali@126.com>
…in tests (flay test on Windows runners) (#29014) **Description:** Limit number of goroutines started during `processor/tailsamplingprocessor` tests. This causes very frequently failures on the Windows tests, see [here](#28682 (comment)) for example. The issue is that the race detector has a hard limit on number of goroutines, see golang/go#23611. The fix limits the concurrency in two tests so this limit is not hit on GH Windows runners. **Link to tracking Issue:** Fix #9126 **Testing:** Increased the concurrency on the two changed tests caused the error and validated that it passed twice on my fork. **Documentation:** N/A
Description: Added a new config item to support the QueueSettings values. Extended the exportHelper.New[Metrics|Logs|Traces]Exporter call to pass in the QueueSettings config, thus enabling persistent_queue for this exporter. Link to tracking Issue: Fixes issue #25859 Testing: Extending unit tests to check configuration changes are picked up. Documentation: Added sending_queue config items to README.md's configuration section.
Updated to match core
Promote collectdreceiver as beta component Fixes #28658
Bump github.com/DataDog/datadog-agent/pkg/proto from 0.49.0-rc.2 to 0.50.0-devel in /exporter/datadogexporter Bump github.com/IBM/sarama from 1.41.3 to 1.42.0 in /exporter/kafkaexporter Bump github.com/IBM/sarama from 1.41.3 to 1.42.0 in /receiver/kafkareceiver Bump github.com/IBM/sarama from 1.41.3 to 1.42.1 in /receiver/kafkametricsreceiver Bump github.com/aws/aws-sdk-go from 1.46.7 to 1.47.3 in /exporter/awscloudwatchlogsexporter Bump github.com/aws/aws-sdk-go from 1.46.7 to 1.47.3 in /exporter/awsemfexporter Bump github.com/aws/aws-sdk-go from 1.46.7 to 1.47.3 in /exporter/awsxrayexporter Bump github.com/aws/aws-sdk-go from 1.46.7 to 1.47.3 in /exporter/datadogexporter Bump github.com/aws/aws-sdk-go from 1.46.7 to 1.47.3 in /extension/observer/ecsobserver Bump github.com/aws/aws-sdk-go from 1.46.7 to 1.47.3 in /internal/aws/awsutil Bump github.com/aws/aws-sdk-go from 1.46.7 to 1.47.3 in /internal/aws/cwlogs Bump github.com/aws/aws-sdk-go from 1.46.7 to 1.47.3 in /internal/aws/k8s Bump github.com/aws/aws-sdk-go from 1.46.7 to 1.47.3 in /internal/aws/proxy Bump github.com/aws/aws-sdk-go from 1.46.7 to 1.47.3 in /internal/aws/xray Bump github.com/aws/aws-sdk-go from 1.46.7 to 1.47.3 in /internal/aws/xray/testdata/sampleapp Bump github.com/aws/aws-sdk-go from 1.46.7 to 1.47.3 in /internal/metadataproviders Bump github.com/aws/aws-sdk-go from 1.46.7 to 1.47.3 in /processor/resourcedetectionprocessor Bump github.com/aws/aws-sdk-go from 1.46.7 to 1.47.3 in /receiver/awsecscontainermetricsreceiver Bump github.com/aws/aws-sdk-go from 1.46.7 to 1.47.3 in /receiver/awsxrayreceiver Bump github.com/aws/aws-sdk-go from 1.46.7 to 1.47.4 in /receiver/awscontainerinsightreceiver Bump github.com/aws/aws-sdk-go-v2 from 1.21.2 to 1.22.1 in /exporter/awskinesisexporter Bump github.com/aws/aws-sdk-go-v2 from 1.21.2 to 1.22.1 in /extension/sigv4authextension Bump github.com/aws/aws-sdk-go-v2/config from 1.19.1 to 1.22.0 in /exporter/awskinesisexporter Bump github.com/aws/aws-sdk-go-v2/config from 1.19.1 to 1.22.0 in /extension/sigv4authextension Bump github.com/aws/aws-sdk-go-v2/credentials from 1.13.43 to 1.15.1 in /exporter/awskinesisexporter Bump github.com/aws/aws-sdk-go-v2/credentials from 1.13.43 to 1.15.1 in /extension/sigv4authextension Bump github.com/aws/aws-sdk-go-v2/service/kinesis from 1.20.0 to 1.22.0 in /exporter/awskinesisexporter Bump github.com/aws/aws-sdk-go-v2/service/sts from 1.23.2 to 1.25.0 in /exporter/awskinesisexporter Bump github.com/aws/aws-sdk-go-v2/service/sts from 1.23.2 to 1.25.0 in /extension/sigv4authextension Bump github.com/golangci/golangci-lint from 1.55.1 to 1.55.2 in /internal/tools Bump github.com/gorilla/mux from 1.8.0 to 1.8.1 in /receiver/jaegerreceiver Bump github.com/gorilla/mux from 1.8.0 to 1.8.1 in /receiver/sapmreceiver Bump github.com/gorilla/mux from 1.8.0 to 1.8.1 in /receiver/signalfxreceiver Bump github.com/gorilla/mux from 1.8.0 to 1.8.1 in /receiver/skywalkingreceiver Bump github.com/gorilla/mux from 1.8.0 to 1.8.1 in /receiver/splunkhecreceiver Bump github.com/gorilla/mux from 1.8.0 to 1.8.1 in /testbed/mockdatareceivers/mockawsxrayreceiver Bump github.com/influxdata/influxdb-client-go/v2 from 2.12.3 to 2.12.4 in /receiver/influxdbreceiver Bump github.com/mattn/go-sqlite3 from 1.14.17 to 1.14.18 in /extension/storage Bump github.com/prometheus/procfs from 0.11.1 to 0.12.0 in /receiver/hostmetricsreceiver Bump github.com/shirou/gopsutil/v3 from 3.23.9 to 3.23.10 in /exporter/signalfxexporter Bump github.com/shirou/gopsutil/v3 from 3.23.9 to 3.23.10 in /extension/observer/hostobserver Bump github.com/shirou/gopsutil/v3 from 3.23.9 to 3.23.10 in /processor/resourcedetectionprocessor Bump github.com/shirou/gopsutil/v3 from 3.23.9 to 3.23.10 in /receiver/awscontainerinsightreceiver Bump github.com/shirou/gopsutil/v3 from 3.23.9 to 3.23.10 in /receiver/hostmetricsreceiver Bump github.com/shirou/gopsutil/v3 from 3.23.9 to 3.23.10 in /receiver/jmxreceiver Bump github.com/shirou/gopsutil/v3 from 3.23.9 to 3.23.10 in /testbed Bump github.com/spf13/cobra from 1.7.0 to 1.8.0 in /cmd/telemetrygen Bump github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common from 1.0.777 to 1.0.782 in /exporter/tencentcloudlogserviceexporter Bump go.mongodb.org/atlas from 0.34.0 to 0.35.0 in /receiver/mongodbatlasreceiver Bump golang.org/x/mod from 0.13.0 to 0.14.0 in /cmd/configschema Bump golang.org/x/sys from 0.13.0 to 0.14.0 in /exporter/signalfxexporter Bump golang.org/x/sys from 0.13.0 to 0.14.0 in /pkg/stanza Bump golang.org/x/sys from 0.13.0 to 0.14.0 in /pkg/winperfcounters Bump golang.org/x/sys from 0.13.0 to 0.14.0 in /receiver/hostmetricsreceiver Bump golang.org/x/sys from 0.13.0 to 0.14.0 in /receiver/windowseventlogreceiver Bump golang.org/x/text from 0.13.0 to 0.14.0 in /cmd/configschema Bump golang.org/x/text from 0.13.0 to 0.14.0 in /cmd/mdatagen Bump golang.org/x/text from 0.13.0 to 0.14.0 in /internal/coreinternal Bump golang.org/x/text from 0.13.0 to 0.14.0 in /pkg/stanza Bump golang.org/x/text from 0.13.0 to 0.14.0 in /testbed Bump golang.org/x/time from 0.3.0 to 0.4.0 in /cmd/telemetrygen Bump google.golang.org/api from 0.148.0 to 0.149.0 in /exporter/f5cloudexporter Bump google.golang.org/api from 0.148.0 to 0.149.0 in /exporter/googlecloudpubsubexporter Bump google.golang.org/api from 0.148.0 to 0.149.0 in /receiver/googlecloudpubsubreceiver Bump google.golang.org/api from 0.148.0 to 0.149.0 in /receiver/googlecloudspannerreceiver
Bump github.com/aws/aws-sdk-go from 1.47.4 to 1.47.5 in /exporter/awscloudwatchlogsexporter Bump github.com/aws/aws-sdk-go from 1.47.4 to 1.47.5 in /exporter/awsemfexporter Bump github.com/aws/aws-sdk-go from 1.47.4 to 1.47.5 in /exporter/awsxrayexporter Bump github.com/aws/aws-sdk-go from 1.47.4 to 1.47.5 in /exporter/datadogexporter Bump github.com/aws/aws-sdk-go from 1.47.4 to 1.47.5 in /extension/observer/ecsobserver Bump github.com/aws/aws-sdk-go from 1.47.4 to 1.47.5 in /internal/aws/awsutil Bump github.com/aws/aws-sdk-go from 1.47.4 to 1.47.5 in /internal/aws/cwlogs Bump github.com/aws/aws-sdk-go from 1.47.4 to 1.47.5 in /internal/aws/k8s Bump github.com/aws/aws-sdk-go from 1.47.4 to 1.47.5 in /internal/aws/proxy Bump github.com/aws/aws-sdk-go from 1.47.4 to 1.47.5 in /internal/aws/xray Bump github.com/aws/aws-sdk-go from 1.47.4 to 1.47.5 in /internal/aws/xray/testdata/sampleapp Bump github.com/aws/aws-sdk-go from 1.47.4 to 1.47.5 in /internal/metadataproviders Bump github.com/aws/aws-sdk-go from 1.47.4 to 1.47.5 in /processor/resourcedetectionprocessor Bump github.com/aws/aws-sdk-go from 1.47.4 to 1.47.5 in /receiver/awscontainerinsightreceiver Bump github.com/aws/aws-sdk-go from 1.47.4 to 1.47.5 in /receiver/awsecscontainermetricsreceiver Bump github.com/aws/aws-sdk-go from 1.47.4 to 1.47.5 in /receiver/awsxrayreceiver Bump github.com/aws/aws-sdk-go-v2/config from 1.22.0 to 1.22.2 in /exporter/awskinesisexporter Bump github.com/aws/aws-sdk-go-v2/config from 1.22.0 to 1.22.2 in /extension/sigv4authextension Bump github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common from 1.0.782 to 1.0.786 in /exporter/tencentcloudlogserviceexporter Bump google.golang.org/api from 0.149.0 to 0.150.0 in /exporter/f5cloudexporter Bump google.golang.org/api from 0.149.0 to 0.150.0 in /exporter/googlecloudpubsubexporter Bump google.golang.org/api from 0.149.0 to 0.150.0 in /receiver/googlecloudpubsubreceiver Bump google.golang.org/api from 0.149.0 to 0.150.0 in /receiver/googlecloudspannerreceiver
Bump github.com/aws/aws-sdk-go from 1.47.5 to 1.47.6 in /exporter/awscloudwatchlogsexporter Bump github.com/aws/aws-sdk-go from 1.47.5 to 1.47.6 in /exporter/awsemfexporter Bump github.com/aws/aws-sdk-go from 1.47.5 to 1.47.6 in /exporter/awsxrayexporter Bump github.com/aws/aws-sdk-go from 1.47.5 to 1.47.6 in /exporter/datadogexporter Bump github.com/aws/aws-sdk-go from 1.47.5 to 1.47.6 in /extension/observer/ecsobserver Bump github.com/aws/aws-sdk-go from 1.47.5 to 1.47.6 in /internal/aws/awsutil Bump github.com/aws/aws-sdk-go from 1.47.5 to 1.47.6 in /internal/aws/cwlogs Bump github.com/aws/aws-sdk-go from 1.47.5 to 1.47.6 in /internal/aws/k8s Bump github.com/aws/aws-sdk-go from 1.47.5 to 1.47.6 in /internal/aws/proxy Bump github.com/aws/aws-sdk-go from 1.47.5 to 1.47.6 in /internal/aws/xray Bump github.com/aws/aws-sdk-go from 1.47.5 to 1.47.6 in /internal/aws/xray/testdata/sampleapp Bump github.com/aws/aws-sdk-go from 1.47.5 to 1.47.6 in /internal/metadataproviders Bump github.com/aws/aws-sdk-go from 1.47.5 to 1.47.6 in /processor/resourcedetectionprocessor Bump github.com/aws/aws-sdk-go from 1.47.5 to 1.47.6 in /receiver/awscontainerinsightreceiver Bump github.com/aws/aws-sdk-go from 1.47.5 to 1.47.6 in /receiver/awsecscontainermetricsreceiver Bump github.com/aws/aws-sdk-go from 1.47.5 to 1.47.6 in /receiver/awsxrayreceiver
I fixed linter issue by following this document. https://google.github.io/styleguide/go/decisions.html#indent-error-flow
**Description:** `gopsutil` recently added the capability to pass environment vars through context. This is now done everywhere. This environment variable setting function is no longer used or necessary. This PR removes it. **Link to tracking Issue:** #23055 Signed-off-by: Braydon Kains <braydonk@google.com>
This fixes security vulnerabilities found via govulncheck in the standard library when running against the previous patch versions of golang. While these vulnerabilities don't actually present themselves in the binary, the workflows when running govuln check fail and thus taking in the latest patches fix the issue. Testing gets caught in workflow run. Noticed the issue originally when running workflows on this pr: #28885
Additionally added a golangci-lint.yaml update to automatically apply this change to new code going forward Fixes #23811 --------- Co-authored-by: Alex Boten <aboten@lightstep.com>
) **Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> This feature adds provider resource attributes `mongodb_atlas.provider.name` and `mongodb_atlas.region.name` to add additional context and filtering capabilities. **Link to tracking Issue:** <Issue number if applicable> #28833 **Testing:** <Describe what testing was performed and which tests were added.> Test were automatically updated. Live testing was performed and validated on clusters. **Documentation:** <Describe the documentation added.> Docs were automatically updated.
**Description:** Promote syslogexporter to alpha and add it to otelcontribcol **Link to tracking Issue:** related to: #21242, #21244, #21245 **Testing:** <Describe what testing was performed and which tests were added.> Manual tests: Configuration: ```yaml exporters: syslog: network: tcp port: 514 endpoint: 127.0.0.1 protocol: rfc5424 receivers: filelog: start_at: beginning include: - /Users/kkujawa/git/opentelemetry-collector-contrib/test.txt operators: - type: syslog_parser protocol: rfc5424 service: pipelines: logs: receivers: - filelog exporters: - syslog ``` Logs: ``` ./bin/otelcontribcol_darwin_amd64 --config /Users/kkujawa/git/opentelemetry-collector-contrib/bin/config.yaml 2023-11-06T12:59:31.656+0100 info service@v0.88.1-0.20231026220224-6405e152a2d9/telemetry.go:84 Setting up own telemetry... 2023-11-06T12:59:31.656+0100 info service@v0.88.1-0.20231026220224-6405e152a2d9/telemetry.go:201 Serving Prometheus metrics {"address": ":8888", "level": "Basic"} 2023-11-06T12:59:31.656+0100 info exporter@v0.88.1-0.20231026220224-6405e152a2d9/exporter.go:275 Development component. May change in the future. {"kind": "exporter", "data_type": "logs", "name": "syslog"} 2023-11-06T12:59:31.656+0100 info syslogexporter@v0.88.0/exporter.go:42 Syslog Exporter configured {"kind": "exporter", "data_type": "logs", "name": "syslog", "endpoint": "127.0.0.1", "Protocol": "rfc5424", "port": 514} 2023-11-06T12:59:31.657+0100 info service@v0.88.1-0.20231026220224-6405e152a2d9/service.go:143 Starting otelcontribcol... {"Version": "0.88.0-dev", "NumCPU": 16} 2023-11-06T12:59:31.657+0100 info extensions/extensions.go:33 Starting extensions... 2023-11-06T12:59:31.657+0100 info adapter/receiver.go:45 Starting stanza receiver {"kind": "receiver", "name": "filelog", "data_type": "logs"} 2023-11-06T12:59:31.657+0100 info service@v0.88.1-0.20231026220224-6405e152a2d9/service.go:169 Everything is ready. Begin running and processing data. 2023-11-06T12:59:31.858+0100 info fileconsumer/file.go:263 Started watching file {"kind": "receiver", "name": "filelog", "data_type": "logs", "component": "fileconsumer", "path": "/Users/kkujawa/git/opentelemetry-collector-contrib/test.txt"} ```
…28889) **Description:** Fix a bug when duplicate readers are added to the active list even after the underlying file is closed. To fix this, continue from the outer loop. This doesn't result in any duplicates, but this will keep producing the following annoying error every time. ```2023-11-05T02:34:03.530+0530 ERROR Failed to seek {"component": "fileconsumer", "path": "/var/folders/fs/njj5c3xx7vdcsr28n19vykw00000gn/T/TestStalePartialFingerprintDiscarded2443925830/001/1616317274.log2", "error": "seek /var/folders/fs/njj5c3xx7vdcsr28n19vykw00000gn/T/TestStalePartialFingerprintDiscarded2443925830/001/1616317274.log2: file already closed"}``` **Testing:** Update the test to check the previouPollFiles
**Description:** Fixing a bug in udp async mode only (but didn't affect the default non-async mode). Udp-receiver reuses the same buffer when each packet is processed. While that's working fine when running it without async config, it cause a significant amount of duplicate packets and corrupted packets being sent downstream. The reader-async thread is reading a packet from the udp port into the buffer, places that buffer in the channel, and reads another packet into the same buffer and pushes it to the channel. Let's say that the processor-async thread was a bit slow, so it only tries to read from the channel after the 2 items were placed in the channel. In that case, the processor thread will read 2 items from the channel, but it will be the same 2nd packet (since the 1st one was overwritten). In some cases, it seems the processor is reading a corrupted buffer (since the reader is currently writing into it). We can't fix it by having the reader allocate a new buffer before each time it reads a packet from the udp port, since that hurts performance significantly (reducing it up to ~50%). Instead, use a pool so the buffers are reused. Before reading a packet, the reader get a buffer from the pool. The processor returns it back to the pool after it has been successfully processed **Link to tracking Issue:** 27613 **Testing:** Ran existing unitests. Ran ran stress tests (sending 250k udp packets per second) duplicate/corruption issue didn't happen; performance wasn't hurt. **Documentation:** None
…d skip flaky tests (#29101) See https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/28898/files#r1389614720. Looks like a merge conflict.
The `needs triage` label is directly related to how we define triaging. Added a link to the triaging definition to make the label's usage more clear. (Even though the triaging process paragraph is just above this table in the document, it's easy to miss).
**Description:** Fixes misleading documentation about which RBAC role is required and other invalid YAML I found along the way
**Description:** typo for kubernetes label in k8sattributesprocessor **Link to tracking Issue:** n/a **Testing:** n/a docs **Documentation:** n/a
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description:
Link to tracking Issue:
Testing:
Documentation: