[receiver/file] add file receiver skeleton 1/3 #16827

pmcollins · 2022-12-08T22:09:31Z

Description:
This change adds a skeleton, the first PR, for the File Receiver.

Link to tracking Issue: #14638

Testing: Just a few units tests added since no functionality has been added yet.

Documentation: I have added a small amount of documentation to the README. More content will be added as features are submitted for review.

runforesight · 2022-12-08T22:11:29Z

Foresight Summary

Major Impacts

build-and-test-windows duration(4 seconds) has decreased 40 minutes 37 seconds compared to main branch avg(40 minutes 41 seconds).

View More Details

✅ tracegen workflow has finished in 1 minute 58 seconds and finished at 26th Jan, 2023.

Job	Failed Steps	Tests
build-dev	- 🔗	N/A	See Details
publish-stable	- 🔗	N/A	See Details
publish-latest	- 🔗	N/A	See Details

✅ telemetrygen workflow has finished in 2 minutes 5 seconds and finished at 26th Jan, 2023.

Job	Failed Steps	Tests
build-dev	- 🔗	N/A	See Details
publish-latest	- 🔗	N/A	See Details
publish-stable	- 🔗	N/A	See Details

✅ check-links workflow has finished in 2 minutes 40 seconds (⚠️ 50 seconds more than `main` branch avg.) and finished at 26th Jan, 2023.

Job	Failed Steps	Tests
changed files	- 🔗	N/A	See Details
check-links	- 🔗	N/A	See Details

✅ prometheus-compliance-tests workflow has finished in 3 minutes 52 seconds (3 minutes 38 seconds less than `main` branch avg.) and finished at 26th Jan, 2023.

Job	Failed Steps	Tests
prometheus-compliance-tests	- 🔗	✅ 21 ❌ 0 ⏭ 0 🔗	See Details

✅ build-and-test workflow has finished in 38 minutes 53 seconds (13 minutes 9 seconds less than `main` branch avg.) and finished at 26th Jan, 2023.

Job	Failed Steps	Tests
unittest-matrix (1.18, internal)	- 🔗	✅ 561 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.19, internal)	- 🔗	✅ 561 ❌ 0 ⏭ 0 🔗	See Details
correctness-metrics	- 🔗	✅ 2 ❌ 0 ⏭ 0 🔗	See Details
correctness-traces	- 🔗	✅ 17 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.18, extension)	- 🔗	✅ 547 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.19, extension)	- 🔗	✅ 547 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.18, processor)	- 🔗	✅ 1471 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.19, processor)	- 🔗	✅ 1471 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.19, receiver-0)	- 🔗	✅ 2573 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.18, receiver-0)	- 🔗	✅ 2573 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.18, exporter)	- 🔗	✅ 2416 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.19, exporter)	- 🔗	✅ 2416 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.18, receiver-1)	- 🔗	✅ 1944 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.18, other)	- 🔗	✅ 4659 ❌ 0 ⏭ 0 🔗	See Details
integration-tests	- 🔗	✅ 55 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.19, other)	- 🔗	✅ 4659 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.19, receiver-1)	- 🔗	✅ 1944 ❌ 0 ⏭ 0 🔗	See Details
setup-environment	- 🔗	N/A	See Details
checks	- 🔗	N/A	See Details
lint-matrix (receiver-0)	- 🔗	N/A	See Details
lint-matrix (receiver-1)	- 🔗	N/A	See Details
lint-matrix (processor)	- 🔗	N/A	See Details
lint-matrix (exporter)	- 🔗	N/A	See Details
lint-matrix (extension)	- 🔗	N/A	See Details
lint-matrix (internal)	- 🔗	N/A	See Details
lint-matrix (other)	- 🔗	N/A	See Details
build-examples	- 🔗	N/A	See Details
check-codeowners	- 🔗	N/A	See Details
check-collector-module-version	- 🔗	N/A	See Details
lint	- 🔗	N/A	See Details
unittest (1.19)	- 🔗	N/A	See Details
unittest (1.18)	- 🔗	N/A	See Details
cross-compile (darwin, amd64)	- 🔗	N/A	See Details
cross-compile (darwin, arm64)	- 🔗	N/A	See Details
cross-compile (linux, 386)	- 🔗	N/A	See Details
cross-compile (linux, amd64)	- 🔗	N/A	See Details
cross-compile (linux, arm)	- 🔗	N/A	See Details
cross-compile (linux, arm64)	- 🔗	N/A	See Details
cross-compile (linux, ppc64le)	- 🔗	N/A	See Details
cross-compile (windows, 386)	- 🔗	N/A	See Details
cross-compile (windows, amd64)	- 🔗	N/A	See Details
build-package (deb)	- 🔗	N/A	See Details
build-package (rpm)	- 🔗	N/A	See Details
windows-msi	- 🔗	N/A	See Details
publish-check	- 🔗	N/A	See Details
publish-dev	- 🔗	N/A	See Details
publish-stable	- 🔗	N/A	See Details

✅ load-tests workflow has finished in 13 minutes 13 seconds and finished at 26th Jan, 2023.

Job	Failed Steps	Tests
loadtest (TestIdleMode)	- 🔗	✅ 1 ❌ 0 ⏭ 0 🔗	See Details
loadtest (TestTraceAttributesProcessor)	- 🔗	✅ 3 ❌ 0 ⏭ 0 🔗	See Details
loadtest (TestMetric10kDPS\|TestMetricsFromFile)	- 🔗	✅ 6 ❌ 0 ⏭ 0 🔗	See Details
loadtest (TestMetricResourceProcessor\|TestTrace10kSPS)	- 🔗	✅ 12 ❌ 0 ⏭ 0 🔗	See Details
loadtest (TestTraceNoBackend10kSPS\|TestTrace1kSPSWithAttrs)	- 🔗	✅ 8 ❌ 0 ⏭ 0 🔗	See Details
loadtest (TestTraceBallast1kSPSWithAttrs\|TestTraceBallast1kSPSAddAttrs)	- 🔗	✅ 10 ❌ 0 ⏭ 0 🔗	See Details
loadtest (TestBallastMemory\|TestLog10kDPS)	- 🔗	✅ 19 ❌ 0 ⏭ 0 🔗	See Details
setup-environment	- 🔗	N/A	See Details

⭕ build-and-test-windows workflow has finished in 3 seconds (40 minutes 38 seconds less than `main` branch avg.) and finished at 26th Jan, 2023.

Job	Failed Steps	Tests
windows-unittest-matrix	- 🔗	N/A	See Details
windows-unittest	- 🔗	N/A	See Details

✅ changelog workflow has finished in 1 minute 31 seconds and finished at 26th Jan, 2023.

Job	Failed Steps	Tests
changelog	- 🔗	N/A	See Details

🔎 See details on Foresight

^{*You can configure Foresight comments in your organization settings page.}

atoulme

LGTM!

MovieStoreGuy · 2022-12-28T03:31:00Z

receiver/filereceiver/README.md

+| Status                   |               |
+|--------------------------|---------------|
+| Stability                | [development] |
+| Supported pipeline types | metrics       |


Considering that the exporter emits logs, metrics, and traces, should this also support these pipelines?

Hi @MovieStoreGuy, thanks for your feedback. I was thinking that although eventually we'll want the file receiver to be able to handle all three, for this initial version, it would just support metrics, and the other pipelines would be added later. Do you think we should indicate the additional pipelines as supported now?

It seems worthwhile to include support for all types from the start, since there shouldn't be much additional work.

MovieStoreGuy

Just a couple thing to follow up.

MovieStoreGuy · 2022-12-28T03:34:21Z

receiver/filereceiver/README.md

+
+The following settings are required:
+
+- `path` [no default]: the file containing the metrics written by a File Exporter.


What should happen if the path doesn't exist or the file doesn't exist? Is it expected for the path to exist?

If the file doesn't exist there will be nothing for this receiver to do, since that file is the only input source for this receiver. I'm thinking that if the file doesn't exist, the receiver should both fail to start and produce an informative error.

Since the exporter supports rotation, I suggest that the receiver should also. Would it be possible to use the pkg/stanza/fileconsumer package?

I don't mind the course of action provided the behaviour is clear to the user :)

MovieStoreGuy · 2022-12-28T03:35:21Z

receiver/filereceiver/go.mod

+
+require (
+	github.com/stretchr/testify v1.8.1
+	go.opentelemetry.io/collector v0.67.1-0.20221216200611-892d07dffbb8


I believe there is a more recent version that this will need to be updated to.

Will update, thank you.

MovieStoreGuy · 2022-12-28T03:36:45Z

receiver/filereceiver/receiver.go

+	"go.opentelemetry.io/collector/component"
+)
+
+type fileReceiver struct{}


If you'd like the reduce the lines of code you need to write, you can rerwrite this as:

Suggested change

type fileReceiver struct{}

type fileReceiver struct {

component.StartFunc

component.ShutdownFunc

}

Interesting, I wasn't familiar with that. Updated.

MovieStoreGuy · 2023-01-08T05:24:48Z

Happy new year! I hope you managed to enjoy some holidays.

Once you get a moment, could I ask you to rebase with main please?

github-actions · 2023-01-26T05:17:32Z

This PR was marked stale due to lack of activity. It will be closed in 14 days.

MovieStoreGuy

If you can just update the go mod and rebase against main against, happy to get this in :)

MovieStoreGuy · 2023-01-26T11:16:52Z

receiver/filereceiver/go.mod

+	go.opentelemetry.io/collector/component v0.69.0
+	go.opentelemetry.io/collector/confmap v0.69.0
+	go.opentelemetry.io/collector/consumer v0.69.0


Suggested change

go.opentelemetry.io/collector/component v0.69.0

go.opentelemetry.io/collector/confmap v0.69.0

go.opentelemetry.io/collector/consumer v0.69.0

go.opentelemetry.io/collector/component v0.70.0

go.opentelemetry.io/collector/confmap v0.70.0

go.opentelemetry.io/collector/consumer v0.70.0

You'll need to run make tidy after this.

MovieStoreGuy

I'm happy for this to go in like so and any comments be followed in the next PRs.

djaglowski

I'm fine with my comments being addressed in subsequent PRs.

pmcollins marked this pull request as ready for review December 19, 2022 23:15

pmcollins requested review from a team and bogdandrutu December 19, 2022 23:15

github-actions bot assigned MovieStoreGuy Dec 19, 2022

atoulme approved these changes Dec 23, 2022

View reviewed changes

MovieStoreGuy reviewed Dec 28, 2022

View reviewed changes

github-actions bot added the Stale label Jan 26, 2023

MovieStoreGuy reviewed Jan 26, 2023

View reviewed changes

pmcollins added 2 commits January 26, 2023 17:16

add file receiver skeleton

599bf91

update readme

65c0731

MovieStoreGuy approved these changes Jan 26, 2023

View reviewed changes

MovieStoreGuy added ready to merge Code review completed; ready to merge by maintainers and removed Stale labels Jan 26, 2023

djaglowski approved these changes Jan 27, 2023

View reviewed changes

djaglowski merged commit e2dfaa5 into open-telemetry:main Jan 27, 2023

crobert-1 mentioned this pull request Oct 16, 2023

[receiver/file] logs and traces are not supported #27682

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[receiver/file] add file receiver skeleton 1/3 #16827

[receiver/file] add file receiver skeleton 1/3 #16827

pmcollins commented Dec 8, 2022

runforesight bot commented Dec 8, 2022 •

edited

Loading

`build-and-test-windows` duration(4 seconds) has decreased 40 minutes 37 seconds compared to main branch avg(40 minutes 41 seconds).

✅ tracegen workflow has finished in 1 minute 58 seconds and finished at 26th Jan, 2023.

✅ telemetrygen workflow has finished in 2 minutes 5 seconds and finished at 26th Jan, 2023.

✅ check-links workflow has finished in 2 minutes 40 seconds (⚠️ 50 seconds more than `main` branch avg.) and finished at 26th Jan, 2023.

✅ prometheus-compliance-tests workflow has finished in 3 minutes 52 seconds (3 minutes 38 seconds less than `main` branch avg.) and finished at 26th Jan, 2023.

✅ build-and-test workflow has finished in 38 minutes 53 seconds (13 minutes 9 seconds less than `main` branch avg.) and finished at 26th Jan, 2023.

✅ load-tests workflow has finished in 13 minutes 13 seconds and finished at 26th Jan, 2023.

⭕ build-and-test-windows workflow has finished in 3 seconds (40 minutes 38 seconds less than `main` branch avg.) and finished at 26th Jan, 2023.

✅ changelog workflow has finished in 1 minute 31 seconds and finished at 26th Jan, 2023.

atoulme left a comment

MovieStoreGuy Dec 28, 2022

pmcollins Jan 3, 2023

djaglowski Jan 4, 2023

MovieStoreGuy left a comment

MovieStoreGuy Dec 28, 2022

pmcollins Jan 3, 2023

djaglowski Jan 4, 2023

MovieStoreGuy Jan 8, 2023

MovieStoreGuy Dec 28, 2022

pmcollins Jan 3, 2023

MovieStoreGuy Dec 28, 2022

pmcollins Jan 3, 2023

MovieStoreGuy commented Jan 8, 2023

github-actions bot commented Jan 26, 2023

MovieStoreGuy left a comment

MovieStoreGuy Jan 26, 2023

MovieStoreGuy left a comment

djaglowski left a comment


		The following settings are required:

		- `path` [no default]: the file containing the metrics written by a File Exporter.

-type fileReceiver struct{}
+type fileReceiver struct {
+    component.StartFunc
+    component.ShutdownFunc
+}

[receiver/file] add file receiver skeleton 1/3 #16827

[receiver/file] add file receiver skeleton 1/3 #16827

Conversation

pmcollins commented Dec 8, 2022

runforesight bot commented Dec 8, 2022 • edited Loading

Foresight Summary

build-and-test-windows duration(4 seconds) has decreased 40 minutes 37 seconds compared to main branch avg(40 minutes 41 seconds).

✅ tracegen workflow has finished in 1 minute 58 seconds and finished at 26th Jan, 2023.

✅ telemetrygen workflow has finished in 2 minutes 5 seconds and finished at 26th Jan, 2023.

✅ check-links workflow has finished in 2 minutes 40 seconds (⚠️ 50 seconds more than main branch avg.) and finished at 26th Jan, 2023.

✅ prometheus-compliance-tests workflow has finished in 3 minutes 52 seconds (3 minutes 38 seconds less than main branch avg.) and finished at 26th Jan, 2023.

✅ build-and-test workflow has finished in 38 minutes 53 seconds (13 minutes 9 seconds less than main branch avg.) and finished at 26th Jan, 2023.

✅ load-tests workflow has finished in 13 minutes 13 seconds and finished at 26th Jan, 2023.

⭕ build-and-test-windows workflow has finished in 3 seconds (40 minutes 38 seconds less than main branch avg.) and finished at 26th Jan, 2023.

✅ changelog workflow has finished in 1 minute 31 seconds and finished at 26th Jan, 2023.

atoulme left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MovieStoreGuy left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MovieStoreGuy commented Jan 8, 2023

github-actions bot commented Jan 26, 2023

MovieStoreGuy left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MovieStoreGuy left a comment

Choose a reason for hiding this comment

djaglowski left a comment

Choose a reason for hiding this comment

runforesight bot commented Dec 8, 2022 •

edited

Loading

`build-and-test-windows` duration(4 seconds) has decreased 40 minutes 37 seconds compared to main branch avg(40 minutes 41 seconds).

✅ check-links workflow has finished in 2 minutes 40 seconds (⚠️ 50 seconds more than `main` branch avg.) and finished at 26th Jan, 2023.

✅ prometheus-compliance-tests workflow has finished in 3 minutes 52 seconds (3 minutes 38 seconds less than `main` branch avg.) and finished at 26th Jan, 2023.

✅ build-and-test workflow has finished in 38 minutes 53 seconds (13 minutes 9 seconds less than `main` branch avg.) and finished at 26th Jan, 2023.

⭕ build-and-test-windows workflow has finished in 3 seconds (40 minutes 38 seconds less than `main` branch avg.) and finished at 26th Jan, 2023.