Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Counter Output Operator #570

Merged
merged 20 commits into from
Feb 23, 2022
Merged

Add Counter Output Operator #570

merged 20 commits into from
Feb 23, 2022

Conversation

schmikei
Copy link
Contributor

@schmikei schmikei commented Feb 21, 2022

Description of Changes

Adds a new output operator: count_output

The purpose of this operator is to count the number of entries processed over an interval.

Proposed config:

pipeline:
- type: count_output
   path: /optional/path/to/output/file # otherwise stdout
   duration: 1m # duration of the output 

Please check that the PR fulfills these requirements

  • Tests for the changes have been added (for bug fixes / features)
  • Docs have been added / updated (for bug fixes / features)
  • Add a changelog entry (for non-trivial bug fixes / features)
  • CI passes

@codecov
Copy link

codecov bot commented Feb 21, 2022

Codecov Report

Merging #570 (65fa531) into master (3b23e5c) will decrease coverage by 1.80%.
The diff coverage is 86.29%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #570      +/-   ##
==========================================
- Coverage   76.69%   74.88%   -1.80%     
==========================================
  Files         132      133       +1     
  Lines        8308    10076    +1768     
==========================================
+ Hits         6371     7545    +1174     
- Misses       1473     2065     +592     
- Partials      464      466       +2     
Impacted Files Coverage Δ
operator/buffer/buffer.go 75.86% <ø> (-1.92%) ⬇️
operator/buffer/memory.go 71.60% <ø> (-3.74%) ⬇️
...builtin/input/aws/cloudwatch/cloudwatch_persist.go 75.00% <ø> (-6.82%) ⬇️
operator/builtin/input/azure/event_hub.go 0.00% <0.00%> (ø)
operator/builtin/input/azure/event_hub_config.go 90.91% <ø> (-1.09%) ⬇️
operator/builtin/input/file/finder.go 100.00% <ø> (ø)
operator/builtin/input/file/reader.go 53.05% <0.00%> (-5.73%) ⬇️
operator/builtin/input/http/config.go 97.56% <ø> (-0.29%) ⬇️
operator/builtin/input/journald/journald.go 58.76% <ø> (-1.83%) ⬇️
operator/builtin/input/udp/udp.go 73.49% <ø> (-1.51%) ⬇️
... and 157 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5ff3f35...65fa531. Read the comment docs.

@schmikei
Copy link
Contributor Author

May look into adding https://pkg.go.dev/expvar as another flag and thats what we publish. Not sure if its necessary on this first pass.

@schmikei schmikei marked this pull request as ready for review February 22, 2022 19:54
@schmikei schmikei requested a review from a team as a code owner February 22, 2022 19:54
Copy link
Contributor

@cpheps cpheps left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good just come comments

operator/builtin/output/count/count.go Show resolved Hide resolved
operator/builtin/output/count/count.go Outdated Show resolved Hide resolved
operator/builtin/output/count/count.go Outdated Show resolved Hide resolved
operator/builtin/output/count/count.go Show resolved Hide resolved
operator/builtin/output/count/count.go Outdated Show resolved Hide resolved
cpheps
cpheps previously approved these changes Feb 23, 2022
Mrod1598
Mrod1598 previously approved these changes Feb 23, 2022
@schmikei schmikei dismissed stale reviews from Mrod1598 and cpheps via dee28f7 February 23, 2022 16:19
@schmikei schmikei merged commit dbd3ef0 into master Feb 23, 2022
@schmikei schmikei deleted the counter_output branch February 23, 2022 16:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants