Filebeat inputv2 integration #19686
Merged
Filebeat inputv2 integration #19686
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Pinging @elastic/integrations-services (Team:Services) |
This change integrates the v2 input API with the existing filebeat architecture. The log input and the v2 architecure both use the statestore as a common storage for read-states between restarts. The integration wraps the input v2 API into a RunnerFactory, such that new inputs are automatically available to config file relading, autodiscovery, and fleet. Note: this PR does not introduce or link any input yet. The full list of changes will include: - Introduce v2 API interfaces - Introduce [compatibility layer](https://github.com/urso/beats/tree/fb-input-v2-combined/filebeat/input/v2/compat) to integrate API with existing functionality - Introduce helpers for writing [stateless](https://github.com/urso/beats/blob/fb-input-v2-combined/filebeat/input/v2/input-stateless/stateless.go) inputs. - Introduce helpers for writing [inputs that store a state](https://github.com/urso/beats/tree/fb-input-v2-combined/filebeat/input/v2/input-cursor) between restarts. - Integrate new API with [existing inputs and modules](https://github.com/urso/beats/blob/fb-input-v2-combined/filebeat/beater/filebeat.go#L301) in filebeat.
21adcab to
1820e36
Compare
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
|
kvch
reviewed
Jul 7, 2020
|
|
||
| func init() { | ||
| // Filebeat build the beat root command for executing filebeat and it's subcommands. | ||
| func Filebeat(inputs beater.PluginFactory) *cmd.BeatsRootCmd { |
There was a problem hiding this comment.
I am on the fence about this change. The code itself is better. I am just a bit disappointed that the consistency over different beats cmd/root.go files are gone. But I guess I can live with that. :D
There was a problem hiding this comment.
yeah, don't like it either. This last integration step is kinda ugly TBH. But trying to not use globals and x-pack filebeat relying on globals kinda forced me to do this. Long term I think libbeat/cmd and the <beats>/cmd packages will be removed.
kvch
approved these changes
Jul 7, 2020
2 tasks
urso
pushed a commit
to urso/beats
that referenced
this pull request
Jul 8, 2020
(cherry picked from commit bce6dea)
urso
pushed a commit
that referenced
this pull request
Jul 9, 2020
(cherry picked from commit bce6dea)
melchiormoulin
pushed a commit
to melchiormoulin/beats
that referenced
this pull request
Oct 14, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This change integrates the v2 input API with the existing filebeat
architecture. The log input and the v2 architecure both use the
statestore as a common storage for read-states between restarts.
The integration wraps the input v2 API into a RunnerFactory, such that
new inputs are automatically available to config file relading,
autodiscovery, and fleet.
Note: this PR does not introduce or link any input yet.
The full list of changes will include:
Why is it important?
Add support for running v2 inputs in Filebeat and Agent.
Checklist
- [ ] I have made corresponding changes to the documentation- [ ] I have made corresponding change to the default configuration files- [ ] I have added tests that prove my fix is effective or that my feature works- [ ] I have added an entry inCHANGELOG.next.asciidocorCHANGELOG-developer.next.asciidoc.Author's Checklist
How to test this PR locally
Related issues