Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(parsers.xpath): Handle protobuf maps correctly #13719

Merged
merged 4 commits into from
Aug 7, 2023

Conversation

srebhan
Copy link
Member

@srebhan srebhan commented Aug 3, 2023

resolves #13715

Currently, Telegraf panics if you use maps (such as map<string, string>) in your protocol-buffers. This is mainly due to a bug in the protocol-buffer parsing code of the protobufquery library.

This PR bumps the dependency of the library and adds a unit-test based on the data of #13715. While at it I unified the location of the xpath-querying libraries under the srebhan account.

@telegraf-tiger telegraf-tiger bot added fix pr to fix corresponding bug plugin/parser 1. Request for new parser plugins 2. Issues/PRs that are related to parser plugins labels Aug 3, 2023
@srebhan srebhan added the ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review. label Aug 3, 2023
@jcpunk
Copy link

jcpunk commented Aug 3, 2023

I can confirm this parses for my use case.

@@ -138,7 +138,6 @@ func (d *protobufDocument) GetNodePath(node, relativeTo dataNode, sep string) st
if n.Parent != nil && n.Parent.Value() != nil {
kind = reflect.TypeOf(n.Parent.Value()).Kind()
}
fmt.Printf("node %q: %v (value=%v) parent: %v\n", n.Data, kind, n.Value(), n.Parent.Value())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these helpful? should they be log messages? Same question in xpath/parsers.go

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd love to have those at the debug level.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These will only pollute the output. If in doubt you can always print the XML equivalent of the input document using the xpath_print_document setting.

@srebhan srebhan force-pushed the xpath_issue_13715 branch from 21925c1 to 842759b Compare August 7, 2023 20:54
@srebhan srebhan requested a review from powersj August 7, 2023 20:57
@telegraf-tiger
Copy link
Contributor

telegraf-tiger bot commented Aug 7, 2023

@powersj powersj merged commit 806b7cd into influxdata:master Aug 7, 2023
@github-actions github-actions bot added this to the v1.27.4 milestone Aug 7, 2023
powersj pushed a commit that referenced this pull request Aug 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
fix pr to fix corresponding bug plugin/parser 1. Request for new parser plugins 2. Issues/PRs that are related to parser plugins ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] unable to parse complex protobuf message
3 participants