Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Support timestamp protobuf serde #6927

Merged
merged 4 commits into from
Feb 10, 2021
Merged

Conversation

jzaralim
Copy link
Contributor

@jzaralim jzaralim commented Feb 1, 2021

Description

One of the reasons the decimal and timestamp types were not enabled in protobuf was because those two types have dependencies that need to be registered first in order for schema registry to be able to read schemas involving these types. This PR adds an extra step before registering protobuf schemas that resolves dependencies.

Testing done

manually, unit and QTT tests

Reviewer checklist

  • Ensure docs are updated if necessary. (eg. if a user visible feature is being added or changed).
  • Ensure relevant issues are linked (description should include text like "Fixes #")

@jzaralim jzaralim requested a review from a team as a code owner February 1, 2021 22:29
@jzaralim
Copy link
Contributor Author

jzaralim commented Feb 1, 2021

cc @rayokota

Copy link
Member

@rayokota rayokota left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jzaralim LGTM! Left a minor nit

final ProtobufSchema resolved = AbstractKafkaProtobufSerializer.resolveDependencies(
srClient,
true,
true,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: the arg for useLatestVersion is not used because autoRegisterSchema is true, but it would probably be best to pass false anyway (since false is the default for useLatestVersion in general). So the args should read srClient, true, false, true.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rayokota - looking at AbstractKafkProtobufSerializer I noticed this method is "Visible for testing". If we're going to be depending on it here, can we remove that comment and make it clear that it's public API? Also would be nice to document it since I'm not entirely sure what it does from the method name. Does it just flatten it into one big protobuf schema?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@agavra , sure I've filed https://confluentinc.atlassian.net/browse/DGS-1285 to track. The method recursively registers referenced schemas.

Copy link
Contributor

@agavra agavra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Thanks @jzaralim - I have one comment for @rayokota inline

final ProtobufSchema resolved = AbstractKafkaProtobufSerializer.resolveDependencies(
srClient,
true,
true,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rayokota - looking at AbstractKafkProtobufSerializer I noticed this method is "Visible for testing". If we're going to be depending on it here, can we remove that comment and make it clear that it's public API? Also would be nice to document it since I'm not entirely sure what it does from the method name. Does it just flatten it into one big protobuf schema?

Copy link
Member

@spena spena left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm! Thanks Zara

@jzaralim jzaralim merged commit 5ea1ce4 into confluentinc:master Feb 10, 2021
@jzaralim jzaralim deleted the proto branch February 10, 2021 19:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants