Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dataflow: extensible template with javascript udf #8291

Merged
merged 6 commits into from
Aug 30, 2022
Merged

Conversation

anguillanneuf
Copy link
Member

@anguillanneuf anguillanneuf commented Aug 27, 2022

Toward internal b/238353202 and b/244173472 (region tags are created on Monday, Aug 29)

@anguillanneuf anguillanneuf requested review from davidcavazos and a team as code owners August 27, 2022 01:16
@product-auto-label product-auto-label bot added api: dataflow Issues related to the Dataflow API. samples Issues that are directly related to samples. labels Aug 27, 2022
Copy link
Collaborator

@dandhlee dandhlee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will you be adding the region tag in the future?

@snippet-bot
Copy link

snippet-bot bot commented Aug 29, 2022

Here is the summary of changes.

You are about to add 1 region tag.

This comment is generated by snippet-bot.
If you find problems with this result, please file an issue at:
https://github.com/googleapis/repo-automation-bots/issues.
To update this comment, add snippet-bot:force-run label or use the checkbox below:

  • Refresh this comment

@anguillanneuf anguillanneuf requested a review from dandhlee August 29, 2022 16:58
@anguillanneuf
Copy link
Member Author

@dandhlee Oh yes, region tags were created (this morning) and added.

Copy link
Collaborator

@dandhlee dandhlee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for Python.
Could you get a review for the JS file if you think it's needed? If not, just a product review would be fine.

Copy link
Contributor

@davidcavazos davidcavazos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great! Thanks. I made a few minor comments but overall looks good.

From a branding perspective, the non-Flex templates are called "Classic Templates" so maybe we want to rename this from extensible to classic?

* @param {string} inJson input Pub/Sub JSON message (stringified)
*/
function process(inJson) {
var obj = JSON.parse(inJson),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like Nashorn (partially) supports ECMA6, so let's see if we can modernize this Javascript a little bit.

Does it work if we change var to const? This forces the "variable" to be immutable, and makes it respect naming scopes.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it doesn't unfortunately. Tests failed after I made the switch. Keeping var but we should file a bug.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, let's keep var for now. Although if Nashorn is getting deprecated in the next Java version, maybe we can let it be. It looks like Java 11 will end of life in around 1 year and alongside with that Nashorn will be deprecated, the templates and other UDFs will most likely have to adapt GraalVM or something similar since Nashorn won't be around.

@anguillanneuf
Copy link
Member Author

Thanks @dandhlee and @davidcavazos for the review. Addressed all the review comments.

For the naming, we agreed on not updating extensible template to classic template because the correct name is actually "Google-provided templates". All "Google-provided templates" are now extensible and we can also create flex templates that are extensible. "extensible" templates aren't really a type of templates defined in the same sense as a feature but a concept.

@davidcavazos
Copy link
Contributor

LGTM! Thank you! This looks great :)

@anguillanneuf anguillanneuf merged commit 0f79dfe into main Aug 30, 2022
@anguillanneuf anguillanneuf deleted the beam-udf branch August 30, 2022 23:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: dataflow Issues related to the Dataflow API. samples Issues that are directly related to samples.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants