Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: update substrait to 0.57.1 #274

Merged
merged 15 commits into from
Oct 10, 2024

Conversation

wackywendell
Copy link
Contributor

The validator seemed to be a bit out of date with the main substrait repo, so I decided to see what it would take to update it to current substrait. And... it wasn't too bad.

For new features, I simply added them to the validator as NotYetImplemented, so they should emit warnings (not errors).

The only place I added actual support was for IntervalDayToSeconds, which should now accept the precision/subseconds variety, and not just the deprecated microseconds.

Copy link
Member

@EpsilonPrime EpsilonPrime left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on getting this moving into the right direction. This is a great first step towards bringing the validator up to where it should be.

I suspect we will want to avoid releases until we address all of the basic requirements of the latest version (such as enforcing that function names have their accepted argument types in their names) but that shouldn't hold up this first, great step.

rs/src/parse/expressions/literals.rs Outdated Show resolved Hide resolved
@@ -38,8 +38,13 @@ enum LiteralValue {
/// May be used only for binary and FixedBinary.
Binary(Vec<u8>),

/// May be used only for IntervalYearToMonth and IntervalDayToSecond.
Interval(i64, i64),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we reject use of Interval as deprecated at this point or just return an error as we would likely do once the old fields are removed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still new to this code base, but I think this is an internal class that is not exported; it does not have a 1:1 match with the public protobuf files. So there isn't a way to do Interval (and not IntervalYearToMonth and IntervalDayToSecond) in protobufs right now - there's nothing to deprecate or return an error on.

Does that make sense? (Or am I missing something?)

rs/src/parse/relations/set.rs Outdated Show resolved Hide resolved
Copy link
Member

@mbrobbel mbrobbel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this!

rs/src/parse/expressions/literals.rs Outdated Show resolved Hide resolved
rs/src/parse/relations/join.rs Outdated Show resolved Hide resolved
rs/src/parse/relations/join.rs Outdated Show resolved Hide resolved
rs/src/parse/relations/mod.rs Outdated Show resolved Hide resolved
@wackywendell
Copy link
Contributor Author

Thanks for the comments! I'm working through them now 👍

wackywendell and others added 4 commits October 9, 2024 12:46
Co-authored-by: Matthijs Brobbel <m1brobbel@gmail.com>
Required for test case tpc-h01 to pass, where `days: 1` is specified, but subseconds and precision are not.
@wackywendell
Copy link
Contributor Author

OK - I've addressed all the comments from code review. There's a couple above that I'm not sure I fully handled - let me know what you think!

And thank you both for your quick reviews, @EpsilonPrime and @mbrobbel !

@wackywendell
Copy link
Contributor Author

Thanks for the reviews! The last thing seems to be the commit lints. I can do some squashing and rebasing to address, although I'm generally hesitant to change history on publicly available commits; it will make the review comments above hard to track. Is that worth doing here?

@mbrobbel
Copy link
Member

Thanks for the reviews! The last thing seems to be the commit lints. I can do some squashing and rebasing to address, although I'm generally hesitant to change history on publicly available commits; it will make the review comments above hard to track. Is that worth doing here?

No, it doesn't matter. This repo uses squash merge, so it'll use the PR title as merge commit message.

@mbrobbel mbrobbel changed the title Update to 0.57.1 feat: update substrait to 0.57.1 Oct 10, 2024
@mbrobbel mbrobbel requested a review from EpsilonPrime October 10, 2024 17:40
@mbrobbel mbrobbel merged commit 5f6b8fc into substrait-io:main Oct 10, 2024
36 of 37 checks passed
@wackywendell wackywendell deleted the update-0.57.1 branch October 16, 2024 20:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants