Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor(optimizer): Optimize USING expansion #4115

Merged
merged 2 commits into from
Sep 13, 2024

Conversation

VaggelisD
Copy link
Collaborator

@VaggelisD VaggelisD commented Sep 13, 2024

This PR is an optimization follow-up on #4113

When processing multiple joins such as A JOIN B JOIN C USING(col), we gather the columns from the previous sources (A, B etc) to expand USING (col) to ON c.col = <source>.col. Previously, this operation would happen from scratch on each iteration, which can be avoided. A few key observations:

  1. The ordered list is constructed as a subset of scope.selected_sources; It can have [0, N] elements before the join loop
  2. At each iteration, we'll add the current join_table to ordered; It is implied that join_table is a member of scope.selected_sources as it participates in the scope's JOIN clause(s) so (1) continues to hold

The combination of (1) and (2) means that we can avoid checking if source_name is both in scope.selected_sources and ordered; Besides that, (2) drives incremental construction of columns at each iteration.

This PR also adds one more test case of a longer JOIN chain; This is to ensure that columns from all intermediate join tables are gathered, even if there is no USING clause.

@VaggelisD VaggelisD force-pushed the vaggelisd/qualify_cols branch from ce1e48a to 1f01944 Compare September 13, 2024 13:36
@VaggelisD VaggelisD force-pushed the vaggelisd/qualify_cols branch from 1f01944 to 2b5c8d0 Compare September 13, 2024 13:45
@georgesittas georgesittas merged commit 7cf1d70 into main Sep 13, 2024
6 checks passed
@georgesittas georgesittas deleted the vaggelisd/qualify_cols branch September 13, 2024 15:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants