feat: Add initial index lookup join operator implementation #12218
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
✅ Deploy Preview for meta-velox canceled.
|
|
This pull request was exported from Phabricator. Differential Revision: D68906030 |
xiaoxmeng
added a commit
to xiaoxmeng/velox
that referenced
this pull request
Jan 30, 2025
…incubator#12218) Summary: Add the initial index lookup operator implementation which supports inner join and left join from an index source. The index lookup join operator takes input from the probe side with one batch at a time. For each probe input batch, it sends the lookup request from the index source and gets a lookup result iterator. The result iterator produces the output in batches through a future for async processing at the storage backend. For each lookup result, the index join operator produces one or more output batches based on the operator's output batch size limit. For left join, the index join operator needs to detect the input request rows that have no matches and produce output with nulls for the columns from the lookup table. Differential Revision: D68906030
a750671 to
bfad28f
Compare
|
This pull request was exported from Phabricator. Differential Revision: D68906030 |
bfad28f to
666488e
Compare
xiaoxmeng
added a commit
to xiaoxmeng/velox
that referenced
this pull request
Jan 30, 2025
…incubator#12218) Summary: Add the initial index lookup operator implementation which supports inner join and left join from an index source. The index lookup join operator takes input from the probe side with one batch at a time. For each probe input batch, it sends the lookup request from the index source and gets a lookup result iterator. The result iterator produces the output in batches through a future for async processing at the storage backend. For each lookup result, the index join operator produces one or more output batches based on the operator's output batch size limit. For left join, the index join operator needs to detect the input request rows that have no matches and produce output with nulls for the columns from the lookup table. Differential Revision: D68906030
|
This pull request was exported from Phabricator. Differential Revision: D68906030 |
mbasmanova
approved these changes
Jan 30, 2025
…incubator#12218) Summary: Add the initial index lookup operator implementation which supports inner join and left join from an index source. The index lookup join operator takes input from the probe side with one batch at a time. For each probe input batch, it sends the lookup request from the index source and gets a lookup result iterator. The result iterator produces the output in batches through a future for async processing at the storage backend. For each lookup result, the index join operator produces one or more output batches based on the operator's output batch size limit. For left join, the index join operator needs to detect the input request rows that have no matches and produce output with nulls for the columns from the lookup table. Reviewed By: mbasmanova Differential Revision: D68906030
666488e to
c4b44fa
Compare
|
This pull request was exported from Phabricator. Differential Revision: D68906030 |
|
This pull request has been merged in dcafd32. |
ArnavBalyan
pushed a commit
to ArnavBalyan/velox
that referenced
this pull request
Jan 31, 2025
…incubator#12218) Summary: Pull Request resolved: facebookincubator#12218 Add the initial index lookup operator implementation which supports inner join and left join from an index source. The index lookup join operator takes input from the probe side with one batch at a time. For each probe input batch, it sends the lookup request from the index source and gets a lookup result iterator. The result iterator produces the output in batches through a future for async processing at the storage backend. For each lookup result, the index join operator produces one or more output batches based on the operator's output batch size limit. For left join, the index join operator needs to detect the input request rows that have no matches and produce output with nulls for the columns from the lookup table. Reviewed By: mbasmanova Differential Revision: D68906030 fbshipit-source-id: 4038c31653bc96893497b3c26d935fa2f7b58f43
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
Add the initial index lookup operator implementation which supports inner join and left join
from an index source.
The index lookup join operator takes input from the probe side with one batch at a time. For each
probe input batch, it sends the lookup request from the index source and gets a lookup result iterator.
The result iterator produces the output in batches through a future for async processing at the
storage backend. For each lookup result, the index join operator produces one or more output batches
based on the operator's output batch size limit. For left join, the index join operator needs to detect the
input request rows that have no matches and produce output with nulls for the columns from the lookup
table.
Differential Revision: D68906030