Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: do brute force search on unindexed data #3036

Merged
merged 15 commits into from
Oct 31, 2024
Prev Previous commit
Next Next commit
fix
Signed-off-by: BubbleCal <bubble-cal@outlook.com>
BubbleCal committed Oct 24, 2024

Verified

This commit was signed with the committer’s verified signature.
wjones127 Will Jones
commit a636d128eafade1e83ba8ec5398b7f5da282e9f8
11 changes: 0 additions & 11 deletions rust/lance/src/dataset/scanner.rs
Original file line number Diff line number Diff line change
@@ -1404,17 +1404,6 @@ impl Scanner {
))
}

// fn flat_fts(
// &self,
// input: Arc<dyn ExecutionPlan>,
// column: String,
// index: Index,
// query: FullTextSearchQuery,
// ) -> Result<Arc<dyn ExecutionPlan>> {
// let flat_fts_plan = FlatFtsExec::new(input, column, index, query);
// Ok(Arc::new(flat_fts_plan))
// }

// ANN/KNN search execution node with optional prefilter
async fn knn(&self, filter_plan: &FilterPlan) -> Result<Arc<dyn ExecutionPlan>> {
let Some(q) = self.nearest.as_ref() else {
4 changes: 2 additions & 2 deletions rust/lance/src/io/exec/fts.rs
Original file line number Diff line number Diff line change
@@ -186,9 +186,9 @@ impl ExecutionPlan for FtsExec {
}
}

/// An execution node that performs full text search
/// An execution node that performs flat full text search
///
/// This node would perform full text search with inverted index on the dataset.
/// This node would perform flat full text search on unindexed rows.
/// The result is a stream of record batches containing the row ids that match the search query,
/// and scores of the matched rows.
#[derive(Debug)]

Unchanged files with check annotations Beta

pub fn remove_stream<'a>(
&'a self,
locations: BoxStream<'a, Result<Path>>,
) -> BoxStream<Result<Path>> {

Check warning on line 638 in rust/lance-io/src/object_store.rs

GitHub Actions / linux-build (nightly)

elided lifetime has a name
self.inner
.delete_stream(locations.err_into::<ObjectStoreError>().boxed())
.err_into::<Error>()
fn cast_dictionary_arrays<'a, T: ArrowDictionaryKeyType + 'static>(
arrays: &'a [&'a ArrayRef],
) -> Vec<&Arc<dyn Array>> {

Check warning on line 489 in rust/lance-file/src/writer/statistics.rs

GitHub Actions / linux-build (nightly)

elided lifetime has a name
arrays
.iter()
.map(|x| x.as_dictionary::<T>().values())
fn search_values<'a>(
&'a self,
values: &'a Vec<ScalarValue>,
) -> BoxStream<Result<RowIdTreeMap>> {

Check warning on line 81 in rust/lance-index/src/scalar/label_list.rs

GitHub Actions / linux-build (nightly)

elided lifetime has a name
futures::stream::iter(values)
.then(move |value| {
let value_query = SargableQuery::Equals(value.clone());