Fix NPE when LeafReader return null VectorValues #13162
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
benwtrent
requested changes
Mar 7, 2024
| // The field does not exist or does not index vectors | ||
| FloatVectorValues floatVectorValues = getFloatVectorValues(fi.name); | ||
| if (floatVectorValues == null) { | ||
| FloatVectorValues.checkField(this, field); |
There was a problem hiding this comment.
The leaf reader here shouldn't throw. Especially since the companion method that accepts a KnnCollector doesn't.
| // The field does not exist or does not index vectors | ||
| ByteVectorValues byteVectorValues = getByteVectorValues(fi.name); | ||
| if (byteVectorValues == null) { | ||
| ByteVectorValues.checkField(this, field); |
There was a problem hiding this comment.
The leaf reader here shouldn't throw. Especially since the companion method that accepts a KnnCollector doesn't.
Comment on lines
43
to
45
| if (vectorValues == null) { | ||
| return DoubleValues.EMPTY; | ||
| } |
There was a problem hiding this comment.
Why do you throw in the byte similarity source, but not here? We need to be consistent. I think throwing here is acceptable as well (via FloatVectorValues.check).
There was a problem hiding this comment.
Nice catch. I will fix it.
benwtrent
approved these changes
Mar 8, 2024
benwtrent
pushed a commit
that referenced
this pull request
Mar 11, 2024
`LeafReader#getXXXVectorValues` may return null value.
**Reproduction**:
```
public class TestKnnByteVectorQuery extends BaseKnnVectorQueryTestCase {
public void testVectorEncodingMismatch() throws IOException {
try (Directory indexStore =
getIndexStore("field", new float[] {0, 1}, new float[] {1, 2}, new float[] {0, 0});
IndexReader reader = DirectoryReader.open(indexStore)) {
AbstractKnnVectorQuery query =
new KnnFloatVectorQuery("field", new float[] {0, 1}, 10);
IndexSearcher searcher = newSearcher(reader);
searcher.search(query, 10);
}
}
}
```
**Output**:
```
java.lang.NullPointerException: Cannot invoke "org.apache.lucene.index.FloatVectorValues.size()" because the return value of "org.apache.lucene.index.LeafReader.getFloatVectorValues(String)" is null
```
benwtrent
added a commit
that referenced
this pull request
Mar 20, 2024
Related to: #13162 Since this is unreleased, no changelog entry is necessary.
benwtrent
added a commit
that referenced
this pull request
Mar 20, 2024
Related to: #13162 Since this is unreleased, no changelog entry is necessary.
|
FWIW: This commit seems to have duplicated These should probably be refactored to eliminate the duplication? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
LeafReader#getXXXVectorValuesmay return null value.Reproduction:
Output: