Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for KNN vector similarity search #513

Merged
merged 4 commits into from
Jul 12, 2023
Merged

Conversation

Pwuts
Copy link
Contributor

@Pwuts Pwuts commented May 18, 2023

Added components

  • KNNExpression, to be passed to RedisModel.find(). Can be used in conjunction with normal operators, e.g.:
    v_query = openai.Embedding.create(input=query).data[0]["embedding"]
    v_query = numpy.array(v_query, numpy.float32)   # may be redundant
    
    MyRedisModel.find(
        MyRedisModel.prop == "value",
        knn=KNNExpression(
            k=5,
            vector_field=MyRedisModel.__fields__["embedding"],
            reference_vector=v_query.tobytes(),
        )
    )
  • VectorFieldOptions, a simple dataclass to encapsulate the settings for a vector field; passed to Field(vector_options=.

Changed components

  • JsonModel and HashModel, to accommodate vector fields
    • RedisModel.from_redis(), to add a field containing the vector score when applicable
  • FindQuery, to incorporate KNN vector search support

@Pwuts
Copy link
Contributor Author

Pwuts commented May 18, 2023

@dvora-h @chayim 0:)

@Pwuts Pwuts changed the title Add support for KNN vector search Add support for KNN vector similarity search May 19, 2023
aredis_om/model/model.py Outdated Show resolved Hide resolved
@Pwuts
Copy link
Contributor Author

Pwuts commented May 22, 2023

@chayim where should I add instructions for using this feature?

@chayim
Copy link
Contributor

chayim commented May 22, 2023

IMHO we should start a separate vectorsimilarity doc... @elena-kolevska given what you're up to - any preference?

@elena-kolevska
Copy link

Looping in @bsbodden, who's starting to work on a VSS quickstart for redis.io.

@Pwuts Pwuts requested a review from chayim May 25, 2023 11:08
@Pwuts
Copy link
Contributor Author

Pwuts commented May 30, 2023

bump

Copy link
Contributor

@chayim chayim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Type hint asks mostly

@codecov-commenter
Copy link

codecov-commenter commented Jun 5, 2023

Codecov Report

Patch coverage: 70.10% and project coverage change: -0.82 ⚠️

Comparison is base (70f6401) 78.94% compared to head (583ae4a) 78.13%.

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #513      +/-   ##
==========================================
- Coverage   78.94%   78.13%   -0.82%     
==========================================
  Files          14       14              
  Lines        1178     1267      +89     
==========================================
+ Hits          930      990      +60     
- Misses        248      277      +29     
Flag Coverage Δ
unit 78.13% <70.10%> (-0.82%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
aredis_om/__init__.py 100.00% <ø> (ø)
aredis_om/model/__init__.py 100.00% <ø> (ø)
aredis_om/model/model.py 85.30% <70.10%> (-1.79%) ⬇️

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@XChikuX XChikuX mentioned this pull request Jun 20, 2023
@XChikuX
Copy link

XChikuX commented Jul 6, 2023

Okay, it's been more than a month. And this hasn't been merged even though all that's being corrected are type hints.
@chayim, I'm aware you are a very busy individual. Please spare 5 minutes of your busy schedule for approving this!

@chayim
Copy link
Contributor

chayim commented Jul 11, 2023

Hi folks. I just merged in #534 so that we can have a CI that is properly reflective and runs this, and tests. That was the first step.

Logic wise, this looks proper, and as if it fits with the rest of this library. Merging in the master changes, and giving it another look.

I'm pro merging this - and issuing a release.

@chayim chayim merged commit 89b6c84 into redis:main Jul 12, 2023
@chayim chayim added the enhancement New feature or request label Jul 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants