Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Add recall calculation as part of benchmark summary for vectorsearch workloads. #282

Closed
akashsha1 opened this issue Apr 19, 2024 · 1 comment
Labels
enhancement New feature or request

Comments

@akashsha1
Copy link

Is your feature request related to a problem?

Vector search workloads presently do not support recall calculation in the summary section.

What solution would you like?

VectorSearch outcomes rely on recall, which is an important metric in determining if the search was successful. It would be ideal to include this metric, as new workloads and benchmarks in this space evolve (e.g. semantic search)

What alternatives have you considered?

The current alternative is to manually calculate the recall offline based on the data being published, or using opensearch queries to aggregrate the information.

@akashsha1 akashsha1 added enhancement New feature or request untriaged labels Apr 19, 2024
@gkamat
Copy link
Collaborator

gkamat commented Apr 23, 2024

Will look into this issue further shortly, whether this could be just another metric that is reported.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants