Can't Search on GPU

[cr-intezra]

Summary

Following the provided notebooks for training I have a trained and 'added' index based on sift1b. It matches the benchmarks for size and what not. However at search time I am unable to get it to search from the GPUs getting the 'time-saving' benefit of using GPUs.

Platform

OS: Ubuntu 22.04

Faiss version: faiss-gpu 1.9.0

Installed from: conda

Running with 2 A6000 Ada GPUs.

Running on:

• CPU
• GPU

Interface:

• C++
• Python

Reproduction instructions

Training:
index = faiss.index_factory(d, "OPQ16_64,IVF1048576_HNSW32,PQ16x4fsr")
clustering.train_ivf_index_with_2level(index, learn, gpu=True)

I think add the sift1b data and confirm the ntotal is correct.

for search I have a benchmark script that does different batch query sizes [1, 10, 50, 100, 500, 1000, 5000, 10000]

for search on gpu i've tried:

index_gpu = faiss.index_cpu_to_all_gpus(index)

ngpus = faiss.get_num_gpus()

index_ivf = faiss.extract_index_ivf(index_cpu)
quantizer = index_ivf.quantizer

if ngpus > 0:
print(f"{ngpus} GPU(s) detected. Benchmarking GPU...")
index_gpu = faiss.index_cpu_to_all_gpus(quantizer) # distribute to all GPUs
index_ivf.quantizer = index_gpu

Based off of the benchmarks and watching nvidia-smi the index and or search is not occurring on GPU.

I am using GPU successfully for training and adding data just not search?

What am I missing here...?

[asadoughi]

Only a subset of the algorithms are available on GPU: https://github.com/facebookresearch/faiss/wiki/Faiss-on-the-GPU#implemented-indexes

[cr-intezra]

I guess I'm getting confused from the graphic at the bottom of https://github.com/facebookresearch/faiss/wiki/Guidelines-to-choose-an-index

following the flow and assuming a very large dataset should I be leaving off the _HNSW32?

That would then allow to do search on GPU, correct?