Use custom centroids in IVFFlat

[JingyuanHe1222]

Hi, I wonder if there's any interface we can use to construct IVF clusters with custom centroids vectors?

So instead of initialize the IVF index and add corpus vectors for train (which construct corpus centroids), can we manually designate some vectors as the centroids and update the clustering accordingly?

Seemingly this could be achieved by constructing a IndexFlat clusters on the custom centroids as quantizer, and manually construct nlist IndexFlat for each of these clusters after calculating the vector assignments using KNN.

But I wonder if there's a prebuild interface, which might be faster?

Thanks in advance!

[JingyuanHe1222]

I am not sure if this is the correct way of doing this, but I have been adding the custom centroids into the quantizer, and then initialize the IVF index with this "trained" quantizer, as in the following:

coarse_quantizer = faiss.IndexFlatL2(vector_dim)
coarse_quantizer.add(centroids)
index = faiss.IndexIVFFlat(coarse_quantizer, vector_dim, n_clusters, metrics)

[asitjain]

Did you find a solution? Does you approach work? I tried it and it seems to work but after training final index (IVFPQ), quantizer's cluster count doubles.

[mdouze]

Sure that works!