Milvus vs FAISS

[ACraig7]

@adinasurya @008784796 @Jagadeesh-ajjada @Boganabi @BandaruDinesh @vamsikrishnabandaru3 @008445808 @ritvik180 @smrchanda877 @Dinesh0614 @itsnotmik @Jenildabhi5103 @Adenkri45 @tejaswi-142

Milvus

• Dedicated vector database for managing large-scale vector data
• Complete storage and indexing solution, which can handle massive datasets with features like distributed storage and cloud-native deployment
• Horizontal scaling, allowing it to manage billions of vectors across distributed nodes
• Designed to handle high-throughput queries in a cloud environment
• Variety of indexing methods, such as IVF (Inverted File), HNSW (Hierarchical Navigable Small World), and others, allowing users to choose the best method for their use case
• Automatically manages indexing and can dynamically optimize indices based on data changes
• Complex query capabilities, including filtering, aggregation, and combination of vector searches with traditional database queries
• SQL-like interface for querying
• User-friendly interface and APIs (RESTful, gRPC) for easy integration
• Comprehensive solution with less emphasis on low-level implementations

FAISS

• Library for efficient similarity search and clustering of dense vectors
• Focuses on performance and memory efficiency for searching and indexing.
• Doesn’t provide a full-fledged database but can be integrated with other databases for storage
• Suited for single-node environments
• Focus is on performance optimizations for vector search
• Rich set of indexing methods, including Flat, IVFPQ (Inverted File with Product Quantization), and HNSW
• Does not support complex query types directly
• Requires more technical expertise to integrate and use, as it’s primarily a library
• Suited for users comfortable with programming and wanting to fine-tune performance
• Users need to integrate FAISS with other tools for data management

From research FAISS is a better at searching and clustering vectors but is just a library and is not a database. So we could use FAISS for its searching capabilities and Milvus for its database or we would have to use another technology for a database. Although, Milvus combines both a database searching of vectors of FAISS, so we would only need to use Milvus unless the professor wants us to use both. Also, Milvus does seem to be more user friendly and all around easier to use. So, if we can get away with just using Milvus, I suggest we do that.