Adding one document to an existing index

[Hazoom]

Hi,

I have an index with many document that I built, and I want to add one document to it.
The naive approach will be of course to run the index command on all my documents together with the next index, but the issue is that my system runs in production environment and I cannot allow it because it takes too much time.

Could you please assist?

[duongkstn]

same issue. any advice ?

[e-tornike]

I tried the following, but it seems to overwrite the index at each run:

>>>from pyserini.index.lucene import LuceneIndexer, IndexReader
>>>indexer = LuceneIndexer("index")
>>>indexer.add("{'id': '0', 'contents': 'Document 0'}")
>>>indexer.close()
>>>reader = IndexReader("index")
>>>print(reader.stats())
{'total_terms': 2, 'documents': 1, 'non_empty_documents': 1, 'unique_terms': 2}

>>>indexer = LuceneIndexer("index")
>>>indexer.add("{'id': '1', 'contents': 'Document 1'}")
>>>indexer.close()
>>>reader = IndexReader("index")
>>>print(reader.stats())
{'total_terms': 2, 'documents': 1, 'non_empty_documents': 1, 'unique_terms': 2}

>>>indexer = LuceneIndexer("index")
>>>indexer.add("{'id': '0', 'contents': 'Document 0'}")
>>>indexer.add("{'id': '1', 'contents': 'Document 1'}")
>>>indexer.close()
>>>reader = IndexReader("index")
>>>print(reader.stats())
{'total_terms': 4, 'documents': 2, 'non_empty_documents': 2, 'unique_terms': 3}

Based on: #1344

[lintool]

Hi all, thanks for all your interest! If this is a feature that many people want... then we'll be happy to prioritize!

[lintool]

Hi @Hazoom @duongkstn @e-tornike I think this is what you're looking for? #1451

[e-tornike]

Awesome! Thanks a lot for the quick implementation!