Code

Sort Index/Docids By Field#1026

Code

Merged

fulmicoton merged 53 commits intoquickwit-oss:mainquickwit-oss/tantivy:mainfrom PSeitz:indexmetaPSeitz/tantivy:indexmetaCopy head branch name to clipboardMay 17, 2021

+1,945-351

Commits

Commits on Apr 29, 2021

• sort index by field

  Show description for 521075d
  PSeitz

  committed
  521075d

  Copy full SHA for 521075d

• support docid mapping in multivalue fastfield

  Show description for 5643ee2
  PSeitz

  committed
  5643ee2

  Copy full SHA for 5643ee2

• handle docid map in bytes fastfield

  PSeitz

  committed
  60bf3f8

  Copy full SHA for 60bf3f8

Commits on May 3, 2021

• forward docid mapping, remap postings

  PSeitz

  committed
  8469223

  Copy full SHA for 8469223

• Merge remote-tracking branch 'upstream/main' into indexmeta

  PSeitz

  committed
  3cd436e

  Copy full SHA for 3cd436e

• fix merge conflicts

  PSeitz

  committed
  045dfee

  Copy full SHA for 045dfee

• move test to index_sorter

  PSeitz

  committed
  e97bdc9

  Copy full SHA for e97bdc9

• add docid index mapping old->new

  Show description for 62224fb
  PSeitz

  committed
  62224fb

  Copy full SHA for 62224fb

• remap docid in fielnorm

  PSeitz

  committed
  1ec2e61

  Copy full SHA for 1ec2e61

Commits on May 4, 2021

• resort docids in recorder, more extensive tests

  PSeitz

  committed
  8a7dc78

  Copy full SHA for 8a7dc78

• handle index sorting in docstore

  Show description for 179e859
  PSeitz

  committed
  179e859

  Copy full SHA for 179e859

Commits on May 5, 2021

• refactor

  Show description for 855680b
  PSeitz

  committed
  855680b

  Copy full SHA for 855680b

• u32 to DocId

  PSeitz

  committed
  77b1aa1

  Copy full SHA for 77b1aa1

• better doc_id_map creation

  Show description for a7766fb
  PSeitz

  committed
  a7766fb

  Copy full SHA for a7766fb

• add non mut method to FastFieldWriters

  Show description for 18ef88c
  PSeitz

  committed
  18ef88c

  Copy full SHA for 18ef88c

• remove sort_index

  PSeitz

  committed
  d6775cd

  Copy full SHA for d6775cd

• fix clippy issues

  PSeitz

  committed
  451479b

  Copy full SHA for 451479b

• fix SegmentComponent iterator

  Show description for 0ef02cd
  PSeitz

  committed
  0ef02cd

  Copy full SHA for 0ef02cd

• fix test

  PSeitz

  committed
  3d537bf

  Copy full SHA for 3d537bf

• fmt

  PSeitz

  committed
  b0b0129

  Copy full SHA for b0b0129

• handle indexsettings deserialize

  PSeitz

  committed
  0bbdd42

  Copy full SHA for 0bbdd42

• add reading, writing bytes to doc store

  Show description for b954fa6
  PSeitz

  committed
  b954fa6

  Copy full SHA for b954fa6

Commits on May 6, 2021

• rename index_sorter to doc_id_mapping

  Show description for 8e9278d
  PSeitz

  committed
  8e9278d

  Copy full SHA for 8e9278d

• fix compile issue, make sort_by_field optional

  PSeitz

  committed
  aca8cb8

  Copy full SHA for aca8cb8

• fix test compile

  PSeitz

  committed
  a34cd0e

  Copy full SHA for a34cd0e

Commits on May 7, 2021

• validate index settings on merge

  Show description for ba5a0e6
  PSeitz

  committed
  ba5a0e6

  Copy full SHA for ba5a0e6

• fix doctest

  PSeitz

  committed
  20f10e0

  Copy full SHA for 20f10e0

• add itertools, use kmerge

  Show description for 69dab3d
  PSeitz

  committed
  69dab3d

  Copy full SHA for 69dab3d

• implement/test merge for fastfield

  Show description for 38c178f
  PSeitz

  committed
  38c178f

  Copy full SHA for 38c178f

• Use precalculated docid mapping in merger

  Show description for e7468e5
  PSeitz

  committed
  e7468e5

  Copy full SHA for e7468e5

Commits on May 10, 2021

• fix fast field reader docs

  Show description for 9aab1b9
  PSeitz

  committed
  9aab1b9

  Copy full SHA for 9aab1b9

• add test for multifast field merge

  Show description for f8a3022
  PSeitz

  committed
  f8a3022

  Copy full SHA for f8a3022

• add num_bytes to BytesFastFieldReader

  Show description for 4f77067
  PSeitz

  committed
  4f77067

  Copy full SHA for 4f77067

• add MultiValueLength trait

  Show description for 39bcf13
  PSeitz

  committed
  39bcf13

  Copy full SHA for 39bcf13

• Add ReaderWithOrdinal, fix

  Show description for c576e88
  PSeitz

  committed
  c576e88

  Copy full SHA for c576e88

• add test for merging bytes with sorted docids

  PSeitz

  committed
  1b410d4

  Copy full SHA for 1b410d4

• Merge fieldnorm for sorted index

  PSeitz

  committed
  32a3a91

  Copy full SHA for 32a3a91

• handle posting list in merge in sorted index

  Show description for 76f8de9
  PSeitz

  committed
  76f8de9

  Copy full SHA for 76f8de9

• handle doc store order in merge in sorted index

  PSeitz

  committed
  b1c1c0d

  Copy full SHA for b1c1c0d

Commits on May 11, 2021

• fix typo, cleanup

  PSeitz

  committed
  eb0357c

  Copy full SHA for eb0357c

• make IndexSetting non-optional

  PSeitz

  committed
  3129885

  Copy full SHA for 3129885

• fix type, rename test file

  Show description for 00aab07
  PSeitz

  committed
  00aab07

  Copy full SHA for 00aab07

• remove SegmentReaderWithOrdinal accessors

  PSeitz

  committed
  cd2711c

  Copy full SHA for cd2711c

• cargo fmt

  PSeitz

  committed
  4816fc4

  Copy full SHA for 4816fc4

• add index sort & merge test to include deletes

  PSeitz

  committed
  5fc0ac4

  Copy full SHA for 5fc0ac4

• Fix posting list merge issue

  Show description for ade0ac0
  PSeitz

  committed
  ade0ac0

  Copy full SHA for ade0ac0

• performance: cache field readers, use bytes for doc store merge

  PSeitz

  committed
  b2a7fff

  Copy full SHA for b2a7fff

Commits on May 12, 2021

• change facet merge test to cover index sorting

  PSeitz

  committed
  aab65f0

  Copy full SHA for aab65f0

• add RawDocument abstraction to access bytes in doc store

  PSeitz

  committed
  25cb568

  Copy full SHA for 25cb568

• Merge remote-tracking branch 'upstream/main' into indexmeta

  PSeitz

  committed
  ea65dc1

  Copy full SHA for ea65dc1

• fix deserialization, update changelog

  Show description for b6a0f42
  PSeitz

  committed
  b6a0f42

  Copy full SHA for b6a0f42

• cache store readers to utilize lru cache (4x performance)

  Show description for 84da0be
  PSeitz

  committed
  84da0be

  Copy full SHA for 84da0be

Commits on May 14, 2021

• add include_temp_doc_store flag in InnerSegmentMeta

  Show description for de0ea84
  PSeitz

  committed
  de0ea84

  Copy full SHA for de0ea84