Query CAR database according to epoch #3555
Labels
Performance
Priority: 2 - High
Very important and should be addressed ASAP
Ready
Issue is ready for work and anyone can freely assign it to themselves
Type: Enhancement
Issue summary
There are two kinds of data stores in Forest: Read-only CAR files and writeable ParityDB databases. Since all values are uniquely determined by their key, it does not matter for correctness which data store we query first. It does matter for performance, though, and we want to query as few data stores as possible.
We first query the CAR data stores in the order they were added. If a key isn't present in the CAR data stores, we then query the ParityDB database. However, in regular operation, we're significantly more likely to query new data than old data. As such, querying the data stores with the latest data first is a better option. The ParityDB database contains current data and is, therefore, the newest and should be queried first. The CAR data stores should be sorted from highest to lowest by epoch.
Other information and links
The text was updated successfully, but these errors were encountered: