make index shipper read path handle range of tables by type of index

[sandeepsukhani]

What this PR does / why we need it:
We run one indexshipper instance for each boltdb-shipper and tsdb instance. While initializing indexshipper, both the stores pass an index specific OpenIndexFileFunc to help indexshipper open the index files without having to make it aware of the file types or the way to interpret them.

If Loki is using both boltdb-shipper and tsdb for the index store and has the same local and/or object storage path configured for storing the index files, it fails to run because the generic indexshipper running for both the store types tries to open both types of index files using the same OpenIndexFileFunc.

This PR makes indexshipper accept a range of table numbers to handle only specific tables i.e it would sync and load-on-startup only specific tables. The table number ranges are calculated based on the configured schema start/end date and the index period of both the index types.

Checklist

• Tests updated

[grafanabot]

./tools/diff_coverage.sh ../loki-main/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
+        distributor	0%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0.3%
+           chunkenc	0%
+              logql	0%
+               loki	0%

[grafanabot]

./tools/diff_coverage.sh ../loki-main/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
+        distributor	0%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0.3%
+           chunkenc	0%
+              logql	0%
+               loki	0%

[owen-d]

This looks great. One thing I think we should add is support for skipping files within a table. For instance, consider two period configs (I know we're hardcoding period to 24h now, but it's a configurable field and should/can be configurable in the future IMO):

1. boltdb, period=24h
2. tsdb, period=48h

It's possible for an earlier timestamp (i.e. 72h) to have the same bucket as a later one (144h) because 72h / 24h = 3 and 144h / 48h = 3.

Approving in the meantime :)

[sandeepsukhani]

One thing I think we should add is support for skipping files within a table.

Yeah, I think we will have to keep the file name format unique to be able to distinguish between different index file types.

[grafanabot]

./tools/diff_coverage.sh ../loki-main/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
+        distributor	0%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0.4%
+           chunkenc	0%
+              logql	0%
+               loki	0%