Skip to content

Commit

Permalink
SupportsRowIndexFilters
Browse files Browse the repository at this point in the history
  • Loading branch information
jaceklaskowski committed Dec 16, 2023
1 parent affe799 commit 8aa1545
Show file tree
Hide file tree
Showing 3 changed files with 25 additions and 2 deletions.
23 changes: 22 additions & 1 deletion docs/SupportsRowIndexFilters.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,24 @@
# SupportsRowIndexFilters

`SupportsRowIndexFilters` is...FIXME
`SupportsRowIndexFilters` is an [abstraction](#contract) of [file indices](#implementations) that can support [row index filters](#rowIndexFilters).

!!! note "Appears Change Data Feed Only"
The only implementation of `SupportsRowIndexFilters` is [TahoeFileIndex](TahoeFileIndex.md) yet [rowIndexFilters](#rowIndexFilters) is only used to create [CdcAddFileIndex](change-data-feed/CdcAddFileIndex.md#rowIndexFilters) and [TahoeRemoveFileIndex](change-data-feed/TahoeRemoveFileIndex.md#rowIndexFilters) for [Change Data Feed](change-data-feed/index.md) for [CDCReaderImpl](change-data-feed/CDCReaderImpl.md#processDeletionVectorActions).

## Contract

### Row Index Filters { #rowIndexFilters }

```scala
rowIndexFilters: Option[Map[String, RowIndexFilterType]] = None
```

A mapping of `URI`-encoded file paths to a row index filter type.

Used when:

* `CDCReaderImpl` is requested to [processDeletionVectorActions](change-data-feed/CDCReaderImpl.md#processDeletionVectorActions) (to create a [CdcAddFileIndex](change-data-feed/CdcAddFileIndex.md#rowIndexFilters) and a [TahoeRemoveFileIndex](change-data-feed/TahoeRemoveFileIndex.md#rowIndexFilters))

## Implementations

* [TahoeFileIndex](TahoeFileIndex.md)
2 changes: 2 additions & 0 deletions docs/TahoeFileIndex.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@

The aim of `TahoeFileIndex` (and `FileIndex` in general) is to reduce usage of very expensive disk access for file-related information using Hadoop [FileSystem]({{ hadoop.api }}/org/apache/hadoop/fs/FileSystem.html) API.

`TahoeFileIndex` is [SupportsRowIndexFilters](SupportsRowIndexFilters.md).

## Contract

### Matching Files { #matchingFiles }
Expand Down
2 changes: 1 addition & 1 deletion mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,6 @@ nav:
- FileFormat:
- DeltaFileFormat: DeltaFileFormat.md
- DeltaParquetFileFormat: DeltaParquetFileFormat.md
- SupportsRowIndexFilters.md
- Transaction Log (DeltaLog):
- DeltaLog: DeltaLog.md
- Operation: Operation.md
Expand Down Expand Up @@ -247,6 +246,7 @@ nav:
- PreprocessTimeTravel: PreprocessTimeTravel.md
- StagedDeltaTableV2: StagedDeltaTableV2.md
- File Indices:
- SupportsRowIndexFilters.md
- TahoeFileIndex: TahoeFileIndex.md
- TahoeBatchFileIndex: TahoeBatchFileIndex.md
- TahoeFileIndexWithSnapshotDescriptor: TahoeFileIndexWithSnapshotDescriptor.md
Expand Down

0 comments on commit 8aa1545

Please sign in to comment.