You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When saving the data as a table (either with CREATE TABLE SQL statement or a saveAsTable method) we save columnsToIndex, cubeSize and columnStats in the Metastore. This metadata might not be retrievable if not stored in the Delta Log, making it unreliable for other sources.
We should discuss and understand:
Which format should save the Properties (columnsToIndex, for example, should contain the column type)?
Where should we store this data? Both in Metastore and Commit Log? Just in the Commit Log?
The text was updated successfully, but these errors were encountered:
When saving the data as a table (either with CREATE TABLE SQL statement or a saveAsTable method) we save
columnsToIndex
,cubeSize
andcolumnStats
in the Metastore. This metadata might not be retrievable if not stored in the Delta Log, making it unreliable for other sources.We should discuss and understand:
The text was updated successfully, but these errors were encountered: