Code

Add Table properties to Qbeast#124

Code

Merged

osopardo1 merged 51 commits intoQbeast-io:mainQbeast-io/qbeast-spark:mainfrom osopardo1:42-table-writes-catalogosopardo1/qbeast-spark:42-table-writes-catalogCopy head branch name to clipboardNov 25, 2022

+2,721-243

Commits

Commits on Jul 29, 2022

• Wip on adding datasourceV2 compatibility

  osopardo1

  committed
  b627bab

  Copy full SHA for b627bab

Commits on Aug 3, 2022

• WIP on merging V1 and V2 methods

  osopardo1

  committed
  99adfd7

  Copy full SHA for 99adfd7

• Add tests

  osopardo1

  committed
  037a290

  Copy full SHA for 037a290

• Fix SQL in tests

  osopardo1

  committed
  7baadd2

  Copy full SHA for 7baadd2

Commits on Aug 4, 2022

• Add ReadSupport and TODOs on the code

  osopardo1

  committed
  6812a85

  Copy full SHA for 6812a85

• Add headers

  osopardo1

  committed
  e7fb0dd

  Copy full SHA for e7fb0dd

• Ammend some commints and create new test class for QbeastSparkIntegration

  osopardo1

  committed
  c73d4b5

  Copy full SHA for c73d4b5

• Moved Correctness tests to new class

  osopardo1

  committed
  74f70de

  Copy full SHA for 74f70de

Commits on Aug 5, 2022

• Extend DeltaCatalog and modify buildScan() method

  osopardo1

  committed
  f6188ff

  Copy full SHA for f6188ff

• Rollback QbeastCatalog

  osopardo1

  committed
  35b7143

  Copy full SHA for 35b7143

• Add select statement to SQLIntegrationTest

  osopardo1

  committed
  c49d20d

  Copy full SHA for c49d20d

• Fix Table Properties

  osopardo1

  committed
  983e5ce

  Copy full SHA for 983e5ce

• Return rdd in buildScan()

  osopardo1

  committed
  275eca1

  Copy full SHA for 275eca1

• Add new spark rule for fallback reads to V1 implementation

  osopardo1

  committed
  569f96f

  Copy full SHA for 569f96f

• Create headers

  osopardo1

  committed
  6f29bdd

  Copy full SHA for 6f29bdd

• Add path to getTable for tests

  osopardo1

  committed
  1a82a17

  Copy full SHA for 1a82a17

• Rollback to old createRelation implementation

  osopardo1

  committed
  2cec710

  Copy full SHA for 2cec710

Commits on Aug 6, 2022

• Add comments to the code and rename QbeastScanRDD to QbeastScan

  osopardo1

  committed
  2772bfe

  Copy full SHA for 2772bfe

• Remove spark val from QbeastTableImpl

  osopardo1

  committed
  d1e4db4

  Copy full SHA for d1e4db4

Commits on Aug 8, 2022

• Add INSERT INTO tests using more statements, add INSERT OVERWRITE tests, add tests for both INSERT INTO and OVERWRITE using real data

  Jiaweihu08

  committed
  7fa0bb1

  Copy full SHA for 7fa0bb1

• Add comments

  Jiaweihu08

  committed
  de3242d

  Copy full SHA for de3242d

Commits on Aug 9, 2022

• Use existing method to read csv data

  Jiaweihu08

  committed
  5467986

  Copy full SHA for 5467986

Commits on Aug 10, 2022

• Add INSERT INTO test for managed table

  Jiaweihu08

  committed
  f23dbdd

  Copy full SHA for f23dbdd

Commits on Aug 11, 2022

• Add 'INSERT OVERWRITE'-awareness to managed tables

  Jiaweihu08

  committed
  7b60c1b

  Copy full SHA for 7b60c1b

• Add test for INSERT OVERWRITE on managed tables

  Jiaweihu08

  committed
  8894e91

  Copy full SHA for 8894e91

Commits on Sep 1, 2022

• Merge branch 'main' into 42-table-writes-catalog

  Show description for 6fbd37a
  osopardo1

  committed
  6fbd37a

  Copy full SHA for 6fbd37a

Commits on Sep 5, 2022

• Use same test data

  osopardo1

  committed
  10f1f41

  Copy full SHA for 10f1f41

Commits on Sep 6, 2022

• Merge branch 'main' into 42-table-writes-catalog

  osopardo1

  committed
  9650ab1

  Copy full SHA for 9650ab1

• Adapting QbeastWriteBuilder to version 3.2.2 of Spark

  osopardo1

  committed
  88dd7f7

  Copy full SHA for 88dd7f7

• Armonize tests

  osopardo1

  committed
  13f86b9

  Copy full SHA for 13f86b9

Commits on Sep 13, 2022

• Add staging properties to the table and change QbeastCatalog to QbeastDeltaCatalog

  Show description for 88087fa
  osopardo1

  committed
  88087fa

  Copy full SHA for 88087fa

• Renamed QbeastCatalog to QbeastCatalogUtils

  osopardo1

  committed
  7f03e96

  Copy full SHA for 7f03e96

• Add scaladocs and comments

  osopardo1

  committed
  4923773

  Copy full SHA for 4923773

Commits on Sep 14, 2022

• Rename Catalog

  osopardo1

  committed
  e2448ed

  Copy full SHA for e2448ed

• Remove duplication of CreateTable

  osopardo1

  committed
  8f9218b

  Copy full SHA for 8f9218b

• Resolving build errors

  osopardo1

  committed
  9276b59

  Copy full SHA for 9276b59

Commits on Sep 19, 2022

• Extend CatalogExtenssion instead of DelegatingCatalogExtenssion or DeltaCatalog

  Show description for 1b5bebf
  osopardo1

  committed
  1b5bebf

  Copy full SHA for 1b5bebf

• Update readme

  osopardo1

  committed
  b355478

  Copy full SHA for b355478

Commits on Sep 20, 2022

• Fix creation of empty tables and add more test

  Show description for 3b10dee
  osopardo1

  committed
  3b10dee

  Copy full SHA for 3b10dee

• Update readme and add SupportsTruncate

  osopardo1

  committed
  737efda

  Copy full SHA for 737efda

Commits on Sep 28, 2022

• Shorten code comment

  Jiaweihu08

  committed
  93d4e7b

  Copy full SHA for 93d4e7b

Commits on Oct 5, 2022

• Add basic test for Catalog and DefaultStagedTable

  Show description for d0f84cb
  

  osopardo1

  authored and

  osopardo1

  committed
  d0f84cb

  Copy full SHA for d0f84cb

Commits on Oct 6, 2022

• Test for QbeastStagedTableTest and other exceptions tests

  

  osopardo1

  authored and

  osopardo1

  committed
  de86bd8

  Copy full SHA for de86bd8

• Add test on commitStagedChanges

  

  osopardo1

  authored and

  osopardo1

  committed
  c46ffbd

  Copy full SHA for c46ffbd

• Fixed bug on newWriteBuilder() creation and fixed tests

  

  osopardo1

  authored and

  osopardo1

  committed
  0aa8e59

  Copy full SHA for 0aa8e59

Commits on Oct 11, 2022

• Remove SparkTransformations and add tests for QbeastTableImpl, QbeastCatalogUtils and general QbeastCatalog behaviour

  

  osopardo1

  authored and

  osopardo1

  committed
  92f8988

  Copy full SHA for 92f8988

Commits on Oct 13, 2022

• Add Adria as Developer in pomExtra

  

  osopardo1

  authored and

  osopardo1

  committed
  f0dc97b

  Copy full SHA for f0dc97b

Commits on Oct 14, 2022

• Add test without location

  

  osopardo1

  authored and

  osopardo1

  committed
  6501112

  Copy full SHA for 6501112

Commits on Oct 25, 2022

• Add different method inputs for isQbeastProvider method

  

  osopardo1

  authored and

  osopardo1

  committed
  7f58c4d

  Copy full SHA for 7f58c4d

• Add different method inputs for isQbeastProvider

  

  osopardo1

  authored and

  osopardo1

  committed
  509aa02

  Copy full SHA for 509aa02

• Add rule for passing properties on ReplaceAsTableStatement

  

  osopardo1

  authored and

  osopardo1

  committed
  2b5c00b

  Copy full SHA for 2b5c00b