Release v0.3.0

[nfx]

• Added support for save_table(..., mode="overwrite") to StatementExecutionBackend (#74). In this release, we've added support for overwriting a table when saving data using the save_table method in the StatementExecutionBackend. Previously, attempting to use the overwrite mode would raise a NotImplementedError. Now, when this mode is specified, the method first truncates the table before inserting the new rows. The truncation is done using the execute method to run a TRUNCATE TABLE SQL command. Additionally, we've added a new integration test, test_overwrite, to the test_deployment.py file to verify the new overwrite mode functionality. A new option, mode="overwrite", has been added to the save_table method, allowing for the existing data in the table to be deleted and replaced with the new data being written. We've also added two new test cases, test_statement_execution_backend_save_table_overwrite_empty_table and test_mock_backend_overwrite, to verify the new functionality. It's important to note that the method signature has been updated to include a default value for the mode parameter, setting it to append by default. This change does not affect the functionality and only provides a more convenient default behavior for users of the method.

[github-actions]

❌ 17/18 passed, 1 failed, 2 skipped, 11m28s total

❌ test_overwrite: databricks.sdk.errors.platform.BadRequest: [INSUFFICIENT_PERMISSIONS] Insufficient privileges: (589ms)

databricks.sdk.errors.platform.BadRequest: [INSUFFICIENT_PERMISSIONS] Insufficient privileges:
User does not have permission CREATE,USAGE on database `default`.
13:28 DEBUG [databricks.sdk] Loaded from environment
13:28 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
13:28 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
13:28 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
13:28 INFO [databricks.sdk] Using Databricks Metadata Service authentication
[gw9] linux -- Python 3.10.13 /home/runner/work/lsql/lsql/.venv/bin/python
13:28 DEBUG [databricks.sdk] Loaded from environment
13:28 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
13:28 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
13:28 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
13:28 INFO [databricks.sdk] Using Databricks Metadata Service authentication
13:28 DEBUG [databricks.labs.lsql.backends] [api][execute] CREATE TABLE IF NOT EXISTS default.foo (first STRING NOT NULL, second BOOLEAN NOT NULL) USING DE... (3 more bytes)
13:28 DEBUG [databricks.labs.lsql.core] Executing SQL statement: CREATE TABLE IF NOT EXISTS default.foo (first STRING NOT NULL, second BOOLEAN NOT NULL) USING DELTA
13:28 DEBUG [databricks.sdk] POST /api/2.0/sql/statements/
> {
>   "format": "JSON_ARRAY",
>   "statement": "CREATE TABLE IF NOT EXISTS default.foo (first STRING NOT NULL, second BOOLEAN NOT NULL) USING DE... (3 more bytes)",
>   "warehouse_id": "TEST_DEFAULT_WAREHOUSE_ID"
> }
< 200 OK
< {
<   "statement_id": "01eeec3d-f431-12d4-a824-ce3aa42c40c0",
<   "status": {
<     "error": {
<       "error_code": "BAD_REQUEST",
<       "message": "[INSUFFICIENT_PERMISSIONS] Insufficient privileges:\nUser does not have permission CREATE,USAGE o... (21 more bytes)"
<     },
<     "state": "FAILED"
<   }
< }
13:28 DEBUG [databricks.sdk] Loaded from environment
13:28 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
13:28 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
13:28 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
13:28 INFO [databricks.sdk] Using Databricks Metadata Service authentication
13:28 DEBUG [databricks.labs.lsql.backends] [api][execute] CREATE TABLE IF NOT EXISTS default.foo (first STRING NOT NULL, second BOOLEAN NOT NULL) USING DE... (3 more bytes)
13:28 DEBUG [databricks.labs.lsql.core] Executing SQL statement: CREATE TABLE IF NOT EXISTS default.foo (first STRING NOT NULL, second BOOLEAN NOT NULL) USING DELTA
13:28 DEBUG [databricks.sdk] POST /api/2.0/sql/statements/
> {
>   "format": "JSON_ARRAY",
>   "statement": "CREATE TABLE IF NOT EXISTS default.foo (first STRING NOT NULL, second BOOLEAN NOT NULL) USING DE... (3 more bytes)",
>   "warehouse_id": "TEST_DEFAULT_WAREHOUSE_ID"
> }
< 200 OK
< {
<   "statement_id": "01eeec3d-f431-12d4-a824-ce3aa42c40c0",
<   "status": {
<     "error": {
<       "error_code": "BAD_REQUEST",
<       "message": "[INSUFFICIENT_PERMISSIONS] Insufficient privileges:\nUser does not have permission CREATE,USAGE o... (21 more bytes)"
<     },
<     "state": "FAILED"
<   }
< }
[gw9] linux -- Python 3.10.13 /home/runner/work/lsql/lsql/.venv/bin/python

_{Running from acceptance #47}