feat(gsheets): file upload

[betodealmeida]

SUMMARY

Add a df_to_sql method to the GSheets DB engine spec, so we can upload CSV files. When a file is uploaded, a new sheet is created, and the database catalog is updated to point the table name to the URL of the new sheet.

The user can also choose to replace an existing table, in which case the sheet gets cleared before the data is inserted.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

Inserting some data:

Querying the table:

Uploading a different CSV and replacing the table:

And querying it (note that the metadata is cached and shows the old schema):

And the sheet after the second upload:

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

• Has associated issue:
• Required feature flags:
• Changes UI
• Includes DB Migration (follow approval process in SIP-59)
  • Migration is atomic, supports rollback & is backwards-compatible
  • Confirm DB migration upgrade and downgrade tested
  • Runtime estimates and downtime expectations provided
• Introduces new feature or API
• Removes existing feature or API

[codecov]

Codecov Report

Merging #24921 (165a0c0) into master (81bf2f0) will decrease coverage by 1.74%.
Report is 1 commits behind head on master.
The diff coverage is 29.31%.

❗ Current head 165a0c0 differs from pull request most recent head 9472c76. Consider uploading reports for the commit 9472c76 to get more accurate results

@@            Coverage Diff             @@
##           master   #24921      +/-   ##
==========================================
- Coverage   69.00%   67.26%   -1.74%     
==========================================
  Files        1906     1906              
  Lines       74132    74183      +51     
  Branches     8208     8208              
==========================================
- Hits        51153    49902    -1251     
- Misses      20856    22158    +1302     
  Partials     2123     2123              

Flag	Coverage Δ
hive	54.13% <29.82%> (-0.05%)	⬇️
mysql	79.12% <29.82%> (-0.09%)	⬇️
postgres	79.22% <29.31%> (-0.09%)	⬇️
presto	54.03% <29.82%> (-0.05%)	⬇️
python	79.72% <29.31%> (-3.65%)	⬇️
sqlite	77.81% <29.82%> (-0.09%)	⬇️
unit	?

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed	Coverage Δ
...tend/plugins/legacy-preset-chart-nvd3/src/utils.js	16.02% <ø> (+0.10%)	⬆️
superset/db_engine_specs/gsheets.py	38.82% <29.31%> (-37.65%)	⬇️

... and 97 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[Antonio-RiveroMartnez]

Just curious why moving from Optional ? Because I see Optional being used in several other TypedDict classes. Want to confirm if this is a pattern we aim to follow moving forward, thanks.

[betodealmeida]

We bumped the minimum Python version we support in Superset (I think to 3.9?), and in it we can use the syntax foo | None instead of Optional[foo]. This way we don't need to import Optional, and the type declaration looks more natural (for example, it's similar to Typescript's foo | null for an optional foo).

[Antonio-RiveroMartnez]

Oh cool! Good to know moving forward then. Thanks @betodealmeida