Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(stdlib/sql): add BigQuery support #2925

Merged
merged 8 commits into from
Aug 4, 2020

Conversation

alespour
Copy link
Contributor

@alespour alespour commented Jun 18, 2020

This PR add BigQuery support to sql package.

  • It uses forked driver bonitoo-io/go-sql-bigquery because the original was incomplete/ready for integration and seems kind of unmaintaned or inactive at least
  • The driver name is bigquery and uses URL format with bigquery scheme as connection string

Done checklist

  • docs/SPEC.md updated
  • Test cases written

@alespour alespour force-pushed the feat/bigquery branch 9 times, most recently from a370aac to f036833 Compare June 19, 2020 13:29
@alespour alespour marked this pull request as ready for review June 19, 2020 13:46
@codecov-commenter
Copy link

codecov-commenter commented Jun 19, 2020

Codecov Report

Merging #2925 into master will increase coverage by 0.31%.
The diff coverage is 51.92%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2925      +/-   ##
==========================================
+ Coverage   51.75%   52.06%   +0.31%     
==========================================
  Files         332      333       +1     
  Lines       41398    41502     +104     
==========================================
+ Hits        21426    21610     +184     
+ Misses      17511    17416      -95     
- Partials     2461     2476      +15     
Impacted Files Coverage Δ
stdlib/sql/bigquery.go 45.65% <45.65%> (ø)
stdlib/sql/from.go 33.62% <100.00%> (+1.16%) ⬆️
stdlib/sql/source_validator.go 78.94% <100.00%> (+3.43%) ⬆️
stdlib/sql/to.go 45.56% <100.00%> (+0.44%) ⬆️
stdlib/universe/pivot.gen.go 10.14% <0.00%> (+0.21%) ⬆️
stdlib/sql/mysql.go 42.47% <0.00%> (+1.76%) ⬆️
stdlib/sql/postgres.go 48.48% <0.00%> (+2.02%) ⬆️
stdlib/sql/mssql.go 53.19% <0.00%> (+28.36%) ⬆️
stdlib/sql/snowflake.go 44.03% <0.00%> (+40.36%) ⬆️
... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update be25922...45f6ddc. Read the comment docs.

@russorat russorat requested review from nathanielc and wolffcm July 15, 2020 15:30
Copy link

@wolffcm wolffcm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this is very fine work, as usual. I just had some questions below.

@@ -38,7 +40,7 @@ require (
github.com/pkg/browser v0.0.0-20180916011732-0a3d74bf9ce4 // indirect
github.com/pkg/errors v0.9.1
github.com/pkg/term v0.0.0-20180730021639-bffc007b7fd5 // indirect
github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious why you needed to bump the version of this? Maybe as I review it will become clear.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was not intentional, I do not know how to make mod tidy not update it... :(

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I guess that maybe one of the new dependencies needed a newer version of it then.

github.com/Azure/go-autorest/autorest v0.10.1
github.com/Azure/go-autorest/autorest/adal v0.8.3
github.com/Azure/go-autorest/autorest/azure/auth v0.4.2
github.com/DATA-DOG/go-sqlmock v1.4.1
github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883
github.com/apache/arrow/go/arrow v0.0.0-20191024131854-af6fa24be0db
github.com/bonitoo-io/go-sql-bigquery v0.3.4
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that you forked solcates repository, but also merged some changes to there, does that mean we can use the original github.com/solcates/go-sql-bigquery now?

Copy link
Contributor Author

@alespour alespour Jul 23, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since I wasn't getting a response from the maintainer who seem to have abandoned the project, I gave up on hoping on the merge and continued with modifications in the bonitoo-io fork. The owner then commented later that he _ had totally forgotten about this project_ and merged the PR, but the fork has diverted more from the original before that, into more pure SQL golang driver unlike the original which also contained some ORM stuff etc.

The original project does not seem to be active (no commits since Oct 27, 2019) and maintained (eg. PR was merge despite failing check due to some missing 3rd party lib).

Copy link

@wolffcm wolffcm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great. Thanks for this fine work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants