Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GCSToBigQueryOperator no longer uses field_delimiter or time_partitioning #27842

Closed
2 tasks done
mdering opened this issue Nov 22, 2022 · 2 comments · Fixed by #27961
Closed
2 tasks done

GCSToBigQueryOperator no longer uses field_delimiter or time_partitioning #27842

mdering opened this issue Nov 22, 2022 · 2 comments · Fixed by #27961
Labels
area:providers kind:bug This is a clearly a bug provider:google Google (including GCP) related issues

Comments

@mdering
Copy link
Contributor

mdering commented Nov 22, 2022 •
edited
Loading

Apache Airflow Provider(s)

google

Versions of Apache Airflow Providers

google=8.5.0

Apache Airflow version

2.4.3

Operating System

Debian GNU/Linux 11 (bullseye)

Deployment

Other Docker-based deployment

Deployment details

No response

What happened

The newest version of the google providers no longer provides the field_delimiter or time_partitioning fields to the bq job configuration for the GCStoBQ transfers. Looking at the code it seems like this behavior was removed during the change to use deferrable operations

What you think should happen instead

These fields should continue to be provided

How to reproduce

No response

Anything else

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct
@mdering mdering added area:providers kind:bug This is a clearly a bug labels Nov 22, 2022
@boring-cyborg
Copy link

boring-cyborg bot commented Nov 22, 2022

Thanks for opening your first issue here! Be sure to follow the issue template!

@uranusjr uranusjr added the provider:google Google (including GCP) related issues label Nov 23, 2022
@Wolff-Lucas
Copy link

I have the same issues with the 8.0.5.

@mdering mdering mentioned this issue Nov 28, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area:providers kind:bug This is a clearly a bug provider:google Google (including GCP) related issues
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix: re-enable use of parameters in gcs_to_bq which had been disabled mdering/airflow
3 participants
@uranusjr @mdering @Wolff-Lucas