Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BigQuery: removes LoadJob error for autodetect=True and schema set #4213

Merged

Conversation

alixhami
Copy link
Contributor

Validations in the LoadJob class were set to disallow a schema to be set while autodetect is True. This raised an error when running client.get_job() and client.list_jobs() for LoadJobs that were created with autodetect=True because the jobs came back with the schema having been set. The additional code added in system tests shows how to reproduce the bug.

@alixhami alixhami added the api: bigquery Issues related to the BigQuery API. label Oct 18, 2017
@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Oct 18, 2017
Copy link
Contributor

@jba jba left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK I guess, but I wonder if there's a way to distinguish between a user setting a schema when autodetect=True, which should be an error, vs. reading a LoadJob. Maybe the from_api_repr method could set _schema instead of schema to avoid triggering the check?

@tswast
Copy link
Contributor

tswast commented Oct 18, 2017

You're right that it could be possible to bypass the check on load. It would be to change

config.schema = _parse_schema_resource(schema)

to

config._schema = _parse_schema_resource(schema)

I think the server-side error message is pretty clear if someone does try to set a schema and use autodetect. I'm okay removing the client-side check.

@tswast tswast merged commit f192cb6 into googleapis:bigquery-b2 Oct 18, 2017
@alixhami alixhami deleted the bq-fix-load-job-schema-setter branch January 5, 2018 01:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the BigQuery API. cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants