BigQueryJsonDataWriter should use specified bigQueryWriteClient when creating JsonStreamWriter

[davidfevre-gouv-nc]

Describe the bug
I have an error java.io.IOException: Your default credentials were not found. on bigQueryTemplate.writeJsonStream when using the configuration parameter cloud.gcp.bigquery.credentials.encoded-key.

After debugging BigQueryTemplate.writeJsonStream(), it seems to me that BigQueryJsonDataWriter should use specified bigQueryWriteClient when creating JsonStreamWriter.

I would suggest to replace lines 80-81 in BigQueryJsonDataWriter constructor from :

    streamWriter =
        JsonStreamWriter.newBuilder(writeStream.getName(), writeStream.getTableSchema()).build();

to

    streamWriter =
        JsonStreamWriter.newBuilder(writeStream.getName(), writeStream.getTableSchema(), bigQueryWriteClient).build();

** Complete error stack**

Caused by: java.io.IOException: Your default credentials were not found. To set up Application Default Credentials for your environment, see https://cloud.google.com/docs/authentication/external/set-up-adc.
at com.google.auth.oauth2.DefaultCredentialsProvider.getDefaultCredentials(DefaultCredentialsProvider.java:127)
at com.google.auth.oauth2.GoogleCredentials.getApplicationDefault(GoogleCredentials.java:152)
at com.google.auth.oauth2.GoogleCredentials.getApplicationDefault(GoogleCredentials.java:124)
at com.google.api.gax.core.GoogleCredentialsProvider.getCredentials(GoogleCredentialsProvider.java:70)
at com.google.api.gax.rpc.ClientContext.create(ClientContext.java:162)
at com.google.cloud.bigquery.storage.v1.stub.GrpcBigQueryWriteStub.create(GrpcBigQueryWriteStub.java:132)
at com.google.cloud.bigquery.storage.v1.stub.BigQueryWriteStubSettings.createStub(BigQueryWriteStubSettings.java:147)
at com.google.cloud.bigquery.storage.v1.BigQueryWriteClient.(BigQueryWriteClient.java:259)
at com.google.cloud.bigquery.storage.v1.BigQueryWriteClient.create(BigQueryWriteClient.java:241)
at com.google.cloud.bigquery.storage.v1.ConnectionWorker.(ConnectionWorker.java:330)
at com.google.cloud.bigquery.storage.v1.StreamWriter.(StreamWriter.java:235)
at com.google.cloud.bigquery.storage.v1.StreamWriter.(StreamWriter.java:57)
at com.google.cloud.bigquery.storage.v1.StreamWriter$Builder.build(StreamWriter.java:823)
at com.google.cloud.bigquery.storage.v1.SchemaAwareStreamWriter.(SchemaAwareStreamWriter.java:105)
at com.google.cloud.bigquery.storage.v1.SchemaAwareStreamWriter.(SchemaAwareStreamWriter.java:56)
at com.google.cloud.bigquery.storage.v1.SchemaAwareStreamWriter$Builder.build(SchemaAwareStreamWriter.java:660)
at com.google.cloud.bigquery.storage.v1.JsonStreamWriter.(JsonStreamWriter.java:50)
at com.google.cloud.bigquery.storage.v1.JsonStreamWriter.(JsonStreamWriter.java:38)
at com.google.cloud.bigquery.storage.v1.JsonStreamWriter$Builder.build(JsonStreamWriter.java:410)
at com.google.cloud.spring.bigquery.core.BigQueryJsonDataWriter.(BigQueryJsonDataWriter.java:81)
at com.google.cloud.spring.bigquery.core.BigQueryTemplate.getBigQueryJsonDataWriter(BigQueryTemplate.java:288)
at com.google.cloud.spring.bigquery.core.BigQueryTemplate.getWriteApiResponse(BigQueryTemplate.java:298)
at com.google.cloud.spring.bigquery.core.BigQueryTemplate.lambda$writeJsonStream$0(BigQueryTemplate.java:256)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
at java.base/java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:317)

[meltsufin]

You're absolutely right. This seems like an oversight. Raising PR to fix as you suggested.

[davidfevre-gouv-nc]

Hi @meltsufin,
Do you have an idea about when this fix will be released ?

[meltsufin]

Hi @meltsufin, Do you have an idea about when this fix will be released ?

@davidfevre-gouv-nc We forgot to merge it before the last release. It should be released in about 2-3 weeks.