Support BigQuery authorized datasets

[pietrodn]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment. If the issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If the issue is assigned to a user, that user is claiming responsibility for the issue. If the issue is assigned to "hashibot", a community member has claimed the issue already.

Description

Please support BigQuery authorized datasets in the resources defining the dataset access permissions for BigQuery.

New or Affected Resource(s)

• google_bigquery_dataset
• google_bigquery_dataset_access

Potential Terraform Configuration

# Propose what you think the configuration to take advantage of this feature should look like.
# We may not use it verbatim, but it's helpful in understanding your intent.

resource "google_bigquery_dataset_access" "access" {
  dataset_id    = google_bigquery_dataset.private.dataset_id
  dataset {
    project_id = google_bigquery_table.public.project
    dataset_id = google_bigquery_dataset.public.dataset_id
  }
}

resource "google_bigquery_dataset" "private_dataset" {
  dataset_id                  = "private_dataset"
  location                    = "EU"

  access {
    role          = "OWNER"
    user_by_email = google_service_account.bqowner.email
  }

  access {
    dataset {
      project_id = google_bigquery_table.public.project
      dataset_id = google_bigquery_dataset.public.dataset_id
    }
  }
}

References

• b/209498732
• https://cloud.google.com/bigquery/docs/authorized-datasets
• https://cloud.google.com/bigquery/docs/reference/rest/v2/datasets/update

[slevenick]

Hey @pietrodn is this a new feature that was recently released?

I'm not seeing the structure described here and in the linked documentation guide in the REST API itself. The Dataset object in the API only has view and routine as possible object values. Possibly the documentation is just outdated though

[pietrodn]

@slevenick Yes, it was officially marked as GA on November 12. See the changelog.

[slevenick]

Cool, it should just be an addition of this object to a couple of resources then. Likely both bigquery_dataset & bigquery_dataset_access

[rileykarson]

Note: This may mean changing the BQ IAM resources, assigning size/m instead of size/s

[Ziemowit-LH]

If it matters:
The .tfstate file shows something like this now on importing the a dataset with authorized ones set:

             {
                "domain": "",
                "group_by_email": "",
                "role": "",
                "special_group": "",
                "user_by_email": "",
                "view": []
              },

[rileykarson]

@Ziemowit-LH: Can you file a bug for that?

[jamiet-msm]

Hi @slevenick, I note that this issue has been closed by GoogleCloudPlatform/magic-modules#5700 and there is a commit in this repo at 6416cb6 on 2022-02-11 however I don't see any mention of authorized datasets in the CHANGELOG for 4.11.0 which was released on 2022-02-16.

Hence, a question....is there additional work still to be done to support authorized datasets in the terraform provider?

[rileykarson]

This will go out in the next release, 4.12.0. There's a slight delay between merge and releasing, as we cut release branches a few days prior to a release. 4.12.0 was meant to be released today, but we were unable to qualify a release branch last week due to upstream GCP issues causing 10-20% of our nightly tests to fail. I'm following up on trying to resolve those failures so that we can make 4.12.0 next week.

[jamiet-msm]

Good news, thank you Riley

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.