google_project googleapi: Error 403: User is not authorized., forbidden.

[jurschel]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment
• If an issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to "hashibot", a community member has claimed the issue already.

Terraform Version

Terraform v0.12.6

• provider.google v2.11.0
• provider.google-beta v2.11.0
• provider.random v2.1.2

Affected Resource(s)

google_project

Terraform Configuration Files

provider "google" {
region = "${var.region}"
credentials = "${file("${var.credentials_file_path}")}"
}

provider "random" {}

resource "random_id" "prod_host_project" {
byte_length = 8
}

resource "random_id" "prod_service_project_1" {
byte_length = 8
}

resource "random_id" "prod_service_project_2" {
byte_length = 8
}

resource "random_id" "non-prod_host_project" {
byte_length = 8
}

resource "random_id" "non-prod_service_project_1" {
byte_length = 8
}

resource "random_id" "non-prod_service_project_2" {
byte_length = 8
}

resource "google_project" "prod_host_project" {
name = "something"
project_id = "something"
org_id = "${var.org_id}"
billing_account = "${var.billing_account_id}"
}

resource "google_project" "non-prod_host_project" {
name = "something"
project_id = "something"
org_id = "${var.org_id}"
billing_account = "${var.billing_account_id}"
}
resource "google_project" "non-prod_service_project_1" {
name = "something"
project_id = "something"
org_id = "${var.org_id}"
billing_account = "${var.billing_account_id}"
}

resource "google_project" "non-prod_service_project_2" {
name = "something"
project_id = "something"
org_id = "${var.org_id}"
billing_account = "${var.billing_account_id}"
}

resource "google_project" "prod_service_project_1" {
name = "something"
project_id = "something"
org_id = "${var.org_id}"
billing_account = "${var.billing_account_id}"
}

resource "google_project" "prod_service_project_2" {
name = "something"
project_id = "something"
org_id = "${var.org_id}"
billing_account = "${var.billing_account_id}"
}

resource "google_project_services" "prod_host_project" {
project = "${google_project.prod_host_project.project_id}"
services = ["compute.googleapis.com", "cloudapis.googleapis.com" , "cloudfunctions.googleapis.com" , "cloudresourcemanager.googleapis.com", "cloudtrace.googleapis.com", "container.googleapis.com", "iam.googleapis.com" , "iamcredentials.googleapis.com", "logging.googleapis.com" , "monitoring.googleapis.com", "stackdriver.googleapis.com", "storage-api.googleapis.com", "storage-component.googleapis.com" ]
}

resource "google_project_services" "prod_service_project_1" {
project = "${google_project.prod_service_project_1.project_id}"
services = ["compute.googleapis.com", "cloudapis.googleapis.com" , "cloudfunctions.googleapis.com" , "cloudresourcemanager.googleapis.com", "cloudtrace.googleapis.com", "container.googleapis.com", "iam.googleapis.com" , "iamcredentials.googleapis.com", "logging.googleapis.com" , "monitoring.googleapis.com", "stackdriver.googleapis.com", "storage-api.googleapis.com", "storage-component.googleapis.com" ]
}

resource "google_project_services" "prod_service_project_2" {
project = "${google_project.prod_service_project_2.project_id}"
services = ["compute.googleapis.com", "cloudapis.googleapis.com" , "cloudfunctions.googleapis.com" , "cloudresourcemanager.googleapis.com", "cloudtrace.googleapis.com", "container.googleapis.com", "iam.googleapis.com" , "iamcredentials.googleapis.com", "logging.googleapis.com" , "monitoring.googleapis.com", "stackdriver.googleapis.com", "storage-api.googleapis.com", "storage-component.googleapis.com" ]
}

resource "google_project_services" "non-prod_host_project" {
project = "${google_project.non-prod_host_project.project_id}"
services = ["compute.googleapis.com", "cloudapis.googleapis.com" , "cloudfunctions.googleapis.com" , "cloudresourcemanager.googleapis.com", "cloudtrace.googleapis.com", "container.googleapis.com", "iam.googleapis.com" , "iamcredentials.googleapis.com", "logging.googleapis.com" , "monitoring.googleapis.com", "stackdriver.googleapis.com", "storage-api.googleapis.com", "storage-component.googleapis.com" ]
}

resource "google_project_services" "non-prod_service_project_1" {
project = "${google_project.non-prod_service_project_1.project_id}"
services = ["compute.googleapis.com", "cloudapis.googleapis.com" , "cloudfunctions.googleapis.com" , "cloudresourcemanager.googleapis.com", "cloudtrace.googleapis.com", "container.googleapis.com", "iam.googleapis.com" , "iamcredentials.googleapis.com", "logging.googleapis.com" , "monitoring.googleapis.com", "stackdriver.googleapis.com", "storage-api.googleapis.com", "storage-component.googleapis.com" ]
}

resource "google_project_services" "non-prod_service_project_2" {
project = "${google_project.non-prod_service_project_2.project_id}"
services = ["compute.googleapis.com", "cloudapis.googleapis.com" , "cloudfunctions.googleapis.com" , "cloudresourcemanager.googleapis.com", "cloudtrace.googleapis.com", "container.googleapis.com", "iam.googleapis.com" , "iamcredentials.googleapis.com", "logging.googleapis.com" , "monitoring.googleapis.com", "stackdriver.googleapis.com", "storage-api.googleapis.com", "storage-component.googleapis.com" ]
}

resource "google_compute_shared_vpc_host_project" "prod_host_project" {
project = "${google_project.prod_host_project.project_id}"
depends_on = ["google_project_services.prod_host_project"]
}

resource "google_compute_shared_vpc_host_project" "non-prod_host_project" {
project = "${google_project.non-prod_host_project.project_id}"
depends_on = ["google_project_services.non-prod_host_project"]
}

resource "google_compute_shared_vpc_service_project" "prod_service_project_1" {
host_project = "${google_project.prod_host_project.project_id}"
service_project = "${google_project.prod_service_project_1.project_id}"

depends_on = ["google_compute_shared_vpc_host_project.prod_host_project",
"google_project_services.prod_service_project_1",
]
}

resource "google_compute_shared_vpc_service_project" "prod_service_project_2" {
host_project = "${google_project.prod_host_project.project_id}"
service_project = "${google_project.prod_service_project_2.project_id}"

depends_on = ["google_compute_shared_vpc_service_project.prod_service_project_2",
"google_project_services.prod_service_project_2",
]
}

resource "google_compute_shared_vpc_service_project" "non-prod_service_project_1" {
host_project = "${google_project.non-prod_host_project.project_id}"
service_project = "${google_project.non-prod_service_project_1.project_id}"

depends_on = ["google_compute_shared_vpc_host_project.non-prod_host_project",
"google_project_services.non-prod_service_project_1",
]
}

resource "google_compute_shared_vpc_service_project" "non-prod_service_project_2" {
host_project = "${google_project.non-prod_host_project.project_id}"
service_project = "${google_project.non-prod_service_project_2.project_id}"

depends_on = ["google_compute_shared_vpc_host_project.non-prod_host_project",
"google_project_services.non-prod_service_project_2",
]
}

resource "google_compute_network" "prod_shared_network" {
name = "prod_shared-network"
auto_create_subnetworks = "true"
project = "${google_compute_shared_vpc_host_project.prod_host_project.project}"

depends_on = ["google_compute_shared_vpc_service_project.prod_service_project_1",
"google_compute_shared_vpc_service_project.prod_service_project_2",
]
}

resource "google_compute_network" "non-prod_shared_network" {
name = "non-prod-shared-network"
auto_create_subnetworks = "true"
project = "${google_compute_shared_vpc_host_project.non-prod_host_project.project}"

depends_on = ["google_compute_shared_vpc_service_project.non-prod_service_project_1",
"google_compute_shared_vpc_service_project.non-prod_service_project_2",
]
}

resource "google_compute_firewall" "prod_shared_network" {
name = "allow-dns-ssh-icmp-http-https"
network = "${google_compute_network.prod_shared_network.self_link}"
project = "${google_compute_network.prod_shared_network.project}"

allow {
protocol = "icmp"
}

allow {
protocol = "tcp"
ports = ["22", "53", "80", "443"]
}
allow {
protocol = "udp"
ports = ["53"]
}
}

resource "google_compute_firewall" "non-prod_shared_network" {
name = "allow-dns-ssh-icmp-http-https"
network = "${google_compute_network.non-prod_shared_network.self_link}"
project = "${google_compute_network.non-prod_shared_network.project}"

allow {
protocol = "icmp"
}
allow {
protocol = "tcp"
ports = ["22", "53", "80", "443"]
}
allow {
protocol = "udp"
ports = ["53"]
}
}

Expected Behavior

This should have created two shared vpc projects and 4 service projects that were attached.

Actual Behavior

Error: error creating project something (something): googleapi: Error 403: User is not authorized., forbidden. If you received a 403 error, make sure you have the roles/resourcemanager.projectCreator permission

Steps to Reproduce

1. terraform apply

Important Factoids

I am using the same credential file that I use in Ansible to create GCP projects on the same org and it has project.creator and billing.user and it works just fine.

[Chupaka]

Your debug log shows that... Wait, where's the debug log?.. Isn't there too many projects with the same name ("something")?

[jurschel]

I apologize, I'm pretty new to submitting bug reports here. There is no "crash" it just produces the error above. Is there some other way I should run terraform apply to get more detailed reasons why it's producing the Error 403? The "something" is just a redaction so I don't put my real config out...

[Chupaka]

https://www.terraform.io/docs/internals/debugging.html

Could you simplify your code (down to two demo projects, for example) and try to repeat the problem? Or the problem disappears in that case? Are there any succeeded actions before the error during the apply? There's no telepathists, sorry.

[jurschel]

Yep there are items in the above config that succeed since when I do a destroy about 6 items are destroyed. If you noticed this config is essentially the example config for shared vpc creation from the GitHub repo. Also I've seen that debug link as well already and it doesn't really apply as I said there is no crashing. Terraform just basically says the credential I'm using does not have project creator. However, I know that not to be the case as ansible uses the exact same credential in a playbook using gcloud commands to create projects...

[jurschel]

So upon your recommendation I actually just tried to use the example with the credentials I have and get the same 403 access denied. So it's just the example and I still get the same 403 error.

Here you can see proof that using gcloud this same service account has no problem creating projects...

[jurschel]

I believe I think I understand more of what is happening. It appears that for org_id and billing_account_id it isn't pulling in a default variable from the variables.tf file. I ran the command and manually entered all the -var= variables and it did start working. If the variables have a default set in them all the other times I've used terraform apply it just works.

[jurschel]

Yes that's the issue. If I hardcode the org_id and billing_account_id into the tf instead of using
org_id = "${var.org_id}"
billing_account = "${var.billing_account_id}"
it works as expected

[jurschel]

Wanted to give an update. It really is strange because during the planning it does pull the org_id and billing_account id as you can see in the below output. It will not however work unless I hardcode those two items. So the credential file isn't changing but if those two items are hardcoded and not variables it all works fine.

google_project.prod_service_project_1 will be created

• resource "google_project" "prod_service_project_1" {
  • app_engine = (known after apply)
  • auto_create_network = true
  • billing_account = "013DB0-XXXXXX-XXXXXX"
  • folder_id = (known after apply)
  • id = (known after apply)
  • name = "something-devops"
  • number = (known after apply)
  • org_id = "694XXXXXXXXX"
  • policy_data = (known after apply)
  • policy_etag = (known after apply)
  • project_id = "something-devops-ac17"
  • skip_delete = (known after apply)

[jurschel]

I just moved to another machine and tried this and it appears to work. Difference is I'm on an Ubuntu laptop now vs a windows machine that all of the above came from.

[paddycarver]

Also I've seen that debug link as well already and it doesn't really apply as I said there is no crashing.

We use the debug log information (notably, these parts:

Terraform has detailed logs which can be enabled by setting the TF_LOG environment variable to any value. This will cause detailed logs to appear on stderr.

You can set TF_LOG to one of the log levels TRACE, DEBUG, INFO, WARN or ERROR to change the verbosity of the logs. TRACE is the most verbose and it is the default if TF_LOG is set to something other than a log level name.

To persist logged output you can set TF_LOG_PATH in order to force the log to always be appended to a specific file when logging is enabled. Note that even when TF_LOG_PATH is set, TF_LOG must be set in order for any logging to be enabled.

If you find a bug with Terraform, please include the detailed log by using a service such as gist.

) to get more information about how Terraform is actually behaving, not just how it's appearing to behave. It helps us get more information about what actual HTTP requests it's making, along with helpful messages for common problems we leave for ourselves in the code. That's why we ask for it regardless of whether there's a crash or not.

In this case, it sounds like there's an upstream Terraform bug around Windows machines and using variable files. I'll dig into this to make sure, and open an issue upstream about it if I can reproduce it.

[jurschel]

I discovered what the problem was just now. Apparently at some point something set TF_VAR_org_id and TF_VAR_billing_account to something that was different than what I needed to use for this plan. I wiped those environment variables out and now it's working. Thanks for your help!

[paddycarver]

Glad this got resolved! Thanks for reporting back.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!