Add advanced_machine_features to GCE Instances

[upodroid]

Fixes: hashicorp/terraform-provider-google#9251

If this PR is for Terraform, I acknowledge that I have:

• Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
• Generated Terraform, and ran make test and make lint to ensure it passes unit and linter tests.
• Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
• Ran relevant acceptance tests (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
• Read the Release Notes Guide before writing my release note below.

Release Note Template for Downstream PRs (will be copied)

compute: added `advanced_machine_features` to `google_compute_instance`

[modular-magician]

Hello! I am a robot who works on Magic Modules PRs.

I have detected that you are a community contributor, so your PR will be assigned to someone with a commit-bit on this repo for initial review.

Thanks for your contribution! A human will be with you soon.

@c2thorn, please review this PR or find an appropriate assignee.

[upodroid]

According to the docs, the fields are updatable but it doesn't work.

https://cloud.google.com/compute/docs/instances/nested-virtualization/enabling#enabling_nested_virtualization_directly
https://cloud.google.com/compute/docs/instances/disabling-smt#limitations

---[ REQUEST ]---------------------------------------
PUT /compute/beta/projects/projects/REDACTED/zones/us-central1-a/instances/tf-test-g0fq94jg1s?alt=json&prettyPrint=false HTTP/1.1
Host: www.googleapis.com
User-Agent: google-api-go-client/0.5 Terraform/0.14.7 (+https://www.terraform.io) Terraform-Plugin-SDK/2.5.0 terraform-provider-google-beta/acc
Content-Length: 2083
Content-Type: application/json
X-Goog-Api-Client: gl-go/1.15.5 gdcl/20210502
Accept-Encoding: gzip

{
 "advancedMachineFeatures": {
  "enableNestedVirtualization": true,
  "threadsPerCore": 1
 },
 "cpuPlatform": "Unknown CPU Platform",
 "creationTimestamp": "2021-06-07T06:10:53.918-07:00",
 "disks": [
  {
   "autoDelete": true,
   "boot": true,
   "deviceName": "persistent-disk-0",
   "diskSizeGb": "10",
   "guestOsFeatures": [
    {
     "type": "UEFI_COMPATIBLE"
    },
    {
     "type": "VIRTIO_SCSI_MULTIQUEUE"
    }
   ],
   "interface": "SCSI",
   "kind": "compute#attachedDisk",
   "licenses": [
    "https://www.googleapis.com/compute/beta/projects/debian-cloud/global/licenses/debian-10-buster"
   ],
   "mode": "READ_WRITE",
   "source": "https://www.googleapis.com/compute/beta/projects/REDACTED/zones/us-central1-a/disks/tf-test-g0fq94jg1s",
   "type": "PERSISTENT"
  }
 ],
 "fingerprint": "Fut0wt3-_Cc=",
 "id": "2036102861302244402",
 "kind": "compute#instance",
 "labelFingerprint": "42WmSpB8rSM=",
 "lastStartTimestamp": "2021-06-07T06:11:01.000-07:00",
 "lastStopTimestamp": "2021-06-07T06:13:24.791-07:00",
 "machineType": "https://www.googleapis.com/compute/beta/projects/REDACTED/zones/us-central1-a/machineTypes/n1-standard-2",
 "metadata": {
  "fingerprint": "2ore-lqyvN0=",
  "kind": "compute#metadata"
 },
 "name": "tf-test-g0fq94jg1s",
 "networkInterfaces": [
  {
   "fingerprint": "jHpMQX6gdrM=",
   "kind": "compute#networkInterface",
   "name": "nic0",
   "network": "https://www.googleapis.com/compute/beta/projects/REDACTED/global/networks/default",
   "networkIP": "10.128.0.20",
   "subnetwork": "https://www.googleapis.com/compute/beta/projects/REDACTED/regions/us-central1/subnetworks/default"
  }
 ],
 "scheduling": {
  "automaticRestart": true,
  "onHostMaintenance": "MIGRATE"
 },
 "selfLink": "https://www.googleapis.com/compute/beta/projects/REDACTED/zones/us-central1-a/instances/tf-test-g0fq94jg1s",
 "shieldedInstanceConfig": {
  "enableIntegrityMonitoring": true,
  "enableVtpm": true
 },
 "shieldedInstanceIntegrityPolicy": {
  "updateAutoLearnPolicy": true
 },
 "shieldedVmConfig": {
  "enableIntegrityMonitoring": true,
  "enableVtpm": true
 },
 "shieldedVmIntegrityPolicy": {
  "updateAutoLearnPolicy": true
 },
 "status": "TERMINATED",
 "tags": {
  "fingerprint": "42WmSpB8rSM="
 },
 "zone": "https://www.googleapis.com/compute/beta/projects/REDACTED/zones/us-central1-a"
}

-----------------------------------------------------
2021/06/07 14:13:34 [DEBUG] Google API Response Details:
---[ RESPONSE ]--------------------------------------
HTTP/2.0 400 Bad Request
Alt-Svc: h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Mon, 07 Jun 2021 13:13:34 GMT
Server: ESF
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

{
  "error": {
    "code": 400,
    "message": "Invalid value for field 'resource.advancedMachineFeatures': '{  \"enableNestedVirtualization\": true,  \"threadsPerCore\": 1}'. Updating 'advancedMachineFeatures' is not supported.",
    "errors": [
      {
        "message": "Invalid value for field 'resource.advancedMachineFeatures': '{  \"enableNestedVirtualization\": true,  \"threadsPerCore\": 1}'. Updating 'advancedMachineFeatures' is not supported.",
        "domain": "global",
        "reason": "invalid"
      }
    ]
  }
}

-----------------------------------------------------
2021/06/07 14:13:34 [DEBUG] Retry Transport: Stopping retries, last request failed with non-retryable error: googleapi: got HTTP response code 400 with body: HTTP/2.0 400 Bad Request
Alt-Svc: h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Mon, 07 Jun 2021 13:13:34 GMT
Server: ESF
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

{
  "error": {
    "code": 400,
    "message": "Invalid value for field 'resource.advancedMachineFeatures': '{  \"enableNestedVirtualization\": true,  \"threadsPerCore\": 1}'. Updating 'advancedMachineFeatures' is not supported.",
    "errors": [
      {
        "message": "Invalid value for field 'resource.advancedMachineFeatures': '{  \"enableNestedVirtualization\": true,  \"threadsPerCore\": 1}'. Updating 'advancedMachineFeatures' is not supported.",
        "domain": "global",
        "reason": "invalid"
      }
    ]
  }
}
2021/06/07 14:13:34 [DEBUG] Retry Transport: Returning after 1 attempts
2021/06/07 14:13:34 [WARN] Got error running Terraform: exit status 1

Error: Error updating instance: googleapi: Error 400: Invalid value for field 'resource.advancedMachineFeatures': '{  "enableNestedVirtualization": true,  "threadsPerCore": 1}'. Updating 'advancedMachineFeatures' is not supported., invalid

  on terraform_plugin_test.tf line 7, in resource "google_compute_instance" "foobar":
   7: resource "google_compute_instance" "foobar" {


    provider_test.go:276: Step 3/4 error: Error running apply: exit status 1
        
        Error: Error updating instance: googleapi: Error 400: Invalid value for field 'resource.advancedMachineFeatures': '{  "enableNestedVirtualization": true,  "threadsPerCore": 1}'. Updating 'advancedMachineFeatures' is not supported., invalid
        
          on terraform_plugin_test.tf line 7, in resource "google_compute_instance" "foobar":
           7: resource "google_compute_instance" "foobar" {
        
        
2021/06/07 14:13:38 [INFO] Authenticating using DefaultClient...

Do you want to wait for API support or shall I mark the fields as ForceNew ?

I'll update the Instance Template after this issue is fixed.

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 4 files changed, 186 insertions(+), 3 deletions(-))
Terraform Beta: Diff ( 4 files changed, 186 insertions(+), 3 deletions(-))
TF Conversion: Diff ( 1 file changed, 23 insertions(+))

[modular-magician]

I have triggered VCR tests based on this PR's diffs. See the results here: "https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=190963"

[upodroid]

I raised the Instance template as a separate PR. Templates can't be updated so it is easier to work with #4850

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 4 files changed, 185 insertions(+), 3 deletions(-))
Terraform Beta: Diff ( 4 files changed, 185 insertions(+), 3 deletions(-))
TF Conversion: Diff ( 1 file changed, 23 insertions(+))

[modular-magician]

I have triggered VCR tests based on this PR's diffs. See the results here: "https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=190966"

[c2thorn]

Let me reach out to the API team, but in the meantime it shouldn't be a breaking change to label this block as ForceNew, and then switch to an updatable version afterwards.

[c2thorn]

We should utilize AtLeastOneOf here to avoid an empty block

[karlkfi]

If we add AtLeastOneOf here, we should probably also add it to the template to keep them in sync with similar diff/recreate behavior.
#4850

[karlkfi]

AtLeastOneOf might also resolve this issue: hashicorp/terraform-provider-google#9436
But it would make the block required for users to set, right?

[c2thorn]

My intention with AtLeastOneOf is to apply it to enable_nested_virtualization and threads_per_core, so that if advanced_machine_features is specified at all, one of those fields will be required.

[upodroid]

It has been a while and docs have been unclear. I'll rebase this and test it again on Monday

[karlkfi]

This description could probably be clearer.

Perhaps:

The number of threads per physical core. To disable simultaneous multithreading (SMT) set this to 1. If unset, the maximum number of threads per core supported by the underlying processor is used by default. The number of threads per core is not configurable on machine types that have fewer than 2 vCPUs.`,

It's unclear to me whether this field will be populated with the maximum number of threads per core supported by the underlying processor or not. I assume, since it's marked as not computed that the API never returns a value that wasn't specified by the user. Is this correct?

It's also unclear from the docs when exactly the API will error. Does it allow specifying 1 for a 1 vCPU machine type, like n1-standard-1, or is it actually unconfigurable and error if any value is specified in that case? We might need to change "not configurable" to "must be 1 or unspecified".

[karlkfi]

After some testing, I don't think the API actually computes any of these fields, for instances or instance templates.
hashicorp/terraform-provider-google#9436 (comment)

So I would recommend removing Computed here and making a new PR to fix instance templates.

This also probably means we don't need AtLeastOneOf. The API does not require these fields to be specified.

[c2thorn]

@karlkfi

So I would recommend removing Computed here and making a new PR to fix instance templates.

Thanks for looking into this. I agree if there is no case where the API returns a value, we should not mark it computed.

This also probably means we don't need AtLeastOneOf. The API does not require these fields to be specified.

This may be true, but it is our own team's convention to not allow an empty block to be specified. Like

advanced_machine_features {}

[upodroid]

The API bug seems to have been fixed now.

I tried 4 cases:

• No advanced_machine_config block set.
• Both advanced_machine_config nested fields have been set. API returns values for both.
• advanced_machine_config.0.enable_nested_virtualisation set to true alone. API doesn't return value for threads_per_core which is good and removes the need for it to be Computed.
• advanced_machine_config.0.threads_per_core set to 1 alone. API doesn't return value for enable_nested_virtualisation which is good and removes the need for it to be Computed.

This was tested with the v1 API. I hope beta works the same way too 😃

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 5 files changed, 159 insertions(+), 7 deletions(-))
Terraform Beta: Diff ( 6 files changed, 160 insertions(+), 8 deletions(-))
TF Conversion: Diff ( 1 file changed, 2 deletions(-))

[c2thorn]

LGTM, thanks. Would you mind doing one last rebase off of master? I made changes to our CI and I want to make sure there aren't any hiccups. @upodroid

[upodroid]

Done

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 5 files changed, 159 insertions(+), 7 deletions(-))
Terraform Beta: Diff ( 6 files changed, 160 insertions(+), 8 deletions(-))
TF Conversion: Diff ( 1 file changed, 2 deletions(-))