Metadata attribute does not act as a computed attribute

[krlydm]

Terraform, Provider, Kubernetes and Helm Versions

Terraform version: v1.3.9
Provider version: v2.9.0
Kubernetes version: v0.23.0

Affected Resource(s)

• helm_release

Terraform Configuration Files

resource "helm_release" "example" {
  name                = "example"
  repository          = "https://helm.github.io/examples"
  chart               = "hello-world"
  version             = "0.1.0"
  namespace           = "default"
}

resource "aws_ssm_parameter" "example" {
  name  = "example"
  type  = "String"
  tier  = "Standard"
  insecure_value = yamlencode(helm_release.example.metadata)
}

Steps to Reproduce

1. Deploy any helm chart with helm_release and store its metadata attribute somewhere. Run terraform apply.
2. Once deployment is successful, update the helm_release to include a set attribute.

   resource "helm_release" "example" {
     name                = "example"
     repository          = "https://helm.github.io/examples"
     chart               = "hello-world"
     version             = "0.1.0"
     namespace           = "default"
   
     set {
       name  = "test"
       value = "test"
     }
   }

3. Run terraform apply and it will throw an Error: Provider produced inconsistent final plan error.

Expected Behavior

metadata attribute should be treated as a computed attribute after modifying a helm_release. Doing some debugging in the current implementation, I think the problem is that the metadata output piped as input to a resource is not showing as (known after apply) as shown during the first apply. This is a false assumption, because we should only know these values after the modification is completed on the resource.
Based on the documentation, metadata is block status of the deployed release, so values will be known after apply only, at least that's what I expect.

Actual Behavior

Output from first apply:

...
Terraform will perform the following actions:

  # aws_ssm_parameter.example will be created
  + resource "aws_ssm_parameter" "example" {
      + arn            = (known after apply)
      + data_type      = (known after apply)
      + id             = (known after apply)
      + insecure_value = (known after apply)
      + key_id         = (known after apply)
      + name           = "example"
      + tags_all       = (known after apply)
      + tier           = "Standard"
      + type           = "String"
      + value          = (sensitive value)
      + version        = (known after apply)
    }

  # helm_release.example will be created
  + resource "helm_release" "example" {
      + atomic                     = false
      + chart                      = "hello-world"
      + cleanup_on_fail            = false
      + create_namespace           = false
      + dependency_update          = false
      + disable_crd_hooks          = false
      + disable_openapi_validation = false
      + disable_webhooks           = false
      + force_update               = false
      + id                         = (known after apply)
      + lint                       = false
      + manifest                   = (known after apply)
      + max_history                = 0
      + metadata                   = (known after apply)
      + name                       = "example"
      + namespace                  = "default"
      + pass_credentials           = false
      + recreate_pods              = false
      + render_subchart_notes      = true
      + replace                    = false
      + repository                 = "https://helm.github.io/examples"
      + reset_values               = false
      + reuse_values               = false
      + skip_crds                  = false
      + status                     = "deployed"
      + timeout                    = 300
      + verify                     = false
      + version                    = "0.1.0"
      + wait                       = true
      + wait_for_jobs              = false
    }

Plan: 2 to add, 0 to change, 0 to destroy.


------------------------------------------------------------------------

Cost Estimation:

...

------------------------------------------------------------------------

Do you want to perform these actions in workspace "example"?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

helm_release.example: Still creating... [10s elapsed]
helm_release.example: Still creating... [20s elapsed]
helm_release.example: Still creating... [30s elapsed]
helm_release.example: Creation complete after 38s [id=example]
aws_ssm_parameter.example: Creating...
aws_ssm_parameter.example: Creation complete after 1s [id=example]

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

Output from second apply (modified with adding a new set parameter):

...
Terraform will perform the following actions:

  # aws_ssm_parameter.example will be updated in-place
  ~ resource "aws_ssm_parameter" "example" {
        id             = "example"
      ~ insecure_value = <<-EOT
            - "app_version": "1.16.0"
              "chart": "hello-world"
              "name": "example"
              "namespace": "default"
              "revision": 1
          -   "values": "{}"
          +   "values": "null"
              "version": "0.1.0"
        EOT
        name           = "example"
        tags           = {}
      + value          = (sensitive value)
        # (6 unchanged attributes hidden)
    }

  # helm_release.example will be updated in-place
  ~ resource "helm_release" "example" {
        id                         = "example"
        name                       = "example"
        # (27 unchanged attributes hidden)

      + set {
          + name  = "test"
          + value = "test"
        }
    }

Plan: 0 to add, 2 to change, 0 to destroy.


------------------------------------------------------------------------

Cost Estimation:

...

------------------------------------------------------------------------

Do you want to perform these actions in workspace "example"?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

helm_release.example: Still modifying... [id=example, 10s elapsed]
helm_release.example: Still modifying... [id=example, 20s elapsed]
helm_release.example: Modifications complete after 26s [id=example]
╷
│ Error: Provider produced inconsistent final plan
│ 
│ When expanding the plan for aws_ssm_parameter.example to include new values
│ learned so far during apply, provider "registry.terraform.io/hashicorp/aws"
│ produced an invalid new value for .insecure_value: was cty.StringVal("-
│ \"app_version\": \"1.16.0\"\n  \"chart\": \"hello-world\"\n  \"name\":
│ \"example\"\n  \"namespace\": \"default\"\n  \"revision\": 1\n  \"values\":
│ \"null\"\n  \"version\": \"0.1.0\"\n"), but now cty.StringVal("-
│ \"app_version\": \"1.16.0\"\n  \"chart\": \"hello-world\"\n  \"name\":
│ \"example\"\n  \"namespace\": \"default\"\n  \"revision\": 2\n  \"values\":
│ \"{\\\"test\\\":\\\"test\\\"}\"\n  \"version\": \"0.1.0\"\n").
│ 
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
Operation failed: failed running terraform apply (exit 1)

Error says it is related to aws_ssm_parameter, however is independent form the resource you use to input metadata attribute, because I was able to reproduce the same error with other providers and resources as well.

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[BBBmau]

Hello, thank you for opening this issue @krlydm. Based on the trace provided it seems like this is more of a aws provider issue rather than a k8s issue. I would open this in the aws repo instead.

[krlydm]

@BBBmau This is not an AWS provider issue, because you can reproduce with any other resource as I mentioned. Here is an example:

resource "helm_release" "example" {
  name                = "example"
  repository          = "https://helm.github.io/examples"
  chart               = "hello-world"
  version             = "0.1.0"
  namespace           = "default"
}

resource "local_file" "example" {
  content  = yamlencode(helm_release.example.metadata)
  filename = "${path.module}/foo.bar"
}

Output of first apply:

Terraform will perform the following actions:

  # helm_release.example will be created
  + resource "helm_release" "example" {
      + atomic                     = false
      + chart                      = "hello-world"
      + cleanup_on_fail            = false
      + create_namespace           = false
      + dependency_update          = false
      + disable_crd_hooks          = false
      + disable_openapi_validation = false
      + disable_webhooks           = false
      + force_update               = false
      + id                         = (known after apply)
      + lint                       = false
      + manifest                   = (known after apply)
      + max_history                = 0
      + metadata                   = (known after apply)
      + name                       = "example"
      + namespace                  = "default"
      + pass_credentials           = false
      + recreate_pods              = false
      + render_subchart_notes      = true
      + replace                    = false
      + repository                 = "https://helm.github.io/examples"
      + reset_values               = false
      + reuse_values               = false
      + skip_crds                  = false
      + status                     = "deployed"
      + timeout                    = 300
      + verify                     = false
      + version                    = "0.1.0"
      + wait                       = true
      + wait_for_jobs              = false
    }

  # local_file.example will be created
  + resource "local_file" "example" {
      + content              = (known after apply)
      + content_base64sha256 = (known after apply)
      + content_base64sha512 = (known after apply)
      + content_md5          = (known after apply)
      + content_sha1         = (known after apply)
      + content_sha256       = (known after apply)
      + content_sha512       = (known after apply)
      + directory_permission = "0777"
      + file_permission      = "0777"
      + filename             = "./foo.bar"
      + id                   = (known after apply)
    }

Plan: 2 to add, 0 to change, 0 to destroy.

------------------------------------------------------------------------

Cost Estimation:

...

------------------------------------------------------------------------

Do you want to perform these actions in workspace "example"?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

helm_release.example: Still creating... [10s elapsed]
helm_release.example: Still creating... [20s elapsed]
helm_release.example: Still creating... [30s elapsed]
helm_release.example: Creation complete after 34s [id=example]
local_file.example: Creating...
local_file.example: Creation complete after 0s [id=f040e3d00368267f40cc51722e6d5bfcfaba95e8]

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

Adding a set parameter:

resource "helm_release" "example" {
  name                = "example"
  repository          = "https://helm.github.io/examples"
  chart               = "hello-world"
  version             = "0.1.0"
  namespace           = "default"

  set {
    name  = "test"
    value = "test"
  }
}

resource "local_file" "example" {
  content  = yamlencode(helm_release.example.metadata)
  filename = "${path.module}/foo.bar"
}

Output of the second apply:

Terraform will perform the following actions:

  # helm_release.example will be updated in-place
  ~ resource "helm_release" "example" {
        id                         = "example"
        name                       = "example"
        # (27 unchanged attributes hidden)

      + set {
          + name  = "test"
          + value = "test"
        }
    }

  # local_file.example will be created
  + resource "local_file" "example" {
      + content              = <<-EOT
            - "app_version": "1.16.0"
              "chart": "hello-world"
              "name": "example"
              "namespace": "default"
              "revision": 1
              "values": "null"
              "version": "0.1.0"
        EOT
      + content_base64sha256 = (known after apply)
      + content_base64sha512 = (known after apply)
      + content_md5          = (known after apply)
      + content_sha1         = (known after apply)
      + content_sha256       = (known after apply)
      + content_sha512       = (known after apply)
      + directory_permission = "0777"
      + file_permission      = "0777"
      + filename             = "./foo.bar"
      + id                   = (known after apply)
    }

Plan: 1 to add, 1 to change, 0 to destroy.
------------------------------------------------------------------------

Cost Estimation:

...

------------------------------------------------------------------------

Do you want to perform these actions in workspace "example"?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

helm_release.example: Still modifying... [id=example, 10s elapsed]
helm_release.example: Still modifying... [id=example, 20s elapsed]
helm_release.example: Modifications complete after 29s [id=example]
╷
│ Error: Provider produced inconsistent final plan
│ 
│ When expanding the plan for local_file.example to include new values
│ learned so far during apply, provider
│ "registry.terraform.io/hashicorp/local" produced an invalid new value for
│ .content: was cty.StringVal("- \"app_version\": \"1.16.0\"\n  \"chart\":
│ \"hello-world\"\n  \"name\": \"example\"\n  \"namespace\": \"default\"\n
│ \"revision\": 1\n  \"values\": \"null\"\n  \"version\": \"0.1.0\"\n"), but
│ now cty.StringVal("- \"app_version\": \"1.16.0\"\n  \"chart\":
│ \"hello-world\"\n  \"name\": \"example\"\n  \"namespace\": \"default\"\n
│ \"revision\": 2\n  \"values\": \"{\\\"test\\\":\\\"test\\\"}\"\n
│ \"version\": \"0.1.0\"\n").
│ 
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
Operation failed: failed running terraform apply (exit 1)

Do you think this is an issue with both AWS and local file resources?

[BBBmau]

There seems to be something very odd happening within the metadata block.

When running terraform apply with helm_release resource by itself the metadata has value set as {} but on the second apply it changes to null with no mention of a diff occurring when running the second terraform apply

"metadata": [
              {
                "app_version": "1.16.0",
                "chart": "hello-world",
                "name": "example",
                "namespace": "default",
                "revision": 1,
                "values": "null",
                "version": "0.1.0"
              }
            ],

I'll be opening this issue and investigating it a bit further. This should be the main reason why the Provider produced inconsistent final plan is occurring