Error when updating a service backend: missing required field 'ServiceVersion'

[cdodd]

I'm getting an issue when I attempt to update a backend of an existing service. When updating the backend I get the error missing required field 'ServiceVersion'. However, if I run pulumi up a second time it appears to update the service, but doesn't, I can see the service is still on version 1 in the Fastly UI. If I run pulumi up for a third time I get the error again, then the cycle continues.

Steps to reproduce

I've created an example project here https://github.com/cdodd/fastly-issue

1. Run pulumi up and create the service
2. Update the backend origin name to something else (e.g. "origin2.myservice.com")
3. Run pulumi up, apply the change, you should receive the error missing required field 'ServiceVersion'
4. Run pulumi up, apply the change, change will show complete.
5. Verify service is not updated in Fastly UI
6. Run pulumi up, get the error message again
7. Cycle continues

Example output on failure

$ pulumi up

Please choose a stack, or create a new one: dev
Previewing update (dev):
     Type                       Name              Plan       Info
     pulumi:pulumi:Stack        fastly-issue-dev
 ~   └─ fastly:index:Servicev1  my-service        update     [diff: ~backends]

Resources:
    ~ 1 to update
    1 unchanged

Do you want to perform this update? yes
Updating (dev):
     Type                       Name              Status                  Info
     pulumi:pulumi:Stack        fastly-issue-dev  **failed**              1 error
 ~   └─ fastly:index:Servicev1  my-service        **updating failed**     [diff: ~backends]; 1 err

Diagnostics:
  pulumi:pulumi:Stack (fastly-issue-dev):
    error: update failed

  fastly:index:Servicev1 (my-service):
    error: 1 error occurred:
    	* updating urn:pulumi:dev::fastly-issue::fastly:index/servicev1:Servicev1::my-service: 1 error occurred:
    	* missing required field 'ServiceVersion'

Resources:
    1 unchanged

Duration: 1s

Example output on update that appears to work, but fails

$ pulumi up

Please choose a stack, or create a new one: dev
Previewing update (dev):
     Type                       Name              Plan       Info
     pulumi:pulumi:Stack        fastly-issue-dev
 ~   └─ fastly:index:Servicev1  my-service        update     [diff: ~backends]

Resources:
    ~ 1 to update
    1 unchanged

Do you want to perform this update? yes
Updating (dev):
     Type                       Name              Status      Info
     pulumi:pulumi:Stack        fastly-issue-dev
 ~   └─ fastly:index:Servicev1  my-service        updated     [diff: ~backends]

Resources:
    ~ 1 updated
    1 unchanged

Duration: 18s

[leezen]

@cdodd Do you mind providing what the state looks like (pulumi stack export > export.json) for the Servicev1 resource? I'm curious what clonedVersion is set to.

[farvour]

This seems like it is due to the following Terraform change:
fastly/terraform-provider-fastly@a893204

It appears that Pulumi is never going to work correctly with this since the logic in the underlying provider added a an OR condition if the value is not set and I'm unable to actually pass it in as a parameter (nor would I want to).

I had to roll back to an older provider (3.4.x) to work with the pulumi provider again.

[StarpTech]

Hi, I run into the same issue. It's unusable.

[leezen]

My understanding from looking at that change is that the only scenario in which you're stuck is if you're using a version of the provider prior to 3.0 since that's when the change mentioned above gets picked up. If you're upgrading from a previous version where the state didn't include ServiceVersion then you should be able to continue via pulumi up --refresh to pick up the correct value and move forward. Does that work for you?

[StarpTech]

My understanding from looking at that change is that the only scenario in which you're stuck is if you're using a version of the provider prior to 3.0

I run into that issue with the latest version. I started a few days ago with the development. Did you test it?

[StarpTech]

@leezen doesn't work

1. Upgrade
2. pulumi refresh
3. pulumi up

Diagnostics:
  fastly:index:ServiceCompute (service):
    error: 1 error occurred:
        * updating urn:pulumi:dev::service::fastly:index/serviceCompute:ServiceCompute::service: 1 error occurred:
        * missing required field 'ServiceVersion'

[leezen]

Apologies -- I thought that refresh should bring this value into state, but I was wrong and it does not. Even with using legacy plugin search. A few experiments:

1. Create a Servicev1 from scratch at fastly version 3.10.0 -- this is successful.
2. Make a modification and run an update. This fails w/ the ServiceVersion issue mentioned above.
3. Change a value and run an update. This does not work.
4. Create a Servicev1 from scratch at fastly version 3.4.0 -- this is successful.
5. Change a value and run an update. This works.
6. Upgrade to 3.10.0, make a modification, and run an update. This fails w/ the ServiceVersion issue mentioned above.

Looking at the state, it doesn't appear we ever actually write activeVersion or clonedVersion to state in the newer version of the provider (whereas in 3.4.0, it's present in the state)

[leezen]

I think the issue is actually introduced via fastly/terraform-provider-fastly#388

[farvour]

@leezen thank you for digging further into this. Do you know if this is something that Pulumi's tf2pulumi has to account for or is this something the upstream provider is going to have to be patched to compensate for? It seems more of a pulumi issue as you describe, the value never gets written to state even with a refresh and I think that logic is independent of the upstream provider since the pulumi generator creates a bridge library if I recall...

What is the recommended course-of-action to ensure those of us who were using the provider pre-3.10.0 so that we can upgrade to 3.10.0? It appears that 3.10.0 simply doesn't work at all with Pulumi at this point since you can't even run an update against a freshly created service w/ the 3.10.0 provider...

[oatnog]

I'm also running into this. It's my first time using Pulumi.

[shaidar]

Any updates on this as I just tried 4.0.0 version of the provider with a clean state and still seeing the same error.

[shaidar]

Tried downgrading to 3.7.0 and then to 3.4.0 and in both cases received a not so descriptive error:

error: unknown resource type '<nil>'

So currently am blocked on being able to use this provider.

[jkodroff]

@shaidar Can you provide some sample code? I believe the code that the OP provided is pre-V4. (I don't see ServiceV1 in the provider docs anymore.)

[shaidar]

Tested out new version of pulumi-fastly provider (4.0.1) and ran into the same issue. Ran pulumi up with the minimal example below. It created the Fastly distro successfully, however after tweaking the code by changing the domain comment, it threw the infamous ServiceVersion error

# Original minimal example

import pulumi
import pulumi_fastly as fastly

ol_devops_qa = fastly.ServiceVcl("ol-devops-qa",
    backends=[fastly.ServiceVclBackendArgs(
        address="127.0.0.1",
        name="localhost",
        port=80,
    )],
    domains=[fastly.ServiceVclDomainArgs(
        comment="ol-devops-qa",
        name="ol-devops-qa.odl.mit.edu",
    )],
    force_destroy=True)

# Tweaked minimal example by changing comment

import pulumi
import pulumi_fastly as fastly

ol_devops_qa = fastly.ServiceVcl("ol-devops-qa",
    backends=[fastly.ServiceVclBackendArgs(
        address="127.0.0.1",
        name="localhost",
        port=80,
    )],
    domains=[fastly.ServiceVclDomainArgs(
        comment="ol-devops-test",
        name="ol-devops-qa.odl.mit.edu",
    )],
    force_destroy=True)

[shaidar]

Just checking in to see whether someone had a chance to come up with a potential solution to this?

[blarghmatey]

So, I've been digging through the Terraform provider code and the schema.json for this plugin to try and understand what is happening. The overall problem is that the ServiceVersion parameter that the TF provider and Fastly library are relying on is an implicit attribute that the TF provider computes automatically. My assumption is that the missing piece is that the Pulumi provider needs to access that attribute in the generated state from the TF provider and include it in its own state management to pass back and forth.

Because it's not an explicit attribute that is managed through the schema.json I'm not sure if that's possible/straightforward? Perhaps we just need to add the ServiceVersion to the schema.json to see if we can pass it through explicitly to the TF provider? Or is there some other configuration that we can specify to control which state values are saved from the TF provider output?

[jkodroff]

Still experiencing this issue after releasing v4.0.2, in which I attempted to correct a version mismatch in our fork of terraform-plugin-sdk. Posting some additional details on the legwork I've done on this issue so far:

After the first update to comment only, we get the following details which erroneously show that name has changed:

Do you want to perform this update? details
  pulumi:pulumi:Stack: (same)
    [urn=urn:pulumi:dev::fastly-pulumi::pulumi:pulumi:Stack::fastly-pulumi-dev]
    ~ fastly:index/serviceVcl:ServiceVcl: (update)
        [id=2FSJizTKzDIqzqqGd8XtVW]
        [urn=urn:pulumi:dev::fastly-pulumi::fastly:index/serviceVcl:ServiceVcl::test]
        [provider=urn:pulumi:dev::fastly-pulumi::pulumi:providers:fastly::default_4_0_2::c9deac51-03d5-4ca4-8e7e-4d8b31e701d0]
      ~ domains: [
          ~ [0]: {
                  ~ comment: "foo" => "foo2"
                  ~ name   : "demo2.myservice.com" => "demo2.myservice.com"
                }
        ]

This update fails. Then, after running pulumi up again, we get the following (correct) details:

Do you want to perform this update? details
  pulumi:pulumi:Stack: (same)
    [urn=urn:pulumi:dev::fastly-pulumi::pulumi:pulumi:Stack::fastly-pulumi-dev]
    ~ fastly:index/serviceVcl:ServiceVcl: (update)
        [id=2FSJizTKzDIqzqqGd8XtVW]
        [urn=urn:pulumi:dev::fastly-pulumi::fastly:index/serviceVcl:ServiceVcl::test]
        [provider=urn:pulumi:dev::fastly-pulumi::pulumi:providers:fastly::default_4_0_2::c9deac51-03d5-4ca4-8e7e-4d8b31e701d0]
      ~ domains        : [
          ~ [0]: {
                  ~ comment   : "foo" => "foo2"
                }
        ]

This update succeeds.

Some possibly relevant links from the TF provider:

• Definition of active_version (which presumably translates to ServiceVersion when the call is made to the to the Fastly API): https://github.com/fastly/terraform-provider-fastly/blob/8fe4296b28f386dfadb1f159c57c9379569c4187/fastly/base_fastly_service.go#L100-L108
• There's a custom diff function that might have something to do with the incorrect behavior: https://github.com/fastly/terraform-provider-fastly/blob/8fe4296b28f386dfadb1f159c57c9379569c4187/fastly/base_fastly_service.go#L60
• Here's the definition of domain, where we see the erroneous diff: https://github.com/fastly/terraform-provider-fastly/blob/8fe4296b28f386dfadb1f159c57c9379569c4187/fastly/block_fastly_service_domain.go#L27

Last thing I can think of is maybe that domain and backend upstream are being translated as arrays in Pulumi, but they seem to be singleton blocks upstream, so maybe we're missing a MaxItemsOne.

Folks, thanks for your patience on this. I'm gonna ask for some additional help from some colleagues and we'll get this figured out.

[blarghmatey]

@jkodroff thank you for the detailed reply. Just curious if you and/or your colleagues have had the opportunity to look into this further? Is there anything that an external contributor can do to help keep this moving forward?

[jkodroff]

We did look into this further. @pgavlin has a PR on our fork of the upstream provider that should fix the (very difficult to diagnose) cause of the error and I'll push a fix out today.

[blarghmatey]

Thanks for the update, that's great news! We'll help to test it out once you have the time to push out the updated release.

[jkodroff]

@blarghmatey This should be fixed in pulumi-fastly v4.0.3.

[kevinrobbins]

This appears to be fixed for me in v4.0.3!

[stack72]

Thanks for taking the time to see this through @jkodroff