[BUG] publishing API failes with Microsoft.WindowsAzure.ApiManagement.Management.Core.Exceptions.PreconditionFailedException

[shawnyu5]

Release version

v5.1.1

Describe the bug

Publishing an api with a revision included in the name causes the publisher to fail with the following logs:

info: Publisher[0]
      Putting API my-api-v1-b2b;rev=1...
dbug: PutRestResource[0]
      Beginning request to put REST resource URI https://management.azure.com/subscriptions/***/resourceGroups/***/providers/Microsoft.ApiManagement/service/poc-apiops-dev/apis/my-api-v1-b2b;rev=1?api-version=2022-04-01-preview...
dbug: PutRestResource[0]
      Beginning request to put REST resource URI https://management.azure.com/subscriptions/***/resourceGroups/***/providers/Microsoft.ApiManagement/service/poc-apiops-dev/apis/my-api-v1-b2b?api-version=2022-04-01-preview...
dbug: PutRestResource[0]
      Successfully put REST resource at URI https://management.azure.com/subscriptions/***/resourceGroups/***/providers/Microsoft.ApiManagement/service/poc-apiops-dev/apis/my-api-v1-b2b;rev=1?api-version=2022-04-01-preview.
crit: Publisher[0]
      System.Net.Http.HttpRequestException: HTTP request to URI https://management.azure.com/subscriptions/***/resourceGroups/***/providers/Microsoft.ApiManagement/service/poc-apiops-dev/apis/my-api-v1-b2b?api-version=2022-04-01-preview&asyncId=65d505a9dab0190ea44116c4&asyncCode=200&t=63844056233[460](https://github.corp.fundserv.com/FundSERV/api_ops_poc/actions/runs/17734/job/47941#step:8:461)9245&c=MIIHADCCBeigAwIBAgITfARmUs4KPxjtgZmnLQAABGZSzjANBgkqhkiG9w0BAQsFADBEMRMwEQYKCZImiZPyLGQBGRYDR0JMMRMwEQYKCZImiZPyLGQBGRYDQU1FMRgwFgYDVQQDEw9BTUUgSW5mcmEgQ0EgMDUwHhcNMjQwMjAxMDAxMjE1WhcNMjUwMTI2MDAxMjE1WjBAMT4wPAYDVQQDEzVhc3luY29wZXJhdGlvbnNpZ25pbmdjZXJ0aWZpY2F0ZS5tYW5hZ2VtZW50LmF6dXJlLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMq-pifiTvVwj7Jv-8SkcHMQ5okiyv7c79aKWRSvZiipjV4GUeL1152yyCcfI8C2GBVqN1Lm_FZAPul8xhXyxs-w73Njmk1UlAVg_D-P2RD-cx5q-pbvLdGq7b-cMW_f2EogXnhpGACn8c3m3-MsrMswCt31m-5GidX__OHaufvROTAPzrjYa7rqbASotZKRhrCZID9XLjrgu1mEsyG0ierPSeAI-zTICLU-kty3pwttNZjw_iCD2A4BCN9GWZZhApX5hyZLVaDt7nHEDtNliLti1kq_XB7cxyNFmtZSDAZRmJWHfNQomt5Co_qXyxPCIVGhlZzVqNmj_RIZLkVYLakCAwEAAaOCA-0wggPpMCcGCSsGAQQBgjcVCgQaMBgwCgYIKwYBBQUHAwEwCgYIKwYBBQUHAwIwPQYJKwYBBAGCNxUHBDAwLgYmKwYBBAGCNxUIhpDjDYTVtHiE8Ys-hZvdFs6dEoFggvX2K4Py0SACAWQCAQowggHLBggrBgEFBQcBAQSCAb0wggG5MGMGCCsGAQUFBzAChldodHRwOi8vY3JsLm1pY3Jvc29mdC5jb20vcGtpaW5mcmEvQ2VydHMvQ08xUEtJSU5UQ0EwMS5BTUUuR0JMX0FNRSUyMEluZnJhJTIwQ0ElMjAwNS5jcnQwUwYIKwYBBQUHMAKGR2h0dHA6Ly9jcmwxLmFtZS5nYmwvYWlhL0NPMVBLSUlOVENBMDEuQU1FLkdCTF9BTUUlMjBJbmZyYSUyMENBJTIwMDUuY3J0MFMGCCsGAQUFBzAChkdodHRwOi8vY3JsMi5hbWUuZ2JsL2FpYS9DTzFQS0lJTlRDQTAxLkFNRS5HQkxfQU1FJTIwSW5mcmElMjBDQSUyMDA1LmNydDBTBggrBgEFBQcwAoZHaHR0cDovL2NybDMuYW1lLmdibC9haWEvQ08xUEtJSU5UQ0EwMS5BTUUuR0JMX0FNRSUyMEluZnJhJTIwQ0ElMjAwNS5jcnQwUwYIKwYBBQUHMAKGR2h0dHA6Ly9jcmw0LmFtZS5nYmwvYWlhL0NPMVBLSUlOVENBMDEuQU1FLkdCTF9BTUUlMjBJbmZyYSUyMENBJTIwMDUuY3J0MB0GA1UdDgQWBBQzzZmU3WY_Pa_PtbBddSxWOdCGmjAOBgNVHQ8BAf8EBAMCBaAwggEmBgNVHR8EggEdMIIBGTCCARWgggERoIIBDYY_aHR0cDovL2NybC5taWNyb3NvZnQuY29tL3BraWluZnJhL0NSTC9BTUUlMjBJbmZyYSUyMENBJTIwMDUuY3JshjFodHRwOi8vY3JsMS5hbWUuZ2JsL2NybC9BTUUlMjBJbmZyYSUyMENBJTIwMDUuY3JshjFodHRwOi8vY3JsMi5hbWUuZ2JsL2NybC9BTUUlMjBJbmZyYSUyMENBJTIwMDUuY3JshjFodHRwOi8vY3JsMy5hbWUuZ2JsL2NybC9BTUUlMjBJbmZyYSUyMENBJTIwMDUuY3JshjFodHRwOi8vY3JsNC5hbWUuZ2JsL2NybC9BTUUlMjBJbmZyYSUyMENBJTIwMDUuY3JsMBcGA1UdIAQQMA4wDAYKKwYBBAGCN3sBATAfBgNVHSMEGDAWgBR61hmFKHlscXYeYPjzS--iBUIWHTAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDQYJKoZIhvcNAQELBQADggEBACOqbheswhOnKXLTkIutiIcteO6UoShFvwHPuLKn1BKMJ_lg2KTmeWeMNXwGuAM1Z4tgXp4IasV0v5l_PNdIkE__sX0pUO1ZM3PhdRXALW5KrTbR5tceOYbro0Wzdm1skFE-tjtTwB9fDTiGSf9Yona8SX88VKN9FFL40XZssGFD4vyUUlwgOWsjcBSY6sVQRB-WFGKyibrdxjNd18-Uro0FhS6OjPYdjzqmigmAx6BfL-wNEghkFhiYCpFFSaT1pbaju8cBhg25iAJyHywC0mWg8rm274lLY8frHDtGELz8cxuXgOaxMy_10uFR3dL7DTBtNBXmK4R3L5NEipHyg90&s=OB8COr2tKR3xyhtAJUZfzCqjGQSfrL3fMjIpqMSk90BXAi3DQZ6jDGt4Za1W9PcX-uw9QlJN4_i8OSo7vNkv9viPbffJKIR0fV7kSfWb70qzJd5fH21JFvWQLEcGmrhWLg5W_7x5SlOIWskTRnSqtgOof8vqWq47afd0MPZ62tMFMKopcxRExoVg68rFFTF9MGT8Sqtv_vkBZmQwkvZLy1EvFsMA-vCnSLVRE_FqszObVJSXOdaiNKaXCP-cPP8qU0corhei_waoO-Tm2eyF7cA3WD7tWJqc7muG7-eJtxrX6JPJPH-0BngZHxnku_ao581iW9349j2AW2NEaqJqdQ&h=mm-PzeXa8XekpP-9fAXlFCpZHJYrXr17KQaOnUkqp20 failed with status code 412. Content is '{"error":{"code":"PreconditionFailed","message":"Exception of type 'Microsoft.WindowsAzure.ApiManagement.Management.Core.Exceptions.PreconditionFailedException' was thrown.","details":null}}'.
         at common.HttpPipelineExtensions.Validate(Response response, Uri requestUri)
         at common.HttpPipelineExtensions.WaitForLongRunningOperation(HttpPipeline pipeline, Response response, CancellationToken cancellationToken)
         at common.HttpPipelineExtensions.PutResource(HttpPipeline pipeline, Uri uri, JsonObject resource, CancellationToken cancellationToken)
         at publisher.Program.<>c__DisplayClass13_0.<<GetPutRestResource>b__0>d.MoveNext()
      --- End of stack trace from previous location ---
         at publisher.Api.PutApi(ApiName apiName, ApiInformationFile apiInformationFile, ApiSpecificationFile specificationFile, JsonObject configurationApiJson, ServiceUri serviceUri, GetRestResource getRestResource, PutRestResource putRestResource, ILogger logger, CancellationToken cancellationToken)
         at publisher.Api.<>c__DisplayClass24_0.<<ProcessArtifactsToPut>b__4>d.MoveNext()
      --- End of stack trace from previous location ---
         at System.Threading.Tasks.Parallel.<>c__50`1.<<ForEachAsync>b__50_0>d.MoveNext()
      --- End of stack trace from previous location ---
         at common.IEnumerableExtensions.ForEachParallel[T](IEnumerable`1 enumerable, Func`2 action, CancellationToken cancellationToken)
         at publisher.Api.ProcessArtifactsToPut(IReadOnlyCollection`1 files, JsonObject configurationJson, ServiceDirectory serviceDirectory, ServiceUri serviceUri, GetRestResource getRestResource, PutRestResource putRestResource, ILogger logger, CancellationToken cancellationToken)
         at publisher.Service.ProcessArtifactsToPut(IReadOnlyCollection`1 files, JsonObject configurationJson, ServiceDirectory serviceDirectory, ServiceUri serviceUri, ListRestResources listRestResources, GetRestResource getRestResource, PutRestResource putRestResource, DeleteRestResource deleteRestResource, ILogger logger, CancellationToken cancellationToken)
         at publisher.Publisher.RunWithoutCommitId(CancellationToken cancellationToken)
         at publisher.Publisher.Run(CancellationToken cancellationToken)
         at publisher.Publisher.ExecuteAsync(CancellationToken cancellationToken)
info: Microsoft.Hosting.Lifetime[0]
      Application is shutting down...

Where as publishing my-api-v1-b2b, without the ;rev=1 at the end succeeds.

Expected behavior

Publishing an API with the revision in the name should succeed.

Actual behavior

Publishing an API with the revision in the name fails.

Reproduction Steps

With a file structure looking like this, use the publisher to publish an API both with and without a revision

.
├── apis
├── my-api-v1
│   │   ├── apiInformation.json
│   │   ├── policy.xml
│   │   └── specification.json
│   ├── my-api-v1;rev=1
│   │   ├── apiInformation.json
│   │   ├── policy.xml
│   │   └── specification.json
.....

[github-actions]

  Thank you for opening this issue! Please be patient while we will look into it and get back to you as this is an open source project. In the meantime make sure you take a look at the [closed issues](https://github.com/Azure/apiops/issues?q=is%3Aissue+is%3Aclosed) in case your question has already been answered. Don't forget to provide any additional information if needed (e.g. scrubbed logs, detailed feature requests,etc.).
  Whenever it's feasible, please don't hesitate to send a Pull Request (PR) our way. We'd greatly appreciate it, and we'll gladly assess and incorporate your changes.

[shawnyu5]

Did some more testing. This only happens when my-api-v1 and my-api-v1;rev=1 have different configuration. If both API configuration are exactly the same, then the publisher successfully publishes both APIs.

[ghost]

@shawnyu5 what do you mean by 'different configuration'? I am having the same issue with same error message. However, the order of which my API is loaded in is different to yours.

I was previously using v4.10.3 when I first encountered the error. I just tried v.5.1.1 and it is the same

info: Publisher[0]
      Putting API myapi;rev=1...
info: Publisher[0]
      Putting API myapi...
dbug: PutRestResource[0]
      Beginning request to put REST resource URI urimyapi;rev=1...
dbug: PutRestResource[0]
      Beginning request to put REST resource URI urimyapi...
dbug: PutRestResource[0]
      Successfully put REST resource at URI urimyapi...
crit: Publisher[0]
      System.Net.Http.HttpRequestException: HTTP request to URI urimyapi;rev=1 failed with status code 409. Content is '{"error":{"code":"IdentifierAlreadyInUse","message":"Resource already exists.","details":null}}'.

[shawnyu5]

I tried to copy my-api-v1/, and rename it to my-api-v1;rev=1, so both base and rev 1 would have the exact same files. This publishes successfully. However the publisher throws an error when my-api-v1/ and my-api-v1;rev=1 have different configuration values.

[ghost]

ah right, so the api spec is the exactly same. but the whole point of using revision is that the api spec is different so need to wait for a fix/reply from them

[guythetechie]

@shawnyu5 & @eyvictorye - Publishing APIs with revisions is currently bugged. We will include a fix in our next major release (sometime in the next few weeks).

[ics-development]

Hi, I was having this issue with 4.10.2. Having updated to 5.1.4, I'm still having this exact issue with 5.1.4 - was this fixed in any version?

[waelkdouh]

We are going to publish the fix in v6. We just released v6 alpha but it's having some issues so it's not quite ready for testing yet. We are getting close. Stays tuned for v6 soon.

[ics-development]

@waelkdouh awesome, will do, thank you. Appreciate your hard work.

[username-anonymous]

Hi, Please could someone suggest when would v6 be releasing, any date and would v6 have fix for above issue?

[waelkdouh]

We are actively working on it. We are hoping to release it in the next couple weeks. We can't promise any timelines as this is an open source project.

[guythetechie]

v6 was recently released.