API Readiness errors and questions for management plane

[ladonnaq]

I am working with the Palo Alto service team to use the release planner app to validate public preview API readiness for their initial beta SDK releases. I have encountered some errors, questions related to function, etc.

• API readiness validation blocked by breaking changes. PR used was Updating specs for PAN azure-rest-api-specs#23435.
  This scenario is confusing because there are 2 PRs in the private repo which have approved breaking changes for changes that are in this PR for the public repo. See comment from service team below.
  "Please provide waiver for Breaking Changes. We are adding changes to the existing api-version as our service is still in private preview. Hence, new api version is not required. These changes are already approved in private repo as well -
  https://github.com/Azure/azure-rest-api-specs-pr/pull/11696
  https://github.com/Azure/azure-rest-api-specs-pr/pull/11240"

• Can't input text into review ticket field to add link to the GitHub issue for namespace review by arch board for initial beta => Review Ticket (Arch Board Namespace Approval) is hardcoded to azure-sdk-pr  #6053
  

• The SDKs should NOT be generated in the API Readiness application for management plane or data plane. I believe there is a GitHub issue open for this but just wanted to make sure that we move this asap. The service teams I have been working with found this confusing. SDK Breaking changes are reviewed as part of API readiness for management plane BUT since the information relating to todo's for SDK release are in the SDK release app we should not generate and display the SDKs in the API Readiness app.

• @josefree @maririos Question about what is required for a API Spec PR merge and how to provide guidance on which PR a service team should use for API readiness. I noticed that PR 23435 is merged but the ARM sign off label is not on the PR so an error is displayed in the API Readiness app that ARM has not signed off. I noticed that ARM signed off the 2 PRs that were in the private repo and are part of this version of the API Spec. So which PR should be used to validate API Readiness? For public preview today we ask teams to send us the link to the either the PR in the private repo that is related to the API Spec that will be published for public preview (for services that are new and do not want to have anything in the public repo until announce). For other scenarios, we ask for the last PR that is associated with the API Spec version for the public launch for the product.

• Previous error which is now not showing up on 2nd test. "Unsupported repository owner error, management plane, API readiness". #5976

This the PR that was used to publish the API spec from the private repo to the public repo in preparation to release the SDKs for public preview. Azure/azure-rest-api-specs#22878

This is the PR I would expect they would use for API Readiness. However, there is also another PR so I tried that one as well. It also gave me the same error - Azure/azure-rest-api-specs#23435.

[maririos]

All points have been covered in separate issues.
Remaining is:

@josefree @maririos Question about what is required for a API Spec PR merge and how to provide guidance on which PR a service team should use for API readiness. I noticed that PR 23435 is merged but the ARM sign off label is not on the PR so an error is displayed in the API Readiness app that ARM has not signed off. I noticed that ARM signed off the 2 PRs that were in the private repo and are part of this version of the API Spec. So which PR should be used to validate API Readiness? For public preview today we ask teams to send us the link to the either the PR in the private repo that is related to the API Spec that will be published for public preview (for services that are new and do not want to have anything in the public repo until announce). For other scenarios, we ask for the last PR that is associated with the API Spec version for the public launch for the product.

@ladonnaq to create a separate issue for this to start a conversation on process and tooling for this requirement