Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update subscription configuration schema to include new parameters #1163

Merged
13 commits merged into from
Nov 10, 2020
Merged

Update subscription configuration schema to include new parameters #1163

13 commits merged into from
Nov 10, 2020

Conversation

benbp
Copy link
Member

@benbp benbp commented Nov 2, 2020

This change simplifies cloud configurations in our live test pipelines. Currently these parameters get defined per cloud in the live test template, which ends up creating a large, verbose configuration. To simplify these definitions, I'd like to move them into the subscription configuration. The resulting template usage becomes more like:

- name: CloudConfig
  type: object
  default:
    Public:
      SubscriptionConfiguration: $(sub-config-azure-cloud-test-resources)
    Preview:
      SubscriptionConfiguration: $(sub-config-azure-cloud-test-resources-preview)
    Canary:
      SubscriptionConfiguration: $(sub-config-azure-cloud-test-resources-preview)
      Location: 'eastus2euap'                                                                                                                                                                                                                                                                UsGov:
      SubscriptionConfiguration: $(sub-config-gov-test-resources)
    China:
      SubscriptionConfiguration: $(sub-config-cn-test-resources)

This change will require an update to the above 4 subscription configurations. The script change handles both the old and new data structure.

Passing live test referencing the new configuration: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=600922&view=logs&j=bbd7d436-e05d-51a8-99a8-0701dc9dffeb&t=20f065cd-bf8b-5b9d-5626-dd2113fcfcf1

Passing live test referencing the old configuration: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=600932&view=results

@azure-sdk
Copy link
Collaborator

The following pipelines have been queued for testing:
java - template
js - template
net - template
python - template
You can sign off on the approval gate to test the release stage of each pipeline.
See eng/common workflow

@benbp benbp requested review from weshaggard and heaths November 4, 2020 02:12
@benbp
Copy link
Member Author

benbp commented Nov 4, 2020 •
edited
Loading

@heaths @weshaggard the latest update includes a couple of things, based on our conversations.

  1. Support the old syntax used in a few places (mainly storage tests for java and js) that involved passing a powershell hash literal down as a parameter string to be rendered directly. This way I can phase this out independently.
  2. Support subscription configuration defining the base set of arm template parameters, if (1.) is not defined.
  3. Add pipeline and/or platform specific arm parameters, passing yaml objects down as pipeline parameters, to be converted. These will all get merged with the arm template parameters from the subscription configuration and passed down to the New-TestResources script as one object.

Passing test using the old subscription configuration and the old template parameter syntax: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=603748&view=logs&j=bbd7d436-e05d-51a8-99a8-0701dc9dffeb&t=5c636125-15aa-53f0-715c-4a154a225488
Passing test using the new subscription configuration: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=603748&view=logs&j=bbd7d436-e05d-51a8-99a8-0701dc9dffeb&t=5c636125-15aa-53f0-715c-4a154a225488

@azure-sdk
Copy link
Collaborator

The following pipelines have been queued for testing:
java - template
js - template
net - template
python - template
You can sign off on the approval gate to test the release stage of each pipeline.
See eng/common workflow

@benbp benbp requested a review from weshaggard November 4, 2020 18:06
@azure-sdk
Copy link
Collaborator

The following pipelines have been queued for testing:
java - template
js - template
net - template
python - template
You can sign off on the approval gate to test the release stage of each pipeline.
See eng/common workflow

heaths
heaths previously requested changes Nov 4, 2020
View reviewed changes
Copy link
Member

@heaths heaths left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no need to split parameters and environment variables - a breaking change. Parameters are already filtered to what the ARM template declares.

@azure-sdk
Copy link
Collaborator

The following pipelines have been queued for testing:
java - template
js - template
net - template
python - template
You can sign off on the approval gate to test the release stage of each pipeline.
See eng/common workflow

@heaths heaths dismissed their stale review November 6, 2020 01:19

Expecting some changes but would like to see them.

@benbp
Copy link
Member Author

benbp commented Nov 6, 2020

@heaths @weshaggard updated based on our meeting. For context: due to the object->string rendering limitations with devops yaml, and breaking change concerns with dev usage of the script, I'm adding ArmTemplateParameters as a net new parameter, that's defined as a stringified powershell hash in pipeline yaml like we do in other repos today.

I'm also setting all the new environment variables defined in the new subscription configurations as deployment outputs. Looking at these values more closely, I'm wondering if they are better off being defined in the sdk pipelines instead. Another discussion though.

heaths
heaths approved these changes Nov 6, 2020
View reviewed changes
Copy link
Member

@heaths heaths left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remember to regenerate the markdown as well. IIRC, the README has instructions how.

@azure-sdk
Copy link
Collaborator

The following pipelines have been queued for testing:
java - template
js - template
net - template
python - template
You can sign off on the approval gate to test the release stage of each pipeline.
See eng/common workflow

This was referenced Nov 6, 2020
Merged
Merged
Merged
Merged
Merged
@benbp benbp mentioned this pull request Nov 6, 2020
Merged
@benbp benbp requested a review from weshaggard November 9, 2020 22:27
@azure-sdk
Copy link
Collaborator

The following pipelines have been queued for testing:
java - template
js - template
net - template
python - template
You can sign off on the approval gate to test the release stage of each pipeline.
See eng/common workflow

@benbp benbp requested a review from weshaggard November 9, 2020 23:10
@azure-sdk
Copy link
Collaborator

The following pipelines have been queued for testing:
java - template
js - template
net - template
python - template
You can sign off on the approval gate to test the release stage of each pipeline.
See eng/common workflow

@azure-sdk
Copy link
Collaborator

The following pipelines have been queued for testing:
java - template
js - template
net - template
python - template
You can sign off on the approval gate to test the release stage of each pipeline.
See eng/common workflow

@azure-sdk
Copy link
Collaborator

The following pipelines have been queued for testing:
java - template
js - template
net - template
python - template
You can sign off on the approval gate to test the release stage of each pipeline.
See eng/common workflow

@ghost
Copy link

ghost commented Nov 10, 2020

Hello @azure-sdk!

Because this pull request has the auto-merge label, I will be glad to assist with helping to merge this pull request once all check-in policies pass.

p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (@msftbot) and give me an instruction to get started! Learn more here.

@ghost ghost merged commit 34ee8c5 into Azure:master Nov 10, 2020
@benbp benbp deleted the benbp/new-subscription-configuration branch November 10, 2020 01:54
benbp added a commit that referenced this pull request Dec 1, 2020
@benbp
Update subscription configuration schema to include new parameters (#…
444c8ca 
…1163)

This change simplifies cloud configurations in our live test pipelines. Currently these parameters get defined per cloud in the live test template, which ends up creating a large, verbose configuration. To simplify these definitions, I'd like to move them into the subscription configuration. The resulting template usage becomes more like:

```
- name: CloudConfig
  type: object
  default:
    Public:
      SubscriptionConfiguration: $(sub-config-azure-cloud-test-resources)
    Preview:
      SubscriptionConfiguration: $(sub-config-azure-cloud-test-resources-preview)
    Canary:
      SubscriptionConfiguration: $(sub-config-azure-cloud-test-resources-preview)
      Location: 'eastus2euap'                                                                                                                                                                                                                                                                UsGov:
      SubscriptionConfiguration: $(sub-config-gov-test-resources)
    China:
      SubscriptionConfiguration: $(sub-config-cn-test-resources)
```

This change will require an update to the above 4 subscription configurations. The script change handles both the old and new data structure.

Passing live test referencing the new configuration: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=600922&view=logs&j=bbd7d436-e05d-51a8-99a8-0701dc9dffeb&t=20f065cd-bf8b-5b9d-5626-dd2113fcfcf1

Passing live test referencing the old configuration: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=600932&view=results
sima-zhu pushed a commit to sima-zhu/azure-sdk-tools that referenced this pull request Dec 3, 2020
@danewalton-msft
Move pnp sample code into own folder (Azure#1163)
c85fb41
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@heaths heaths heaths approved these changes

@weshaggard weshaggard weshaggard approved these changes

@danieljurek danieljurek Awaiting requested review from danieljurek

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@benbp @azure-sdk @heaths @weshaggard