Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use key vault task in CI yaml (#1037) #1166

Merged
merged 1 commit into from
May 3, 2019

Conversation

myagley
Copy link
Contributor

@myagley myagley commented May 3, 2019

Currently our integration tests expect a certificate to be installed on the test machine. The certificate gives us access to a key vault containing the secrets we need for the tests. The certificate itself is also in the key vault, so we have to give the pipeline access to get the certificate. All this was done early in the project, before the key vault task existed in Azure Pipelines.

Using the Azure Pipelines Key Vault task allows us to remove key vault credentials from the pipeline. It also simplifies the build machine setup: we no longer need to install the certificate, and on Linux we don't need to install things like Python, the az CLI, and PowerShell that our scripts require to get/install the certificate. Instead we use the task to list exactly the secrets we need, and they are available to the pipeline. For script tasks, we map the secrets into the environment so they don't get passed as arguments and potentially exposed.

Other changes:

  • Use the .NET Core installer task to pin the version of .NET Core for builds
  • Remove "Install dependencies" and "install test dependencies" tasks since they're no longer needed. For Linux I had to add a script task to install libsnappy, which installPrereqs.sh was doing. It's needed for the rocksdb integration tests. I also had to add a task to install the protocol head certificate, which several integration tests need. This could arguably be checked in as a PS1 script rather than inline script in the YAML file, but this works for now.
  • Remove tasks that copy and publish artifacts for the end-to-end tests (IotEdgeQuickstart and friends). No other pipelines curently consume it right now (they rely on the Build Images pipeline). We can add these back if/when they're needed.
  • Move the Windows job onto a hosted agent
  • Update SecretsHelper to look in the environment first for the needed secrets. When the integration tests are run in the CI pipeline, the KeyVaultHelper is never invoked. Elsewhere, things continue to work like they always have.
  • Our build scripts expect dotnet.exe to live in a certain path, but the .NET Core Installer task puts it somewhere funny, and then prepends that location to the PATH. So I updated the build/test scripts to (1) look for an environment variable, then (2) look on the PATH, then (3) look in some default locations.

Currently our integration tests expect a certificate to be installed on the test machine. The certificate gives us access to a key vault containing the secrets we need for the tests. The certificate itself is also in the key vault, so we have to give the pipeline access to get the certificate. All this was done early in the project, before the key vault task existed in Azure Pipelines.

Using the Azure Pipelines Key Vault task allows us to remove key vault credentials from the pipeline. It also simplifies the build machine setup: we no longer need to install the certificate, and on Linux we don't need to install things like Python, the `az` CLI, and PowerShell that our scripts require to get/install the certificate. Instead we use the task to list exactly the secrets we need, and they are available to the pipeline. For script tasks, we map the secrets into the environment so they don't get passed as arguments and potentially exposed.

Other changes:
- Use the .NET Core installer task to pin the version of .NET Core for builds
- Remove "Install dependencies" and "install test dependencies" tasks since they're no longer needed. For Linux I had to add a script task to install libsnappy, which installPrereqs.sh was doing. It's needed for the rocksdb integration tests. I also had to add a task to install the protocol head certificate, which several integration tests need. This could arguably be checked in as a PS1 script rather than inline script in the YAML file, but this works for now.
- Remove tasks that copy and publish artifacts for the end-to-end tests (IotEdgeQuickstart and friends). No other pipelines curently consume it right now (they rely on the Build Images pipeline). We can add these back if/when they're needed.
- Move the Windows job onto a hosted agent
- Update SecretsHelper to look in the environment first for the needed secrets. When the integration tests are run in the CI pipeline, the KeyVaultHelper is never invoked. Elsewhere, things continue to work like they always have.
- Our build scripts expect dotnet.exe to live in a certain path, but the .NET Core Installer task puts it somewhere funny, and then prepends that location to the PATH. So I updated the build/test scripts to (1) look for an environment variable, then (2) look on the PATH, then (3) look in some default locations.
@myagley myagley merged commit 6f6250c into Azure:release/1.0.7 May 3, 2019
@myagley myagley deleted the miyagley/fix-ci-kv branch May 3, 2019 17:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants