Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move to OneBranch Signing and SBOM generation #1982

Merged
merged 33 commits into from
Mar 14, 2024
Merged
Changes from 30 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
1ed48b1
Updates to signing
adityapatwardhan Mar 12, 2024
b74f202
fix files to sign
TravisEz13 Mar 13, 2024
9a24c68
sbom changes
TravisEz13 Mar 13, 2024
b9cdd49
format changes
TravisEz13 Mar 13, 2024
83cc690
Make packaging wait on build
TravisEz13 Mar 13, 2024
ed91b82
clarify task display names
TravisEz13 Mar 13, 2024
80473e2
Make sure signing is setup
TravisEz13 Mar 13, 2024
18232e3
move other tasks to restore phase
TravisEz13 Mar 13, 2024
a570c56
move checkout to restore phase
TravisEz13 Mar 13, 2024
4331d7b
update container
TravisEz13 Mar 13, 2024
473498e
move signing to it's own job
TravisEz13 Mar 13, 2024
4330f12
add flag required for newest container
TravisEz13 Mar 13, 2024
1bdc123
clarify copy files task displaynames
TravisEz13 Mar 13, 2024
3d2c9dd
reuse existing variable
TravisEz13 Mar 13, 2024
a9cadb3
removed unneeded code
TravisEz13 Mar 13, 2024
dd9e7a6
Use correct path is signing job
TravisEz13 Mar 13, 2024
5062efd
sign the nupkg
TravisEz13 Mar 13, 2024
1465ca9
fix syntax
TravisEz13 Mar 13, 2024
fa48f5d
Revert "clarify copy files task displaynames"
TravisEz13 Mar 13, 2024
9c1cfad
Disable code sign validation on first job
TravisEz13 Mar 13, 2024
d5d03f0
search for where OBP put dotnet.exe
TravisEz13 Mar 13, 2024
f3ef9e7
Move SDK search to it's own job
TravisEz13 Mar 13, 2024
ac44341
delete SDK search
TravisEz13 Mar 13, 2024
440ceb6
make nupkg signing a todo
TravisEz13 Mar 13, 2024
9c65c5b
fix nupkg publishing
TravisEz13 Mar 13, 2024
663f093
try disabling codeql to get signing working per docs
TravisEz13 Mar 13, 2024
1ed8039
don't setup signing where not needed
TravisEz13 Mar 13, 2024
a03237d
combine sign and build job
TravisEz13 Mar 14, 2024
504c5df
set version for SBOM
TravisEz13 Mar 14, 2024
dd8c136
make comments more accurate
TravisEz13 Mar 14, 2024
9a68ea5
Apply suggestions from code review
TravisEz13 Mar 14, 2024
49d6365
fix variable name
TravisEz13 Mar 14, 2024
368a7e8
address pr comments
TravisEz13 Mar 14, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
228 changes: 143 additions & 85 deletions .pipelines/OSS_Microsoft_PSSA-Official.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,30 +9,29 @@ pr:
- master
- release*
variables:
- name: DOTNET_CLI_TELEMETRY_OPTOUT
value: 1
- name: POWERSHELL_TELEMETRY_OPTOUT
value: 1
- name: WindowsContainerImage
value: onebranch.azurecr.io/windows/ltsc2019/vse2022:latest
- name: DOTNET_CLI_TELEMETRY_OPTOUT
value: 1
- name: POWERSHELL_TELEMETRY_OPTOUT
value: 1
- name: WindowsContainerImage
value: onebranch.azurecr.io/windows/ltsc2022/vse2022:latest
resources:
repositories:
- repository: ComplianceRepo
type: github
endpoint: ComplianceGHRepo
name: PowerShell/compliance
ref: master
- repository: onebranchTemplates
type: git
name: OneBranch.Pipelines/GovernedTemplates
ref: refs/heads/main
extends:
template: v2/OneBranch.Official.CrossPlat.yml@onebranchTemplates
parameters:
featureFlags:
WindowsHostVersion: '1ESWindows2022'
customTags: 'ES365AIMigrationTooling'
globalSdl:
disableLegacyManifest: true
sbom:
enabled: false
enabled: true
packageName: Microsoft.PowerShell.ScriptAnalyzer
codeql:
compiled:
enabled: true
Expand All @@ -48,113 +47,172 @@ extends:
enabled: false

stages:
- stage: Build
displayName: Build
- stage: stagebuild
displayName: Build and Package Microsoft.PowerShell.ScriptAnalyzer
jobs:
- job: Build_Job
displayName: Build Microsoft.PowerShell.ScriptAnalyzer
- job: jobbuild
displayName: Build Microsoft.PowerShell.ScriptAnalyzer Files
variables:
- group: ESRP
- name: ob_outputDirectory
value: '$(Build.ArtifactStagingDirectory)/ONEBRANCH_ARTIFACT'
- name: repoRoot
value: $(Build.SourcesDirectory)\OSS_Microsoft_PSSA
- name: ob_sdl_tsa_configFile
value: $(Build.SourcesDirectory)\OSS_Microsoft_PSSA\.config\tsaoptions.json
- name: signSrcPath
value: $(repoRoot)/out
- name: ob_sdl_sbom_enabled
value: true
- name: ob_signing_setup_enabled
value: true
#CodeQL tasks added manually to workaround signing failures
- name: ob_codeql_compiled_enabled
value: false

pool:
type: windows
steps:
- checkout: self
- checkout: ComplianceRepo
env:
ob_restore_phase: true # Set ob_restore_phase to run this step before '🔒 Setup Signing' step.

- pwsh: |
if (-not (Test-Path $(repoRoot)/.config/tsaoptions.json)) {
Get-ChildItem $(Build.SourcesDirectory) -recurse -ErrorAction SilentlyContinue
throw "tsaoptions.json does not exist under $(Build.SourcesDirectory)/OSS_Microsoft_PSSA/.config"
throw "tsaoptions.json does not exist under $(repoRoot)/.config"
}
displayName: Test if tsaoptions.json exists
env:
ob_restore_phase: true # Set ob_restore_phase to run this step before '🔒 Setup Signing' step.

- task: UseDotNet@2
displayName: 'Install .NET dependencies'
env:
ob_restore_phase: true # Set ob_restore_phase to run this step before '🔒 Setup Signing' step.
inputs:
packageType: 'sdk'
useGlobalJson: true
# this is to ensure that we are installing the dotnet at the same location as container by default install the dotnet sdks
installationPath: 'C:\Program Files\dotnet\'
workingDirectory: $(repoRoot)

#- pwsh: |
# New-Item -ItemType Directory -Path $(Build.SourcesDirectory)/.config -Force -Verbose
# Copy-Item '$(Build.SourcesDirectory)/OSS_Microsoft_PSSA/.config/tsaoptions.json' '$(Build.SourcesDirectory)/.config/tsaoptions.json' -Force -Verbose
# displayName: Copy tsaoptions
- task: CodeQL3000Init@0 # Add CodeQL Init task right before your 'Build' step.
env:
ob_restore_phase: true # Set ob_restore_phase to run this step before '🔒 Setup Signing' step.
inputs:
Enabled: true
AnalyzeInPipeline: true
Language: csharp

# this is installing .NET
- pwsh: |
Set-Location "$(Build.SourcesDirectory)/OSS_Microsoft_PSSA"
Set-Location "$(repoRoot)"
try { ./build.ps1 -Configuration Release -All } catch { throw $_ }
displayName: Execute build
env:
ob_restore_phase: true # Set ob_restore_phase to run this step before '🔒 Setup Signing' step.

- task: CodeQL3000Finalize@0 # Add CodeQL Finalize task right after your 'Build' step.
condition: always()
env:
ob_restore_phase: true # Set ob_restore_phase to run this step before '🔒 Setup Signing' step.

- task: onebranch.pipeline.signing@1
displayName: Sign 1st party files
inputs:
command: 'sign'
signing_profile: external_distribution
files_to_sign: '**\*.psd1;**\*.psm1;**\*.ps1xml;**\Microsoft*.dll'
search_root: $(signSrcPath)

- task: onebranch.pipeline.signing@1
displayName: Sign 3rd Party files
inputs:
command: 'sign'
signing_profile: 135020002
files_to_sign: '**/Pluralize*.dll;**/Newtonsoft*.dll'
search_root: $(signSrcPath)

- task: CopyFiles@2
displayName: "Copy signed files to ob_outputDirectory"
inputs:
SourceFolder: "$(signSrcPath)"
Contents: '**'
TargetFolder: $(ob_outputDirectory)

- pwsh: |
$signSrcPath = "$(Build.SourcesDirectory)/OSS_Microsoft_PSSA/out"
# Set signing src path variable
$vstsCommandString = "vso[task.setvariable variable=signSrcPath]${signSrcPath}"
Write-Host "sending $vstsCommandString"
Write-Host "##$vstsCommandString"
$signOutStep1 = "$(Build.SourcesDirectory)/OSS_Microsoft_PSSA/Step1"
$null = New-Item -ItemType Directory -Path $signOutStep1
# Set signing out path variable
$vstsCommandString = "vso[task.setvariable variable=signOutStep1]${signOutStep1}"
Write-Host "sending $vstsCommandString"
Write-Host "##$vstsCommandString"
$signOutPath = "$(Build.SourcesDirectory)/OSS_Microsoft_PSSA/signed"
$null = New-Item -ItemType Directory -Path $signOutPath
# Set signing out path variable
$vstsCommandString = "vso[task.setvariable variable=signOutPath]${signOutPath}"
Write-Host "sending $vstsCommandString"
Write-Host "##$vstsCommandString"
# Set path variable for guardian codesign validation
$vstsCommandString = "vso[task.setvariable variable=GDN_CODESIGN_TARGETDIRECTORY]${signOutPath}"
Write-Host "sending $vstsCommandString"
Write-Host "##$vstsCommandString"
# Get version and create a variable
$moduleData = Import-PowerShellDataFile "$(Build.SourcesDirectory)/OSS_Microsoft_PSSA/Engine/PSScriptAnalyzer.psd1"
$moduleData = Import-PowerShellDataFile "$(repoRoot)/Engine/PSScriptAnalyzer.psd1"
$moduleVersion = $moduleData.ModuleVersion
$vstsCommandString = "vso[task.setvariable variable=moduleVersion]${moduleVersion}"
$vstsCommandString = "vso[task.setvariable variable=ob_sdl_sbom_packageversion]${moduleVersion}"

Write-Host "sending $vstsCommandString"
Write-Host "##$vstsCommandString"
displayName: Setup variables for signing

- template: EsrpSign.yml@ComplianceRepo
parameters:
buildOutputPath: $(signSrcPath)
signOutputPath: $(signOutStep1)
certificateId: "CP-230012"
useMinimatch: true
pattern: |
**\*.psd1
**\*.psm1
**\*.ps1xml
**\Microsoft*.dll

- template: EsrpSign.yml@ComplianceRepo
parameters:
buildOutputPath: $(signOutStep1)
signOutputPath: $(signOutPath)
certificateId: "CP-231522"
useMinimatch: true
pattern: |
**/Pluralize*.dll
**/Newtonsoft*.dll

- template: Sbom.yml@ComplianceRepo
parameters:
BuildDropPath: $(signOutPath)
Build_Repository_Uri: 'https://github.com/powershell/PSScriptAnalyzer'
displayName: Setup SBOM Package Version

- job: nupkg
dependsOn: jobbuild
displayName: Package Microsoft.PowerShell.ScriptAnalyzer
variables:
- name: ob_outputDirectory
value: '$(Build.ArtifactStagingDirectory)/ONEBRANCH_ARTIFACT'
- name: repoRoot
value: $(Build.SourcesDirectory)\OSS_Microsoft_PSSA
- name: ob_sdl_tsa_configFile
value: $(Build.SourcesDirectory)\OSS_Microsoft_PSSA\.config\tsaoptions.json
- name: ob_sdl_sbom_enabled
TravisEz13 marked this conversation as resolved.
Show resolved Hide resolved
value: false
- name: ob_sdl_codeql_compiled_enabled
TravisEz13 marked this conversation as resolved.
Show resolved Hide resolved
value: false
TravisEz13 marked this conversation as resolved.
Show resolved Hide resolved
TravisEz13 marked this conversation as resolved.
Show resolved Hide resolved
- name: signOutPath
value: $(repoRoot)/signed
- name: ob_signing_setup_enabled
value: true
- name: ob_codeql_compiled_enabled
TravisEz13 marked this conversation as resolved.
Show resolved Hide resolved
value: false

pool:
type: windows
steps:
- checkout: self

- pwsh: |
if (-not (Test-Path $(repoRoot)/.config/tsaoptions.json)) {
Get-ChildItem $(Build.SourcesDirectory) -recurse -ErrorAction SilentlyContinue
throw "tsaoptions.json does not exist under $(repoRoot)/.config"
}
displayName: Test if tsaoptions.json exists

- task: DownloadPipelineArtifact@2
displayName: 'Download build files'
inputs:
targetPath: $(signOutPath)
artifact: drop_stagebuild_jobbuild

- pwsh: |
Set-Location "$(Build.SourcesDirectory)/OSS_Microsoft_PSSA"
Set-Location "$(signOutPath)"
dir -dir -recurse *
displayName: Capture artifacts structure

- pwsh: |
Set-Location "$(repoRoot)"
./build -BuildNupkg -CopyManifest -signed
displayName: Create nupkg for publishing

- task: CopyFiles@2
displayName: "Copy Files for 'publish build directory' publish task"
- task: onebranch.pipeline.signing@1
displayName: Sign nupkg
inputs:
SourceFolder: "$(Build.SourcesDirectory)/OSS_Microsoft_PSSA"
Contents: '**'
TargetFolder: $(Build.ArtifactStagingDirectory)/ONEBRANCH_ARTIFACT/build
command: 'sign'
signing_profile: external_distribution
files_to_sign: '**\*.nupkg'
search_root: $(signOutPath)

- pwsh: |
Set-Location "$(repoRoot)"
dir -file -recurse *.nupkg
displayName: Find Nupkg

- task: CopyFiles@2
displayName: "Copy Files for 'Publish module nupkg' publish task"
displayName: "Copy nupkg to ob_outputDirectory"
TravisEz13 marked this conversation as resolved.
Show resolved Hide resolved
inputs:
Contents: "$(signOutPath)/PSScriptAnalyzer.$(moduleVersion).nupkg"
TargetFolder: $(Build.ArtifactStagingDirectory)/ONEBRANCH_ARTIFACT/nupkg
Contents: "$(repoRoot)/**/PSScriptAnalyzer.*.nupkg"
TargetFolder: $(ob_outputDirectory)