Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge CoreCLR-based official builds legs into one leg per platform #92901

Merged
merged 33 commits into from
Oct 4, 2023
Merged
Show file tree
Hide file tree
Changes from 29 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
6c9698f
Move Mac and Linux CoreCLR builds to use single jobs for each all-up …
jkoritzinsky Sep 21, 2023
120d210
Fix binlog argument for OSX jobs.
jkoritzinsky Sep 21, 2023
f504858
Build clr.aot in the CoreCLR official build legs because we NativeAOT…
jkoritzinsky Sep 21, 2023
2646ffc
Remove libraries legs that were only kept around for PGO
jkoritzinsky Sep 21, 2023
71507b9
Fix indentation
jkoritzinsky Sep 21, 2023
eeae55c
Fix indentation
jkoritzinsky Sep 21, 2023
58c9c3b
Move variables from xplat-job up to xplat-setup and fix our condition…
jkoritzinsky Sep 21, 2023
4ab5925
Include more subsets in runtime-official.
jkoritzinsky Sep 22, 2023
8a444bd
Add missing arguments for linux packaging jobs (they are specified di…
jkoritzinsky Sep 22, 2023
6361cf5
Update Alpine crossdac to output to linux_musl folder directly.
jkoritzinsky Sep 27, 2023
d276528
Build the Windows CrossDacs and pack the CrossDac package in a separa…
jkoritzinsky Sep 27, 2023
e3294cc
Pass variables to a higher scope template.
jkoritzinsky Sep 27, 2023
a4e16ad
Fix dependencies
jkoritzinsky Sep 27, 2023
646c519
Fix paths and don't copy the "sharedFramework" folder.
jkoritzinsky Sep 27, 2023
d2d58c2
Refactor more names into variables.
jkoritzinsky Sep 27, 2023
49d5bbc
Don't try publishing the CrossDac for Windows. Remove the publishing …
jkoritzinsky Sep 27, 2023
7c23c69
Remove old crossdac-pack.yml script as we do it differently now.
jkoritzinsky Sep 27, 2023
d80d945
Condition macos signing on the same condition as previously.
jkoritzinsky Sep 27, 2023
fb40f73
Fix typo for post build steps
jkoritzinsky Sep 27, 2023
8695585
Move Windows CoreCLR build legs to the single-job format and remove a…
jkoritzinsky Sep 28, 2023
d41620f
Don't try to build NativeAOT on win-x86. Add comment labels for each …
jkoritzinsky Sep 29, 2023
386e5b2
Convert AllConfigurations leg to the global build template.
jkoritzinsky Sep 29, 2023
882a992
Pass config for AllConfigurations leg
jkoritzinsky Sep 30, 2023
6190812
Refine comment
jkoritzinsky Oct 2, 2023
28ba7cb
Entitle dotnet and apphost.
jkoritzinsky Oct 3, 2023
99acea4
Use variable not template syntax
jkoritzinsky Oct 3, 2023
9898c47
Remove logic for signing macos hosts in the installer build job. We d…
jkoritzinsky Oct 3, 2023
25fa308
Build all of the crossdacs in a separate build from the CoreCLR produ…
jkoritzinsky Oct 3, 2023
22782a7
Fix job suffix name
jkoritzinsky Oct 3, 2023
20fee47
Build checked Cross-OS dacs in PRs for diagnostic purposes.
jkoritzinsky Oct 4, 2023
a8769ad
PR feedback
jkoritzinsky Oct 4, 2023
9cebeb1
Use pipeline artifacts instead of build artifacts for the CrossDac an…
jkoritzinsky Oct 4, 2023
9e64389
Don't use pipeline artifacts. You can't have multiple legs publish to…
jkoritzinsky Oct 4, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions eng/pipelines/common/global-build-job.yml
Original file line number Diff line number Diff line change
Expand Up @@ -237,10 +237,11 @@ jobs:
- task: CodeQL3000Init@0
displayName: Initialize CodeQL (manually-injected)

- script: $(Build.SourcesDirectory)$(dir)build$(scriptExt) -ci $(_archParameter) $(_osParameter) $(crossArg) ${{ parameters.buildArgs }} $(_officialBuildParameter) $(_buildDarwinFrameworksParameter) $(_overrideTestScriptWindowsCmdParameter)
displayName: Build product
${{ if eq(parameters.useContinueOnErrorDuringBuild, true) }}:
continueOnError: ${{ parameters.shouldContinueOnError }}
- template: /eng/pipelines/common/templates/global-build-step.yml
parameters:
buildArgs: ${{ parameters.buildArgs }}
useContinueOnErrorDuringBuild: $${{ parameters.useContinueOnErrorDuringBuild }}
shouldContinueOnError: ${{ parameters.shouldContinueOnError }}

- ${{ if eq(parameters.isManualCodeQLBuild, true) }}:
- task: CodeQL3000Finalize@0
Expand Down
11 changes: 11 additions & 0 deletions eng/pipelines/common/templates/global-build-step.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
parameters:
buildArgs: ''
useContinueOnErrorDuringBuild: false
shouldContinueOnError: false
archParameter: $(_archParameter)

steps:
- script: $(Build.SourcesDirectory)$(dir)build$(scriptExt) -ci ${{ parameters.archParameter }} $(_osParameter) $(crossArg) ${{ parameters.buildArgs }} $(_officialBuildParameter) $(_buildDarwinFrameworksParameter) $(_overrideTestScriptWindowsCmdParameter)
displayName: Build product
${{ if eq(parameters.useContinueOnErrorDuringBuild, true) }}:
continueOnError: ${{ parameters.shouldContinueOnError }}
9 changes: 0 additions & 9 deletions eng/pipelines/common/templates/runtimes/xplat-job.yml
Original file line number Diff line number Diff line change
Expand Up @@ -77,15 +77,6 @@ jobs:
- name: buildConfig
value: ${{ parameters.buildConfig }}

- name: archType
value: ${{ parameters.archType }}

- name: osGroup
value: ${{ parameters.osGroup }}

- name: osSubgroup
value: ${{ parameters.osSubgroup }}

- ${{ if and(eq(variables['System.TeamProject'], 'internal'), ne(variables['Build.Reason'], 'PullRequest')) }}:
- name: _HelixSource
value: official/dotnet/runtime/$(Build.SourceBranch)
Expand Down
9 changes: 9 additions & 0 deletions eng/pipelines/common/xplat-setup.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,15 @@ jobs:

- name: _BuildConfig
value: $(buildConfigUpper)

- name: archType
value: ${{ parameters.archType }}

- name: osGroup
value: ${{ parameters.osGroup }}

- name: osSubgroup
value: ${{ parameters.osSubgroup }}

- name: _runSmokeTestsOnlyArg
value: '/p:RunSmokeTestsOnly=$(isRunSmokeTestsOnly)'
Expand Down
21 changes: 0 additions & 21 deletions eng/pipelines/coreclr/templates/build-job.yml
Original file line number Diff line number Diff line change
Expand Up @@ -229,27 +229,6 @@ jobs:
artifactName: $(buildProductArtifactName)
displayName: 'product build'

- ${{ if and(in(parameters.osGroup, 'windows', 'linux'), ne(parameters.archType, 'x86')) }}:
- template: /eng/pipelines/coreclr/templates/crossdac-build.yml
parameters:
archType: ${{ parameters.archType }}
osGroup: ${{ parameters.osGroup }}
osSubgroup: ${{ parameters.osSubgroup }}
isOfficialBuild: ${{ parameters.signBinaries }}
${{ if eq(parameters.archType, 'arm') }}:
hostArchType: x86
${{ else }}:
hostArchType: x64

- ${{ if and(in(parameters.osGroup, 'windows'), eq(parameters.archType, 'x86')) }}:
- template: /eng/pipelines/coreclr/templates/crossdac-build.yml
parameters:
archType: arm
osGroup: ${{ parameters.osGroup }}
osSubgroup: ${{ parameters.osSubgroup }}
isOfficialBuild: ${{ parameters.signBinaries }}
hostArchType: x86

- ${{ if and(ne(parameters.testGroup, ''), ne(parameters.disableClrTest, true)) }}:
# Publish test native components for consumption by test execution.
- ${{ if ne(parameters.isOfficialBuild, true) }}:
Expand Down
80 changes: 0 additions & 80 deletions eng/pipelines/coreclr/templates/crossdac-build.yml

This file was deleted.

9 changes: 9 additions & 0 deletions eng/pipelines/coreclr/templates/crossdac-hostarch.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
parameters:
archType: ''

variables:
- name: crossDacHostArch
value: x64
- ${{ if eq(parameters.archType, 'arm') }}:
- name: crossDacHostArch
value: x86
75 changes: 0 additions & 75 deletions eng/pipelines/coreclr/templates/crossdac-pack.yml

This file was deleted.

6 changes: 0 additions & 6 deletions eng/pipelines/coreclr/templates/xplat-pipeline-job.yml
Original file line number Diff line number Diff line change
Expand Up @@ -69,12 +69,6 @@ jobs:
- name: buildProductRootFolderPath
value: '$(Build.SourcesDirectory)/artifacts/bin/coreclr/$(osGroup).$(archType).$(buildConfigUpper)'

- name: buildCrossDacArtifactName
value: CoreCLRCrossDacArtifacts

- name: crossDacArtifactPath
value: $(Build.SourcesDirectory)/artifacts/$(buildCrossDacArtifactName)

# We need this because both mono and coreclr build currently depends on CoreClr
- name: coreClrProductArtifactName
value: 'CoreCLRProduct_${{ parameters.runtimeVariant }}_$(osGroup)$(osSubgroup)_$(archType)_$(buildConfig)'
Expand Down
29 changes: 3 additions & 26 deletions eng/pipelines/installer/jobs/build-job.yml
Original file line number Diff line number Diff line change
Expand Up @@ -339,32 +339,9 @@ jobs:
df -h
displayName: Disk Usage before Build

# Build the default subset non-MacOS platforms
- ${{ if ne(parameters.osGroup, 'osx') }}:
- script: $(BaseJobBuildCommand)
displayName: Build
continueOnError: ${{ and(eq(variables.SkipTests, false), eq(parameters.shouldContinueOnError, true)) }}

# Build corehost, sign and add entitlements to MacOS binaries
- ${{ if eq(parameters.osGroup, 'osx') }}:
- script: $(BaseJobBuildCommand) -subset host.native
displayName: Build CoreHost
continueOnError: ${{ and(eq(variables.SkipTests, false), eq(parameters.shouldContinueOnError, true)) }}

- ${{ if and(ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- template: /eng/pipelines/common/macos-sign-with-entitlements.yml
parameters:
filesToSign:
- name: dotnet
path: $(Build.SourcesDirectory)/artifacts/bin/osx-${{ parameters.archType }}.$(_BuildConfig)/corehost
entitlementsFile: $(Build.SourcesDirectory)/eng/pipelines/common/entitlements.plist
- name: apphost
path: $(Build.SourcesDirectory)/artifacts/bin/osx-${{ parameters.archType }}.$(_BuildConfig)/corehost
entitlementsFile: $(Build.SourcesDirectory)/eng/pipelines/common/entitlements.plist

- script: $(BaseJobBuildCommand) -subset host.pkg+host.tools+host.pretest+host.tests+packs
displayName: Build and Package
continueOnError: ${{ and(eq(variables.SkipTests, false), eq(parameters.shouldContinueOnError, true)) }}
- script: $(BaseJobBuildCommand)
displayName: Build
continueOnError: ${{ and(eq(variables.SkipTests, false), eq(parameters.shouldContinueOnError, true)) }}

- ${{ if and(eq(parameters.isOfficialBuild, true), eq(parameters.osGroup, 'windows')) }}:
- powershell: ./eng/collect_vsinfo.ps1 -ArchiveRunName postbuild_log
Expand Down
2 changes: 2 additions & 0 deletions eng/pipelines/installer/jobs/steps/build-linux-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ parameters:
packageStepDescription: null
packagingArgs: ''
subsetArg: ''
condition: succeeded()

steps:
## Run NuGet Authentication for each of the side containers
Expand All @@ -20,6 +21,7 @@ steps:
/bl:artifacts/log/$(_BuildConfig)/msbuild.${{ parameters.packageType }}.installers.binlog
displayName: Package ${{ parameters.packageStepDescription }} - ${{ parameters.packageType }}
target: ${{ parameters.target }}
condition: ${{ parameters.condition }}
# Broken symbolic links break the SBOM processing
# We make some symlinks during the installer generation process,
# but they aren't always valid on disk afterwards. Some of our tooling,
Expand Down
Loading