Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use -native Mill binary in Mill's own build #4294

Merged
merged 58 commits into from
Jan 13, 2025
Merged
Changes from 1 commit
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
af3f67e
.
lihaoyi Jan 12, 2025
c0c6fc5
.
lihaoyi Jan 12, 2025
03b22dd
.
lihaoyi Jan 12, 2025
aca064d
.
lihaoyi Jan 12, 2025
aad0b8e
.
lihaoyi Jan 12, 2025
a20453f
.
lihaoyi Jan 12, 2025
2466c4e
.
lihaoyi Jan 12, 2025
d72821e
.
lihaoyi Jan 12, 2025
f57a34e
.
lihaoyi Jan 12, 2025
6798c51
.
lihaoyi Jan 12, 2025
88f473c
.
lihaoyi Jan 12, 2025
9256259
.
lihaoyi Jan 12, 2025
f0a80b1
.
lihaoyi Jan 12, 2025
97d27bf
.
lihaoyi Jan 12, 2025
2daa594
.
lihaoyi Jan 12, 2025
0f2a784
.
lihaoyi Jan 12, 2025
b69e6e6
.
lihaoyi Jan 12, 2025
9abc762
.
lihaoyi Jan 12, 2025
5968aea
.
lihaoyi Jan 12, 2025
fbd8c6c
.
lihaoyi Jan 12, 2025
e46349f
.
lihaoyi Jan 12, 2025
a3d16ad
.
lihaoyi Jan 12, 2025
8f65a00
.
lihaoyi Jan 12, 2025
b5da7c7
.
lihaoyi Jan 12, 2025
d577683
.
lihaoyi Jan 12, 2025
28bc05b
.
lihaoyi Jan 12, 2025
6748893
.
lihaoyi Jan 12, 2025
1cacb98
.
lihaoyi Jan 12, 2025
cf7fb05
.
lihaoyi Jan 12, 2025
e58b2f3
.
lihaoyi Jan 12, 2025
4c2cee8
.
lihaoyi Jan 12, 2025
f941b90
.
lihaoyi Jan 12, 2025
33d8d3b
.
lihaoyi Jan 12, 2025
6172a04
.
lihaoyi Jan 12, 2025
3ffb4eb
.
lihaoyi Jan 12, 2025
a06d1b5
.
lihaoyi Jan 12, 2025
4479608
.
lihaoyi Jan 12, 2025
25eec9d
.
lihaoyi Jan 12, 2025
2b972fb
.
lihaoyi Jan 12, 2025
4b5216b
.
lihaoyi Jan 12, 2025
c334af0
.
lihaoyi Jan 12, 2025
0c02493
.
lihaoyi Jan 12, 2025
c847f3b
.
lihaoyi Jan 12, 2025
39cb6b7
.
lihaoyi Jan 12, 2025
12caa19
.
lihaoyi Jan 12, 2025
d78619c
.
lihaoyi Jan 12, 2025
795f80b
.
lihaoyi Jan 12, 2025
a69aaa9
.
lihaoyi Jan 12, 2025
85724ed
.
lihaoyi Jan 12, 2025
7e7e119
.
lihaoyi Jan 12, 2025
81d728b
disable problematic test on windows
lihaoyi Jan 12, 2025
fedcec4
consolidate mill-init-error tests
lihaoyi Jan 12, 2025
256ad10
.
lihaoyi Jan 12, 2025
5a6e4b3
Merge branch 'cleanup-tests' into native-bootstrap
lihaoyi Jan 12, 2025
9bba18e
.
lihaoyi Jan 13, 2025
2cb62b0
.
lihaoyi Jan 13, 2025
597dc14
.
lihaoyi Jan 13, 2025
a956d54
.
lihaoyi Jan 13, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
.
lihaoyi committed Jan 12, 2025
commit 97d27bf47d95a27c6f76d4f4eebaa65ec01eb810
5 changes: 1 addition & 4 deletions .github/actions/post-build-setup/action.yml
Original file line number Diff line number Diff line change
@@ -14,10 +14,7 @@ inputs:
runs:
using: "composite"
steps:
- uses: actions/setup-java@v4
with:
java-version: ${{ inputs.java-version }}
distribution: temurin
- run: "echo temurin:${{ inputs.java-version }} > .mill-jvm-version"

# Need to fix cached artifact file permissions because github actions screws it up
# https://github.com/actions/upload-artifact/issues/38
5 changes: 1 addition & 4 deletions .github/workflows/pre-build.yml
Original file line number Diff line number Diff line change
@@ -21,10 +21,7 @@ jobs:
runs-on: ${{ inputs.os }}
timeout-minutes: ${{ inputs.timeout-minutes }}
steps:
- uses: actions/setup-java@v4
with:
java-version: ${{ inputs.java-version }}
distribution: temurin
- run: "echo temurin:${{ inputs.java-version }} > .mill-jvm-version"

- uses: actions/checkout@v4
with:
20 changes: 4 additions & 16 deletions .github/workflows/publish-artifacts.yml
Original file line number Diff line number Diff line change
@@ -33,10 +33,7 @@ jobs:

- uses: coursier/cache-action@v6

- uses: actions/setup-java@v4
with:
java-version: '11'
distribution: temurin
- run: "echo temurin:11 > .mill-jvm-version"

- run: ./mill -i __.publishArtifacts

@@ -81,10 +78,7 @@ jobs:

- uses: coursier/cache-action@v6

- uses: actions/setup-java@v4
with:
java-version: '11'
distribution: temurin
- run: "echo temurin:11 > .mill-jvm-version"

- run: ./mill -i mill.scalalib.PublishModule/

@@ -119,10 +113,7 @@ jobs:
- uses: actions/checkout@v4
with: { fetch-depth: 0 }

- uses: actions/setup-java@v4
with:
java-version: '11'
distribution: temurin
- run: "echo temurin:11 > .mill-jvm-version"

- run: ./mill -i mill.scalalib.PublishModule/ --publishArtifacts dist.native.publishArtifacts

@@ -151,9 +142,6 @@ jobs:

- uses: coursier/cache-action@v6

- uses: actions/setup-java@v4
with:
java-version: '11'
distribution: temurin
- run: "echo temurin:11 > .mill-jvm-version"

- run: ./mill -i dist.uploadToGithub --authKey $REPO_ACCESS_TOKEN
5 changes: 1 addition & 4 deletions .github/workflows/publish-docs.yml
Original file line number Diff line number Diff line change
@@ -27,9 +27,6 @@ jobs:

- uses: coursier/cache-action@v6

- uses: actions/setup-java@v4
with:
java-version: '11'
distribution: temurin
- run: "echo temurin:11 > .mill-jvm-version"

- run: ci/publish-docs.sh
403 changes: 200 additions & 203 deletions .github/workflows/run-tests.yml
Original file line number Diff line number Diff line change
@@ -1,220 +1,217 @@
# Uncommment this to replace the rest of the file when you want to debug stuff in CI
name: Run Debug

on:
push:
pull_request:
workflow_dispatch:

jobs:
debug-windows:
# runs-on: ubuntu-latest
runs-on: windows-latest
steps:
- uses: actions/checkout@v4
with: { fetch-depth: 1 }

- run: ./mill 'integration.feature[launcher-old-versions].packaged.fork.test'
env:
COURSIER_ARCHIVE_CACHE: "C:/coursier-arc"

debug-ubuntu:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with: { fetch-depth: 1 }

- run: ./mill 'integration.feature[launcher-old-versions].packaged.fork.test'

#name: Run Tests
#
## We run full CI on push builds to main and on all pull requests
##
## Manual builds (workflow_dispatch) to the main branch are also published
##
## To maximize bug-catching changes while keeping CI times reasonable, we run:
## - All tests on Linux/Java17
## - Fewer tests on Linux/Java11 and Windows/Java17
## - Fewest tests on Windows/Java11
#name: Run Debug
#
#on:
# push:
# pull_request:
# workflow_dispatch:
#
## cancel older runs of a pull request;
## this will not cancel anything for normal git pushes
#concurrency:
# group: cancel-old-pr-runs-${{ github.workflow }}-${{ github.head_ref || github.run_id }}
# cancel-in-progress: true
#
#jobs:
# # Jobs are listed in rough order of priority: if multiple jobs fail, the first job
# # in the list should be the one that's most worth looking into
# build-linux:
# uses: ./.github/workflows/pre-build.yml
# with:
# os: ubuntu-latest
#
# build-windows:
# uses: ./.github/workflows/pre-build.yml
# with:
# os: windows-latest
#
# test-docs:
# runs-on: ubuntu-latest
# debug-windows:
## runs-on: ubuntu-latest
# runs-on: windows-latest
# steps:
# - uses: actions/checkout@v4
# with: { fetch-depth: 0 }
# with: { fetch-depth: 1 }
#
# - run: ./mill -i docs.githubPages + docs.checkBrokenLinks
# - run: ./mill 'integration.feature[launcher-old-versions].packaged.fork.test'
# env:
# COURSIER_ARCHIVE_CACHE: "C:/coursier-arc"
#
# mac:
# runs-on: macos-latest
# debug-ubuntu:
# runs-on: ubuntu-latest
# steps:
# - uses: actions/checkout@v4
# with: { fetch-depth: 0 }
#
# - uses: actions/setup-java@v4
# with:
# distribution: 'temurin'
# java-version: '17'
#
# - run: ./mill -i example.javalib.basic.__.local.server.test
#
# linux:
# needs: build-linux
# strategy:
# fail-fast: false
# matrix:
#
# include:
# # For most tests, run them arbitrarily on Java 11 or Java 17 on Linux, and
# # on the opposite version on Windows below, so we get decent coverage of
# # each test on each Java version and each operating system
# # We also try to group tests together to manually balance out the runtimes of each jobs
# - java-version: 17
# millargs: "'{main,scalalib,testrunner,bsp,testkit}.__.test'"
# install-android-sdk: false
#
# - java-version: 11
# millargs: "'{scalajslib,scalanativelib,kotlinlib,pythonlib,javascriptlib}.__.test'"
# install-android-sdk: false
#
# - java-version: 17
# millargs: "contrib.__.test"
# install-android-sdk: false
#
# - java-version: 17 # Run this with Mill native launcher as a smoketest
# millargs: "'example.javalib.__.native.server.test'"
# install-android-sdk: false
#
# - java-version: 17
# millargs: "'example.scalalib.__.local.server.test'"
# install-android-sdk: false
#
# - java-version: 17
# millargs: "'example.kotlinlib.__.local.server.test'"
# install-android-sdk: false
#
# - java-version: 17
# millargs: "'example.android.__.local.server.test'"
# install-android-sdk: true
#
# - java-version: 17
# millargs: "'example.{pythonlib,javascriptlib}.__.local.server.test'"
# install-android-sdk: false
#
# - java-version: 11
# millargs: "'example.thirdparty[{mockito,acyclic,commons-io}].local.server.test'"
# install-android-sdk: false
#
# - java-version: 17
# millargs: "'example.thirdparty[{fansi,jimfs,netty,gatling}].local.server.test'"
# install-android-sdk: false
#
# - java-version: '17'
# millargs: "'example.thirdparty[arrow].local.server.test'"
# install-android-sdk: false
#
# - java-version: 11
# millargs: "'example.{cli,fundamentals,depth,extending}.__.local.server.test'"
# install-android-sdk: false
# # Most of these integration tests should not depend on which mode they
# # are run in, so just run them in `local`
# - java-version: '17'
# millargs: "'integration.{failure,feature,ide}.__.native.server.test'"
# install-android-sdk: false
# # These invalidation tests need to be exercised in both execution modes
# # to make sure they work with and without -i/--no-server being passed
# - java-version: 17
# millargs: "'integration.invalidation.__.packaged.fork.test'"
# install-android-sdk: false
#
# - java-version: 17
# millargs: "'integration.invalidation.__.packaged.server.test'"
# install-android-sdk: false
#
# uses: ./.github/workflows/post-build-selective.yml
# with:
# install-android-sdk: ${{ matrix.install-android-sdk }}
# java-version: ${{ matrix.java-version }}
# millargs: ${{ matrix.millargs }}
#
# windows:
# needs: build-windows
# strategy:
# fail-fast: false
# matrix:
# include:
# # just run a subset of examples/ on Windows, because for some reason running
# # the whole suite can take hours on windows v.s. half an hour on linux
# - java-version: 11
# millargs: '"{main,scalalib,bsp}.__.test"'
#
# - java-version: 11
# millargs: '"example.scalalib.{basic,publishing}.__.packaged.fork.test"'
#
# - java-version: 17
# millargs: "'integration.{feature,failure}.__.packaged.fork.test'"
#
# - java-version: 11
# millargs: "'integration.invalidation.__.packaged.server.test'"
#
# - java-version: 11
# millargs: "contrib.__.test"
#
# uses: ./.github/workflows/post-build-selective.yml
# with:
# os: windows-latest
# java-version: ${{ matrix.java-version }}
# millargs: ${{ matrix.millargs }}
# # Provide a shorter coursier archive folder to avoid hitting path-length bugs when
# # running the graal native image binary on windows
# coursierarchive: "C:/coursier-arc"
# with: { fetch-depth: 1 }
#
# itest:
# needs: build-linux
# strategy:
# fail-fast: false
# matrix:
# include:
# # bootstrap tests
# - java-version: 11 # Have one job on oldest JVM
# buildcmd: ci/test-mill-dev.sh && ci/test-mill-release.sh && ./mill -i -k __.ivyDepsTree && ./mill -i -k __.ivyDepsTree --withRuntime
# - java-version: 17 # Have one job on default JVM
# buildcmd: ci/test-mill-bootstrap.sh
# - run: ./mill 'integration.feature[launcher-old-versions].packaged.fork.test'

name: Run Tests

# We run full CI on push builds to main and on all pull requests
#
# uses: ./.github/workflows/post-build-raw.yml
# with:
# java-version: ${{ matrix.java-version }}
# buildcmd: ${{ matrix.buildcmd }}
# Manual builds (workflow_dispatch) to the main branch are also published
#
# # Scalafmt, Mima, and Scalafix job runs last because it's the least important:
# # usually just an automated or mechanical manual fix to do before merging
# lint-autofix:
# needs: build-linux
# uses: ./.github/workflows/post-build-raw.yml
# with:
# java-version: '17'
# buildcmd: ./mill -i mill.scalalib.scalafmt.ScalafmtModule/checkFormatAll __.sources + __.mimaReportBinaryIssues + __.fix --check + mill.javalib.palantirformat.PalantirFormatModule/ --check + mill.kotlinlib.ktlint.KtlintModule/checkFormatAll
# To maximize bug-catching changes while keeping CI times reasonable, we run:
# - All tests on Linux/Java17
# - Fewer tests on Linux/Java11 and Windows/Java17
# - Fewest tests on Windows/Java11

on:
push:
pull_request:
workflow_dispatch:

# cancel older runs of a pull request;
# this will not cancel anything for normal git pushes
concurrency:
group: cancel-old-pr-runs-${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true

jobs:
# Jobs are listed in rough order of priority: if multiple jobs fail, the first job
# in the list should be the one that's most worth looking into
build-linux:
uses: ./.github/workflows/pre-build.yml
with:
os: ubuntu-latest

build-windows:
uses: ./.github/workflows/pre-build.yml
with:
os: windows-latest

test-docs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with: { fetch-depth: 0 }

- run: ./mill -i docs.githubPages + docs.checkBrokenLinks

mac:
runs-on: macos-latest
steps:
- uses: actions/checkout@v4
with: { fetch-depth: 0 }

- run: "echo temurin:17 > .mill-jvm-version"

- run: ./mill -i example.javalib.basic.__.local.server.test

linux:
needs: build-linux
strategy:
fail-fast: false
matrix:

include:
# For most tests, run them arbitrarily on Java 11 or Java 17 on Linux, and
# on the opposite version on Windows below, so we get decent coverage of
# each test on each Java version and each operating system
# We also try to group tests together to manually balance out the runtimes of each jobs
- java-version: 17
millargs: "'{main,scalalib,testrunner,bsp,testkit}.__.test'"
install-android-sdk: false

- java-version: 11
millargs: "'{scalajslib,scalanativelib,kotlinlib,pythonlib,javascriptlib}.__.test'"
install-android-sdk: false

- java-version: 17
millargs: "contrib.__.test"
install-android-sdk: false

- java-version: 17 # Run this with Mill native launcher as a smoketest
millargs: "'example.javalib.__.native.server.test'"
install-android-sdk: false

- java-version: 17
millargs: "'example.scalalib.__.local.server.test'"
install-android-sdk: false

- java-version: 17
millargs: "'example.kotlinlib.__.local.server.test'"
install-android-sdk: false

- java-version: 17
millargs: "'example.android.__.local.server.test'"
install-android-sdk: true

- java-version: 17
millargs: "'example.{pythonlib,javascriptlib}.__.local.server.test'"
install-android-sdk: false

- java-version: 11
millargs: "'example.thirdparty[{mockito,acyclic,commons-io}].local.server.test'"
install-android-sdk: false

- java-version: 17
millargs: "'example.thirdparty[{fansi,jimfs,netty,gatling}].local.server.test'"
install-android-sdk: false

- java-version: '17'
millargs: "'example.thirdparty[arrow].local.server.test'"
install-android-sdk: false

- java-version: 11
millargs: "'example.{cli,fundamentals,depth,extending}.__.local.server.test'"
install-android-sdk: false
# Most of these integration tests should not depend on which mode they
# are run in, so just run them in `local`
- java-version: '17'
millargs: "'integration.{failure,feature,ide}.__.native.server.test'"
install-android-sdk: false
# These invalidation tests need to be exercised in both execution modes
# to make sure they work with and without -i/--no-server being passed
- java-version: 17
millargs: "'integration.invalidation.__.packaged.fork.test'"
install-android-sdk: false

- java-version: 17
millargs: "'integration.invalidation.__.packaged.server.test'"
install-android-sdk: false

uses: ./.github/workflows/post-build-selective.yml
with:
install-android-sdk: ${{ matrix.install-android-sdk }}
java-version: ${{ matrix.java-version }}
millargs: ${{ matrix.millargs }}

windows:
needs: build-windows
strategy:
fail-fast: false
matrix:
include:
# just run a subset of examples/ on Windows, because for some reason running
# the whole suite can take hours on windows v.s. half an hour on linux
- java-version: 11
millargs: '"{main,scalalib,bsp}.__.test"'

- java-version: 11
millargs: '"example.scalalib.{basic,publishing}.__.packaged.fork.test"'

- java-version: 17
millargs: "'integration.{feature,failure}.__.packaged.fork.test'"

- java-version: 11 # Run this with Mill native launcher as a smoketest
millargs: "'integration.invalidation.__.native.server.test'"

- java-version: 11
millargs: "contrib.__.test"

uses: ./.github/workflows/post-build-selective.yml
with:
os: windows-latest
java-version: ${{ matrix.java-version }}
millargs: ${{ matrix.millargs }}
# Provide a shorter coursier archive folder to avoid hitting path-length bugs when
# running the graal native image binary on windows
coursierarchive: "C:/coursier-arc"

itest:
needs: build-linux
strategy:
fail-fast: false
matrix:
include:
# bootstrap tests
- java-version: 11 # Have one job on oldest JVM
buildcmd: ci/test-mill-dev.sh && ci/test-mill-release.sh && ./mill -i -k __.ivyDepsTree && ./mill -i -k __.ivyDepsTree --withRuntime
- java-version: 17 # Have one job on default JVM
buildcmd: ci/test-mill-bootstrap.sh

uses: ./.github/workflows/post-build-raw.yml
with:
java-version: ${{ matrix.java-version }}
buildcmd: ${{ matrix.buildcmd }}

# Scalafmt, Mima, and Scalafix job runs last because it's the least important:
# usually just an automated or mechanical manual fix to do before merging
lint-autofix:
needs: build-linux
uses: ./.github/workflows/post-build-raw.yml
with:
java-version: '17'
buildcmd: ./mill -i mill.scalalib.scalafmt.ScalafmtModule/checkFormatAll __.sources + __.mimaReportBinaryIssues + __.fix --check + mill.javalib.palantirformat.PalantirFormatModule/ --check + mill.kotlinlib.ktlint.KtlintModule/checkFormatAll