Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-33156][INFRA] Upgrade GithubAction image from 18.04 to 20.04 #30050

Closed
wants to merge 2 commits into from
Closed

[SPARK-33156][INFRA] Upgrade GithubAction image from 18.04 to 20.04 #30050

wants to merge 2 commits into from

Conversation

dongjoon-hyun
Copy link
Member

@dongjoon-hyun dongjoon-hyun commented Oct 15, 2020

What changes were proposed in this pull request?

This PR aims to upgrade Github Action runner image from Ubuntu 18.04 (LTS) to Ubuntu 20.04 (LTS).

Why are the changes needed?

ubuntu-latest in GitHub Action is still Ubuntu 18.04 (LTS).

This upgrade will help Apache Spark 3.1+ preparation for vote and release on the latest OS.

This is tested here.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Pass the Github Action in this PR.

@SparkQA
Copy link

SparkQA commented Oct 15, 2020

Test build #129812 has finished for PR 30050 at commit 8a154db.

  • This patch fails due to an unknown error code, -9.
  • This patch merges cleanly.
  • This patch adds no public classes.

Copy link
Member

@HyukjinKwon HyukjinKwon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like ubuntu-20.04 is a preview one and that's why ubuntu-latest doesn't point this out (I read actions/runner-images#1430 and https://docs.github.com/en/free-pro-team@latest/actions/reference/specifications-for-github-hosted-runners#supported-runners-and-hardware-resources).

Should we just maybe leave it as is so GitHub Actions will updates it in their side? In this way, we could avoid updating this version later when another Ubuntu comes out.

@dongjoon-hyun
Copy link
Member Author

dongjoon-hyun commented Oct 15, 2020

The reason why they are still there officially is that they couldn't verify that how many plugins (or actions) in their market place fail to support it.

However, it's not true for us. I verified that Apache Spark don't use none of the problematic actions in dongjoon-hyun#36 .

And, ubuntu_20.04 is LTS. I don't think we need to upgrade this until next LTS (22.04). It's almost two years.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/34417/
Test FAILed.

@HyukjinKwon
Copy link
Member

Okay, I don't have a strong opinion here. cc @maropu or @srowen

@@ -17,7 +17,7 @@ jobs:
# Build: build Spark and run the tests for specified modules.
build:
name: "Build modules: ${{ matrix.modules }} ${{ matrix.comment }} (JDK ${{ matrix.java }}, ${{ matrix.hadoop }}, ${{ matrix.hive }})"
runs-on: ubuntu-latest
runs-on: ubuntu-20.04
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: how about leaving comments about the next LTS of ubuntu-20.04 here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO latest was a bit ambiguous, so the new one (ubuntu-20.04) can make it easier to understand in which environment we run tests. Thanks, @dongjoon-hyun !

@maropu
Copy link
Member

maropu commented Oct 15, 2020

Seems fine to me

@dongjoon-hyun
Copy link
Member Author

Thank you, @maropu . Sure!

@dongjoon-hyun
Copy link
Member Author

Today, Scala Repo is extremely unstable. I saw the failure in all CIs.

Error:  https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.eed3si9n/sbt-assembly/scala_2.12/sbt_1.0/0.15.0/jars/sbt-assembly.jar: download error: Caught java.io.IOException: Server returned HTTP response code: 504 for URL: https://dl.bintray.com/sbt/sbt-plugin-releases/com.eed3si9n/sbt-assembly/scala_2.12/sbt_1.0/0.15.0/jars/sbt-assembly.jar (Server returned HTTP response code: 504 for URL: https://dl.bintray.com/sbt/sbt-plugin-releases/com.eed3si9n/sbt-assembly/scala_2.12/sbt_1.0/0.15.0/jars/sbt-assembly.jar) while downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.eed3si9n/sbt-assembly/scala_2.12/sbt_1.0/0.15.0/jars/sbt-assembly.jar

@dongjoon-hyun
Copy link
Member Author

Github Action already passed in the previous commit and the last one is only adding comment. I'll merge this.
Thank you so much, @HyukjinKwon and @maropu !
Merged to master.

@dongjoon-hyun dongjoon-hyun deleted the ubuntu_20.04 branch October 15, 2020 09:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants