Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] Update GCC in the dist-linux builders #58416

Merged
merged 2 commits into from
Feb 17, 2019

Conversation

cuviper
Copy link
Member

@cuviper cuviper commented Feb 13, 2019

Upstream LLVM is planning to raise their minimum toolchain requirements, so they may start using C++14 features. This new policy has already landed in the form of a "soft" error.
http://lists.llvm.org/pipermail/llvm-dev/2019-February/130117.html

For GCC, they will require at least version 5.1. This PR moves our crosstool-ng builders to their max GCC 5.2, with a few small patches to fix compatibility. The dist-x86_64-linux builder is updated to GCC 5.5 and LLVM/Clang 8.0.0-rc2, which also affects dist-i686-linux sharing the same scripts.

r? @alexcrichton

This also updates dist-i686-linux, since it borrows the same scripts.

While we're at it, update llvm+clang+lld to llvm-project 8.0.0-rc2.
@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Feb 13, 2019
- Target options > Target Architecture = arm
- Target options > Architecture level = armv6 -- (+)
- Target options > Floating point = software (no FPU) -- (\*)
- Operating System > Target OS = linux
- Operating System > Linux kernel version = 3.2.72 -- Precise kernel
- C-library > glibc version = 2.14.1
- C compiler > gcc version = 4.9.3
- C-library > glibc version = 2.16.0
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note, the arm builders were already configured for glibc 2.16.0, but the README wasn't in sync.

@alexcrichton
Copy link
Member

@bors: r+

Looks great to me, thanks so much for taking care of this @cuviper!

@bors
Copy link
Contributor

bors commented Feb 13, 2019

📌 Commit adb54c2 has been approved by alexcrichton

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 13, 2019
@Centril
Copy link
Contributor

Centril commented Feb 13, 2019

In-between-rollups-filler, @bors p=1

@bors
Copy link
Contributor

bors commented Feb 15, 2019

⌛ Testing commit adb54c2 with merge f1124a1386ea71ff7ca6e354761e3f97c1d947d4...

@rust-highfive
Copy link
Collaborator

The job dist-x86_64-apple-alt of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
[00:01:38]    Compiling serde_derive v1.0.81
[00:01:41]    Compiling toml v0.4.10
[00:01:41]    Compiling serde_json v1.0.33
[00:01:50]    Compiling bootstrap v0.0.0 (/Users/travis/build/rust-lang/rust/src/bootstrap)
No output has been received in the last 30m0s, this potentially indicates a stalled build or something wrong with the build itself.
Check the details on how to adjust your build configuration on: https://docs.travis-ci.com/user/common-build-problems/#Build-times-out-because-no-output-was-received
The build has been terminated

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@bors
Copy link
Contributor

bors commented Feb 15, 2019

💔 Test failed - checks-travis

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Feb 15, 2019
@alexcrichton
Copy link
Member

@bors: retry

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 15, 2019
@kennytm
Copy link
Member

kennytm commented Feb 15, 2019

@bors p=187

rollup fairness

@bors
Copy link
Contributor

bors commented Feb 15, 2019

⌛ Testing commit adb54c2 with merge b196a004f22a522dae16d794aeae64179ebfd95f...

@bors
Copy link
Contributor

bors commented Feb 15, 2019

💔 Test failed - checks-travis

@rust-highfive
Copy link
Collaborator

Your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Feb 15, 2019
@pietroalbini
Copy link
Member

@bors retry -- apparently we can't clone the repo anymore on macOS

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 15, 2019
@pietroalbini
Copy link
Member

@bors treeclosed=100

@kennytm
Copy link
Member

kennytm commented Feb 17, 2019

@bors retry p=1

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 17, 2019
@bors
Copy link
Contributor

bors commented Feb 17, 2019

⌛ Testing commit adb54c2 with merge 5eb74cf050c07af96156394ca278aa7f70950a22...

@kennytm
Copy link
Member

kennytm commented Feb 17, 2019

@bors p=19 treeclosed=9

Let's get this merged as soon as possible to avoid invalidating more docker image cache.

@bors
Copy link
Contributor

bors commented Feb 17, 2019

💔 Test failed - checks-travis

@rust-highfive
Copy link
Collaborator

The job dist-i686-linux of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
 86 67.8M   86 58.9M    0     0  4539k      0  0:00:15  0:00:13  0:00:02 6188k
 98 67.8M   98 66.7M    0     0  4780k      0  0:00:14  0:00:14 --:--:-- 6801k
100 67.8M  100 67.8M    0     0  4817k      0  0:00:14  0:00:14 --:--:-- 6804k
[00:03:59] + cd gcc-5.5.0
[00:03:59] + sed -i 's|ftp://gcc\.gnu\.org/|http://gcc.gnu.org/|g' ./contrib/download_prerequisites
[00:03:59] --2019-02-17 11:23:58--  http://gcc.gnu.org/pub/gcc/infrastructure/mpfr-2.4.2.tar.bz2
[00:03:59] Resolving gcc.gnu.org... 209.132.180.131
[00:03:59] Connecting to gcc.gnu.org|209.132.180.131|:80... connected.
[00:04:00] HTTP request sent, awaiting response... 200 OK
---
[01:10:22]  ---> e460d83ddb40
[01:10:22] Step 25/41 : RUN ./build-clang.sh
[01:10:22]  ---> Running in d90c9a6c2e3f
[01:10:23] + source shared.sh
[01:10:23] + LLVM=llvmorg-8.0.0-rc2
[01:10:23] + mkdir llvm-project
[01:10:23] + cd llvm-project
[01:10:23] + curl -L https://github.com/llvm/llvm-project/archive/llvmorg-8.0.0-rc2.tar.gz
[01:10:23] + tar xzf - --strip-components=1
[01:10:23]                                  Dload  Upload   Total   Spent    Left  Speed
[01:10:23] 
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100   136    0   136    0     0    649      0 --:--:-- --:--:-- --:--:--   743
---
[01:10:40] + cd clang-build
[01:10:40] + INC=/rustroot/include
[01:10:40] + INC=/rustroot/include:/rustroot/lib/gcc/x86_64-unknown-linux-gnu/5.5.0/include-fixed
[01:10:40] + INC=/rustroot/include:/rustroot/lib/gcc/x86_64-unknown-linux-gnu/5.5.0/include-fixed:/usr/include
[01:10:40] + hide_output cmake ../llvm -DCMAKE_C_COMPILER=/rustroot/bin/gcc -DCMAKE_CXX_COMPILER=/rustroot/bin/g++ -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/rustroot -DLLVM_TARGETS_TO_BUILD=X86 '-DLLVM_ENABLE_PROJECTS=clang;lld' -DC_INCLUDE_DIRS=/rustroot/include:/rustroot/lib/gcc/x86_64-unknown-linux-gnu/5.5.0/include-fixed:/usr/include
[01:11:10] Sun Feb 17 12:31:09 UTC 2019 - building ...
[01:11:20] + hide_output make -j10
[01:11:20] + set +x
[01:11:50] Sun Feb 17 12:31:48 UTC 2019 - building ...
---
[02:35:03]  ---> 5dd2fb62d40f
[02:35:03] Step 32/41 : RUN ./build-perl.sh
[02:35:03]  ---> Running in 304f169ed4ca
[02:35:04] + source shared.sh
[02:35:04] + curl https://www.cpan.org/src/5.0/perl-5.28.0.tar.gz
[02:35:04]   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
[02:35:04]                                  Dload  Upload   Total   Spent    Left  Speed
[02:35:05] 
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  7 17.0M    7 1325k    0     0  2843k      0  0:00:06 --:--:--  0:00:06 3553k
100 17.0M  100 17.0M    0     0  14.3M      0  0:00:01  0:00:01 --:--:-- 15.5M
[02:35:05] + cd perl-5.28.0
[02:35:05] + CC=gcc
[02:35:05] + CFLAGS='-I /rustroot/include -fgnu89-inline'
[02:35:05] + hide_output ./configure.gnu
[02:35:05] + set +x
[02:35:35] Sun Feb 17 13:55:34 UTC 2019 - building ...
[02:35:45] + hide_output make -j10

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Feb 17, 2019
@kennytm
Copy link
Member

kennytm commented Feb 17, 2019

@bors retry

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 17, 2019
@bors
Copy link
Contributor

bors commented Feb 17, 2019

⌛ Testing commit adb54c2 with merge 9a3392e...

bors added a commit that referenced this pull request Feb 17, 2019
[CI] Update GCC in the dist-linux builders

Upstream LLVM is planning to raise their minimum toolchain requirements, so they may start using C++14 features. This new policy has already landed in the form of a "soft" error.
http://lists.llvm.org/pipermail/llvm-dev/2019-February/130117.html

For GCC, they will require at least version 5.1. This PR moves our crosstool-ng builders to their max GCC 5.2, with a few small patches to fix compatibility. The dist-x86_64-linux builder is updated to GCC 5.5 and LLVM/Clang 8.0.0-rc2, which also affects dist-i686-linux sharing the same scripts.

r? @alexcrichton
@cuviper
Copy link
Member Author

cuviper commented Feb 17, 2019

Any idea what's invalidating it? It looks like dist-i686-linux has gotten through the docker image a few times, but keeps rebuilding it. The others appear to be cached fine.

@bors
Copy link
Contributor

bors commented Feb 17, 2019

☀️ Test successful - checks-travis, status-appveyor
Approved by: alexcrichton
Pushing 9a3392e to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Feb 17, 2019
@bors bors merged commit adb54c2 into rust-lang:master Feb 17, 2019
@rust-highfive

This comment has been minimized.

@pietroalbini
Copy link
Member

@bors treeclosed- 🎉

bors added a commit that referenced this pull request Feb 18, 2019
…crum

ci: fix docker cache hash collision

#58416 uncovered a bug in our caching for docker images: if the image `foo` pulls files from the image `bar` and a file in `bar` changed, the hash of `foo` will be the same even though it should be different. In that PR's case, `dist-i686-linux` pulls scripts from `dist-x86_64-linux`, and the PR only changed those scripts, causing an hash collision for `dist-i686-linux`.

We have to fix this, since the image will be rebuilt every time bors switches from testing master to testing beta/stable (and when it switches back), making CI way more painful than it currently is.

The approach used by this PR is to just include all the files in `src/ci/docker` in the hash. It's a bit heavy-handed and it will cause a rebuild of all the images every time a single image changes, but it's the best I can think of.

r? @Mark-Simulacrum
cc @alexcrichton @kennytm
@cuviper cuviper deleted the dist-linux-gcc branch March 27, 2019 22:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants