Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x86_64-linux: lower glibc version requirement to 2.17 (compatible with centos 7) #501

Merged
merged 10 commits into from
Dec 18, 2020

Conversation

japaric
Copy link
Contributor

@japaric japaric commented Dec 14, 2020

This PR lowers the glibc version requirement of the x86_64-linux-gnu binaries produced by cross to v2.17 (from v2.23). With this change x86_64-linux applications build with cross can run on LTS distributions like centos 7.

The implementation consists of changing the base image of the x86_64-gnu image from ubuntu:16.04 to centos:7
The existing scripts were adapted as follows:

  • package management has been refactored into install_packages and purge_packages functions that work under ubuntu and centos
  • added if_centos and if_ubuntu helper functions to handle steps specific to either distribution
  • qemu.sh: centos doesn't provide libattr, libcap and libpixman as static libraries so those libraries have to be compiled when the script runs on centos
  • linux-image.sh: this script is very ubuntu/debian specific as it relies on apt-get to build a minimal debian sysroot. Instead of modifying the script to make it work on centos, I opted for running the script in a fresh ubuntu:16.04 docker container and copying the output debian sysroot (/qemu) into the centos based image

@japaric japaric requested review from Dylan-DPC-zz and a team as code owners December 14, 2020 15:09
@japaric
Copy link
Contributor Author

japaric commented Dec 14, 2020

note that due #499 the qemu-system runner is not being tested in CI. To ensure this PR is not breaking qemu-system on x86_64, PR #500 should land first.

can't be used within the CI environment (?)
@japaric
Copy link
Contributor Author

japaric commented Dec 16, 2020

https://dev.azure.com/rust-embedded/cross/_build/results?buildId=791&view=logs&j=14306e81-fbe4-59fd-bc5f-032f2af5fc08&t=883603b3-149d-599e-2f7b-6080333ed52c&l=49

$ cargo install --path . --force
(..)
$ cross build --target x86_64-unknown-linux-gnu --release
(..)
/target/release/build/cross-dbf0d01025c300c1/build-script-build: /lib64/libc.so.6: version `GLIBC_2.18' not found (required by /target/release/build/cross-dbf0d01025c300c1/build-script-build)

interesting the first cargo install command, which runs outside docker, creates artifacts that depend on glibc 2.18+ (CI env is ubuntu 20.04), then the second cross command reuses and runs those artifacts (the build scripts) which fails because there's only a glibc 2.17 inside the docker. a cargo clean should fix the issue (you should not mix cargo build --target $T and cross build --target $T commands; that's a known issue/limitation)

(I was under the impression that cargo install was the same as cargo build, which confusingly is not the same as cargo build --target x86_64-unknown-linux-gnu (where target = host) because they use different output target directories (target/debug vs target/x86_64-unknown-linux-gnu/debug. maybe some subtle semantic changed in one of those commands at some point -- or I may be misremembering things)

before invoking cross
avoids runtime failures. only required when target = x86_64-linux-gnu
@japaric
Copy link
Contributor Author

japaric commented Dec 16, 2020

https://dev.azure.com/rust-embedded/cross/_build/results?buildId=792&view=logs&j=8f8c9133-e1c4-5d42-4562-da6ba7afb96e&t=3c0ce6f7-5e36-5685-3cbb-bc4b1fce52ae&l=16436

curl --retry 3 -sSfL https://pkg.freebsd.org/FreeBSD:12:i386/quarterly/All/openssl-1.1.1h,1.txz -O
curl: (22) The requested URL returned error: 404 Not Found

that has failed twice in a row so the URL is probably gone; let's figure out which new URL to use ...

@japaric
Copy link
Contributor Author

japaric commented Dec 16, 2020

PR #502 should fix the freebsd failure

@japaric
Copy link
Contributor Author

japaric commented Dec 17, 2020

Now that #502 has landed, CI for this PR should be green. Could I get some feedback on this PR? (@reitermarkus maybe?)

@reitermarkus
Copy link
Member

bors try

bors bot added a commit that referenced this pull request Dec 17, 2020
@bors
Copy link
Contributor

bors bot commented Dec 17, 2020

try

Build failed:

@japaric
Copy link
Contributor Author

japaric commented Dec 17, 2020

(the "try" build failed due to a network error but the 'multi-stage' commit is green on CI)

@reitermarkus
Copy link
Member

bors r+

bors bot added a commit that referenced this pull request Dec 17, 2020
501: x86_64-linux: lower glibc version requirement to 2.17 (compatible with centos 7) r=reitermarkus a=japaric

This PR lowers the glibc version requirement of the x86_64-linux-gnu binaries produced by cross to v2.17 (from v2.23). With this change x86_64-linux applications build with cross can run on LTS distributions like centos 7.

The implementation consists of changing the base image of the x86_64-gnu image from ubuntu:16.04 to centos:7
The existing scripts were adapted as follows:

- package management has been refactored into `install_packages` and `purge_packages` functions that work under ubuntu and centos
- added `if_centos` and `if_ubuntu` helper functions to handle steps specific to either distribution
- `qemu.sh`: centos doesn't provide libattr, libcap and libpixman as static libraries so those libraries have to be compiled when the script runs on centos
- `linux-image.sh`: this script is very ubuntu/debian specific as it relies on apt-get to build a minimal debian sysroot. Instead of modifying the script to make it work on centos, I opted for running the script in a fresh ubuntu:16.04 docker container and copying the output debian sysroot (`/qemu`) into the centos based image

Co-authored-by: Jorge Aparicio <jorge.aparicio@ferrous-systems.com>
Co-authored-by: Jorge Aparicio <jorge@japaric.io>
@bors
Copy link
Contributor

bors bot commented Dec 17, 2020

Build failed:

@reitermarkus
Copy link
Member

bors retry

bors bot added a commit that referenced this pull request Dec 17, 2020
501: x86_64-linux: lower glibc version requirement to 2.17 (compatible with centos 7) r=reitermarkus a=japaric

This PR lowers the glibc version requirement of the x86_64-linux-gnu binaries produced by cross to v2.17 (from v2.23). With this change x86_64-linux applications build with cross can run on LTS distributions like centos 7.

The implementation consists of changing the base image of the x86_64-gnu image from ubuntu:16.04 to centos:7
The existing scripts were adapted as follows:

- package management has been refactored into `install_packages` and `purge_packages` functions that work under ubuntu and centos
- added `if_centos` and `if_ubuntu` helper functions to handle steps specific to either distribution
- `qemu.sh`: centos doesn't provide libattr, libcap and libpixman as static libraries so those libraries have to be compiled when the script runs on centos
- `linux-image.sh`: this script is very ubuntu/debian specific as it relies on apt-get to build a minimal debian sysroot. Instead of modifying the script to make it work on centos, I opted for running the script in a fresh ubuntu:16.04 docker container and copying the output debian sysroot (`/qemu`) into the centos based image

Co-authored-by: Jorge Aparicio <jorge.aparicio@ferrous-systems.com>
Co-authored-by: Jorge Aparicio <jorge@japaric.io>
@bors
Copy link
Contributor

bors bot commented Dec 17, 2020

Build failed:

@japaric
Copy link
Contributor Author

japaric commented Dec 18, 2020

it's always the musl.sh script that's failing due to "curl: (22) The requested URL returned error: 502 Bad Gateway". I cannot repro locally but the CI fails with a different target each time. Let's try one more time?

bors retry

@bors
Copy link
Contributor

bors bot commented Dec 18, 2020

Build succeeded:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants