Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Add support for specifying a cstdlib other than glibc. #7

Merged
merged 2 commits into from
Dec 20, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 11 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,12 +34,14 @@ load("@bazel_bootlin//toolchains:toolchains.bzl", "bootlin_toolchain_deps")

bootlin_toolchain_deps(
architecture = "x86-64-core-i7",
cstdlib = "glibc",
buildroot_version = "2020.08-1",
)
```

* `architecture` - refers to the [architecture
string](https://toolchains.bootlin.com/toolchains.html) used by Bootlin.
* `cstdlib` - refers to the [cstdlib string](https://toolchains.bootlin.com/releases_x86-64-core-i7.html#:~:text=64%2Dcore%2Di7%2D%2D-,glibc%2D%2Dstable,-%2D2022.08%2D1%20Download) used by bootlin.
* `buildroot_version` - refers to the [Buildroot version
string](https://toolchains.bootlin.com/releases_x86-64-core-i7.html#:~:text=i7%2D%2Dglibc%2D%2Dstable%2D-,2021.11%2D1,-Download%20sha256)
used by Bootlin.
Expand All @@ -49,12 +51,12 @@ used by Bootlin.
Currently `bazel_bootlin` only provides the "glibc--stable" version of the following Bootlin
toolchains:

| Architecture | Buildroot Version |
| --- | --- |
| `x86-64` | `2022.08-1`, `2021.11-5` |
| `x86-64-core-i7` | `2020.08-1` |
| `aarch64` | `2021.11-1`, `2020.08-1` |
| `armv7-eabihf` | `2020.08-1` |
| Architecture | cstdlib | Buildroot Version |
| --- | --- | --- |
| `x86-64` | `glibc` | `2022.08-1`, `2021.11-5` |
| `x86-64-core-i7` |`glibc` | `2020.08-1` |
| `aarch64` | `glibc`, `musl` | `2021.11-1`, `2020.08-1` |
| `armv7-eabihf` | `glibc` | `2020.08-1` |

This list is easily expanded. If a toolchain of interest isn't available feel free to submit and
[issue](https://github.com/agoessling/bazel_bootlin/issues), or alternatively take a look at
Expand All @@ -74,7 +76,7 @@ toolchain that is included. The platforms specify `constraint_value` for the ca

```Starlark
platform(
name = "{architecture}-linux-gnu-{buildroot_version}",
name = "{architecture}-linux-{cstdlib}-{buildroot_version}",
constraint_values = [
"@platforms//cpu:{architecture}",
"@platforms//os:linux",
Expand All @@ -91,15 +93,15 @@ In order to enable toolchain selection via platforms, Bazel requires a special f
target platform:

```Shell
bazel build --incompatible_enable_cc_toolchain_resolution --platforms=@bazel_bootlin//platforms:{architecture}-linux-gnu-{buildroot_version} //...
bazel build --incompatible_enable_cc_toolchain_resolution --platforms=@bazel_bootlin//platforms:{architecture}-linux-{cstdlib}-{buildroot_version} //...
```

The ergonomics can be improved by placing the flags in a
[`.bazelrc`](https://bazel.build/docs/bazelrc) file:

```Shell
build --incompatible_enable_cc_toolchain_resolution
build --platforms=@bazel_bootlin//platforms:{architecture}-linux-gnu-{buildroot_version}
build --platforms=@bazel_bootlin//platforms:{architecture}-linux-{cstdlib}-{buildroot_version}
```

Then a simple `bazel build //...` will utilize the desired toolchain.
35 changes: 18 additions & 17 deletions platforms/platforms.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,23 @@ load("@bazel_bootlin//toolchains:toolchain_info.bzl", "AVAILABLE_TOOLCHAINS")

def all_platforms():
for architecture in AVAILABLE_TOOLCHAINS:
for buildroot_version in AVAILABLE_TOOLCHAINS[architecture]:
if buildroot_version not in native.existing_rules():
native.constraint_value(
name = buildroot_version,
constraint_setting = "@bazel_bootlin//platforms:buildroot_version",
for cstdlib in AVAILABLE_TOOLCHAINS[architecture]:
for buildroot_version in AVAILABLE_TOOLCHAINS[architecture][cstdlib]:
if buildroot_version not in native.existing_rules():
native.constraint_value(
name = buildroot_version,
constraint_setting = "@bazel_bootlin//platforms:buildroot_version",
visibility = ["//visibility:public"],
)

native.platform(
name = "{0}-linux-{1}-{2}".format(architecture, cstdlib, buildroot_version),
constraint_values = [
"@platforms//cpu:{0}".format(
AVAILABLE_TOOLCHAINS[architecture][cstdlib][buildroot_version]["platform_arch"],
),
"@platforms//os:linux",
"@bazel_bootlin//platforms:{0}".format(buildroot_version),
],
visibility = ["//visibility:public"],
)

native.platform(
name = "{0}-linux-gnu-{1}".format(architecture, buildroot_version),
constraint_values = [
"@platforms//cpu:{0}".format(
AVAILABLE_TOOLCHAINS[architecture][buildroot_version]["platform_arch"],
),
"@platforms//os:linux",
"@bazel_bootlin//platforms:{0}".format(buildroot_version),
],
visibility = ["//visibility:public"],
)
110 changes: 66 additions & 44 deletions setup_toolchains.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,41 +3,56 @@

_AVAILABLE_TOOLCHAINS = {
'x86-64': {
'2021.11-5': {
'sha256': '6fe812add925493ea0841365f1fb7ca17fd9224bab61a731063f7f12f3a621b0',
'platform_arch': 'x86_64',
'tool_prefix': 'x86_64-buildroot-linux-gnu',
},
'2022.08-1': {
'sha256': '861c1e8ad0a66e4c28e7a1f8319d68080ab0ff8d16a765e65540f1957203a190',
'platform_arch': 'x86_64',
'tool_prefix': 'x86_64-buildroot-linux-gnu',
'glibc': {
'2021.11-5': {
'sha256': '6fe812add925493ea0841365f1fb7ca17fd9224bab61a731063f7f12f3a621b0',
'platform_arch': 'x86_64',
'tool_prefix': 'x86_64-buildroot-linux-gnu',
},
'2022.08-1': {
'sha256': '861c1e8ad0a66e4c28e7a1f8319d68080ab0ff8d16a765e65540f1957203a190',
'platform_arch': 'x86_64',
'tool_prefix': 'x86_64-buildroot-linux-gnu',
},
},
},
'x86-64-core-i7': {
'2020.08-1': {
'sha256': '3dd408e857f5c8e579748995477f2783fcf5ad0aac89719ea3c5c75446dfa63c',
'platform_arch': 'x86_64',
'tool_prefix': 'x86_64-buildroot-linux-gnu',
'glibc': {
'2020.08-1': {
'sha256': '3dd408e857f5c8e579748995477f2783fcf5ad0aac89719ea3c5c75446dfa63c',
'platform_arch': 'x86_64',
'tool_prefix': 'x86_64-buildroot-linux-gnu',
},
},
},
'aarch64': {
'2021.11-1': {
'sha256': 'dec070196608124fa14c3f192364c5b5b057d7f34651ad58ebb8fc87959c97f7',
'platform_arch': 'aarch64',
'tool_prefix': 'aarch64-buildroot-linux-gnu',
'glibc': {
'2021.11-1': {
'sha256': 'dec070196608124fa14c3f192364c5b5b057d7f34651ad58ebb8fc87959c97f7',
'platform_arch': 'aarch64',
'tool_prefix': 'aarch64-buildroot-linux-gnu',
},
'2020.08-1': {
'sha256': '8ab7a2f17cb96621b048ab0a872650dd62faa54cd74c961b9902b8c04bff7dd1',
'platform_arch': 'aarch64',
'tool_prefix': 'aarch64-buildroot-linux-gnu',
},
},
'2020.08-1': {
'sha256': '8ab7a2f17cb96621b048ab0a872650dd62faa54cd74c961b9902b8c04bff7dd1',
'platform_arch': 'aarch64',
'tool_prefix': 'aarch64-buildroot-linux-gnu',
'musl': {
'2021.11-1': {
'sha256': '6919b4cf04b8c5628a2a93bcf4b05e1143ab15dfb4bc2acde02a2e3c075bf041',
'platform_arch': 'aarch64',
'tool_prefix': 'aarch64-buildroot-linux-musl',
},
},
},
'armv7-eabihf': {
'2020.08-1': {
'sha256': '7b6682603af9a9b5c0e46fd57165723483bb68295e827d14d238e63f33a147a8',
'platform_arch': 'armv7',
'tool_prefix': 'arm-buildroot-linux-gnueabihf',
'glibc': {
'2020.08-1': {
'sha256': '7b6682603af9a9b5c0e46fd57165723483bb68295e827d14d238e63f33a147a8',
'platform_arch': 'armv7',
'tool_prefix': 'arm-buildroot-linux-gnueabihf',
},
},
},
}
Expand Down Expand Up @@ -71,31 +86,37 @@


def create_wrappers(wrapper_dir):
for arch, buildroot_versions in _AVAILABLE_TOOLCHAINS.items():
for arch, cstdlibs in _AVAILABLE_TOOLCHAINS.items():
try:
os.mkdir(os.path.join(wrapper_dir, arch))
except FileExistsError:
pass

for version, version_info in buildroot_versions.items():
for cstdlib, buildroot_versions in cstdlibs.items():
try:
os.mkdir(os.path.join(wrapper_dir, arch, version))
os.mkdir(os.path.join(wrapper_dir, arch, cstdlib))
except FileExistsError:
pass

for tool, tool_info in _ALL_TOOLS.items():
toolchain_name = f'{arch}-linux-gnu-{version}'
wrapper_name = f'{toolchain_name}-{tool}'
actual_tool_name = f'{version_info["tool_prefix"]}-{tool_info["buildroot_name"]}'
wrapper_path = os.path.join(wrapper_dir, arch, version, wrapper_name)
for version, version_info in buildroot_versions.items():
try:
os.mkdir(os.path.join(wrapper_dir, arch, cstdlib, version))
except FileExistsError:
pass

for tool, tool_info in _ALL_TOOLS.items():
toolchain_name = f'{arch}-linux-{cstdlib}-{version}'
wrapper_name = f'{toolchain_name}-{tool}'
actual_tool_name = f'{version_info["tool_prefix"]}-{tool_info["buildroot_name"]}'
wrapper_path = os.path.join(wrapper_dir, arch, cstdlib, version, wrapper_name)

with open(wrapper_path, 'w') as f:
f.write('#!/bin/bash\n')
# Uses buildroot's ".br_real" wrapper to allow -no-canonical-prefix,
# -fno-canonical-system-headers, and --sysroot to correctly work (using relative paths).
f.write(f'exec external/{toolchain_name}/bin/{actual_tool_name} $@\n')
with open(wrapper_path, 'w') as f:
f.write('#!/bin/bash\n')
# Uses buildroot's ".br_real" wrapper to allow -no-canonical-prefix,
# -fno-canonical-system-headers, and --sysroot to correctly work (using relative paths).
f.write(f'exec external/{toolchain_name}/bin/{actual_tool_name} $@\n')

os.chmod(wrapper_path, 0o777)
os.chmod(wrapper_path, 0o777)


def write_toolchain_info(filename):
Expand All @@ -110,11 +131,12 @@ def write_test_script(filename):
f.write('set -e\n')
f.write('set -o xtrace\n\n')

for arch, buildroot_versions in _AVAILABLE_TOOLCHAINS.items():
for version in buildroot_versions:
platform = f'@bazel_bootlin//platforms:{arch}-linux-gnu-{version}'
f.write(f'bazel build --verbose_failures --platforms={platform} //test:test_cpp\n')
f.write(f'bazel build --verbose_failures --platforms={platform} //test:test_c\n')
for arch, cstdlibs in _AVAILABLE_TOOLCHAINS.items():
for cstdlib, buildroot_versions in cstdlibs.items():
for version in buildroot_versions:
platform = f'@bazel_bootlin//platforms:{arch}-linux-{cstdlib}-{version}'
f.write(f'bazel build --verbose_failures --platforms={platform} //test:test_cpp\n')
f.write(f'bazel build --verbose_failures --platforms={platform} //test:test_c\n')

os.chmod(filename, 0o777)

Expand Down
26 changes: 14 additions & 12 deletions test_build_all.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,17 @@
set -e
set -o xtrace

bazel build --verbose_failures --platforms=@bazel_bootlin//platforms:x86-64-linux-gnu-2021.11-5 //test:test_cpp
bazel build --verbose_failures --platforms=@bazel_bootlin//platforms:x86-64-linux-gnu-2021.11-5 //test:test_c
bazel build --verbose_failures --platforms=@bazel_bootlin//platforms:x86-64-linux-gnu-2022.08-1 //test:test_cpp
bazel build --verbose_failures --platforms=@bazel_bootlin//platforms:x86-64-linux-gnu-2022.08-1 //test:test_c
bazel build --verbose_failures --platforms=@bazel_bootlin//platforms:x86-64-core-i7-linux-gnu-2020.08-1 //test:test_cpp
bazel build --verbose_failures --platforms=@bazel_bootlin//platforms:x86-64-core-i7-linux-gnu-2020.08-1 //test:test_c
bazel build --verbose_failures --platforms=@bazel_bootlin//platforms:aarch64-linux-gnu-2021.11-1 //test:test_cpp
bazel build --verbose_failures --platforms=@bazel_bootlin//platforms:aarch64-linux-gnu-2021.11-1 //test:test_c
bazel build --verbose_failures --platforms=@bazel_bootlin//platforms:aarch64-linux-gnu-2020.08-1 //test:test_cpp
bazel build --verbose_failures --platforms=@bazel_bootlin//platforms:aarch64-linux-gnu-2020.08-1 //test:test_c
bazel build --verbose_failures --platforms=@bazel_bootlin//platforms:armv7-eabihf-linux-gnu-2020.08-1 //test:test_cpp
bazel build --verbose_failures --platforms=@bazel_bootlin//platforms:armv7-eabihf-linux-gnu-2020.08-1 //test:test_c
bazel build --verbose_failures --platforms=@bazel_bootlin//platforms:x86-64-linux-glibc-2021.11-5 //test:test_cpp
bazel build --verbose_failures --platforms=@bazel_bootlin//platforms:x86-64-linux-glibc-2021.11-5 //test:test_c
bazel build --verbose_failures --platforms=@bazel_bootlin//platforms:x86-64-linux-glibc-2022.08-1 //test:test_cpp
bazel build --verbose_failures --platforms=@bazel_bootlin//platforms:x86-64-linux-glibc-2022.08-1 //test:test_c
bazel build --verbose_failures --platforms=@bazel_bootlin//platforms:x86-64-core-i7-linux-glibc-2020.08-1 //test:test_cpp
bazel build --verbose_failures --platforms=@bazel_bootlin//platforms:x86-64-core-i7-linux-glibc-2020.08-1 //test:test_c
bazel build --verbose_failures --platforms=@bazel_bootlin//platforms:aarch64-linux-glibc-2021.11-1 //test:test_cpp
bazel build --verbose_failures --platforms=@bazel_bootlin//platforms:aarch64-linux-glibc-2021.11-1 //test:test_c
bazel build --verbose_failures --platforms=@bazel_bootlin//platforms:aarch64-linux-glibc-2020.08-1 //test:test_cpp
bazel build --verbose_failures --platforms=@bazel_bootlin//platforms:aarch64-linux-glibc-2020.08-1 //test:test_c
bazel build --verbose_failures --platforms=@bazel_bootlin//platforms:aarch64-linux-musl-2021.11-1 //test:test_cpp
bazel build --verbose_failures --platforms=@bazel_bootlin//platforms:aarch64-linux-musl-2021.11-1 //test:test_c
bazel build --verbose_failures --platforms=@bazel_bootlin//platforms:armv7-eabihf-linux-glibc-2020.08-1 //test:test_cpp
bazel build --verbose_failures --platforms=@bazel_bootlin//platforms:armv7-eabihf-linux-glibc-2020.08-1 //test:test_c

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/bin/bash
exec external/aarch64-linux-glibc-2020.08-1/bin/aarch64-buildroot-linux-gnu-ar $@
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/bin/bash
exec external/aarch64-linux-glibc-2020.08-1/bin/aarch64-buildroot-linux-gnu-cpp.br_real $@
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/bin/bash
exec external/aarch64-linux-glibc-2020.08-1/bin/aarch64-buildroot-linux-gnu-gcc.br_real $@
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/bin/bash
exec external/aarch64-linux-glibc-2020.08-1/bin/aarch64-buildroot-linux-gnu-gcov $@
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/bin/bash
exec external/aarch64-linux-glibc-2020.08-1/bin/aarch64-buildroot-linux-gnu-ld $@
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/bin/bash
exec external/aarch64-linux-glibc-2020.08-1/bin/aarch64-buildroot-linux-gnu-nm $@
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/bin/bash
exec external/aarch64-linux-glibc-2020.08-1/bin/aarch64-buildroot-linux-gnu-objdump $@
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/bin/bash
exec external/aarch64-linux-glibc-2020.08-1/bin/aarch64-buildroot-linux-gnu-strip $@
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/bin/bash
exec external/aarch64-linux-glibc-2021.11-1/bin/aarch64-buildroot-linux-gnu-ar $@
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/bin/bash
exec external/aarch64-linux-glibc-2021.11-1/bin/aarch64-buildroot-linux-gnu-cpp.br_real $@
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/bin/bash
exec external/aarch64-linux-glibc-2021.11-1/bin/aarch64-buildroot-linux-gnu-gcc.br_real $@
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/bin/bash
exec external/aarch64-linux-glibc-2021.11-1/bin/aarch64-buildroot-linux-gnu-gcov $@
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/bin/bash
exec external/aarch64-linux-glibc-2021.11-1/bin/aarch64-buildroot-linux-gnu-ld $@
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/bin/bash
exec external/aarch64-linux-glibc-2021.11-1/bin/aarch64-buildroot-linux-gnu-nm $@
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/bin/bash
exec external/aarch64-linux-glibc-2021.11-1/bin/aarch64-buildroot-linux-gnu-objdump $@
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/bin/bash
exec external/aarch64-linux-glibc-2021.11-1/bin/aarch64-buildroot-linux-gnu-strip $@
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/bin/bash
exec external/aarch64-linux-musl-2021.11-1/bin/aarch64-buildroot-linux-musl-ar $@
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/bin/bash
exec external/aarch64-linux-musl-2021.11-1/bin/aarch64-buildroot-linux-musl-cpp.br_real $@
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/bin/bash
exec external/aarch64-linux-musl-2021.11-1/bin/aarch64-buildroot-linux-musl-gcc.br_real $@
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/bin/bash
exec external/aarch64-linux-musl-2021.11-1/bin/aarch64-buildroot-linux-musl-gcov $@
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/bin/bash
exec external/aarch64-linux-musl-2021.11-1/bin/aarch64-buildroot-linux-musl-ld $@
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/bin/bash
exec external/aarch64-linux-musl-2021.11-1/bin/aarch64-buildroot-linux-musl-nm $@
Loading