Missing fcntl.h headers when compiling rust build scripts #29

omerbenamram · 2023-03-30T10:18:50Z

Hi,

I've started looking into using bazel-zig-cc as a replacement for https://github.com/grailbio/bazel-toolchain.
Most stuff builds correctly (including numerous c deps that we have).
We are targeting libc 2.27, which seems to require a workaround.

However a build script for ring seems to not be able to find the fnctl headers, which wasn't a problem when we were using the llvm toolchain with a sysroot.

I've tried to skim the toolchain definitions here, but it seems to be passing the correct include flags. Is there anything else that you may be aware that can cause this?

Thanks for this amazing project, it is compelling because it is so much lighter compared to the 500mb llvm prebuild binaries!

ERROR: /home/oamram/.cache/bazel/_bazel_oamram/620f14f02be8e83cd8d980488ddc8999/external/crate_index__ring-0.16.20/BUILD.bazel:147:19: Running Cargo build script ring [for tool] failed: (Exit 1): cargo_build_script_runner failed: error executing command (from target @crate_index__ring-0.16.20//:ring_build_script) bazel-out/k8-opt-exec-2B5CBBC6/bin/external/rules_rust/cargo/cargo_build_script_runner/cargo_build_script_runner bazel-out/k8-opt-exec-2B5CBBC6/bin/external/crate_index__ring-0.16.20/ring_build_script_ ... (remaining 9 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
Build script process failed with exit code 101
--stdout:
OPT_LEVEL = Some("3")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
cargo:rerun-if-env-changed=CC_x86_64-unknown-linux-gnu
CC_x86_64-unknown-linux-gnu = None
cargo:rerun-if-env-changed=CC_x86_64_unknown_linux_gnu
CC_x86_64_unknown_linux_gnu = None
cargo:rerun-if-env-changed=HOST_CC
HOST_CC = None
cargo:rerun-if-env-changed=CC
CC = Some("/home/oamram/.cache/bazel/_bazel_oamram/620f14f02be8e83cd8d980488ddc8999/sandbox/linux-sandbox/11058/execroot/__main__/external/zig_sdk/tools/x86_64-linux-gnu.2.27/c++")
cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-linux-gnu
CFLAGS_x86_64-unknown-linux-gnu = None
cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_linux_gnu
CFLAGS_x86_64_unknown_linux_gnu = None
cargo:rerun-if-env-changed=HOST_CFLAGS
HOST_CFLAGS = None
cargo:rerun-if-env-changed=CFLAGS
CFLAGS = Some("-no-canonical-prefixes -Wno-builtin-macro-redefined -D__DATE__=\"redacted\" -D__TIMESTAMP__=\"redacted\" -D__TIME__=\"redacted\" -include external/zig_sdk/glibc-hacks/fcntl.h -include external/zig_sdk/glibc-hacks/res_search-amd64.h -O2 -DNDEBUG")
cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
CRATE_CC_NO_DEFAULTS = None
CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")

--stderr:
<built-in>:387:10: fatal error: 'external/zig_sdk/glibc-hacks/fcntl.h' file not found
#include "external/zig_sdk/glibc-hacks/fcntl.h"
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
thread 'main' panicked at 'execution failed', external/crate_index__ring-0.16.20/build.rs:663:9

The text was updated successfully, but these errors were encountered:

motiejus · 2023-03-30T10:28:00Z

I guess it is building it in a different directory, not at the root of the bazel sandbox?

Edit: check with bazel build -s --sandbox_debug <...> to make sure.

Anyhow, this all linker-header hackery will be removed after ziglang/zig#15101 is merged.

omerbenamram · 2023-03-30T10:28:44Z

manually editing the toolchain and adding the sandbox root to include seems to fix it

Is there a way to do it generally for all invocations?

omerbenamram · 2023-03-30T10:36:04Z

It seems to be running under /home/oamram/.cache/bazel/_bazel_oamram/620f14f02be8e83cd8d980488ddc8999/execroot/__main__.

SUBCOMMAND: # @crate_index__ring-0.16.20//:ring_build_script [action 'Running Cargo build script ring', configuration: fbd0f538ecc15c98923e3004fdbfde064b1382cfc68789cde858b7a03d576b63, execution platform: @local_config_platform//:host]
(cd /home/oamram/.cache/bazel/_bazel_oamram/620f14f02be8e83cd8d980488ddc8999/execroot/__main__ && \
  exec env - \
    AR=external/zig_sdk/tools/x86_64-linux-gnu.2.27/ar \
    BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1 \
    CARGO_CRATE_NAME=ring \
    CARGO_ENCODED_RUSTFLAGS='--sysroot=${pwd}/bazel-out/k8-fastbuild/bin/external/rust_linux_x86_64__x86_64-unknown-linux-gnu__stable_tools/rust_toolchain\x1f--cap-lints=allow' \
    CARGO_FEATURE_ALLOC=1 \
    CARGO_FEATURE_DEFAULT=1 \
    CARGO_FEATURE_DEV_URANDOM_FALLBACK=1 \
    CARGO_FEATURE_ONCE_CELL=1 \
    CARGO_FEATURE_STD=1 \
    CARGO_MANIFEST_DIR=bazel-out/k8-opt-exec-2B5CBBC6/bin/external/crate_index__ring-0.16.20/ring_build_script_.runfiles/crate_index__ring-0.16.20/ \
    CARGO_PKG_NAME=ring \
    CARGO_PKG_VERSION=0.16.20 \
    CARGO_PKG_VERSION_MAJOR=0 \
    CARGO_PKG_VERSION_MINOR=16 \
    CARGO_PKG_VERSION_PATCH=20 \
    CARGO_PKG_VERSION_PRE='' \
    CC=external/zig_sdk/tools/x86_64-linux-gnu.2.27/c++ \
    CFLAGS='-no-canonical-prefixes -Wno-builtin-macro-redefined -D__DATE__="redacted" -D__TIMESTAMP__="redacted" -D__TIME__="redacted" -include external/zig_sdk/glibc-hacks/fcntl.h -include external/zig_sdk/glibc-hacks/res_search-amd64.h -fno-lto -Wl,-S' \
    CXX=external/zig_sdk/tools/x86_64-linux-gnu.2.27/c++ \
    CXXFLAGS='-no-canonical-prefixes -Wno-builtin-macro-redefined -D__DATE__="redacted" -D__TIMESTAMP__="redacted" -D__TIME__="redacted" -include external/zig_sdk/glibc-hacks/fcntl.h -include external/zig_sdk/glibc-hacks/res_search-amd64.h -fno-lto -Wl,-S' \
    DEBUG=true \
    HOST=x86_64-unknown-linux-gnu \
    LD=external/zig_sdk/tools/x86_64-linux-gnu.2.27/c++ \
    LDFLAGS='-pie -ldl -lpthread -fno-lto -Wl,-S' \
    NUM_JOBS=1 \
    OPT_LEVEL=0 \
    PATH=/bin:/usr/bin:/usr/local/bin \
    PROFILE=debug \
    RUSTC=bazel-out/k8-fastbuild/bin/external/rust_linux_x86_64__x86_64-unknown-linux-gnu__stable_tools/rust_toolchain/bin/rustc \
    TARGET=x86_64-unknown-linux-gnu \
  bazel-out/k8-opt-exec-2B5CBBC6/bin/external/rules_rust/cargo/cargo_build_script_runner/cargo_build_script_runner bazel-out/k8-opt-exec-2B5CBBC6/bin/external/crate_index__ring-0.16.20/ring_build_script_ ring-asm bazel-out/k8-fastbuild/bin/external/crate_index__ring-0.16.20/ring_build_script.out_dir bazel-out/k8-fastbuild/bin/external/crate_index__ring-0.16.20/ring_build_script.env bazel-out/k8-fastbuild/bin/external/crate_index__ring-0.16.20/ring_build_script.flags bazel-out/k8-fastbuild/bin/external/crate_index__ring-0.16.20/ring_build_script.linkflags bazel-out/k8-fastbuild/bin/external/crate_index__ring-0.16.20/ring_build_script.linksearchpaths bazel-out/k8-fastbuild/bin/external/crate_index__ring-0.16.20/ring_build_script.depenv bazel-out/k8-fastbuild/bin/external/crate_index__ring-0.16.20/ring_build_script.stdout.log bazel-out/k8-fastbuild/bin/external/crate_index__ring-0.16.20/ring_build_script.stderr.log)
# Configuration: fbd0f538ecc15c98923e3004fdbfde064b1382cfc68789cde858b7a03d576b63
# Execution platform: @local_config_platform//:host```

omerbenamram · 2023-03-30T10:53:19Z

Also, another target we're building (jemalloc) uses rules_foreign_cc to invoke make also fails:

checking whether to use statfs or statvfs... statfs
checking crt_externs.h usability... no
checking crt_externs.h presence... no
checking for crt_externs.h... no
checking for _NSGetEnviron... no
checking for newlocale... yes
checking for uselocale... yes
checking for strtod_l... yes
checking for strtoll_l... yes
checking for strtoull_l... yes
checking for C99 vsnprintf... yes
checking whether printf supports positional parameters... yes
checking value of AF_INET... 2
checking value of AF_INET6... 10
checking value of AF_UNIX... 1
checking value of MSG_PEEK... 2
checking value of MSG_OOB... 1
checking value of MSG_DONTROUTE... 4
checking for getprotobyname_r... yes
checking for endservent... yes
checking for if_nametoindex... yes
checking for if_indextoname... yes
checking if arpa/nameser_compat.h is needed... no
checking for res_query... configure: error: not found
configure: error: ./configure failed for glib
_____ END BUILD LOGS _____
rules_foreign_cc: Build wrapper script location: bazel-out/k8-opt-exec-2B5CBBC6/bin/external/rules_foreign_cc/toolchains/pkgconfig_tool_default_foreign_cc/wrapper_build_script.sh
rules_foreign_cc: Build script location: bazel-out/k8-opt-exec-2B5CBBC6/bin/external/rules_foreign_cc/toolchains/pkgconfig_tool_default_foreign_cc/build_script.sh
rules_foreign_cc: Build log location: bazel-out/k8-opt-exec-2B5CBBC6/bin/external/rules_foreign_cc/toolchains/pkgconfig_tool_default_foreign_cc/BootstrapPkgConfig.log

Target @jemalloc//:jemalloc failed to build

From the issue you've linked this seems to be related (res_ symbols renaming?)

motiejus · 2023-03-30T12:10:00Z

Yes, res_* and fcntl are known to have problems in some environments.

You are welcome to dig in and understand what the problem is (I will accept a patch that adds a test case and fixes it), or, in the meanwhile, run on glibc 2.34+ and get back to your desired glibc version when the problem is fully resolved with #29 (comment)

goniz · 2023-04-09T12:06:41Z

I guess it is building it in a different directory, not at the root of the bazel sandbox?

Edit: check with bazel build -s --sandbox_debug <...> to make sure.

Anyhow, this all linker-header hackery will be removed after ziglang/zig#15101 is merged.

@motiejus whats the next steps now that ziglang/zig#15101 is merged upstream?

motiejus · 2023-04-09T18:10:17Z

I guess it is building it in a different directory, not at the root of the bazel sandbox?
Edit: check with bazel build -s --sandbox_debug <...> to make sure.
Anyhow, this all linker-header hackery will be removed after ziglang/zig#15101 is merged.

@motiejus whats the next steps now that ziglang/zig#15101 is merged upstream?

Upgrade the Zig SDK and remove all of the afore-mentioned hacks from bazel-zig-cc.

I am on PTO until 2023-04-17 though -- feel free to pick it up or wait a week until I get back.

omerbenamram changed the title ~~Missing fnctl headers when compiling rust build scripts~~ Missing fcntl.h headers when compiling rust build scripts Mar 30, 2023

goniz mentioned this issue Apr 11, 2023

remove glibc hacks and update Zig SDK #30

Merged

motiejus closed this as completed in #30 Apr 24, 2023

motiejus added this to the v2.0.0 milestone May 3, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Missing fcntl.h headers when compiling rust build scripts #29

Missing fcntl.h headers when compiling rust build scripts #29

omerbenamram commented Mar 30, 2023

motiejus commented Mar 30, 2023 •

edited

Loading

omerbenamram commented Mar 30, 2023 •

edited

Loading

omerbenamram commented Mar 30, 2023 •

edited

Loading

omerbenamram commented Mar 30, 2023

motiejus commented Mar 30, 2023

goniz commented Apr 9, 2023 •

edited

Loading

motiejus commented Apr 9, 2023 •

edited

Loading

Missing fcntl.h headers when compiling rust build scripts #29

Missing fcntl.h headers when compiling rust build scripts #29

Comments

omerbenamram commented Mar 30, 2023

motiejus commented Mar 30, 2023 • edited Loading

omerbenamram commented Mar 30, 2023 • edited Loading

omerbenamram commented Mar 30, 2023 • edited Loading

omerbenamram commented Mar 30, 2023

motiejus commented Mar 30, 2023

goniz commented Apr 9, 2023 • edited Loading

motiejus commented Apr 9, 2023 • edited Loading

motiejus commented Mar 30, 2023 •

edited

Loading

omerbenamram commented Mar 30, 2023 •

edited

Loading

omerbenamram commented Mar 30, 2023 •

edited

Loading

goniz commented Apr 9, 2023 •

edited

Loading

motiejus commented Apr 9, 2023 •

edited

Loading