Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bazel_5: init at 5.0.0 #156110

Merged
merged 7 commits into from
Jan 26, 2022
Merged

bazel_5: init at 5.0.0 #156110

merged 7 commits into from
Jan 26, 2022

Conversation

r2r-dev
Copy link
Contributor

@r2r-dev r2r-dev commented Jan 22, 2022 •
edited
Loading

Motivation for this change

Add https://github.com/bazelbuild/bazel/releases/tag/5.0.0

Things done

Bazel 5 is going to be a second long term support release.

Changes from bazel_4:

  • compile java_tools (such as ijar) using nix-provided jdk
  • local jvm and jdk are now passed using --tool_java_runtime_version, --tool_java_runtime_version and --extra_toolchains flags
    see https://docs.bazel.build/versions/5.0.0/bazel-and-java.html for details
  • update-srcDeps.py: distdir_tar function renamed to _distdir_tar to properly parse bazel query output executed against bazel 5 sources
  • bazel_5.updater: uses bazel_4 to generate src-deps.json. This change was required to handle json.decode calls in bazel 5 dependencies
  • protobuf-test and java-test adjusted to take into consideration both jdk 8 (for Bazel < 5.0.0) and jdk 11 (for Bazel >= 5.0.0)

https://blog.bazel.build/2022/01/19/bazel-5.0.html
https://github.com/bazelbuild/bazel/releases/tag/5.0.0

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 22.05 Release Notes (or backporting 21.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
    • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • Fits CONTRIBUTING.md.

@r2r-dev r2r-dev requested a review from Profpatsch as a code owner January 22, 2022 00:09
@AleksanderGondek
Copy link
Contributor

Super excited to see this merged!

@pl-misuw
Copy link

👍 Good job, can't wait for it.

@r2r-dev
bazel_5: init at 5.0.0
eeede61 
Bazel 5 is going to be a second long term support release.

Changes from bazel_4:
- compile java_tools (such as ijar) using nix-provided jdk
- local jvm and jdk are now passed using --tool_java_runtime_version, --tool_java_runtime_version and --extra_toolchains flags
  see https://docs.bazel.build/versions/5.0.0/bazel-and-java.html for details
- update-srcDeps.py: distdir_tar function renamed to _distdir_tar to properly parse bazel query output executed against bazel 5 sources
- bazel_5.updater: uses bazel_4 to generate src-deps.json. This change was required to handle json.decode calls in bazel 5 dependencies
- protobuf-test and java-test adjusted to take into consideration both jdk 8 (for Bazel < 5.0.0) and jdk 11 (for Bazel >= 5.0.0)

https://blog.bazel.build/2022/01/19/bazel-5.0.html
https://github.com/bazelbuild/bazel/releases/tag/5.0.0
@ofborg ofborg bot added the 8.has: package (new) This PR adds a new package label Jan 22, 2022
uri-canva
uri-canva requested changes Jan 22, 2022
View reviewed changes
Copy link
Contributor

@uri-canva uri-canva left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't build it, get this error on Monterey, maybe ofborg will have more luck once you remove the references to the BUILD files that don't exist anymore:

ERROR: /private/tmp/nix-build-bazel-5.0.0.drv-0/bazel_src/tools/osx/BUILD:36:8: Executing genrule //tools/osx:xcode-locator-genrule failed: (Exit 1): bash failed: error executing command
  (cd /private/tmp/nix-build-bazel-5.0.0.drv-0/bazel_w2MkV045/out/execroot/io_bazel && \
  exec env - \
    PATH=/nix/store/srv3qk2wrq058z2wwvdj02hm5jlvcj40-coreutils-9.0/bin:/nix/store/fkygdjhm2raz32xrzwnq8b89hkkdc9vv-python3-3.9.9/bin:/nix/store/n5wfzh32h5z8f4cgvzi18bw9z2cx72li-unzip-6.0/bin:/nix/store/j6psba7fhh8s5f5jws10ab4jxib07vyn-which-2.21/bin:/nix
  /nix/store/b0q4wjdw6xwhfsqy8ab106cmhn5xrr8p-bash-5.1-p12/bin/bash -c 'source external/bazel_tools/tools/genrule/genrule-setup.sh;
  /nix/store/scg11r43cghwmwzdxdnfmf9smg85hdqx-clang-wrapper-11.1.0/bin/clang  -frandom-seed=q9a93n662b -Wno-deprecated-declarations -isystem /nix/store/fkygdjhm2raz32xrzwnq8b89hkkdc9vv-python3-3.9.9/include -isystem /nix/store/cx3cahmpcl9zc3wg6z79zv8j1pj
')
# Configuration: 8e55c9b76f8a990b2d0df99e7ea9a26196076ba338ba4436cfcc96e98e157942
# Execution platform: //:default_host_platform
In file included from tools/osx/xcode_locator.m:24:
In file included from /nix/store/488a9qwl2njhxzqc9jkanww8m6ivrzgn-apple-framework-CoreServices/Library/Frameworks/CoreServices.framework/Headers/CoreServices.h:19:
In file included from /nix/store/f1sj4q111pvlz2gc6fh2giy59p4hjx3j-apple-framework-CoreFoundation/Library/Frameworks/CoreFoundation.framework/Headers/CoreFoundation.h:16:
In file included from /nix/store/8xkv6d53n2hznyvf1vsyknqz4awj77i7-Libsystem-1238.60.2/include/sys/types.h:75:
/nix/store/8xkv6d53n2hznyvf1vsyknqz4awj77i7-Libsystem-1238.60.2/include/sys/cdefs.h:761:2: error: Unsupported architecture
#error Unsupported architecture
 ^
In file included from tools/osx/xcode_locator.m:24:
In file included from /nix/store/488a9qwl2njhxzqc9jkanww8m6ivrzgn-apple-framework-CoreServices/Library/Frameworks/CoreServices.framework/Headers/CoreServices.h:19:
In file included from /nix/store/f1sj4q111pvlz2gc6fh2giy59p4hjx3j-apple-framework-CoreFoundation/Library/Frameworks/CoreFoundation.framework/Headers/CoreFoundation.h:16:
In file included from /nix/store/8xkv6d53n2hznyvf1vsyknqz4awj77i7-Libsystem-1238.60.2/include/sys/types.h:78:
/nix/store/8xkv6d53n2hznyvf1vsyknqz4awj77i7-Libsystem-1238.60.2/include/machine/types.h:37:2: error: architecture not supported
#error architecture not supported
 ^
In file included from tools/osx/xcode_locator.m:24:
In file included from /nix/store/488a9qwl2njhxzqc9jkanww8m6ivrzgn-apple-framework-CoreServices/Library/Frameworks/CoreServices.framework/Headers/CoreServices.h:19:
In file included from /nix/store/f1sj4q111pvlz2gc6fh2giy59p4hjx3j-apple-framework-CoreFoundation/Library/Frameworks/CoreFoundation.framework/Headers/CoreFoundation.h:16:
In file included from /nix/store/8xkv6d53n2hznyvf1vsyknqz4awj77i7-Libsystem-1238.60.2/include/sys/types.h:79:
In file included from /nix/store/8xkv6d53n2hznyvf1vsyknqz4awj77i7-Libsystem-1238.60.2/include/sys/_types.h:33:
/nix/store/8xkv6d53n2hznyvf1vsyknqz4awj77i7-Libsystem-1238.60.2/include/machine/_types.h:34:2: error: architecture not supported
#error architecture not supported
 ^
In file included from tools/osx/xcode_locator.m:24:
In file included from /nix/store/488a9qwl2njhxzqc9jkanww8m6ivrzgn-apple-framework-CoreServices/Library/Frameworks/CoreServices.framework/Headers/CoreServices.h:19:
In file included from /nix/store/f1sj4q111pvlz2gc6fh2giy59p4hjx3j-apple-framework-CoreFoundation/Library/Frameworks/CoreFoundation.framework/Headers/CoreFoundation.h:16:
In file included from /nix/store/8xkv6d53n2hznyvf1vsyknqz4awj77i7-Libsystem-1238.60.2/include/sys/types.h:79:
/nix/store/8xkv6d53n2hznyvf1vsyknqz4awj77i7-Libsystem-1238.60.2/include/sys/_types.h:55:9: error: unknown type name '__int64_t'
typedef __int64_t       __darwin_blkcnt_t;      /* total blocks */
        ^
/nix/store/8xkv6d53n2hznyvf1vsyknqz4awj77i7-Libsystem-1238.60.2/include/sys/_types.h:56:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
typedef __int32_t       __darwin_blksize_t;     /* preferred block size */
        ^
note: '__int128_t' declared here
/nix/store/8xkv6d53n2hznyvf1vsyknqz4awj77i7-Libsystem-1238.60.2/include/sys/_types.h:57:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
typedef __int32_t       __darwin_dev_t;         /* dev_t */
        ^
note: '__int128_t' declared here
/nix/store/8xkv6d53n2hznyvf1vsyknqz4awj77i7-Libsystem-1238.60.2/include/sys/_types.h:60:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
typedef __uint32_t      __darwin_gid_t;         /* [???] process and group IDs */
        ^
note: '__uint128_t' declared here
/nix/store/8xkv6d53n2hznyvf1vsyknqz4awj77i7-Libsystem-1238.60.2/include/sys/_types.h:61:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
typedef __uint32_t      __darwin_id_t;          /* [XSI] pid_t, uid_t, or gid_t*/
        ^
note: '__uint128_t' declared here
/nix/store/8xkv6d53n2hznyvf1vsyknqz4awj77i7-Libsystem-1238.60.2/include/sys/_types.h:62:9: error: unknown type name '__uint64_t'
typedef __uint64_t      __darwin_ino64_t;       /* [???] Used for 64 bit inodes */
        ^
/nix/store/8xkv6d53n2hznyvf1vsyknqz4awj77i7-Libsystem-1238.60.2/include/sys/_types.h:68:9: error: unknown type name '__darwin_natural_t'
typedef __darwin_natural_t __darwin_mach_port_name_t; /* Used by mach */
        ^
/nix/store/8xkv6d53n2hznyvf1vsyknqz4awj77i7-Libsystem-1238.60.2/include/sys/_types.h:70:9: error: unknown type name '__uint16_t'; did you mean '__uint128_t'?
typedef __uint16_t      __darwin_mode_t;        /* [???] Some file attributes */
        ^
note: '__uint128_t' declared here
/nix/store/8xkv6d53n2hznyvf1vsyknqz4awj77i7-Libsystem-1238.60.2/include/sys/_types.h:71:9: error: unknown type name '__int64_t'
typedef __int64_t       __darwin_off_t;         /* [???] Used for file sizes */
        ^
/nix/store/8xkv6d53n2hznyvf1vsyknqz4awj77i7-Libsystem-1238.60.2/include/sys/_types.h:72:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
typedef __int32_t       __darwin_pid_t;         /* [???] process and group IDs */
        ^
note: '__int128_t' declared here
/nix/store/8xkv6d53n2hznyvf1vsyknqz4awj77i7-Libsystem-1238.60.2/include/sys/_types.h:73:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
typedef __uint32_t      __darwin_sigset_t;      /* [???] signal set */
        ^
note: '__uint128_t' declared here
/nix/store/8xkv6d53n2hznyvf1vsyknqz4awj77i7-Libsystem-1238.60.2/include/sys/_types.h:74:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
typedef __int32_t       __darwin_suseconds_t;   /* [???] microseconds */
        ^
note: '__int128_t' declared here
/nix/store/8xkv6d53n2hznyvf1vsyknqz4awj77i7-Libsystem-1238.60.2/include/sys/_types.h:75:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
typedef __uint32_t      __darwin_uid_t;         /* [???] user IDs */
        ^
note: '__uint128_t' declared here
/nix/store/8xkv6d53n2hznyvf1vsyknqz4awj77i7-Libsystem-1238.60.2/include/sys/_types.h:76:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
typedef __uint32_t      __darwin_useconds_t;    /* [???] microseconds */
        ^
note: '__uint128_t' declared here
In file included from tools/osx/xcode_locator.m:24:
In file included from /nix/store/488a9qwl2njhxzqc9jkanww8m6ivrzgn-apple-framework-CoreServices/Library/Frameworks/CoreServices.framework/Headers/CoreServices.h:19:
In file included from /nix/store/f1sj4q111pvlz2gc6fh2giy59p4hjx3j-apple-framework-CoreFoundation/Library/Frameworks/CoreFoundation.framework/Headers/CoreFoundation.h:16:
In file included from /nix/store/8xkv6d53n2hznyvf1vsyknqz4awj77i7-Libsystem-1238.60.2/include/sys/types.h:81:
/nix/store/8xkv6d53n2hznyvf1vsyknqz4awj77i7-Libsystem-1238.60.2/include/machine/endian.h:37:2: error: architecture not supported
#error architecture not supported
 ^
In file included from tools/osx/xcode_locator.m:24:
In file included from /nix/store/488a9qwl2njhxzqc9jkanww8m6ivrzgn-apple-framework-CoreServices/Library/Frameworks/CoreServices.framework/Headers/CoreServices.h:19:
In file included from /nix/store/f1sj4q111pvlz2gc6fh2giy59p4hjx3j-apple-framework-CoreFoundation/Library/Frameworks/CoreFoundation.framework/Headers/CoreFoundation.h:16:
/nix/store/8xkv6d53n2hznyvf1vsyknqz4awj77i7-Libsystem-1238.60.2/include/sys/types.h:95:9: error: unknown type name 'u_int64_t'
typedef u_int64_t               u_quad_t;       /* quads */
        ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
Target //src:bazel_nojdk failed to build
INFO: Elapsed time: 181.874s, Critical Path: 30.91s
INFO: 1382 processes: 526 internal, 856 local.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully

ERROR: Could not build Bazel

pkgs/development/tools/build-managers/bazel/bazel_5/default.nix Outdated Show resolved Hide resolved
@r2r-dev
bazel_5: remove patching of tools/xcode/realpath/BUILD and tools/xcod…
a3de5e7 
…e/stdredirect/BUILD in darwinPatches as these files do not exist anymore
@ofborg ofborg bot requested a review from uri-canva January 22, 2022 13:41
@divanorama
Copy link
Contributor

Tested compilation on Gentoo Linux x86_64 with sandbox=true - looks ok

$ ./result/bin/bazel --version
bazel 5.0.0- (@non-git)

@r2r-dev @divanorama
bazel_5: removed trailing spaces
a8de70b 
Co-authored-by: divanorama <divanorama@gmail.com>
@uri-canva
Copy link
Contributor

I had to do this to make it build on my laptop with Monterey:

diff --git a/pkgs/development/tools/build-managers/bazel/bazel_5/no-arc.patch b/pkgs/development/tools/build-managers/bazel/bazel_5/no-arc.patch
index 9eae38c84ac..5377b979409 100644
--- a/pkgs/development/tools/build-managers/bazel/bazel_5/no-arc.patch
+++ b/pkgs/development/tools/build-managers/bazel/bazel_5/no-arc.patch
@@ -28,7 +28,8 @@

  DARWIN_XCODE_LOCATOR_COMPILE_COMMAND = """
 -  /usr/bin/xcrun --sdk macosx clang -mmacosx-version-min=10.9 -fobjc-arc -framework CoreServices \
+-      -framework Foundation -arch arm64 -arch x86_64 -Wl,-no_adhoc_codesign -Wl,-no_uuid -o $@ $< && \
 +  /usr/bin/xcrun --sdk macosx clang -mmacosx-version-min=10.9 -framework CoreServices \
-       -framework Foundation -arch arm64 -arch x86_64 -Wl,-no_adhoc_codesign -Wl,-no_uuid -o $@ $< && \
++      -framework Foundation -arch x86_64 -Wl,-no_uuid -o $@ $< && \
    env -i codesign --identifier $@ --force --sign - $@

I can now repro the same error as ofborg, but I'm not sure what's going on.

@r2r-dev
bazel_5:
6cd1a18 
- do not build xcode-locator as universal binary
- disable layering_check feature preventing grpc from compiling on darwin
@r2r-dev
bazel_5: Build xcode-locator as a universal binary only on "aarch64-d…
04500fd 
…arwin"
@r2r-dev
bazel_5: Build xcode-locator selectively for x86_64 (on "x86_64-darwi…
e1a2800 
…n") or arm64 (on "aarch64-darwin")
@r2r-dev
Copy link
Contributor Author

r2r-dev commented Jan 25, 2022

Should be good now. I had to disable layering checks in order to resolve build errors on darwin.
https://github.com/grpc/grpc/blob/6d96a2221d1f9377a94d20fb0a8823db9e8c9667/BUILD#L27
bazelbuild/bazel@8b9f746

Compilation succeeded on my Monterey based x86_64-darwin builder.

@uri-canva
Copy link
Contributor

Tested on both x86_64-darwin and aarch64-darwin running Monterey, all good.

timothyklim pushed a commit to timothyklim/bazel-flake that referenced this pull request Jan 25, 2022
fix(macos): try changes from NixOS/nixpkgs#156110
5bc3ab1
@r2r-dev @uri-canva
bazel_5: Explicitly add a dependency on bazel-rc
ed55d1f 
Co-authored-by: Uri Baghin <uri@canva.com>
@ofborg ofborg bot requested a review from uri-canva January 25, 2022 11:37
@uri-canva uri-canva merged commit 4ff1e8d into NixOS:master Jan 26, 2022
@r2r-dev r2r-dev deleted the init-bazel_5 branch January 26, 2022 12:49
@Janik-Haag Janik-Haag added the 12. first-time contribution This PR is the author's first one; please be gentle! label Jun 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@divanorama divanorama divanorama left review comments

@Profpatsch Profpatsch Awaiting requested review from Profpatsch

@marsam marsam Awaiting requested review from marsam

@avdv avdv Awaiting requested review from avdv

@groodt groodt Awaiting requested review from groodt

@mboes mboes Awaiting requested review from mboes

@aherrmann aherrmann Awaiting requested review from aherrmann

@olebedev olebedev Awaiting requested review from olebedev

@ylecornec ylecornec Awaiting requested review from ylecornec

@uri-canva uri-canva Awaiting requested review from uri-canva

Assignees
No one assigned
Labels
8.has: package (new) This PR adds a new package 10.rebuild-darwin: 1-10 10.rebuild-darwin: 1 10.rebuild-linux: 1-10 10.rebuild-linux: 1 12. first-time contribution This PR is the author's first one; please be gentle!
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@r2r-dev @AleksanderGondek @pl-misuw @divanorama @uri-canva @Janik-Haag