Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bazel build is failing on gcc 11 #12702

Closed
davido opened this issue Dec 15, 2020 · 17 comments
Closed

bazel build is failing on gcc 11 #12702

davido opened this issue Dec 15, 2020 · 17 comments
Labels
area-java-toolchains javabase, java_toolchain flags, JDK selection, java_toolchain rules, java_tools repository P3 We're not considering working on this, but happy to review a PR. (No assignee) team-Rules-Java Issues for Java rules

Comments

@davido
Copy link
Contributor

davido commented Dec 15, 2020

Trying to build bazel on gcc 11:

$ bazel --version
bazel 4.0.0rc6

$ bazel build src:bazel-bin-dev
[...]
ERROR: /home/davido/projects/bazel/third_party/ijar/BUILD:47:11: Compiling third_party/ijar/zlib_client.cc failed: (Exit 1): gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer '-std=c++0x' -MD -MF ... (remaining 20 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer '-std=c++0x' -MD -MF ... (remaining 20 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
In file included from third_party/ijar/zlib_client.cc:20:
./third_party/ijar/zlib_client.h:64:46: error: 'numeric_limits' is not a member of 'std'
   64 |   static const size_t MAX_BUFFER_SIZE = std::numeric_limits<int32_t>::max();
      |                                              ^~~~~~~~~~~~~~
./third_party/ijar/zlib_client.h:64:68: error: expected primary-expression before '>' token
   64 |   static const size_t MAX_BUFFER_SIZE = std::numeric_limits<int32_t>::max();
      |                                                                    ^
./third_party/ijar/zlib_client.h:64:71: error: '::max' has not been declared; did you mean 'std::max'?
   64 |   static const size_t MAX_BUFFER_SIZE = std::numeric_limits<int32_t>::max();
      |                                                                       ^~~
      |                                                                       std::max
In file included from /usr/lib/gcc/x86_64-redhat-linux/11/../../../../include/c++/11/algorithm:62,
                 from third_party/ijar/zlib_client.cc:16:
/usr/lib/gcc/x86_64-redhat-linux/11/../../../../include/c++/11/bits/stl_algo.h:3467:5: note: 'std::max' declared here
 3467 |     max(initializer_list<_Tp> __l, _Compare __comp)
      |     ^~~
Target //src:bazel-bin-dev failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 1.734s, Critical Path: 1.12s
INFO: 21 processes: 19 internal, 2 linux-sandbox.
FAILED: Build did NOT complete successfully

GCC version:

$ g++ --version
g++ (GCC) 11.0.0 20201204 (Red Hat 11.0.0-0)
@comius
Copy link
Contributor

comius commented Dec 15, 2020

cc @cushon

@comius comius added area-java-toolchains javabase, java_toolchain flags, JDK selection, java_toolchain rules, java_tools repository team-Rules-Java Issues for Java rules untriaged labels Dec 15, 2020
@davido
Copy link
Contributor Author

davido commented Dec 16, 2020

It's failing in abseil now:

[davido@localhost bazel]$ bazel build src:bazel-dev
DEBUG: /home/davido/.cache/bazel/_bazel_davido/0fa756dec521553dbe2dde6b6eac99b4/external/bazel_toolchains/rules/rbe_repo/checked_in.bzl:103:14: rbe_ubuntu1804_java11 not using checked in configs as detect_java_home was set to True 
DEBUG: /home/davido/.cache/bazel/_bazel_davido/0fa756dec521553dbe2dde6b6eac99b4/external/bazel_toolchains/rules/rbe_repo/checked_in.bzl:103:14: rbe_ubuntu1604_java8 not using checked in configs as detect_java_home was set to True 
Loading: 0 packages loaded
    Fetching @local_config_winsdk; fetching
^C
ERROR: build interrupted
INFO: Elapsed time: 2.056s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
[davido@localhost bazel]$ bazel build src:bazel-bin-dev
DEBUG: /home/davido/.cache/bazel/_bazel_davido/0fa756dec521553dbe2dde6b6eac99b4/external/bazel_toolchains/rules/rbe_repo/checked_in.bzl:103:14: rbe_ubuntu1804_java11 not using checked in configs as detect_java_home was set to True 
DEBUG: /home/davido/.cache/bazel/_bazel_davido/0fa756dec521553dbe2dde6b6eac99b4/external/bazel_toolchains/rules/rbe_repo/checked_in.bzl:103:14: rbe_ubuntu1604_java8 not using checked in configs as detect_java_home was set to True 
INFO: Analyzed target //src:bazel-bin-dev (329 packages loaded, 9074 targets configured).
INFO: Found 1 target...
ERROR: /home/davido/.cache/bazel/_bazel_davido/0fa756dec521553dbe2dde6b6eac99b4/external/com_google_absl/absl/synchronization/BUILD.bazel:30:11: Compiling absl/synchronization/internal/graphcycles.cc failed: (Exit 1): gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer '-std=c++0x' -MD -MF ... (remaining 31 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer '-std=c++0x' -MD -MF ... (remaining 31 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
external/com_google_absl/absl/synchronization/internal/graphcycles.cc: In member function 'void absl::lts_2020_02_25::synchronization_internal::GraphCycles::RemoveNode(void*)':
external/com_google_absl/absl/synchronization/internal/graphcycles.cc:451:26: error: 'numeric_limits' is not a member of 'std'
  451 |   if (x->version == std::numeric_limits<uint32_t>::max()) {
      |                          ^~~~~~~~~~~~~~
external/com_google_absl/absl/synchronization/internal/graphcycles.cc:451:49: error: expected primary-expression before '>' token
  451 |   if (x->version == std::numeric_limits<uint32_t>::max()) {
      |                                                 ^
external/com_google_absl/absl/synchronization/internal/graphcycles.cc:451:52: error: '::max' has not been declared; did you mean 'std::max'?
  451 |   if (x->version == std::numeric_limits<uint32_t>::max()) {
      |                                                    ^~~
      |                                                    std::max
In file included from /usr/lib/gcc/x86_64-redhat-linux/11/../../../../include/c++/11/algorithm:62,
                 from external/com_google_absl/absl/synchronization/internal/graphcycles.cc:38:
/usr/lib/gcc/x86_64-redhat-linux/11/../../../../include/c++/11/bits/stl_algo.h:3467:5: note: 'std::max' declared here
 3467 |     max(initializer_list<_Tp> __l, _Compare __comp)
      |     ^~~
Target //src:bazel-bin-dev failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 107.362s, Critical Path: 18.77s
INFO: 731 processes: 32 internal, 580 linux-sandbox, 1 local, 118 worker.
FAILED: Build did NOT complete successfully

@divanorama
Copy link
Contributor

same but for bazel@HEAD #12756

@meisterT meisterT reopened this Jan 26, 2021
@meisterT
Copy link
Member

@comius can you please have a look at the issue that was raised in the 4.0 release bug?

@comius
Copy link
Contributor

comius commented Jan 26, 2021

I don't have gcc11 to test this through. The problem stems from gcc11 requiring <limits> to be imported. See https://www.gnu.org/software/gcc/gcc-11/porting_to.html#header-dep-change

I'm happy to accept changes if somebody adds missing includes, submits PR to abseil, upgrades it in Bazel.

Any idea when fedora/centos is going to include GCC 11? How critical is it that we patch Bazel LTS with it?

@comius comius added P3 We're not considering working on this, but happy to review a PR. (No assignee) and removed untriaged labels Jan 26, 2021
@divanorama
Copy link
Contributor

divanorama commented Jan 26, 2021

gcc 11 doesn't reallly exist yet, right?
https://gcc.gnu.org/gcc-11/changes.html

Note: GCC 11 has not been released yet

no sources in https://ftp.gnu.org/gnu/gcc/

How does one build GCC 11?

@davido
Copy link
Contributor Author

davido commented Jan 26, 2021

Because it was closed, I opened this issue: #12756.

Should we continue the discussion there?

If you install upcoming Fedora Rawhide 34, then GCC is 11 and Bazel is failing to build.

The issue is outdated abseil that is used by gRPC.

Related: abseil/abseil-cpp#887 and grpc/grpc#25114.

@IleanaAldama
Copy link

sorry if this is not the right place to ask, but is it a way to configure bazel to use an older gcc version? in my system I have gcc10 and gcc11 being the later the default one

@davido
Copy link
Contributor Author

davido commented May 29, 2021

The latest failure in Fedora 34 (gcc 11), in Bazel@HEAD (6ac57ca):

ERROR: /bazel/third_party/ijar/BUILD:10:11: Compiling third_party/ijar/mapped_file_unix.cc failed: (Exit 1): gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer '-std=c++0x' -MD -MF ... (remaining 21 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
third_party/ijar/mapped_file_unix.cc: In constructor 'devtools_ijar::MappedOutputFile::MappedOutputFile(const char*, size_t)':
third_party/ijar/mapped_file_unix.cc:115:21: error: 'numeric_limits' is not a member of 'std'
  115 |                std::numeric_limits<size_t>::max());
      |                     ^~~~~~~~~~~~~~
third_party/ijar/mapped_file_unix.cc:115:42: error: expected primary-expression before '>' token
  115 |                std::numeric_limits<size_t>::max());
      |                                          ^
third_party/ijar/mapped_file_unix.cc:115:45: error: '::max' has not been declared; did you mean 'std::max'?
  115 |                std::numeric_limits<size_t>::max());
      |                                             ^~~
      |                                             std::max
In file included from /usr/lib/gcc/x86_64-redhat-linux/11/../../../../include/c++/11/algorithm:62,
                 from third_party/ijar/mapped_file_unix.cc:21:
/usr/lib/gcc/x86_64-redhat-linux/11/../../../../include/c++/11/bits/stl_algo.h:3467:5: note: 'std::max' declared here
 3467 |     max(initializer_list<_Tp> __l, _Compare __comp)
      |     ^~~
Target //src:bazel-bin-dev failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 373.300s, Critical Path: 110.71s
INFO: 1940 processes: 10 internal, 1204 processwrapper-sandbox, 726 worker.
FAILED: Build did NOT complete successfully

after fixing it with:

diff --git a/third_party/ijar/mapped_file_unix.cc b/third_party/ijar/mapped_file_unix.cc
index 6e3a908718..030e9ca59f 100644
--- a/third_party/ijar/mapped_file_unix.cc
+++ b/third_party/ijar/mapped_file_unix.cc
@@ -19,6 +19,7 @@
 #include <sys/mman.h>
 
 #include <algorithm>
+#include <limits>
 
 #include "third_party/ijar/mapped_file.h"

It's failing in absl with:

ERROR: /root/.cache/bazel/_bazel_root/17ca80f5678de367412ae4ed97070adf/external/com_google_absl/absl/synchronization/BUILD.bazel:30:11: Compiling absl/synchronization/internal/graphcycles.cc failed: (Exit 1): gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer '-std=c++0x' -MD -MF ... (remaining 31 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
external/com_google_absl/absl/synchronization/internal/graphcycles.cc: In member function 'void absl::lts_2020_02_25::synchronization_internal::GraphCycles::RemoveNode(void*)':
external/com_google_absl/absl/synchronization/internal/graphcycles.cc:451:26: error: 'numeric_limits' is not a member of 'std'
  451 |   if (x->version == std::numeric_limits<uint32_t>::max()) {
      |                          ^~~~~~~~~~~~~~
external/com_google_absl/absl/synchronization/internal/graphcycles.cc:451:49: error: expected primary-expression before '>' token
  451 |   if (x->version == std::numeric_limits<uint32_t>::max()) {
      |                                                 ^
external/com_google_absl/absl/synchronization/internal/graphcycles.cc:451:52: error: '::max' has not been declared; did you mean 'std::max'?
  451 |   if (x->version == std::numeric_limits<uint32_t>::max()) {
      |                                                    ^~~
      |                                                    std::max
In file included from /usr/lib/gcc/x86_64-redhat-linux/11/../../../../include/c++/11/algorithm:62,
                 from external/com_google_absl/absl/synchronization/internal/graphcycles.cc:38:
/usr/lib/gcc/x86_64-redhat-linux/11/../../../../include/c++/11/bits/stl_algo.h:3467:5: note: 'std::max' declared here
 3467 |     max(initializer_list<_Tp> __l, _Compare __comp)
      |     ^~~
Target //src:bazel-bin-dev failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 32.569s, Critical Path: 12.29s
INFO: 344 processes: 9 internal, 335 processwrapper-sandbox.
FAILED: Build did NOT complete successfully

@davido davido mentioned this issue May 29, 2021
9 tasks
davido added a commit to davido/bazel that referenced this issue May 29, 2021
Fixes bazelbuild#12702.

Also update transitive dependencies:

o netty to 4.1.52
o netty-tcnative to 2.0.24

Change-Id: Iff6d300255b0c5a6732bb310ac99848ee75a45f6
davido added a commit to davido/bazel that referenced this issue May 29, 2021
Fixes bazelbuild#12702.

Without this include the build is failing with:

  third_party/ijar/mapped_file_unix.cc: In constructor 'devtools_ijar::MappedOutputFile::MappedOutputFile(const char*, size_t)':
  third_party/ijar/mapped_file_unix.cc:115:21: error: 'numeric_limits' is not a member of 'std'
  115 |                std::numeric_limits<size_t>::max());
      |                     ^~~~~~~~~~~~~~
davido added a commit to davido/bazel that referenced this issue May 29, 2021
Fixes bazelbuild#12702.

Also update transitive dependencies:

o netty to 4.1.52
o netty-tcnative to 2.0.24

Change-Id: Iff6d300255b0c5a6732bb310ac99848ee75a45f6
davido added a commit to davido/bazel that referenced this issue May 29, 2021
Fixes bazelbuild#12702.

Also update transitive dependencies:

o netty to 4.1.52
o netty-tcnative to 2.0.24

Change-Id: Iff6d300255b0c5a6732bb310ac99848ee75a45f6
@davido
Copy link
Contributor Author

davido commented May 29, 2021

@comius

I don't have gcc11 to test this through.

I have created this docker repository as reproducer: [1]. Note, it includes bazel 4.1.0, but the build on gcc 11 is also failing on Bazel@HEAD.

Abseil is fixed now, and gRPC was bumped to include the right version of abseil library. I bumped the version of gRPC to 1.38.0 in linked PR and it works as expected, at least on Fedora 34/35. I can build Bazel@HEAD with gcc 11 now and I can also build Gerrit Code Review with the result.

[1] https://github.com/davido/fedora-bazel-docker

@davido
Copy link
Contributor Author

davido commented May 31, 2021

After the upgrade to gRPC 1.38.0, we are running into build errors on CI builders that use outdated c++ compiler:

  $ gcc --version
gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44)

I filed this issue to bump c++ compiler baseline and remove/upgrade outdated CI builders.

//CC @oquenchil @philwo @meteorcloudy

@koiuo
Copy link

koiuo commented Jun 1, 2021

@IleanaAldama,

the path is hardcoded here, in the toolchain

https://github.com/bazelbuild/rules_cc/blob/608c7b605fb844a20e96a3eddc9b49ad2542adab/cc/private/toolchain/cc_toolchain_config.bzl#L1327

There's --compiler option, but you can't just specify gcc-10.2.0 there https://docs.bazel.build/versions/master/user-manual.html#flag--compiler, it seems, it is just for selecting one version among versions already defined.

All in all, I believe it is not possible to just override 2 paths values (for cpp and gcc) without defining a custom gcc-10 toolchain
https://docs.bazel.build/versions/master/tutorial/cc-toolchain-config.html

Please share if you have something working before I do. Thanks!

// sorry for offtop folks, I'd be happy to move this discussion elsewhere, just let me know

UPDATE: what I ended up doing is simply hardcoding a different path for the cc toolchain in the bazel cache 🤦 It's a hack, but at least it unblocks the build of my scala + gRPC project after hours of headbanging 😃

To locate the file to edit I simply grepped entire bazel cache by string /usr/bin/gcc

$ cd .cache/bazel
$ grep -r /usr/bin/gcc .
...
./install/129c3d389c2d09bb0ad5b4a301d67a9c/embedded_tools/tools/cpp/cc_toolchain_config.bzl:            tool_path(name = "gcc", path = "/usr/bin/gcc"),
...

@philwo
Copy link
Member

philwo commented Jun 7, 2021

We cannot easily upgrade gRPC to 1.38.0 if it breaks compatibility with the gcc version used by CentOS 7.

Doing so would mean that Bazel has to drop support for this platform (= we no longer build, test or support CentOS 7 on Bazel CI then). This is particularly problematic for CentOS 7 as we use it as the default distribution to build our releases on, because its software is so old that the binaries that it generates work fine on old distributions like Ubuntu 14.04, too.

Is there no way to support gRPC 1.38.0 without breaking backwards compatibility with gcc 4.8.5? If we have to do so, we have to make the conscious decision to drop support for older platforms.

@philwo
Copy link
Member

philwo commented Jun 7, 2021

Oh, and of course we cannot drop support for CentOS 7 at all in Bazel's 4.x LTS branch, only in the master branch and thus the rolling releases.

@ghost
Copy link

ghost commented Jun 11, 2021

Fails here too on Gentoo Linux with GCC11.1 over amd64 and arm64

@samcom12
Copy link

Fails here as well on Centos Linux with GCC11.1.0 over arm64

@Geremia
Copy link

Geremia commented Dec 18, 2021

@davido

after fixing it with: [inserting #include <limits>] […] It's failing in absl

I get exactly this issue building Bazel 4.2.2 on Slackware current x86_64, GCC 11.2.0 ("ERROR: /tmp/bazel_Xt3fSvNY/out/external/com_google_absl/absl/synchronization/BUILD.bazel:30:11: Compiling absl/synchronization/internal/graphcycles.cc failed: (Exit 1): gcc failed: error executing command").

Building Bazel 6.0.0-pre.20211117.1, I get these errors (and many warnings):

src/main/java/com/google/devtools/build/lib/analysis/config/BuildOptions.java:153: error: cannot find symbol
src/main/java/com/google/devtools/build/lib/skyframe/SkyframeBuildView.java:564: error: cannot find symbol
src/main/java/com/google/devtools/build/lib/analysis/NoBuildEvent.java:84: error: cannot find symbol
src/main/java/com/google/devtools/build/lib/bazel/BazelWorkspaceStatusModule.java:116: error: no suitable method found for toString(Charset)
src/main/java/com/google/devtools/build/lib/bazel/repository/RepositoryResolvedModule.java:136: error: cannot find symbol
src/main/java/com/google/devtools/build/lib/bazel/repository/RepositoryResolvedModule.java:141: error: cannot find symbol
src/main/java/com/google/devtools/build/lib/rules/cpp/CppModuleMapAction.java:117: error: cannot find symbol
src/main/java/com/google/devtools/build/lib/buildtool/buildevent/BuildStartingEvent.java:109: error: cannot find symbol
src/main/java/com/google/devtools/build/lib/remote/merkletree/DirectoryTree.java:230: error: cannot find symbol
src/main/java/com/google/devtools/build/lib/remote/merkletree/DirectoryTree.java:236: error: cannot find symbol
src/main/java/com/google/devtools/build/lib/rules/cpp/LibrariesToLinkCollector.java:109: error: cannot find symbol
src/main/java/com/google/devtools/build/lib/rules/cpp/LibrariesToLinkCollector.java:187: error: cannot find symbol
src/main/java/com/google/devtools/build/lib/runtime/commands/CleanCommand.java:212: error: cannot find symbol
src/main/java/com/google/devtools/build/lib/runtime/commands/info/ServerPidInfoItem.java:31: error: cannot find symbol
src/main/java/com/google/devtools/build/lib/shell/JavaSubprocessFactory.java:118: error: cannot find symbol
src/main/java/com/google/devtools/build/lib/skyframe/ToolchainResolutionFunction.java:430: error: cannot find symbol
src/main/java/com/google/devtools/build/lib/util/SimpleLogHandler.java:371: error: cannot find symbol
src/main/java/com/google/devtools/common/options/OptionsUsage.java:59: error: cannot find symbol
src/main/java/net/starlark/java/eval/EvalUtils.java:399: error: cannot find symbol
19 errors

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-java-toolchains javabase, java_toolchain flags, JDK selection, java_toolchain rules, java_tools repository P3 We're not considering working on this, but happy to review a PR. (No assignee) team-Rules-Java Issues for Java rules
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants