Bazel throws exception if no local JDK is installed #6993

jmillikin · 2018-12-22T13:48:35Z

It looks like this was caused by javabase refactorings in these versions, with two distinct breakages occurring in 0.18 -> 0.19 and 0.20 -> 0.21. Representative build logs from a build with Bazel 0.21:

$ bazel-0.21 build //src:bazel
[...]
ERROR: /Users/john/src/bazel/tools/jdk/BUILD:52:1: Executing genrule //tools/jdk:gen_include/darwin/jni_md.h failed (Exit 1) bash failed: error executing command /bin/bash -c 'source external/bazel_tools/tools/genrule/genrule-setup.sh; cp external/local_jdk/include/darwin/jni_md.h bazel-out/darwin-fastbuild/genfiles/tools/jdk/include/darwin/jni_md.h'

Use --sandbox_debug to see verbose messages from the sandbox
cp: external/local_jdk/include/darwin/jni_md.h: No such file or directory
Target //src:bazel failed to build

The genrule comes from a macro:

def java_runtime_files(name, srcs):
    """Copies the given sources out of the current Java runtime."""

    native.filegroup(
        name = name,
        srcs = srcs,
    )   
    for src in srcs:
        native.genrule(
            name = "gen_%s" % src,
            srcs = ["@bazel_tools//tools/jdk:current_java_runtime"],
            toolchains = ["@bazel_tools//tools/jdk:current_java_runtime"],
            cmd = "cp $(JAVABASE)/%s $@" % src, 
            outs = [src],
        )

Editing that genrule to print out what files it sees indicates that $(JAVABASE) has become decoupled from $(locations @bazel_tools//tools/jdk:current_java_runtime).

v0.18
- $(JAVABASE) =external/local_jdk
- genrule srcs has ./external/local_jdk/include/jni.h
v0.19
- $(JAVABASE) =external/local_jdk
- genrule srcs has ./external/embedded_jdk/include/jni.h
v0.20
- $(JAVABASE) =external/local_jdk
- genrule srcs has ./external/embedded_jdk/include/jni.h
v0.21
- $(JAVABASE) =external/local_jdk
- genrule srcs has ./external/remotejdk_macos/include/jni.h

The text was updated successfully, but these errors were encountered:

jmillikin · 2018-12-22T13:53:37Z

I also sometimes see a different, non-genrule error when building with v0.21:

ERROR: /private/var/tmp/_bazel_john/b9bf2dfa6f7c1a3bab4ff5ab00b48067/external/bazel_tools/tools/jdk/BUILD:188:1: SkylarkAction external/bazel_tools/tools/jdk/platformclasspath.jar failed (Exit 1) java failed: error executing command external/remotejdk_macos/bin/java -XX:+IgnoreUnrecognizedVMOptions '--add-exports=jdk.compiler/com.sun.tools.javac.platform=ALL-UNNAMED' -cp ... (remaining 4 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
Exception in thread "main" java.lang.IllegalArgumentException: external/local_jdk
	at jdk.compiler/com.sun.tools.javac.file.Locations$SystemModulesLocationHandler.isCurrentPlatform(Unknown Source)
	at jdk.compiler/com.sun.tools.javac.file.Locations$SystemModulesLocationHandler.update(Unknown Source)
	at jdk.compiler/com.sun.tools.javac.file.Locations$SystemModulesLocationHandler.handleOption(Unknown Source)
	at jdk.compiler/com.sun.tools.javac.file.Locations.handleOption(Unknown Source)
	at jdk.compiler/com.sun.tools.javac.file.BaseFileManager.handleOption(Unknown Source)
	at jdk.compiler/com.sun.tools.javac.file.BaseFileManager$2.handleFileManagerOption(Unknown Source)
	at jdk.compiler/com.sun.tools.javac.main.Option.process(Unknown Source)
	at jdk.compiler/com.sun.tools.javac.main.Option.handleOption(Unknown Source)
	at jdk.compiler/com.sun.tools.javac.file.BaseFileManager.handleOption(Unknown Source)
	at jdk.compiler/com.sun.tools.javac.main.Arguments.doProcessArgs(Unknown Source)
	at jdk.compiler/com.sun.tools.javac.main.Arguments.processArgs(Unknown Source)
	at jdk.compiler/com.sun.tools.javac.main.Arguments.init(Unknown Source)
	at jdk.compiler/com.sun.tools.javac.api.JavacTool.getTask(Unknown Source)
	at DumpPlatformClassPath.dumpJDK9AndNewerBootClassPath(DumpPlatformClassPath.java:106)
	at DumpPlatformClassPath.main(DumpPlatformClassPath.java:67)
Caused by: java.nio.file.NoSuchFileException: external/local_jdk
	at java.base/sun.nio.fs.UnixException.translateToIOException(Unknown Source)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source)
	at java.base/sun.nio.fs.UnixFileSystemProvider.isSameFile(Unknown Source)
	at java.base/java.nio.file.Files.isSameFile(Unknown Source)
	... 15 more
Target //src:bazel failed to build

davido · 2018-12-23T07:42:22Z

Works here on macOS as expected, on bazel 0.21:

$ bazel build src:bazel
[...]
Target //src:bazel up-to-date:
  bazel-bin/src/bazel
INFO: Elapsed time: 714.512s, Critical Path: 117.40s
INFO: 1838 processes: 1503 darwin-sandbox, 1 local, 334 worker.
INFO: Build completed successfully, 1903 total actions

irengrig · 2018-12-23T11:14:07Z

/cc @cushon

cushon · 2018-12-23T11:37:53Z

@meisterT @iirina

The change from external/embedded_jdk/include/jni.h to external/remotejdk_macos/include/jni.h in 0.21 was 1b75762.

jmillikin · 2018-12-23T15:58:09Z

I put together a quick test case to print out $JAVABASE and the available sandbox files. Here's what I get with v0.21, v0.20, and v0.18:

$ cat BUILD 
genrule(
    name = "gen_java",
    srcs = ["@bazel_tools//tools/jdk:current_java_runtime"],
    outs = ["dummy"],
    cmd = """
echo "JAVABASE: $(JAVABASE)"
echo "--------"
find . | grep 'include/jni.h'
""")

$ bazel-0.21 build --announce_rc :gen_java 
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=1 --terminal_columns=121
INFO: Analysed target //:gen_java (7 packages loaded, 164 targets configured).
INFO: Found 1 target...
INFO: From Executing genrule //:gen_java:
JAVABASE: external/local_jdk
--------
./external/remotejdk_macos/include/jni.h

$ bazel-0.20 build --announce_rc :gen_java
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=1 --terminal_columns=121
INFO: Analysed target //:gen_java (8 packages loaded, 164 targets configured).
INFO: Found 1 target...
INFO: From Executing genrule //:gen_java:
JAVABASE: external/local_jdk
--------
./external/embedded_jdk/include/jni.h

$ bazel-0.18 build --announce_rc :gen_java
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=1 --terminal_columns=121
INFO: Analysed target //:gen_java (8 packages loaded).
INFO: Found 1 target...
INFO: From Executing genrule //:gen_java:
JAVABASE: external/local_jdk
--------
./external/embedded_jdk/include/jni.h
./external/local_jdk/include/jni.h

mahnve · 2018-12-30T16:22:53Z

Total newbie, sorry if this is not related, but I get the " warning: could not find a JDK 8 bootclasspath in external/local_jdk, falling back to --release" when running the examples at https://docs.bazel.build/versions/master/tutorial/java.html with anything later than 0.18.1.

I'm on Void Linux and installed bazel with the .sh installer, JDK openjdk-1.8.0_202

davido · 2018-12-31T18:40:33Z

This is a harmless warning and is a known issue that was fixed on 0.21.

meisterT · 2019-01-07T13:39:51Z

John, we tried to repro the build failure on a Mac with JDK10 and on the Macs on buildkite run JDK8 (see https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/729#a8c276a7-b88e-4341-936a-e970e3ecfa4e) and didn't succeed.

So I'm wondering what could be different on your machine.

jmillikin-stripe · 2019-01-08T18:38:05Z

Hmm, on my work laptop the external/local_jdk/ subdir does get created by Bazel v0.21 in the test genrule:

$ bazel build --announce_rc  :gen_java
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=1 --terminal_columns=119
INFO: Analysed target //:gen_java (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
INFO: From Executing genrule //:gen_java:
JAVABASE: external/local_jdk
--------
./external/remotejdk_macos/include/jni.h
./external/local_jdk/include/jni.h

meisterT · 2019-01-09T07:03:41Z

I assume you can also build Bazel there? Can we close the issue then until you have a more specific repro?

jmillikin · 2019-01-09T15:42:24Z

Can you leave this open? I'd like to further debug why my personal laptop isn't getting a local_jdk directory, despite having the same OS and Bazel version.

meisterT · 2019-01-09T15:44:03Z

Sure, I've assigned it to you.

jmillikin · 2019-01-09T15:45:24Z

current reproduction:

# BUILD
genrule(
    name = "gen_java",
    srcs = ["@bazel_tools//tools/jdk:current_java_runtime"],
    outs = ["dummy"],
    cmd = "find .",
)

INFO: From Executing genrule //:gen_java:
.
./bazel-out
./bazel-out/darwin-fastbuild
./bazel-out/darwin-fastbuild/genfiles
./external
./external/bazel_tools
./external/bazel_tools/tools
./external/bazel_tools/tools/genrule
./external/bazel_tools/tools/genrule/genrule-setup.sh
ERROR: /Users/john/src/test-bzl-java/BUILD:1:1: declared output 'dummy' was not created by genrule.
[...]

jmillikin · 2019-01-10T03:09:12Z

Bazel is resolving @local_jdk to the workspace directory instead of to the embedded JDK. The reason it worked on my work laptop is that there's a system-wide JDK installation that gets picked up instead.

If I try to compile a trivial java_binary, it fails in the early Java toolchain setup because the external/local_jdk directory is missing. That directory is present in the Bazel exec root, but contains the contents of the workspace directory.

$ bazel build -s :hello 
INFO: Invocation ID: b8aea1b3-becf-49d7-a13b-8521d6bdcd86
INFO: Analysed target //:hello (1 packages loaded, 14 targets configured).
INFO: Found 1 target...
SUBCOMMAND: # @bazel_tools//tools/jdk:platformclasspath [action 'SkylarkAction external/bazel_tools/tools/jdk/platformclasspath.jar']
(cd /private/var/tmp/_bazel_john/b017119f836a1084c496762e3ab5a352/execroot/__main__ && \
  exec env - \
  external/remotejdk_macos/bin/java -XX:+IgnoreUnrecognizedVMOptions '--add-exports=jdk.compiler/com.sun.tools.javac.platform=ALL-UNNAMED' -cp bazel-out/darwin-fastbuild/bin/external/bazel_tools/tools/jdk/platformclasspath_classes:external/remotejdk_macos/lib/tools.jar DumpPlatformClassPath bazel-out/darwin-fastbuild/bin/external/bazel_tools/tools/jdk/platformclasspath.jar external/local_jdk)
ERROR: /private/var/tmp/_bazel_john/b017119f836a1084c496762e3ab5a352/external/bazel_tools/tools/jdk/BUILD:188:1: SkylarkAction external/bazel_tools/tools/jdk/platformclasspath.jar failed (Exit 1) java failed: error executing command external/remotejdk_macos/bin/java -XX:+IgnoreUnrecognizedVMOptions '--add-exports=jdk.compiler/com.sun.tools.javac.platform=ALL-UNNAMED' -cp ... (remaining 4 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
Exception in thread "main" java.lang.IllegalArgumentException: external/local_jdk
	at jdk.compiler/com.sun.tools.javac.file.Locations$SystemModulesLocationHandler.isCurrentPlatform(Unknown Source)
	at jdk.compiler/com.sun.tools.javac.file.Locations$SystemModulesLocationHandler.update(Unknown Source)
	at jdk.compiler/com.sun.tools.javac.file.Locations$SystemModulesLocationHandler.handleOption(Unknown Source)
	at jdk.compiler/com.sun.tools.javac.file.Locations.handleOption(Unknown Source)
	at jdk.compiler/com.sun.tools.javac.file.BaseFileManager.handleOption(Unknown Source)
	at jdk.compiler/com.sun.tools.javac.file.BaseFileManager$2.handleFileManagerOption(Unknown Source)
	at jdk.compiler/com.sun.tools.javac.main.Option.process(Unknown Source)
	at jdk.compiler/com.sun.tools.javac.main.Option.handleOption(Unknown Source)
	at jdk.compiler/com.sun.tools.javac.file.BaseFileManager.handleOption(Unknown Source)
	at jdk.compiler/com.sun.tools.javac.main.Arguments.doProcessArgs(Unknown Source)
	at jdk.compiler/com.sun.tools.javac.main.Arguments.processArgs(Unknown Source)
	at jdk.compiler/com.sun.tools.javac.main.Arguments.init(Unknown Source)
	at jdk.compiler/com.sun.tools.javac.api.JavacTool.getTask(Unknown Source)
	at DumpPlatformClassPath.dumpJDK9AndNewerBootClassPath(DumpPlatformClassPath.java:106)
	at DumpPlatformClassPath.main(DumpPlatformClassPath.java:67)
Caused by: java.nio.file.NoSuchFileException: external/local_jdk
	at java.base/sun.nio.fs.UnixException.translateToIOException(Unknown Source)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source)
	at java.base/sun.nio.fs.UnixFileSystemProvider.isSameFile(Unknown Source)
	at java.base/java.nio.file.Files.isSameFile(Unknown Source)
	... 15 more
Target //:hello failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 0.623s, Critical Path: 0.30s
INFO: 0 processes.
FAILED: Build did NOT complete successfully

$ ls bazel-test-bzl-java/external/
bazel_tools/        local_config_cc/    local_config_xcode/ local_jdk/          remotejdk_macos/
$ ls bazel-test-bzl-java/external/local_jdk/
BUILD            BUILD.bazel      HelloWorld.java  WORKSPACE
$ ls
BUILD                HelloWorld.java      WORKSPACE            bazel-bin            bazel-genfiles       bazel-out            bazel-test-bzl-java  bazel-testlogs
$ ls -l bazel-test-bzl-java/external/local_jdk/BUILD
lrwxr-xr-x  1 john  wheel  86 Jan  9 19:02 bazel-test-bzl-java/external/local_jdk/BUILD -> /private/var/tmp/_bazel_john/b017119f836a1084c496762e3ab5a352/external/local_jdk/BUILD
$ ls -l /private/var/tmp/_bazel_john/b017119f836a1084c496762e3ab5a352/external/local_jdk/BUILD
lrwxr-xr-x  1 john  wheel  35 Jan  9 19:02 /private/var/tmp/_bazel_john/b017119f836a1084c496762e3ab5a352/external/local_jdk/BUILD -> /Users/john/src/test-bzl-java/BUILD
$

Running Bazel with either --javabase=@embedded_jdk//:jdk or --javabase=@bazel_tools//tools/jdk:remote_jdk changes the value of $(JAVABASE) and successfully runs the genrules in the first post.

I'm pretty confused about why, exactly, Bazel is defaulting @local_jdk to the workspace directory. There's no custom toolchains registered, $JAVA_HOME is unset, and which javac is /usr/bin/javac.

meisterT · 2019-01-10T07:58:00Z

Although it's a bug that the build fails in that way, it is expected that you have to specify a javabase if you don't have a local JDK installed. Also using the embedded JDK for that is no longer possible starting from the next release since it will no longer be exposed (and with 0.23 minimized).

@lberki may have an idea what's wrong with the local_jdk detection since he fixed some expectations around it IIRC.

lberki · 2019-01-10T09:44:32Z

The implementation of the @local_jdk repository is:

new_local_repository(
    name = "local_jdk",
    path = DEFAULT_SYSTEM_JAVABASE,
    build_file = __embedded_dir__ + "/jdk.BUILD",
)

and if DEFAULT_SYSTEM_JAVABASE is empty, it default to the workspace root.

cushon · 2019-01-10T15:46:17Z

if DEFAULT_SYSTEM_JAVABASE is empty, it default to the workspace root.

Related: "improve diagnostics for incorrect java_runtime configuration (#6118)"

jmillikin-stripe · 2019-01-10T18:19:34Z

I feel like failure to detect a mandatory toolchain should result in a specific error from Bazel, similar to how it complains loudly if it can't find a C++ compiler when building a C++ binary. Defaulting the JDK path to the workspace directory is deeply unintuitive.

lberki · 2019-01-10T18:25:39Z

/cc @katre

katre · 2019-01-10T18:33:45Z

I agree that this needs a better error. We're currently changing how Java rules handle the toolchain and runtime dependencies, this will affect a genrule that uses @bazel_tools//tools/jdk:current_java_runtime. However, I am fairly sure that we will still follow the current behavior when the default system javabase is unset.

jkammerl · 2019-02-04T11:03:37Z

Installing the default-jdk on Debian 9 fixed the problem for me. However, the gen_java target/test from an earlier comment still fails.

See bazelbuild/bazel#6993 Change-Id: If81af47c23f8228b94db41e5b0edbcd60f45718c GitOrigin-RevId: e219a10

meisterT · 2020-05-12T07:59:30Z

cc @Wyverald who ran into the same problem recently

irengrig added P1 I'll work on this now. (Assignee required) team-Rules-Java Issues for Java rules untriaged labels Dec 23, 2018

irengrig assigned cushon Dec 23, 2018

cushon removed their assignment Dec 23, 2018

irengrig assigned meisterT Dec 23, 2018

meisterT assigned jmillikin Jan 9, 2019

meisterT added P2 We'll consider working on this in future. (Assignee optional) and removed P1 I'll work on this now. (Assignee required) untriaged labels Jan 9, 2019

meisterT assigned iirina and unassigned meisterT Jan 18, 2019

meisterT mentioned this issue Jan 31, 2019

Bazel build crash with IllegalArgumentException #7304

Closed

ensonic added a commit to googlecloudrobotics/core that referenced this issue Feb 4, 2019

Add default-jdk to fix the build with recent bazel version.

3715768

See bazelbuild/bazel#6993 Change-Id: If81af47c23f8228b94db41e5b0edbcd60f45718c GitOrigin-RevId: e219a10

meisterT unassigned jmillikin and iirina May 12, 2020

meisterT changed the title ~~@bazel_tools/tools/jdk fails to build at HEAD with Bazel v0.19 - v0.21 on macOS~~ Bazel throws exception if no local JDK is installed May 12, 2020

chanseokoh mentioned this issue Jun 8, 2020

Unable to build nodejs example GoogleContainerTools/distroless#523

Closed

This was referenced Jul 13, 2020

FAILED: Build did NOT complete successfully google/j2cl#34

Closed

Fix #7304: Validate that JAVA_HOME points to a JDK. #7324

Closed

IllegalArgumentException if no local jdk installed #7953

Closed

MikhailTymchukFT mentioned this issue Aug 25, 2020

Coursier does not use embedded JDK bazel-contrib/rules_jvm_external#450

Closed

comius self-assigned this Sep 8, 2020

This was referenced Sep 10, 2020

Implement local_java_repository for local version of Java #12078

Closed

Cannot build src/tools/execlog:parser without local JDK #12012

Closed

bazel-io closed this as completed in f259b8a Sep 16, 2020

comius mentioned this issue Nov 19, 2020

Java targets depend on local jdk incorrectly #10468

Closed

comius mentioned this issue May 20, 2022

Unable to build Java example using homebrew #15452

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bazel throws exception if no local JDK is installed #6993

Bazel throws exception if no local JDK is installed #6993

jmillikin commented Dec 22, 2018

jmillikin commented Dec 22, 2018

davido commented Dec 23, 2018 •

edited

Loading

irengrig commented Dec 23, 2018

cushon commented Dec 23, 2018

jmillikin commented Dec 23, 2018

mahnve commented Dec 30, 2018

davido commented Dec 31, 2018

meisterT commented Jan 7, 2019

jmillikin-stripe commented Jan 8, 2019

meisterT commented Jan 9, 2019

jmillikin commented Jan 9, 2019

meisterT commented Jan 9, 2019

jmillikin commented Jan 9, 2019

jmillikin commented Jan 10, 2019

meisterT commented Jan 10, 2019

lberki commented Jan 10, 2019

cushon commented Jan 10, 2019

jmillikin-stripe commented Jan 10, 2019

lberki commented Jan 10, 2019

katre commented Jan 10, 2019

jkammerl commented Feb 4, 2019

meisterT commented May 12, 2020

Bazel throws exception if no local JDK is installed #6993

Bazel throws exception if no local JDK is installed #6993

Comments

jmillikin commented Dec 22, 2018

jmillikin commented Dec 22, 2018

davido commented Dec 23, 2018 • edited Loading

irengrig commented Dec 23, 2018

cushon commented Dec 23, 2018

jmillikin commented Dec 23, 2018

mahnve commented Dec 30, 2018

davido commented Dec 31, 2018

meisterT commented Jan 7, 2019

jmillikin-stripe commented Jan 8, 2019

meisterT commented Jan 9, 2019

jmillikin commented Jan 9, 2019

meisterT commented Jan 9, 2019

jmillikin commented Jan 9, 2019

jmillikin commented Jan 10, 2019

meisterT commented Jan 10, 2019

lberki commented Jan 10, 2019

cushon commented Jan 10, 2019

jmillikin-stripe commented Jan 10, 2019

lberki commented Jan 10, 2019

katre commented Jan 10, 2019

jkammerl commented Feb 4, 2019

meisterT commented May 12, 2020

davido commented Dec 23, 2018 •

edited

Loading