library(rJava) fails on Mac with JDK 8 or 10

[r-suzuki]

After updating to the latest version or rJava (0.9-10), library(rJava) fails. It seems to search JDK 9 but it does not exist.

It was working before updating rJava. And I don't have this issue on my Windows 7 machine (JDK 10 is installed).

System: R-3.5.0 (R GUI) on macOS 10.13.5

Error message (with JDK 8)

> library(rJava)
Error: package or namespace load failed for ‘rJava’:
 .onLoad failed in loadNamespace() for 'rJava', details:
  call: dyn.load(file, DLLpath = DLLpath, ...)
  error: unable to load shared object '/Users/<USER_NAME>/Library/R/3.5/library/rJava/libs/rJava.so':
  dlopen(/Users/<USER_NAME>/Library/R/3.5/library/rJava/libs/rJava.so, 6): Library not loaded: /Library/Java/JavaVirtualMachines/jdk-9.jdk/Contents/Home/lib/server/libjvm.dylib
  Referenced from: /Users/<USER_NAME>/Library/R/3.5/library/rJava/libs/rJava.so
  Reason: image not found

I also installed JDK 10 and got similar error:

Error message (with JDK 10)

> library(rJava)
Error: package or namespace load failed for ‘rJava’:
 .onLoad failed in loadNamespace() for 'rJava', details:
  call: dyn.load(file, DLLpath = DLLpath, ...)
  error: unable to load shared object '/Users/<USER_NAME>/Library/R/3.5/library/rJava/libs/rJava.so':
  dlopen(/Users/<USER_NAME>/Library/R/3.5/library/rJava/libs/rJava.so, 6): Library not loaded: @rpath/libjvm.dylib
  Referenced from: /Users/<USER_NAME>/Library/R/3.5/library/rJava/libs/rJava.so
  Reason: image not found

I really appreciate it if you could check the problem.

[markmyatt]

I also see this problem on R v3.4.3 and R v3.5.0 on macOS 10.13.5 and MacOS 10.11.6 (on an older machine) with both JDK 8u171 and JDK 10.0.1.

The problem appeared with rJava 0.9-10. I have tested rJava 0.9-9 with R v3.3.3 on macOS 10.13.5 and MacOS 10.11.6 with both JDK 8u171 and JDK 10.0.1 and there is no problem.

The message I get is:

Error: package or namespace load failed for ‘rJava’: .onLoad failed in loadNamespace() for 'rJava', details: call: dyn.load(file, DLLpath = DLLpath, ...) error: unable to load shared object '/Library/Frameworks/R.framework/Versions/3.5/Resources/library/rJava/libs/rJava.so': dlopen(/Library/Frameworks/R.framework/Versions/3.5/Resources/library/rJava/libs/rJava.so, 6): Library not loaded: /Library/Java/JavaVirtualMachines/jdk-9.jdk/Contents/Home/lib/server/libjvm.dylib Referenced from: /Library/Frameworks/R.framework/Versions/3.5/Resources/library/rJava/libs/rJava.so Reason: image not found

So it looks like it wants JDK 9. I have never had JDK 9 installed on either machine and it is only available from Oracle archives with a recommendation not to use it.

[s-u]

Have you reconfigured R to match you JDK? See R docs (c.f. R CMD javareconf) - you have to configure R to match your Java. If you don't do anything the default CRAN binary falls back to JDK 9, but if you have anything else, you have to configure R accordingly.

[markmyatt]

Simon,

Thanks you for following up.

You write:

Have you reconfigured R to match you JDK? See R docs (c.f. R CMD javareconf) - you have to configure R to match your Java.

Yes. I did that:

$ sudo R CMD javareconf
Java interpreter : /usr/bin/java
Java version     : 10.0.1
Java home path   : /Library/Java/JavaVirtualMachines/jdk-10.0.1.jdk/Contents/Home
Java compiler    : /usr/bin/javac
Java headers gen.: /usr/bin/javah
Java archive tool: /usr/bin/jar
System Java on macOS

trying to compile and link a JNI program 
detected JNI cpp flags    : -I/System/Library/Frameworks/JavaVM.framework/Headers
detected JNI linker flags : -framework JavaVM
clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I/System/Library/Frameworks/JavaVM.framework/Headers -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c conftest.c -o conftest.o
conftest.c:4:5: warning: 'JNI_CreateJavaVM' is deprecated [-Wdeprecated-declarations]
    JNI_CreateJavaVM(0, 0, 0);
    ^
/System/Library/Frameworks/JavaVM.framework/Headers/jni.h:1936:39: note: 
      'JNI_CreateJavaVM' has been explicitly marked deprecated here
_JNI_IMPORT_OR_EXPORT_ __attribute__((deprecated)) jint JNICALL
                                      ^
1 warning generated.
clang -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o conftest.so conftest.o -framework JavaVM -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation


JAVA_HOME        : /Library/Java/JavaVirtualMachines/jdk-10.0.1.jdk/Contents/Home
Java library path: 
JNI cpp flags    : -I/System/Library/Frameworks/JavaVM.framework/Headers
JNI linker flags : -framework JavaVM
Updating Java configuration in /Library/Frameworks/R.framework/Resources
Done.

Just a warning.

This did not fix the issue.

Do I need to do this and then reinstall rJava?

If you don't do anything the default CRAN binary falls back to JDK 9, but if you have anything else, you have to configure R accordingly.

I tried javaconf and a number of other workarounds that I found on the web. Nothing seemed to work. I can load rJava running R in a terminal and in RStudio but not in the R GUI app or R-AnalyticFlow. That is not much use to me as most of my work is now in R-AnalyticFlow which fails to load rJava.

I have 'downgraded" to R 3.3.3 and rJava 0.9-9. With >= R 3.4.x we get rJava 0.9-10 and that leads to the problem. my impression is that the issue is with rJava 0.9-10 as this failed to load with no change to the JDK giving the same error message. The downgrade is not ideal but it gives me my workflows back.

[r-suzuki]

@s-u, thank you for your feedback.

For years R CMD javareconf was not needed on Mac for me, where I:

• Use binary version of R packages
• Launch R via JRI and then call rJava (R AnalyticFlow)

But anyway I tried R CMD javareconf and got the same result as @markmyatt, that is:

• library(rJava) works with R on the Terminal
• Fails with the same jdk-9 related error on R GUI

And I also find that

• An older version of rJava (0.9-9) has the same issue

So here's the summary of what I've tried. All uses R-3.5.0 on macOS 10.13.5 and JDK 10.

	R on Terminal	R GUI	R AnalyticFlow (call rJava via JRI)
rJava 0.9-9 (default)	Fail	Fail	OK
rJava 0.9-9 (javareconf)	OK	Fail	OK
rJava 0.9-10 (default)	Fail	Fail	Fail
rJava 0.9-10 (javareconf)	OK	Fail	Fail

So I can say:

• Both 0.9-9 and 0.9-10 does not work on R GUI but works on command-line after R CMD javareconf
• 0.9-9 worked via JRI but 0.9-10 not, even after R CMD javareconf

[markmyatt]

It is perhaps inevitable given the complication and brittleness of many systems that I have a different experience from @r-suzuki.

I find no issues with rJava 0.9-9 on any recent version of R in the Terminal, R GUI, RStudio, or R AnalyticFlow. Problems arise with rJava 0.9.10 in R GUI and R AnalyticFlow.

[r-suzuki]

@s-u, finally I have found the source of the problem. On Mac .onLoad only works fine if environment variable JAVA_HOME and DYLD_LIBRARY_PATH are set, but they are not set on R GUI (even after sudo R CMD javareconf).

When calling rJava via JRI, there was no problem with 0.9-9. However this commit (90897df) may have changed the behavior. Setting JAVA_HOME and DYLD_LIBRARY_PATH before calling R (via JRI) solved the problem.

For R GUI users it will be good if rJava can work without JAVA_HOME and DYLD_LIBRARY_PATH, or R GUI should set them.

[r-suzuki]

I updated the comment above to include DYLD_LIBRARY_PATH.

[markmyatt]

Just to confirm ... that does do the trick in R AnalyticFlow.

Many thanks for the excellent support.

[s-u]

@r-suzuki this has nothing to do with JAVA_HOME - what you describe is expected. Please do NOT set JAVA_HOME - you're more likely to get it wrong and break things.

Unless you re-compile rJava against your Java and fix the paths it will only work via the R script, not otherwise - this is due to the binaries that Oracle is supplying.

The gory details: Unlike Apple, Oracle supplies Java binaries that don't work via jni, because they don't include the path to the Java installation in libjvm so it's impossible to link to it. Hence R uses javareconf to detect the correct settings and those are set by the R script. Hence for all Java support you have to always call the R script for the autodetection to work.

If you don't use the R script then you have to use one of the several ways to fix it. The most durable is to set the link path in rJava.so to the major version of the JDK - that's what the CRAN binaries do so they work with JDK 9. If you use any other version, you have to change the path accordingly via install_name_tool -change, so, e.g., if you wanted to switch from JDK 9 to 10 you'd use

install_name_tool -change \
  /Library/Java/JavaVirtualMachines/jdk-9.jdk/Contents/Home/lib/server/libjvm.dylib \
  /Library/Java/JavaVirtualMachines/jdk-10.jdk/Contents/Home/lib/server/libjvm.dylib \ 
  /Library/Frameworks/R.framework/Resources/library/rJava/libs/rJava.so

Unfortunately this is an issue created by Oracle and no one else can fix it other than by modifying their binary to set the correct path (which is what Apple used to do).

@markmyatt please don't set DYLD_LIBRARY_PATHmanually - you should use R CMD to start any jri programs as that guarantees the correct settings - that is the only supported way for starting JRI since you have to set all the environment variables that R needs which only R CMD can do reliably.

[markmyatt]

At the risk of going beyond the level of my ability ... no harm in appearing foolish ... Apple does provide a helper utility:

`/usr/libexec/java_home`

that, AFAIK, will reliably give the correct path to the default JDK. It can be used to find paths to different JDKs using filters for versions number and CPU architectures. I find it surprising that rJava and/or R GUI does not use this utility. A fallback to a hard coded path that may not exist does not seem sensible. This is just carping. I now have a working R-AnalyticFlow installation with current R and libraries running nicely and I am very happy about that.

[s-u]

If you checked the code, you'd find out that it does ... That's exactly why I said don't set JAVA_HOME.

[r-suzuki]

Thank you so much @s-u for details. I tried install_name_tool -change and now library(rJava) works fine on R GUI.

It would be great if it is automatically done in the binary installation process of rJava, but I'm not sure whether it's possible. Anyway thank you for your help!

[markmyatt]

Using install_name_change did the trick. Thanks.

I think the process is a little tricky as the user needs to know the full pathname to libjvm.dylibwhich will vary by Java version and the path to rJava.so which may be buried in the systemwide or user Library folder. This may be beyond many R users.

I agree with @r-suzuki that it would be better if this could be done during package installation. The problem with this is that updating Java might need rJava to be reinstalled.

[s-u]

No, unfortunately, it's not, because binaries are just files that are unpacked - there is no processing involved.

The issue is really with Oracle - before they took over Apple was using a standard OS X framework structure so it worked fine regardless of the version. We had binaries that just worked - as you would expect on macOS. Unfortunately, Oracle doesn't seem to understand macOS at all so they ship their binaries in random locations with each update and without valid ID entry in binary so it's impossible to link against it. They break all the standard rules that Apple makes which is presumably why Apple stopped supporting Java altogether.

[markmyatt]

I'd not like to apportion blame to just one side. Apple can be difficult and arbitrary at times. Did they really serve our best interests by taking their ball home WRT Java? This caused all sorts problems with other applications (e.g. SciLab). Sometimes I think they are not sure where to go with macOS.

The good news here is that we have a working fix for the issue.

[s-u]

IMHO, yes, they did, since Sun at the time had no solution at all. The issues were mostly related to the fact that Java is not really run-anywhere despite its promises - that's not Apple's fault. Apple actually had the only properly working Java on OS X.

What's so frustrating is that if Oracle actually had anyone with a clue of OS X it's easy to fix - just create a standard framework which is a well-defined process and everything would work just fine. It's no rocket science - they could even just look at what Apple did and all would be well. It's also trivial to fix it, but as you said, you can't expect macOS users to do that themselves ...

[s-u]

FWIW I think I'll just switch the CRAN machine to JDK 10 - then at least anyone with the currently supported Java will be fine.

[markmyatt]

The old joke is that Java's "Write once. Run anywhere" promise is the usual case of "Write Once. Debug everywhere".

That last post may have convinced me about Oracle.

With (e.g) Java 8 the updates where on a major version number path so that if you had Java 8 and updated it it was still at:

`/Library/Java/JavaVirtualMachines/jdk-8.jdk ...`.

I fear this may have changed. Your line above had jdk-10.jdk but that did not work on my machines here. I needed jdk-10.0.1.jdk. We may now see different paths for minor upgrades which will break the fix you suggest.

I see a trend for apps to use an embedded JRE to avoid a lot of Java version issues. SciLab now does this in version 6 and later. Recent versions of R AnalyticFlow (RAF) do this. I think that is why the fix from @r-suzuki works but is RAF specific.

[r-suzuki]

@s-u Is there any standard way to use an alternative JRE other than the system default, when I want to use rJava via jri on Mac?

To launch a jri program without rJava, this simply works after sudo R CMD javareconf:

R CMD java -cp <ClassPath> <JavaClass>

And R CMD javareconf -e works perfect if I want to change the JRE:

R CMD javareconf -e JAVA_HOME=<JAVA_HOME> "java -cp <ClassPath> <JavaClass>"

If I need to use rJava via jri with the default JRE, it works afterinstall_name_change to the system's default JRE and if rJava/java/boot/ is included in the class path (as I have reported in #132):

R CMD java -cp <ClassPath>:<R LIBPATH>/rJava/java/boot/ <JavaClass>

However changing JRE does not work in this situation with R CMD javareconf -e:

R CMD javareconf -e JAVA_HOME=<JAVA_HOME> "java -cp <ClassPath>:<R LIBPATH>/rJava/java/boot/ <JavaClass>"

It fails to load rJava as R GUI does:

Error occurred during initialization of VM
Unable to load native library: dlopen(/Library/Java/JavaVirtualMachines/jdk-10.0.1.jdk/Contents/Home/lib/libjava.dylib, 1): Symbol not found: _JVM_GetNanoTimeAdjustment
  Referenced from: /Library/Java/JavaVirtualMachines/jdk-10.0.1.jdk/Contents/Home/lib/libjava.dylib
  Expected in: /Applications/RAnalyticFlow.app/Contents/Resources/ext/jre/lib/server/libjvm.dylib
 in /Library/Java/JavaVirtualMachines/jdk-10.0.1.jdk/Contents/Home/lib/libjava.dylib

I needed to use install_name_change to see the libjvm.dylib of the alternate JRE to make it work in this way. Another workaround that worked for me was to set DYLD_LIBRARY_PATH to change the way dlopen searches the library.

It could be more of R's problem with Java. Thank you in advance for taking your time.

[r-suzuki]

JFYI this is the code used for testing:

import org.rosuda.JRI.REXP;
import org.rosuda.JRI.Rengine;

public class JRIrJavaTest {
	public static void main(String[] args)
	{
		Rengine engine = new Rengine(new String[]{"--no-save"}, false, null);
		REXP result = engine.eval("try({"
				+ "rJava::.jinit();"
				+ "rJava::.jcall(\"java/lang/System\",\"S\",\"getProperty\",\"java.version\")"
				+ "})");
		System.out.println(result);
		engine.end();
	}

}

Output example: [STRING "1.8.0_172"]

[chongliang-luo]

Hi @r-suzuki , I recently run into the same rJava problem as you posted in this issue. After read your discussion with @markmyatt and @s-u I think I'm almost there but still can't library(rJava) in Rstudio. I tried

R CMD javareconf
`Java interpreter : /Library/Java/JavaVirtualMachines/jdk-11.0.1.jdk/Contents/Home/bin/java
Java version : 11.0.1
Java home path : /Library/Java/JavaVirtualMachines/jdk-11.0.1.jdk/Contents/Home
Java compiler : /Library/Java/JavaVirtualMachines/jdk-11.0.1.jdk/Contents/Home/bin/javac
Java headers gen.: /usr/bin/javah
Java archive tool: /Library/Java/JavaVirtualMachines/jdk-11.0.1.jdk/Contents/Home/bin/jar

trying to compile and link a JNI program
detected JNI cpp flags : -I$(JAVA_HOME)/include -I$(JAVA_HOME)/include/darwin
detected JNI linker flags : -L$(JAVA_HOME)/lib/server -ljvm
/usr/local/opt/llvm/bin/clang -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Library/Java/JavaVirtualMachines/jdk-11.0.1.jdk/Contents/Home/include -I/Library/Java/JavaVirtualMachines/jdk-11.0.1.jdk/Contents/Home/include/darwin -I/usr/local/opt/llvm/include -fPIC -Wall -g -O2 -c conftest.c -o conftest.o
/usr/local/opt/llvm/bin/clang -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/opt/llvm/lib -o conftest.so conftest.o -L/Library/Java/JavaVirtualMachines/jdk-11.0.1.jdk/Contents/Home/lib/server -ljvm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation

JAVA_HOME : /Library/Java/JavaVirtualMachines/jdk-11.0.1.jdk/Contents/Home
Java library path: $(JAVA_HOME)/lib/server
JNI cpp flags : -I$(JAVA_HOME)/include -I$(JAVA_HOME)/include/darwin
JNI linker flags : -L$(JAVA_HOME)/lib/server -ljvm
Updating Java configuration in /Library/Frameworks/R.framework/Resources
Done.`

and then

install_name_tool -change /Library/Java/JavaVirtualMachines/jdk-9.jdk/Contents/Home/lib/server/libjvm.dylib /Library/Java/JavaVirtualMachines/jdk-10.jdk/Contents/Home/lib/server/libjvm.dylib /Library/Frameworks/R.framework/Resources/library/rJava/libs/rJava.so

Before seeing your post I tried
export JAVA_HOME=$(/usr/libexec/java_home)
following this link. I assume this is not the problem as it is overwritten by R CMD javareconf (not sure...)

Do you have any idea how should I config this? Thanks!

[s-u]

From your output you're using unsupported compilers so who knows what other issues you have. Please do not set JAVA_HOME and use R CMD javareconf. If you still have issues, please post you exact invocation and errors as those are likely specific to you particular setup.

[chongliang-luo]

Thanks Simon @s-u . I have

Java compiler : /Library/Java/JavaVirtualMachines/jdk-11.0.1.jdk/Contents/Home/bin/javac

Are you saying I should downgrade it to older version of jdk, or it's in the wrong directory?
I will post another issue if this still doesn't work. Thanks again.

[s-u]

Sorry being ambiguous, I meant C compiler (you have some clang in /opt), not Java

[david-bonin]

After reinstalling JDK and rJava, The solution for me was closing Rstudio, running R CMD javareconf and then opening R in terminal, typing library(rJava), exiting the session, and then doing the same in Rstudio, at which point it finally worked.

[lgellis]

@david-bonin that is the only thing that worked for me as well!

[RobertMyles]

@david-bonin crazily, that also was what worked for me! Thanks.

[s-u]

I'm not sure why people are surprised - exactly that is how its documented to work ... you have to make sure your R is configured with Java support and you have to re-run javareconf any time you change your Java configuration ...

[RobertMyles]

I never previously had problems, I only experienced this today while trying to install a package that has an rJava dependency. I ended up having to install JDK 11 and run javareconf, which is something I'm pretty sure I never had to do before with a Mac & Java.

[s-u]

You didn't need to do that for Apple Java, but since Apple dropped Java support and Oracle took over it was always necessary since Oracle doesn't play the the Apple rules.

[dfrail24]

I tried running javareconf in RStudio terminal while restarting my sessions multiple times but still failed to load rJava.
Finally got it to work by doing what was mentioned above: running javareconf then calling library(rJava) in base R using terminal. I can now run rJava without issues on RStudio as well. Odd fix, but happy it's working.

[mgeard]

I had this problem but I was using JDK 13.0.1 and rJava was expecting 11.0.1

My rather stupid solution was to just rename the folder 'jdk-13.0.1.jdk' to 'jdk-11.0.1.jdk' and it's working perfectly fine so far

[NateMietk]

@mgeard brilliant! I was spending ages on this problem - your solution was the only that worked for me :)

[boyanangelov]

@mgeard and @NateMietk , this also worked for me thanks a lot!

[ktbaek]

Worked for me too! Thanks.

[whittkilburn]

renaming the folder worked for me too! Thanks!

[jienagu]

You save my life, @mgeard ! Rename works perfectly!

[christophsax]

I tried this morning, downloaded the updated jdk-13.0.2_osx-x64_bin.dmg, and ran R CMD javareconf. Everything worked smoothly afterwards.