Failing on Windows #5

rado-straka · 2019-08-07T10:58:23Z

On Win10 64, usage failed. Downstream vinhkhuc/JFastText works as expected. I did not tried on Linux yet.

Usage failure on line final JFastText jft = new JFastText();:
java.lang.UnsatisfiedLinkError: no jniFastTextWrapper in java.library.path

Also build failure which is the happening with vinhkhuc/JFastText too, but could help to explain what is happening here:

--- maven-compiler-plugin:3.0:compile (default-compile) @ jfasttext ---
Changes detected - recompiling the module!
Compiling 3 source files to C:\Users\strakar\src\github\JFastText\target\classes
com/github/jfasttext/JFastText.java:[22,38] found raw type: org.bytedeco.javacpp.PointerPointer
  missing type arguments for generic class org.bytedeco.javacpp.PointerPointer<P>
com/github/jfasttext/FastTextWrapper.java:[273,38] position(long) in com.github.jfasttext.FastTextWrapper.FastTextApi overrides <P>position(long) in org.bytedeco.javacpp.Pointer
  return type requires unchecked conversion from com.github.jfasttext.FastTextWrapper.FastTextApi to P
com/github/jfasttext/FastTextWrapper.java:[280,61] found raw type: org.bytedeco.javacpp.PointerPointer
  missing type arguments for generic class org.bytedeco.javacpp.PointerPointer<P>

--- javacpp:1.5.1:build (run-javacpp-parser) @ jfasttext ---
Detected platform "windows-x86_64"
Building for platform "windows-x86_64"
Targeting C:\Users\strakar\src\github\JFastText\src\main\java\com\github\jfasttext\FastTextWrapper.java
Parsing C:\Users\strakar\src\github\JFastText\src\main\cpp\fasttext_wrapper.h
Parsing C:\Users\strakar\src\github\JFastText\src\main\cpp\fasttext_wrapper_javacpp.h

--- javacpp:1.5.1:build (run-javacpp-compiler) @ jfasttext ---
Detected platform "windows-x86_64"
Building for platform "windows-x86_64"
Could not load platform properties for class com.github.jfasttext.JFastText
Generating C:\Users\strakar\src\github\JFastText\target\classes\jnijavacpp.cpp
Generating C:\Users\strakar\src\github\JFastText\target\classes\com\github\jfasttext\jniFastTextWrapper.cpp
Compiling C:\Users\strakar\src\github\JFastText\target\classes\com\github\jfasttext\windows-x86_64\jniFastTextWrapper.dll
cl /IC:\Users\strakar\src\github\JFastText\src\main\java/../cpp "/IC:\Program Files\Java\jdk1.8.0_144\include" "/IC:\Program Files\Java\jdk1.8.0_144\include\win32" C:\Users\strakar\src\github\JFastText\target\classes\com\github\jfasttext\jniFastTextWrapper.cpp C:\Users\strakar\src\github\JFastText\target\classes\jnijavacpp.cpp /Oi /O2 /EHsc /Gy /GL /MD /LD -std=c++0x -IC:\Users\strakar\src\github\JFastText\src\main\java/../cpp /W3 /link /OUT:jniFastTextWrapper.dll psapi.lib 
Failed to execute JavaCPP Builder: Cannot run program "cl" (in directory "C:\Users\strakar\src\github\JFastText\target\classes\com\github\jfasttext\windows-x86_64"): CreateProcess error=2, The system cannot find the file specified
------------------------------------------------------------------------
BUILD FAILURE
------------------------------------------------------------------------

The text was updated successfully, but these errors were encountered:

carschno · 2019-08-07T11:07:37Z

Thanks for your message. I think the issue reported in #4 for Mac OS probably has the same root cause. I'll look into it.

carschno · 2019-08-07T11:10:00Z

@rado-straka Could you try the example in examples/api, to verify?

rado-straka · 2019-08-07T11:50:41Z

------------------------------------------------------------------------
Building java_sandbox 0.1
------------------------------------------------------------------------

--- exec-maven-plugin:1.2.1:exec (default-cli) @ java_sandbox ---
Exception in thread "main" java.lang.UnsatisfiedLinkError: no jniFastTextWrapper in java.library.path
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
	at java.lang.Runtime.loadLibrary0(Runtime.java:870)
	at java.lang.System.loadLibrary(System.java:1122)
	at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:1483)
	at org.bytedeco.javacpp.Loader.load(Loader.java:1186)
	at org.bytedeco.javacpp.Loader.load(Loader.java:1036)
	at com.github.jfasttext.FastTextWrapper.<clinit>(FastTextWrapper.java:10)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:348)
	at org.bytedeco.javacpp.Loader.load(Loader.java:1103)
	at org.bytedeco.javacpp.Loader.load(Loader.java:1036)
	at com.github.jfasttext.FastTextWrapper$FastTextApi.<clinit>(FastTextWrapper.java:267)
	at com.github.jfasttext.JFastText.<init>(JFastText.java:14)
	at ApiExample.main(ApiExample.java:5)
------------------------------------------------------------------------
BUILD FAILURE
------------------------------------------------------------------------

lama0206 · 2019-08-15T16:05:36Z

As far as I see, there are two problems:

the JAR (io.github.carschno:jfasttext:0.5.0) on central does not contain a DLL for Windows, thus using the JAR as dependency and running the example fails on windows (and macos).
building on Windows fails, see below.

The error message from mvn package posted in the first message only shows that no c++ compiler (cl.exe) can be found during the build. When I run mvn package in the Visual C++ command prompt (both, x86 and x64), the c++ compiler runs, but linking fails:

[INFO] --- javacpp:1.5.1:build (run-javacpp-compiler) @ jfasttext ---
[INFO] Detected platform "windows-x86_64"
[INFO] Building for platform "windows-x86_64"
[WARNING] Could not load platform properties for class com.github.jfasttext.JFastText
[INFO] Generating C:\Users\Mathias\VCS\carschno__JFastText\target\classes\jnijavacpp.cpp
[INFO] Generating C:\Users\Mathias\VCS\carschno__JFastText\target\classes\com\github\jfasttext\jniFastTextWrapper.cpp
[INFO] Compiling C:\Users\Mathias\VCS\carschno__JFastText\target\classes\com\github\jfasttext\windows-x86_64\jniFastTextWrapper.dll
[INFO] cl /IC:\Users\Mathias\VCS\carschno__JFastText\src\main\java/../cpp "/IC:\Program Files\Amazon Corretto\jdk11.0.3_7\include" "/IC:\Program Files\Amazon Co
rretto\jdk11.0.3_7\include\win32" C:\Users\Mathias\VCS\carschno__JFastText\target\classes\com\github\jfasttext\jniFastTextWrapper.cpp C:\Users\Mathias\VCS\carsc
hno__JFastText\target\classes\jnijavacpp.cpp /Oi /O2 /EHsc /Gy /GL /MD /LD -std=c++0x -IC:\Users\Mathias\VCS\carschno__JFastText\src\main\java/../cpp /W3 /link
/OUT:jniFastTextWrapper.dll psapi.lib
Microsoft (R) C/C++ Optimizing Compiler Version 19.13.26128 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

cl : Command line warning D9002 : ignoring unknown option '-std=c++0x'
jniFastTextWrapper.cpp

[...]

Microsoft (R) Incremental Linker Version 14.13.26128.0
Copyright (C) Microsoft Corporation.  All rights reserved.

/out:jniFastTextWrapper.dll
/ltcg
/dll
/implib:jniFastTextWrapper.lib
/OUT:jniFastTextWrapper.dll
psapi.lib
jniFastTextWrapper.obj
jnijavacpp.obj
   Creating library jniFastTextWrapper.lib and object jniFastTextWrapper.exp
jniFastTextWrapper.obj : error LNK2001: unresolved external symbol "public: void __cdecl fasttext::Meter::writeGeneralMetrics(class std::basic_ostream<char,struct std::char_traits<char> > &,int)const " (?writeGeneralMetrics@Meter@fasttext@@QEBAXAEAV?$basic_ostream@DU?$char_traits@D@std@@@std@@H@Z)
jniFastTextWrapper.obj : error LNK2001: unresolved external symbol "public: double __cdecl fasttext::Meter::f1Score(int)" (?f1Score@Meter@fasttext@@QEAANH@Z)
jniFastTextWrapper.obj : error LNK2001: unresolved external symbol "public: double __cdecl fasttext::Meter::recall(int)" (?recall@Meter@fasttext@@QEAANH@Z)
jniFastTextWrapper.obj : error LNK2001: unresolved external symbol "public: double __cdecl fasttext::Meter::recall(void)const " (?recall@Meter@fasttext@@QEBANXZ)
jniFastTextWrapper.obj : error LNK2001: unresolved external symbol "public: double __cdecl fasttext::Meter::precision(int)" (?precision@Meter@fasttext@@QEAANH@Z)
jniFastTextWrapper.obj : error LNK2001: unresolved external symbol "public: double __cdecl fasttext::Meter::precision(void)const " (?precision@Meter@fasttext@@QEBANXZ)
jniFastTextWrapper.obj : error LNK2001: unresolved external symbol "public: void __cdecl fasttext::Meter::log(class std::vector<int,class std::allocator<int> >const &,class std::vector<struct std::pair<float,int>,class std::allocator<struct std::pair<float,int> > > const &)" (?log@Meter@fasttext@@QEAAXAEBV?$vector@HV?$allocator@H@std@@@std@@AEBV?$vector@U?$pair@MH@std@@V?$allocator@U?$pair@MH@std@@@2@@4@@Z)
jniFastTextWrapper.dll : fatal error LNK1120: 7 unresolved externals
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------

Comparing the result with vinhkhuc's code base, I see that the jnijavacpp.cpp is located in

target\classes in carschno's build and in
target\classes\com\github\jfasttext in vinhkhuc's build.
In vinhkhuc's build, the jniFastTextWrapper.cpp is in the same directory as the jjnijavacpp.cpp - in carschno's build the jniFastTextWrapper.cpp is in target\classes\com\github\jfasttextas well. The final call to CL seems OK though in both cases (paths stripped and line breaks added by me...):

cl /IJFastText\src\main\java/../cpp "/IC:\Program Files\Amazon Corretto\jdk11.0.3_7\include"
"/IC:\Program Files\Amazon Corretto\jdk11.0.3_7\include\win32"
JFastText\target\classes\com\github\jfasttext\jniFastTextWrapper.cpp
JFastText\target\classes\com\github\jfasttext\jnijavacpp.cpp
/Oi /O2 /EHsc /Gy /GL /MD /LD -std=c++0x
-IJFastText\src\main\java/../cpp /W3 /link /OUT:jniFastTextWrapper.dll psapi.lib

cl /IJFastText\src\main\java/../cpp "/IC:\Program Files\Amazon Corretto\jdk11.0.3_7\include"
"/IC:\Program Files\Amazon Corretto\jdk11.0.3_7\include\win32"
JFastText\target\classes\com\github\jfasttext\jniFastTextWrapper.cpp
JFastText\target\classes\jnijavacpp.cpp
/Oi /O2 /EHsc /Gy /GL /MD /LD -std=c++0x
-IJFastText\src\main\java/../cpp /W3 /link /OUT:jniFastTextWrapper.dll psapi.lib

carschno · 2019-10-21T15:44:44Z

@lama0206 thank you very much for your investigation! I am trying to add the binaries for Mac OS and Windows.
I think the build for Windows fails because you do not have a compiler, or it cannot be found (cl.exe).

lama0206 · 2019-10-25T19:36:53Z

@carschno, sorry for not following up on this.

I think the build for Windows fails because you do not have a compiler, or it cannot be found (cl.exe).

I'd like to disagree - the error message came from cl during its compile run... (I used MS's developer tools, not the full-blown Visual Studio.) If I run cl directly, the compilation fails with the same error.

In the meantime, I was able to build on Windows by changing the call to cl.exe and running it outside the maven build. I changed one parameter in the call to cl: I use /MT (whereas maven uses /MD). Bundling the generated DLLs works fine.

This is neither elegant nor a working solution for an (automatic) build on Windows, but it works for me since I now have a JAR with all native libs included. (The MacOs build runs fine for me.)

So you could either close this bug (as you've removed the claim from the readme) and create a new one for "Build failing on Windows" - or just keep this one open. :) Some day, we'll figure this one out.

Best

carschno mentioned this issue Oct 21, 2019

The Jar package on Maven Central is NOT bundled with precompiled fastText library for Windows, Linux and MacOSX 64bit #6

Closed

carschno closed this as completed in 9f60ef2 Oct 25, 2019

carschno reopened this Oct 25, 2019

carschno mentioned this issue Oct 25, 2019

Update to FastText v0.9.1 #4

Closed

carschno added a commit that referenced this issue Oct 26, 2019

Add instructions for Windows, Mac OS (#5).

5674ed7

carschno closed this as completed in 84f13eb Oct 27, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Failing on Windows #5

Failing on Windows #5

rado-straka commented Aug 7, 2019

carschno commented Aug 7, 2019

carschno commented Aug 7, 2019

rado-straka commented Aug 7, 2019

lama0206 commented Aug 15, 2019

carschno commented Oct 21, 2019

lama0206 commented Oct 25, 2019

Failing on Windows #5

Failing on Windows #5

Comments

rado-straka commented Aug 7, 2019

carschno commented Aug 7, 2019

carschno commented Aug 7, 2019

rado-straka commented Aug 7, 2019

lama0206 commented Aug 15, 2019

carschno commented Oct 21, 2019

lama0206 commented Oct 25, 2019