"The specified procedure could not be found" when linking to 32-bit version of libclang

[Ilia-Kosenkov]

Describe the issue

We at extendr/libR-sys are attempting to build a library for interop between R and rust.
When building for Windows, we have to link libclang coming from msys.

The build pipeline is complicated, but it runs successfully when targeting 64-bit, meaning libclang.dll from mingw64 works fine.
Unfortunately, the build fails when targeting 32-bit.
The failure is reproducible on a variety of machines (including GH Action).
The frustrating part is the problem arises when libclang.dll is linked, and the error message is "The specified procedure could not be found" (OS error 127). We do not invoke any methods from libclang at this point.
Using WinDbg we were able to find what appears to be the issue here: libclang loads a bunch of other libraries, but the loading process terminates after referencing zlib1 (see attached log).
The inclusion of zlib1 triggers an unhandled exception, which can be identified as the source of "The specified procedure could not be found" (at least this is our best guess).

WinDbg log

ModLoad: 00000000`78ed0000 00000000`7a912000   C:\tools\msys64\mingw32\bin\libclang.dll
ModLoad: 00000000`72870000 00000000`72878000   C:\Windows\SysWOW64\VERSION.dll
ModLoad: 00000000`6eb40000 00000000`6eb62000   C:\Users\[redacted]\.rustup\toolchains\nightly-i686-pc-windows-gnu\bin\libgcc_s_dw2-1.dll
ModLoad: 00000000`64b40000 00000000`64b53000   C:\Users\[redacted]\.rustup\toolchains\nightly-i686-pc-windows-gnu\bin\libwinpthread-1.dll
ModLoad: 00000000`038a0000 00000000`08b47000   C:\tools\msys64\mingw32\bin\libLLVM.dll
ModLoad: 00000000`50480000 00000000`5065a000   C:\tools\msys64\mingw32\bin\libstdc++-6.dll
ModLoad: 00000000`038a0000 00000000`08b47000   C:\tools\msys64\mingw32\bin\libLLVM.dll
ModLoad: 00000000`752c0000 00000000`753b7000   C:\Windows\SysWOW64\ole32.dll
ModLoad: 00000000`76890000 00000000`76b05000   C:\Windows\SysWOW64\combase.dll
ModLoad: 00000000`755e0000 00000000`75601000   C:\Windows\SysWOW64\GDI32.dll
ModLoad: 00000000`74ec0000 00000000`74ed7000   C:\Windows\SysWOW64\win32u.dll
ModLoad: 00000000`757e0000 00000000`7593c000   C:\Windows\SysWOW64\gdi32full.dll
ModLoad: 00000000`754e0000 00000000`7555c000   C:\Windows\SysWOW64\msvcp_win.dll
ModLoad: 00000000`76b10000 00000000`76ca8000   C:\Windows\SysWOW64\USER32.dll
ModLoad: 00000000`75a80000 00000000`75ffb000   C:\Windows\SysWOW64\SHELL32.dll
ModLoad: 00000000`75760000 00000000`7579b000   C:\Windows\SysWOW64\cfgmgr32.dll
ModLoad: 00000000`74dd0000 00000000`74e54000   C:\Windows\SysWOW64\shcore.dll
ModLoad: 00000000`762c0000 00000000`76881000   C:\Windows\SysWOW64\windows.storage.dll
ModLoad: 00000000`74fc0000 00000000`75003000   C:\Windows\SysWOW64\powrprof.dll
ModLoad: 00000000`75940000 00000000`7594d000   C:\Windows\SysWOW64\UMPDC.dll
ModLoad: 00000000`74cc0000 00000000`74d04000   C:\Windows\SysWOW64\shlwapi.dll
ModLoad: 00000000`76250000 00000000`7625f000   C:\Windows\SysWOW64\kernel.appcore.dll
ModLoad: 00000000`757a0000 00000000`757b3000   C:\Windows\SysWOW64\cryptsp.dll
ModLoad: 00000000`71240000 00000000`71250000   C:\tools\msys64\mingw32\bin\libffi-7.dll
ModLoad: 00000000`63080000 00000000`6309e000   C:\tools\msys64\mingw32\bin\zlib1.dll
(4424.c44): Unknown exception - code c0000139 (first chance)
ModLoad: 00000000`73d80000 00000000`73f0f000   C:\Windows\SysWOW64\dbghelp.dll
wow64cpu!CpupSyscallStub+0xc:
00000000`77071cbc c3              ret

Steps to Reproduce the Problem

1. Have latest version of R installed (32 bit is required, 64 bit is optional);
2. Have rust installed;
3. Have msys installed;
4. rust has to have pc-windows-gnu target installed:
   a. rustup target add i686-pc-windows-gnu
5. rust should have default toolchain set to nightly-gnu:
   a. rustup default nightly-gnu
6. msys2 should have the following packages:
   a. mingw32/mingw-w64-i686-clang mingw-w64-i686-toolchain
7. The environment should have following variables configured (depending on how msys is installed):
   a. PATH should include references to \msys\mingw32\bin, \msys\usr\bin
   b. R_HOME should be set to the R path (e.g. C:\Program Files\R\R-4.0.3\)
   c. LIBCLANG_PATH to msys\mingw32\bin
8. Get the latest source from github.com/extendr/libR-sys;
9. Navigate into project directory, run cargo build --target=i686-pc-windows-gnu;
10. Observe rust package pulling and compilation right until it hits libR-sys, where it fails with the mentioned above error (see bellow attached output of the compilation process).

Additional Context: Operating System, Screenshots

cargo build --target=i686-pc-windows-gnu

   Compiling winapi-i686-pc-windows-gnu v0.4.0
   Compiling winapi v0.3.9
   Compiling memchr v2.3.4
   Compiling cc v1.0.66
   Compiling glob v0.3.0
   Compiling libc v0.2.81
   Compiling version_check v0.9.2
   Compiling lazy_static v1.4.0
   Compiling bitflags v1.2.1
   Compiling log v0.4.11
   Compiling proc-macro2 v1.0.24
   Compiling quick-error v1.2.3
   Compiling unicode-xid v0.2.1
   Compiling unicode-width v0.1.8
   Compiling regex-syntax v0.6.21
   Compiling winapi-build v0.1.1
   Compiling cfg-if v0.1.10
   Compiling bindgen v0.53.3
   Compiling vec_map v0.8.2
   Compiling strsim v0.8.0
   Compiling winapi v0.2.8
   Compiling shlex v0.1.1
   Compiling peeking_take_while v0.1.2
   Compiling rustc-hash v1.1.0
   Compiling lazycell v1.3.0
   Compiling thread_local v1.0.1
   Compiling nom v5.1.2
   Compiling humantime v1.3.0
   Compiling textwrap v0.11.0
   Compiling clang-sys v0.29.3
   Compiling kernel32-sys v0.2.2
   Compiling which v3.1.1
   Compiling aho-corasick v0.7.15
   Compiling quote v1.0.7
   Compiling libloading v0.5.2
   Compiling regex v1.4.2
   Compiling cexpr v0.4.0
   Compiling winapi-util v0.1.5
   Compiling atty v0.2.14
   Compiling clap v2.33.3
   Compiling termcolor v1.1.2
   Compiling env_logger v0.7.1
   Compiling libR-sys v0.1.10 (C:\Users\[redacted]\source\repos\libR-sys)
error: failed to run custom build command for `libR-sys v0.1.10 (C:\Users\[redacted]\source\repos\libR-sys)`

Caused by:
  process didn't exit successfully: `C:\Users\[redacted]\source\repos\libR-sys\target\debug\build\libR-sys-d03d281092f3c42b\build-script-build` (exit code: 101)
  --- stdout
  cargo:rustc-env=R_HOME=C:\Program Files\R\R-4.0.3\
  cargo:r_home=C:\Program Files\R\R-4.0.3\
  cargo:rustc-link-search=C:\Program Files\R\R-4.0.3\bin\i386
  cargo:rustc-link-lib=dylib=R
  cargo:rerun-if-changed=build.rs
  cargo:rerun-if-changed=wrapper.h

  --- stderr
  thread 'main' panicked at 'Unable to find libclang: "the `libclang` shared library at C:\\tools\\msys64\\mingw32\\bin\\libclang.dll could not be opened: The specified procedure could not be found. (os error 127)"', C:\Users\[redacted]\.cargo\registry\src\github.com-1285ae84e5963aae\bindgen-0.53.3\src/lib.rs:1956:31
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

If applicable, add screenshots to help explain your problem.

• OS: Windows 10 Pro x64 1909/18363.1198
• msys2 20201109.0.0 installed from Chocolatey
• rustup 1.23.1 (3df2264a9 2020-11-30)
• rustc 1.50.0-nightly (3d6705aa5 2020-12-07)
• R 4.0.3
• This problem is also discussed in extednr/libR-sys/#9

UPD01: Fixing incorrect error message used in the title/text

[clauswilke]

Since this is somewhat difficult to reproduce, here are the links to the GH Action builds.

The 64-bit build succeeds:
https://github.com/extendr/libR-sys/runs/1546362277?check_suite_focus=true#step:6:1330

The 32-bit build fails:
https://github.com/extendr/libR-sys/runs/1546362240?check_suite_focus=true#step:6:1297

The GH action scripts are identical except all references to 64-bit architecture have been changed to 32-bit architecture.

GH action script for 64-bit architecture:
https://github.com/extendr/libR-sys/blob/43bf37c547f20f3764967fb1db70ef4a5dc136c9/.github/workflows/test.yml#L56-L90

GH action script for 32-bit architecture:
https://github.com/extendr/libR-sys/blob/43bf37c547f20f3764967fb1db70ef4a5dc136c9/.github/workflows/test.yml#L92-L129