Add support for LLVM's wasm-ld #11862

GuilleX7 · 2023-06-11T19:04:59Z

It's been a long time since some WASM support was added to Meson (see f41bdae). However, there has been support for the Emscripten toolchain only, which ships a stand-alone compiler and linker. Since they moved to LLVM, Emscripten uses its own compiler (emcc) and a custom LLVM's wasm-ld, which is called directly from the compiler, hiding it from the end user.

LLVM's Clang comes with stable, out-of-the-box support for WASM and WASI since version 8.0.0 (see https://releases.llvm.org/8.0.0/docs/ReleaseNotes.html#changes-to-the-webassembly-target). While Meson is moreless able to interact with wasm-ld, it tries to generate options for a generic GNU lld.ld linker, which are invalid for wasm-ld. See #6662, #6847, #10187, #10116, #7489, which would be fixed with this PR.

The main purpose of this PR is to add support for directly using LLVM's Clang + wasm-ld toolchain. The main changes are listed below:

Emscripten does not cover the entire WASM ecosystem anymore:
- WASMDynamicLinker has been renamed to EmscriptenWASMDynamicLinker
- Files generated for the system wasm are no longer appended a .js suffix. Those generated for the emscripten system still are. This is not a breaking change, since emscripten is registered as a stable system in the reference table, but wasm is not. While we can keep the original behaviour, it is "more" correct, IMO, to add .wasm suffix to the unknown WASM system.
- The test folder and test suite wasm has been renamed to emscripten, following the same logic as above
- The cross-file wasm.txt has been renamed to wasm-emscripten.txt, following the same logic as above
Support for freestanding WASM and WASM/WASI has been added:
- LLVMWASMDynamicLinker has been added. It is detected by passing some invalid parameters to Clang. Clang will try, by default, to link with the standard WASI libc, even if wasm32-unknown-unknown is specified. Therefore, two scenarios can happen after asking clang --target=wasm32-unknown-unknown -Wl,: (1) If the Clang installation contains a valid WASI sysroot and can find the WASI libc, wasm-ld will fail to resolve the main symbol, or (2) the Clang installation doesn't contain a valid WASI sysroot and the linker will complain about the libc and the startup files not being resolved. In either case, the name of the linker (wasm-ld) appears, which we can rely on to pick LLVMWASMDynamicLinker.
- Files generated for a wasi or wasm system are appended the .wasm suffix

(Error messages from wasm-ld as mentioned above)

(1) wasm-ld: error: C:/Program Files/LLVM/share/wasi-sdk/bin/../share/wasi-sysroot/lib/wasm32-wasi/libc.a(__main_void.o): undefined symbol: main

(2) wasm-ld: error: cannot open crt1.o: no such file or directory
wasm-ld: error: unable to find library -lc
wasm-ld: error: cannot open C:\Program Files\LLVM\lib\clang\16\lib\libclang_rt.builtins-wasm32.a: no such file or directory

Cross-files for freestanding WASM and WASM/WASI have been added:
- Any LLVM's Clang installation can handle freestanding WASM by just targetting wasm32-unknown-unknown and adding some flags to avoid linking libc and startup files. Compiler-rt builtins might be necessary, but depends on the generated code.
- WASM/WASI is also supported by any LLVM's Clang installation, but this one requires the WASI libc, the startup files and the builtins. All this three items are shipped with the WASI SDK, which also contains a Clang installation. Therefore, the crossfile aims to get all the necessary requirements from a WASI SDK folder.
Some basic cross tests have been added for WASM/WASI. They are skipped unless the WASI environment variable is set to 1. This is done by the cross/wasm-wasi.json, since not every LLVM installation can handle WASI.

I really tried my best here, but since I have little knowledge on this project, I might be missing something. I'll be glad to answer any question, and I'll be open to any suggestion/changes.

ghost · 2023-06-12T09:25:48Z

Oh wow! I have recently decided to switch to Meson for one of my projects, and this patch is exactly what I need, having run into some issues just a few minutes ago (specifically with --start-group and --end-group).

As a bystander, I have a couple question: I normally use -nostdlib -Wl,--no-entry rather than -nostdlib -nostartfiles, is there a difference between them? Also, would it make sense to name things as wasm32 rather than just wasm to future‐proof builds regarding the (currently mostly experimental) wasm64 format?

Edit: Another question: I normally also use --target=wasm32-freestanding -ffreestanding rather than --target=wasm32-unknown-unknown, is there any difference between those?

(I will note that when I ask “is there a difference”, I mean to ask if there is something relevant for the sake of this pull request, i.e. whether it would be better to reconsider those options.)

Edit 2: Also, I think it might be worthwhile to specify llvm-strip as the strip binary.

GuilleX7 · 2023-06-12T11:18:20Z

@zamfofex EDIT: didn't read your first question propertly. -nostdlib already implies -nostartfiles (since they are closely linked to libc), so we may just remove -nostartfiles and leave it as -nostdlib. Good point there. However, -nostdlib -Wl,--no-entry is quite different from just -nostdlib. In the first case, the linker will not even look for a _start entry symbol (which should be a function), so if you declare some random-named functions, everything will be ok. In the latter case, even if we don't link to the libc, the linker would still look for a _start entry symbol, and will complain about it not existing if you haven't provided it. In this case I guess we can just let the user decide whether to pass Wl,--no-entry and not use any entrypoint at all or just use _start.

As per wasm32-freestanding vs wasm32-unknown-unknown, I really doubt there's any kind of difference between those two. AFAIK, freestanding would be parsed as either the subarchitecture, the vendor, the system or the environment. In none of those cases is a valid name (see https://llvm.org/doxygen/classllvm_1_1Triple.html), so LLVM would fallback to "unknown" sub, vendor, system or env. wasm32-unknown-unknown is a direct way to tell LLVM "let's go to the defaults".

About the -ffreestanding argument, I'm not sure if it's convenient or not. AFAIK, this parameters instructs Clang to generate code for a freestanding environment (correct), not assuming anything about libc and its functions (correct), but I'm not sure if it also implies not to use any builtins (In GCC it definitely does, see https://gcc.gnu.org/onlinedocs/gcc/C-Dialect-Options.html#index-ffreestanding-1). If this is true for Clang too, then I guess it would be better to let everyone decide what's more convenient for them.

Regarding wasm32, it would be a good future-proof measure to change, at least, the name of the cross files, even for Emscripten. I'll rename and add the strip binary to both cross files, I completely missed it for some reason :)

laur2000 · 2023-06-12T13:25:53Z

This is exactly what I was looking for! +1

This PR adds support for compiling WebAssembly System Interface (WASI) programs using Meson. WASI is a modular system interface for WebAssembly, designed as a portable target for the compilation of high-level languages like C, allowing them to run on all different types of systems (including embedded systems and servers) without needing to recompile.

tristan957 · 2023-06-13T13:32:38Z

Can you add a release notes snippet in docs/markdown/snippets?

## Added support for the LLVM WASI linker

...

mesonbuild/linkers/linkers.py

@@ -923,7 +923,37 @@
            raise mesonlib.MesonBugException(f'win_subsystem: {value} not handled in lld linker. This should not be possible.')


-class WASMDynamicLinker(GnuLikeDynamicLinkerMixin, PosixDynamicLinkerMixin, DynamicLinker):
+class LLVMWASMDynamicLinker(GnuLikeDynamicLinkerMixin, PosixDynamicLinkerMixin, DynamicLinker):


mesonbuild/linkers/linkers.py

+                         install_rpath: str) -> T.Tuple[T.List[str], T.Set[bytes]]:
+        return ([], set())
+
+class EmscriptenWASMDynamicLinker(GnuLikeDynamicLinkerMixin, PosixDynamicLinkerMixin, DynamicLinker):


GuilleX7 · 2023-06-13T17:44:54Z

@tristan957 Added, thanks for pointing out.

tristan957 · 2023-07-06T13:49:42Z

I have targeted this for 1.3.0. Hopefully someone can review it before then

GuilleX7 · 2023-07-10T19:00:43Z

I have targeted this for 1.3.0. Hopefully someone can review it before then

I'll try to keep it free of conflicts meanwhile. Thanks!

eli-schwartz · 2023-07-10T19:22:36Z

The current conflicts are due to some big refactoring I did a short while ago to delay expensive imports. Once that is resolved there probably should not be any more conflicts. It's basically just imports though, so it should be cheap to rebase and resolve them.

The reason it's being targeted for 1.3.0, for context, is because we're currently in the release candidate stage for the pending release of 1.2.0, which means that the master branch is frozen for non-bugfix changes. This actually makes it a lot less likely for conflicts to happen because we're not touching the affected code either. :D

Hopefully we'll be un-freezing for continued development in about a week, fingers crossed that we don't need an rc4...

tristan957 · 2023-08-16T19:00:15Z

Wanna rebase this PR @GuilleX7?

GuilleX7 · 2023-08-16T19:29:30Z

Sure, I'll rebase and ping you when it's ready. @tristan957

tristan957 · 2023-08-16T19:32:53Z

run_project_tests.py

+    if os.environ.get('WASI') != '1':
+        return True


Seems like you could remove this?

Well, I'm not sure how to proceed here. The idea is to avoid running these tests when the LLVM installation doesn't ship a proper WASI libc, which is the case for out-of-the-box Clang installations. My original idea was to test the LLVM installation instead, but I couldn't get it done properly, so I ended up adding this. I'd be really grateful if you can lend me a hand here.

How does one know if the LLVM installation ships a proper WASI libc? Can you query such information with llvm-config for example?

I guess a pragmatic approach would be to perform a sanity check: look for Clang and then have it compile a single file with a main function using the proper flags to target wasm32-wasi. I remember I tried to do something similar to https://github.com/mesonbuild/meson/blob/master/run_project_tests.py#L965, but for some reason I failed. I'll try to do it again, maybe I just didn't spend too much time on it.

Edit: since standard LLVM installations can target wasm32 by default, the only difference is WASI libc missing, which has to be downloaded manually by the user. I don't think llvm-config could be helpful at all in this case.

Yes, llvm-config would have no way to know if there is a WASI sysroot, that is not its purpose. I think you could run clang -print-resource-dir and check if that has a lib/wasi subdirectory.

tristan957 · 2023-08-16T19:36:59Z

You don't need the extra_files and VERSION in your tests imo. Those are being tested elsewhere.

GuilleX7 · 2023-08-16T19:52:18Z

You don't need the extra_files and VERSION in your tests imo. Those are being tested elsewhere.

I'll get rid of them. Thanks!

It seems that meson only supports emscripten as a compiler to wasm at present, wasi-sdk support is not completed yet, so this only adds ci build scripts for emscripten. References: * <mesonbuild/meson@f41bdae> * <mesonbuild/meson#11862>

ghost · 2024-02-14T09:15:24Z

Hmm, I don’t know what changed, but this code doesn’t seem to be working anymore. Presumably something changed from LLVM’s side on how they handle some options in LLD. I was able to fix this code with the following patch:

diff --git a/mesonbuild/linkers/detect.py b/mesonbuild/linkers/detect.py
index 1094f5e..5b9ea86 100644
--- a/mesonbuild/linkers/detect.py
+++ b/mesonbuild/linkers/detect.py
@@ -161,9 +161,9 @@ def guess_nix_linker(env: 'Environment', compiler: T.List[str], comp_class: T.Ty
             lld_cls = linkers.LLVMLD64DynamicLinker
         else:
             if isinstance(comp_class.LINKER_PREFIX, str):
-                cmd = compiler + override + [comp_class.LINKER_PREFIX]
+                cmd = compiler + override + [comp_class.LINKER_PREFIX + '-h']
             else:
-                cmd = compiler + override + comp_class.LINKER_PREFIX
+                cmd = compiler + override + comp_class.LINKER_PREFIX + ['-h']
             _, _, newerr = Popen_safe(cmd)
             if 'wasm-ld' in newerr:
                 lld_cls = linkers.LLVMWASMDynamicLinker

whitequark · 2025-03-02T01:02:29Z

What is needed to bring this PR forward? The Emscripten toolchain isn't useful if you're building WebAssembly for a non-browser target like Wasmtime, and currently Meson is a blocker in being able to cross-compile some projects this way.

eli-schwartz · 2025-03-02T04:08:03Z

I was hoping another maintainer who is more familiar with wasm would be available to review this... :(

Maybe @jpakkane can review this as the person that initially implemented emscripten support?

whitequark · 2025-03-02T04:18:27Z

I have a lot of Wasm experience. I've looked through this PR and (other than the comment I made) I have one major concern. WASI has a distinction between wasip1 and wasip2 targets. They have different triples, and they also build objects with different ABI (the former imports a bunch of C functions, the latter uses the component model).

I believe they should most likely be distinct systems in Meson, although I've used Meson for the first time today and I may be entirely wrong here.

dcbaker · 2025-03-03T20:00:48Z

@whitequark Based on your description that sounds like we should consider them two different systems. Or two different subsystems? not sure.

whitequark · 2025-03-04T03:57:21Z

Or two different subsystems? not sure.

Having "wasi" as the system and "wasip1"/"wasip2" as the subsystem sounds reasonable to me from first principles. The programming model in C/C++ is essentially the same, so being able to compare just the system seems useful, but for deployment the subsystems would be fairly different as they cannot execute on the same set of Wasm engines.

GuilleX7 requested review from dcbaker and jpakkane as code owners June 11, 2023 19:04

GuilleX7 force-pushed the feat/llvm-wasm-ld branch 2 times, most recently from 521acae to 39ddfed Compare June 12, 2023 11:57

github-advanced-security bot found potential problems Jun 13, 2023

View reviewed changes

GuilleX7 force-pushed the feat/llvm-wasm-ld branch from 39ddfed to bd8ad9c Compare June 13, 2023 17:41

GuilleX7 force-pushed the feat/llvm-wasm-ld branch from bd8ad9c to 41eee7a Compare June 13, 2023 19:19

tristan957 added this to the 1.3.0 milestone Jul 6, 2023

tristan957 reviewed Aug 16, 2023

View reviewed changes

GuilleX7 force-pushed the feat/llvm-wasm-ld branch from 41eee7a to bbccc59 Compare August 16, 2023 21:24

xclaesse modified the milestones: 1.3.0, 1.4.0 Oct 17, 2023

myfreeer mentioned this pull request Dec 2, 2023

ci: initial support of meson and emscripten recp/cglm#362

Merged

Add support for LLVM's wasm-ld

e52c9fb

GuilleX7 force-pushed the feat/llvm-wasm-ld branch from bbccc59 to e52c9fb Compare January 15, 2024 19:27

bruchar1 removed this from the 1.4.0 milestone Mar 3, 2025

bonzini added the Arch:Wasm Compiling to WebAssembly label Mar 10, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for LLVM's wasm-ld #11862

Add support for LLVM's wasm-ld #11862

GuilleX7 commented Jun 11, 2023 •

edited

Loading

ghost commented Jun 12, 2023 •

edited by ghost

Loading

GuilleX7 commented Jun 12, 2023 •

edited

Loading

laur2000 commented Jun 12, 2023 •

edited

Loading

tristan957 commented Jun 13, 2023

GuilleX7 commented Jun 13, 2023

tristan957 commented Jul 6, 2023

GuilleX7 commented Jul 10, 2023

eli-schwartz commented Jul 10, 2023 •

edited

Loading

tristan957 commented Aug 16, 2023

GuilleX7 commented Aug 16, 2023

tristan957 Aug 16, 2023

GuilleX7 Aug 16, 2023

tristan957 Aug 16, 2023

GuilleX7 Aug 16, 2023 •

edited

Loading

whitequark Mar 2, 2025

tristan957 commented Aug 16, 2023

GuilleX7 commented Aug 16, 2023

ghost commented Feb 14, 2024

whitequark commented Mar 2, 2025

eli-schwartz commented Mar 2, 2025

whitequark commented Mar 2, 2025

dcbaker commented Mar 3, 2025

whitequark commented Mar 4, 2025

Add support for LLVM's wasm-ld #11862

Are you sure you want to change the base?

Add support for LLVM's wasm-ld #11862

Conversation

GuilleX7 commented Jun 11, 2023 • edited Loading

ghost commented Jun 12, 2023 • edited by ghost Loading

GuilleX7 commented Jun 12, 2023 • edited Loading

laur2000 commented Jun 12, 2023 • edited Loading

tristan957 commented Jun 13, 2023

GuilleX7 commented Jun 13, 2023

tristan957 commented Jul 6, 2023

GuilleX7 commented Jul 10, 2023

eli-schwartz commented Jul 10, 2023 • edited Loading

tristan957 commented Aug 16, 2023

GuilleX7 commented Aug 16, 2023

tristan957 Aug 16, 2023

Choose a reason for hiding this comment

GuilleX7 Aug 16, 2023

Choose a reason for hiding this comment

tristan957 Aug 16, 2023

Choose a reason for hiding this comment

GuilleX7 Aug 16, 2023 • edited Loading

Choose a reason for hiding this comment

whitequark Mar 2, 2025

Choose a reason for hiding this comment

tristan957 commented Aug 16, 2023

GuilleX7 commented Aug 16, 2023

ghost commented Feb 14, 2024

whitequark commented Mar 2, 2025

eli-schwartz commented Mar 2, 2025

whitequark commented Mar 2, 2025

dcbaker commented Mar 3, 2025

whitequark commented Mar 4, 2025

GuilleX7 commented Jun 11, 2023 •

edited

Loading

ghost commented Jun 12, 2023 •

edited by ghost

Loading

GuilleX7 commented Jun 12, 2023 •

edited

Loading

laur2000 commented Jun 12, 2023 •

edited

Loading

eli-schwartz commented Jul 10, 2023 •

edited

Loading

GuilleX7 Aug 16, 2023 •

edited

Loading