Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wasm2c: add support for module instancing #1814

Merged
merged 1 commit into from
Sep 16, 2022

Conversation

yhdengh
Copy link
Contributor

@yhdengh yhdengh commented Jan 26, 2022

This PR adds support for module instancing to wasm2c as proposed in #1805. The approach is similar to #1721.

  • Globals, memories, and tables are moved into a new structure named module_prefix_ + _instance_t.
  • The structure is passed as the first argument of function calls. The embedder is responsible for initializing the structure.
  • run-spec-wasm2c.py is updated such that main.c generated by the script reflects the changes for module instancing.

Copy link
Member

@kripken kripken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this looks good to me, but I have a few questions.

@shravanrn please take a look as well. Thanks!

@shravanrn
Copy link
Collaborator

Reviewing! Although its getting a bit late in my local timezone (I am currently traveling), so worst case I'll post an update in the next 12 hours or so

@keithw
Copy link
Member

keithw commented Mar 30, 2022

This now passes the current wasm2c tests, including for imports/exports/linking (these are the "old" spec tests from before bulk-memory operations and reference-types were merged).

We're still working on implementing bulk-memory operations and reference-types on top of this, which if successful will hopefully have wasm2c passing the current versions of the spec tests, as well as the remaining multi-memory proposal tests that depend on these features. I've been tracking this at #1853 (comment) .

@shravanrn
Copy link
Collaborator

TLDR: we can go ahead and land this instancing support when ready. I'll deal with the incompatibilities they may introduce in our fork

@kripken @sbc100 Just wanted to briefly update everyone. I was actually away for the last few weeks hence the delayed response here. But I did play a bit more with rebasing some of these changes on our wasm2c fork. In short it should be possible with some work. That said, my current priority for Mozilla is to rewrite RLBox to add more features, so I suspect I won't be able to upstream some of our custom wasm2c features in our fork for a few more weeks. However, I don't want this to block any other ongoing work here, so feel free to land this instancing support. I am happy however to review this and any other PRs you want me to look at.

@keithw keithw force-pushed the module_instancing branch from cf8d31d to 2e8f1bc Compare April 5, 2022 04:09
@keithw keithw requested review from kripken and sbc100 April 6, 2022 23:06
@keithw
Copy link
Member

keithw commented Apr 6, 2022

@shravanrn Thanks, we appreciate this. Of course we'd be grateful for your review. When you do want to upstream the rlbox code, we're happy to help rebase it at the time (if you want our help) and preserve whatever tests are passing. I don't think the two approaches are that different; ours is mostly the same as yours + handling imports/exports/linking, cases that are probably less relevant to the RLBox use-case.

@sbc100 @kripken This is probably ready for another look. @binji gave us a review here (#1877 (review)) that we've since responded to. After this one, the later-stacked PRs are #1877 (bulk memory), #1887 (reference types), and #1888 (tolerating partly invalid modules), after which wasm2c passes all the core spec tests (and all the multi-memory proposal tests) except the SIMD ones.

Copy link
Member

@sbc100 sbc100 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I created a separate PR to remove the signature mangling: #1896. Perhaps we can land that first to simplify this PR?

@sbc100
Copy link
Member

sbc100 commented Apr 14, 2022

Can you rebase now that #1897 and #1896 have landed? I hope it should make this change much more precise.

@yhdengh yhdengh force-pushed the module_instancing branch from f262c76 to 0f66158 Compare July 11, 2022 20:50
@keithw keithw force-pushed the module_instancing branch 3 times, most recently from 881a4dd to bf69a30 Compare July 11, 2022 23:22
@keithw keithw force-pushed the module_instancing branch 6 times, most recently from b12ba7e to 56f4441 Compare July 15, 2022 06:40
@keithw keithw force-pushed the module_instancing branch from 56f4441 to f8b5cef Compare August 14, 2022 23:38
@keithw keithw force-pushed the module_instancing branch 3 times, most recently from e59ec68 to efc90be Compare August 16, 2022 01:14
@keithw keithw force-pushed the module_instancing branch from efc90be to aaafc01 Compare August 24, 2022 15:11
Copy link
Member

@sbc100 sbc100 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK I think this is ready to land now. Any final changes before we do so?

Index memory_index = 0;
for (const Memory* memory : module_->memories) {
bool is_import = memory_index < module_->num_memory_imports;
if (!is_import) {
Write("static ");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be static is the memory is not exported?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These memories are getting declared as members of the instance struct (both the exported and not-exported ones) so I don't think static is needed/valid anymore in that new context.

Co-authored-by: Angela Montemayor <amontema@cs.stanford.edu>
@sbc100 sbc100 enabled auto-merge (squash) September 16, 2022 16:31
@sbc100
Copy link
Member

sbc100 commented Sep 16, 2022

Ok! Marking as auto-merge!

@keithw
Copy link
Member

keithw commented Sep 16, 2022

OK I think this is ready to land now. Any final changes before we do so?

Awesome! I don't think we have any final changes in this one. I just rebased it on the current tip-of-tree to make sure it applies cleanly and then rebased #1877 and #1887 on top of it.

I think after this the wasm2c roadmap from our perspective looks something like:

  1. Bulk memory ops (wasm2c: implement the bulk memory operations proposal #1877)
  2. Reference types (wasm2c: implement the reference-types proposal #1887) [at this point, wasm2c will pass all the current Wasm tests (+ multi-memory and exceptions) except SIMD, and it would be cool to get it added to https://webassembly.org/roadmap/]

Followed by more speculative changes:

  • Changes necessary to accommodate Firefox/RLBox (we chatted a bit with talg a few weeks ago)
  • SIMD [at this point, wasm2c would support all of WebAssembly 2.0 + multi-memory and exceptions]
  • Adding an instance "reset" function (in addition to "instantiate" and "free") to allow the host to to take an instance and more efficiently restore it to initial state without having to free and reallocate all its memories and tables.
  • Adding a mechanism (maybe in the IR or in the CWriter options) to tag memories on a memory-by-memory basis for whether they should be hardware bounds-checked (with mprotect and a signal handler) or software bounds-checked (with explicit checks on every load/store), rather than making this a global choice for all memories. We're finding this useful because we generally want memory#0 to be hardware checked (for performance) but we also like being able to point the other memories at arbitrary places without having to set every one up beforehand with mmap/mprotect. Not sure this is going to be of general interest.
  • ...?

@sbc100 sbc100 merged commit dd44ca9 into WebAssembly:main Sep 16, 2022
@keithw keithw deleted the module_instancing branch October 4, 2022 00:44
matthias-blume pushed a commit to matthias-blume/wabt that referenced this pull request Dec 16, 2022
Co-authored-by: Angela Montemayor <amontema@cs.stanford.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants