Support for M1

[shivanan]

On a M1 Mac - when I try to install I get the following:

npm ERR! code EBADPLATFORM
npm ERR! notsup Unsupported platform for foundationdb@1.0.2: wanted {"os":"any","arch":"x64"} (current: {"os":"darwin","arch":"arm64"})
npm ERR! notsup Valid OS:    any
npm ERR! notsup Valid Arch:  x64
npm ERR! notsup Actual OS:   darwin
npm ERR! notsup Actual Arch: arm64

Is there a way to make it work on this new architecture?

[josephg]

Oh good catch.

There's no good reason why it wouldn't work on M1 macs but I have no idea what configuration changes are needed because I don't have an M1 mac to test and experiment with. I would happy fix this bug in exchange for an M1 mac. Otherwise someone else will need to pick it up!

[josephg]

Made a request for help on the fdb forums

[shivanan]

Thanks. Will do. I feel like we just need to set some npm package settings so that it uses the x64 version on Darwin-arm64

[josephg]

The specific error you're running into comes from this in package.json:

  "cpu": [
    "x64"
  ],

If you remove that, does it build? There may be other changes required in binding.gyp.

[josephg]

There is probably also a way to cross-compile the package for ARM from an intel mac, but I'm not sure how to do this.

Edit: Looks pretty easy - but we'll need to figure out how to express that in gyp. I wonder if lipo can produce universal napi modules...

[josephg]

Ok - I managed to build a universal binary for macos and it seems to work. But foundationdb itself doesn't yet support the apple M1. With 6.2.28 (latest at time of writing):

$ lipo -archs /usr/local/lib/libfdb_c.dylib 
x86_64

There's an open PR for apple silicon support on foundationdb but it hasn't yet been merged.

I've updated package.json to remove the x86_64 restriction and updated the github actions to generate prebuilds for arm64. But I have no idea what Rosetta's behaviour would be in this case - it should be able to use the x86 binaries for everything, but I couldn't speculate on what it actually does. I'm worried with prebuilds for arm nodejs will try to load the native arm version of the module, and that'll fail to run because it can't find a working ARM copy of libfdb_c.dylib.

[josephg]

Mmmf.. looks like github actions is still by default using an old version of macos which doesn't support building for ARM, and we can't move to the newer images until github adds capacity.

The current GH environment does not support arm64.

[josephg]

Anyway, summary:

• Remove x64 limitation in package.json
• Get arm version of the module building (it builds fine already with no changes)
• ARM support in the native bindings code that we depend on (in libfdb_c). Blocked by this PR in apple/foundationdb
• Make ARM prebuilds for the foundationdb npm package. Blocked by old macos images in gh-actions
• Make this package work through Rosetta. (I can't test rosetta. Does it work already? Someone else will need to triage if it doesn't.)

---

To cross-compile the node-foundationdb bindings for ARM and experiment either run:

npx node-gyp rebuild --arch=arm64 

or compile prebuilds with:

yarn prebuild --arch arm64

(This depends on the updated prebuildify - git pull then run yarn first to update.)

Manually specifying --arch should be unnecessary on an arm mac.

Its also possible to build a universal napi module like this:

yarn prebuild --arch x64
yarn prebuild --arch arm64
lipo -create -output uni.node prebuilds/darwin-arm64/node.napi.node prebuilds/darwin-x64/node.napi.node 

But afaik it won't help until we have a universal libfdb_c.dylib.

[shivanan]

Hi
Sorry for the late reply.
Yes I also tried to build in the same way and ran across similar issues.
Running under Rosetta should be fine - I doubt many people are running foundationdb on production on a mac.

Just that I get an error when trying to install the package. I will check your update and see if it works.

Thanks for the help!

[shivanan]

For the time being I've installed node + npm under Rosetta so it all works fine for now. Thanks!

[aikoven]

Hey, we've managed to get a working setup with FoundationDB 6.3.23 for M1 in Docker. To do that, we had to enable knob disable_posix_kernel_aio for fdbserver, fdbcli and node-foundationdb:

fdbserver --knob_disable_posix_kernel_aio=1 <...other args>

fdbcli --exec 'configure new single ssd' --knob_disable_posix_kernel_aio=1

// do this before fdb.open():
fdb.configNetwork({    
  knob: 'disable_posix_kernel_aio=1',
});

[holger-koenig]

Hi, do you have any outlook on this issue? Checking foundationdb downloads, there is now an ARM version for Mac available. This should resolve the first blocker from the list above.

[josephg]

Oh lovely - well, if there are ARM builds of libfdb_c then it should be pretty easy to make an apple silicon build. We just need to adjust some build settings.

Maybe someone with an M1/M2 chip can play around with the build?

A quick glance also suggests we should be able to use github actions to build for apple silicon now that they've updated their images for macos 12. (Macos 11+ is needed for the ARM compiler toolchain).

[josephg]

I've just published foundationdb 2.0 with support via cross-compilation for ARM. Unfortunately, while I can compile the foundationdb library for arm directly on an arm computer (and the resulting artifact works fine), the module compiled on github actions doesn't seem to work. It gets this error:

> f.setAPIVersion(700)
dyld[90234]: missing symbol called
[1] 90234 abort node

I'd love some help figuring this out if anyone has time to debug it! npm install foundationdb@2 will do it.

[atn34]

ld: warning: ignoring file /usr/local/lib/libfdb_c.dylib, building for macOS-arm64 but attempting to link with file built for macOS-x86_64

From yarn prebuild --arch arm64 in https://github.com/josephg/node-foundationdb/actions/runs/4719016895/jobs/8369299925

I have an M1 machine, and if I try to yarn prebuild --arch arm64 with the x86_64 libfdb_c I see the same thing you're seeing. But with the arm64 libfdb_c, it seems to work. Can we try installing the arm64 libfdb_c before yarn prebuild --arch arm64 in github actions?

[josephg]

Sure.

Can x86 and ARM copies of foundationdb sit side-by-side? I guess we could just install x86 foundationdb, build for x86, uninstall, then install arm foundationdb and build for arm.

I wonder if there's some #ifdefs which are set up differently in the ARM / x86 header files or something. The contents of the dynamic library files shouldn't change the compiled output.

[josephg]

Yep, installing the arm version of foundationdb before building for arm did the trick. Thanks @atn34!

foundationdb@2.0.1 works with native apple silicon support now. Sorry this took so long to get sorted! Let me know if there's any problems.