feat(jupyter): send binary data with Deno.jupyter.broadcast

[manzt]

Adds buffers to the Deno.jupyter.broadcast API to send binary data via comms. This affords the ability to send binary data via websockets to the jupyter widget frontend.

Currently, this just implements the correct messaging on the Rust side. I couldn't figure out how to correctly serialize Array<Uint8Array>
from JS to Rust via the op call. Any help there would be greatly appreciated!

[bartlomieju]

Currently, this just implements the correct messaging on the Rust side. I couldn't figure out how to correctly serialize Array
from JS to Rust via the op call. Any help there would be greatly appreciated!

To handle that add #[serde] buffers: Vec<Vec<u8>> to the op and that should do the trick

[manzt]

To handle that add #[serde] buffers: Vec<Vec<u8>> to the op and that should do the trick

Thank you! I tried this, but now I'm getting a serde error:

await Deno.jupyter.broadcast("comm_msg", {
    "comm_id": comm_id,
    "data": {
        "method": "update",
        "state": { "value": 50, "a": null, "b": null },
        "buffer_paths": [["a"], ["b"]]
    },
}, {
    "buffers": [new TextEncoder().encode("Yay"), new TextEncoder().encode("It works")]
});

Stack trace:
TypeError: serde_v8 error: invalid type; expected: array, got: Uint8Array
    at Object.broadcast (ext:cli/40_jupyter.js:13:13)
    at <anonymous>:2:20

[manzt]

For a separate PR (and once we get this working), it would be awesome to avoid copying buffers from JS to Rust when sending the messages (i.e., Vec<&[u8]>) if that's possible. But this is about my limit of my Rust knowledge and how the macros for serializing data work.

[bartlomieju]

For a separate PR (and once we get this working), it would be awesome to avoid copying buffers from JS to Rust when sending the messages (i.e., Vec<&[u8]>) if that's possible. But this is about my limit of my Rust knowledge and how the macros for serializing data work.

Since it's an async API that won't be possible.

What if you try #[serde] Vec<JsBuffer>?

[manzt]

What if you try #[serde] Vec<JsBuffer>

This works :)

[manzt]

Publishing binary text data to the frontend!

Screen.Recording.2023-10-03.at.11.58.10.AM.mov

[manzt]

What's left? would it be good to add some tests for metadata and buffers?

[bartlomieju]

@manzt I think it's enough if you update cli/tests/testdata/jupyter/integration_test.ipynb to use metadata/buffers 👍

[manzt]

Ok. I added the integration tests. As the comment notes, currently they are smoke tests to make sure we can send metadata and buffers without the kernel failing.

[bartlomieju]

LGTM, very nice!