The WebSocketStream Interface

[ricea]

Specify the WebSocketStream API. This exposes backpressure to
JavaScript, avoiding pathological behavior when one side of the
connection is not keeping up. It also provides a modern API using
Streams and Promises to interoperate cleanly with the modern web
ecosystem.

See the explainer at
https://github.com/ricea/websocketstream-explainer/blob/master/README.md
for examples and motivation.

• At least two implementers are interested (and none opposed):
  • Chromium
  • …
• Tests are written and can be reviewed and commented upon at:
  • https://wpt.fyi/results/websockets/stream/
• Implementation bugs are filed:
  • Chromium: https://crbug.com/983030
  • Gecko: …
  • WebKit: …
  • Deno: WebSocketStream API denoland/deno#8831
• MDN issue is filed: …

(See WHATWG Working Mode: Changes for more details.)

---

Preview | Diff

[Lamzin]

Hi Adam,

Thanks for the PR! Looks good in general. However, I left a few comments.

Thanks,
Oleh

[ricea]

The algorithms are complete now.

@nidhijaju @domenic PTAL

The PR Preview is unhappy for some reason, but it builds locally.

[dead-claudia]

Quick question: wouldn't it be better to model it this way?

• ws = new WebSocketStream(url, options?): Open WebSocket
• void await ws.ready: Wait for ready, fails with a OperationError DOMException on premature close.
• void await ws.closed: Wait for closure
• ws.readable: Receive messages like now, reads fail with a NotReadableError DOMException on close.
• ws.writable: Send messages like now, writes fail with a NetworkError DOMException on close. Can be written to before the connection is established, to allow for writing messages immediately upon open.
• To close, just close the writer. The current values of closeCode and closeReason are used for the socket close frame.
• On server close, sets closeCode and closeReason accordingly.

WebIDL:

dictionary WebSocketStreamOptions {
  sequence<USVString> protocols;
  AbortSignal signal;
};

interface WebSocketStream {
  constructor(USVString url,
              optional WebSocketStreamOptions options);
  readonly attribute ReadableStream readable;
  readonly attribute WritableStream writable;
  readonly attribute sequence<DOMString> extensions;
  readonly attribute DOMString protocol;
  readonly attribute Promise<undefined> opened;
  readonly attribute Promise<undefined> closed;
  [Clamp] attribute unsigned short closeCode;
  attribute USVString closeReason;
};

This has a few other added benefits as well:

1. It's closer to fetch and its response = await fetch(request) processing model. Notably, you can write to requests in fetch before the request is even issued.
2. Everything just references a shared object of a single type. Easier for developers to wrap their minds around and easier for implementors to work with.
3. No need for a dedicated close method, and custom close codes and reasons even work with pipeThrough.

Note: I dropped the URL field. Userland code can just employ a Map if they truly need to care about it.

[domenic]

Only got a little ways through before I had to wrap up for the day.

[ricea]

• ws.readable: Receive messages like now, reads fail with a NotReadableError DOMException on close.

I think always throwing an exception on close would lead to poor ergonomics. For example, even a simple function like

const wss = new WebSocketStream('/messages');
for await (const message of wss.readable) {
  const div = new HTMLDivElement();
  div.textContent = message;
  body.appendChild(div);
}

would have to be wrapped in a try { ... } catch { ... } block to deal with the NotReadableError exception.

• ws.writable: Send messages like now, writes fail with a NetworkError DOMException on close. Can be written to before the connection is established, to allow for writing messages immediately upon open.

This is what WebTransport does with datagrams, but it adds a lot of complexity to the standard and implementation. In the case of WebSockets, no messages can be sent until the handshake response has been received anyway, so there's no performance benefit in queuing them early. I consciously chose not to give out the streams until they were useful to avoid the need to think about these issues.

• To close, just close the writer. The current values of closeCode and closeReason are used for the socket close frame.

This means you have to pass around the writer to any place that needs to be able to close the WebSocket. I have the impression that most users of WebSockets don't bother with the close codes but for those who do it feels awkward:

wss.closeCode = 1000;
wss.closeReason = 'Thank you';
writer.close();

• On server close, sets closeCode and closeReason accordingly.

I'm not comfortable that they can be changed asynchronously outside of script:

1. If their timing was unlucky, a developer could overwrite the value sent by the server when trying to close the connection from the user agent, and then there'd be no way to recover it.
2. closeCode and closeReason will change before any reaction to wss.closed is executed, meaning that script could synchronously observe the fact the connection was closed before it had been informed. This sort of racy behaviour leads to hard-to-understand bugs.

This has a few other added benefits as well:

1. It's closer to fetch and its response = await fetch(request) processing model. Notably, you can write to requests in fetch before the request is even issued.

In the case of HTTP, it's necessary to send the request body first, so it's not an exact parallel. fetch() doesn't make the response body available for reading until it actually has a response, which I think is the right choice.

2. Everything just references a shared object of a single type. Easier for developers to wrap their minds around and easier for implementors to work with.

It's a bit easier to implement, yes, but in my opinion it's harder to use. Developers won't have to think about the various dictionaries in my design, they can just copy the calling patterns from examples.

Implementing the construction of a Request from a RequestInit to match the fetch standard is a nightmare, but from the developer's perspective it's just an option bag, it doesn't even require thought.

WebSocketError is a bit of a wart, but I suspect most developers will never have to construct one.

3. No need for a dedicated close method, and custom close codes and reasons even work with pipeThrough.

You can actually pass a WebSocketError through pipeThrough, but I doubt anyone will.

Note: I dropped the URL field. Userland code can just employ a Map if they truly need to care about it.

I'm not sure of the usefulness of the URL field, but it's very easy to implement, and maybe if it saves someone from needing to use a Map it is worth it.

[domenic]

Didn't yet get to the last two sections, but it's all looking quite nice so far; no substantial comments.

[domenic]

LGTM with nits, nice work!

[jhugard]

Any thoughts on taking this as an opportunity to facilitate adding custom HTTP UPGRADE request headers?

For example, web clients currently must provide any auth token as a query parameter on the URL. This conflicts with security best practice, which recommend using an Authorization header to avoid accidental logging of security related info.

[ricea]

Any thoughts on taking this as an opportunity to facilitate adding custom HTTP UPGRADE request headers?

I'm not going to add it in the initial PR.

I've considered adding it initially for WebSocketStream so we can experiment with a smaller and more adaptable user base. But we'd have to add it to the old WebSocket API too eventually. This is getting off-topic for this PR so please take future discussion to #16 (or don't -- that issue has already reached a length where most people won't read the whole thing).

[dontcallmedom-bot]

This issue was mentioned in WEBRTCWG-2024-05-21 (Page 38)

[lucacasonato]

Chrome has shipped this since 124 in stable. Deno is interested in shipping too. Is there any interest in this API from Gecko or WebKit? cc @annevk and @saschanaz

[saschanaz]

This interface is quite similar to WebTransport but still is different in a few ways, I wonder it could be nicer if it was more similar so that potential migration to webtransport would be easier for devs? (e.g. have createBidirectionalStream to get the stream, even though there will only be a single pair of streams. Too hacky maybe? Also kinda interesting that WT somehow doesn't have the url attribute.)

The name is misleading btw, WebSocketStream is not really a stream but an interface to eventually get one.

The official position will be announced in mozilla/standards-positions#970.

[annevk]

@lucacasonato see WebKit/standards-positions#315.

[dead-claudia]

Okay, I revisited #48 (comment), addressed the feedback, and came up with a replacement that also uses streams a bit better.

• stream = new WebSocketStream(url, options?): Open WebSocket
• stream.readable: Receive messages via a readable stream. Comes with closeCode and closeReason properties to read those after close.
• stream.writable: Send messages via a writable stream. Comes with closeCode and closeReason properties to set those for whenever the stream is closed.
• await stream.readable/writable.negotiated: Wait for the WebSocket negotiation to complete and populate stream.readable/writable.protocol and stream.readable/writable.extensions.
  • This promise rejects on open or negotiation failure.
• Both streams close on socket disconnect and server close, as you'd expect.
• Transfer steps for the subtype would require also transmitting negotiation info and close codes/reasons across the transfer boundary.

Here's the WebIDL:

dictionary WebSocketStreamOptions {
    sequence<USVString> protocols = [];
    AbortSignal signal;
};

[Exposed=(Window, Worker)]
interface WebSocketReadableStream : ReadableStream {
    readonly attribute Promise<void> negotiated;
    readonly attribute USVString protocol;
    readonly attribute USVString extensions;
    readonly attribute short closeCode;
    readonly attribute USVString closeReason;
};

[Exposed=(Window, Worker)]
interface WebSocketWritableStream : WritableStream {
    readonly attribute Promise<void> negotiated;
    readonly attribute USVString protocol;
    readonly attribute USVString extensions;
    [Clamp] attribute short closeCode;
    attribute USVString closeReason;
};

[Exposed=(Window, Worker)]
interface WebSocketStream {
    constructor(USVString url, optional WebSocketStreamOptions options = {});
    readonly attribute WebSocketReadableStream readable;
    readonly attribute WebSocketWritableStream writable;
};

This fully separates readers and writers, so you don't need to pass the parent duplex stream around at all to set close codes. The reader and writer operate entirely independently, to the point you can even send them to different threads (and browsers may be able to optimize that to not require an intermediate controlling thread). For high-throughput web sockets, this could offer a major speed-up opportunity.