Add Client and Server Transport Layers based on WebTransport #107

TrustNoOneElse · 2023-09-07T17:31:51Z

Overview

This pull request introduces two transport layers, one for the client and one for the server, built on top of the WebTransport API. These transport layers enable efficient and reliable communication between clients and servers over the web. Additionally, we provide a comprehensive example to showcase their usage.

Todos

Server Transport working (with h3 or wTransport)
Client Transport working
Light Chat Example
Documentation

Additional Notes

We could add as an optional Feature that the Client Webtransport runs in a Webworker. Since im not from the users which often implement games for browsers, im not sure how much sense it would make. Also since the concept is mostly pulling data from the update method which is called trough user code.

Happy for every feedback.

… feature/webtransport

- started example for client - saving progress - fmt

TrustNoOneElse · 2023-09-20T17:07:56Z

Having some certs problem at the moment, 46 certificate unknown a tls handshake problem. Not really sure what broke or what i missed, because last week it was working xD So if anyone knows what could be wrong with the cert generation on the server side, i would be grateful. I will try this go tool for generating the cert later, maybe that will help. But yeah thats why there is currently no progress

TrustNoOneElse · 2023-09-23T12:09:37Z

Finally good news, i got the local certs working again. But chrome somehow is dead on my system (it doesn't accept anymore flags or trusted certs), so i will instead modify the readme, to support firefox for windows. I hope thats okay for everyone, i will link a Linux version from a different repo for chrome.

- certificate generation reworked - non blocking read for server - non blocked read for client WIP

TrustNoOneElse · 2023-09-30T12:38:07Z

Another good update, i finally managed, to have a non blocking client running, which can now send data to the server!
And also recieve!

Now i just need make the server also non blocking when its reading the datagram stream. Afterwards i can develop further the example and finally make clippy/format things and look other things up :D

TrustNoOneElse · 2023-10-02T16:05:34Z

Okay i tested a bit more and the write on the client side is also blocking when the server was shutdown forcefully. So i need to move this also to worker, i taking this back to draft until resolved

- Fix server too many lost clients - Client now doesn't need async

UkoeHB · 2023-10-07T21:09:20Z

We could add as an optional Feature that the Client Webtransport runs in a Webworker. Since im not from the users which often implement games for browsers, im not sure how much sense it would make. Also since the concept is mostly pulling data from the update method which is called trough user code.

Have you done any benchmarks comparing web workers vs wasm bindgen futures? Web workers have a lot of overhead, so I'm not sure they are performant for a mainly IO use-case like this.

TrustNoOneElse · 2023-10-07T21:53:16Z

Have you done any benchmarks comparing web workers vs wasm bindgen futures? Web workers have a lot of overhead, so I'm not sure they are performant for a mainly IO use-case like this.

I didn't find a way which would be able to handle the read outside of the web worker. The main problem was, that the await(JsFuture into Rust async) would hold the wasm ptr for as long as no message is there. This would mean that every next usage of the app would then throw with recursive use and the app would no longer be usable.

For the writing part i didn't test it further with worker, i had one running example but for the simple use case of a chat app it was hard to tell the difference and i wanted to avoid to give the worker more load by also writing and so maybe delay the reading depending on the load.

5tr1k3r · 2023-10-13T16:31:02Z

Hi! Very nice work, hoping to see it merged soon! I spotted a consistent spelling mistake: receive should be spelled like so (and not recieve). I hope this is not bad tone to mention it here.

- imports

UkoeHB

Very nice :)

I'm not sure the worker is actually useful, but putting it in a feature is good.

renet_webtransport/Cargo.toml

UkoeHB · 2023-10-13T16:48:21Z

renet_webtransport/src/bindings.rs

+use js_sys::Promise;
 /// This module contains the bindings to the WebTransport API.


Suggested change

use js_sys::Promise;

/// This module contains the bindings to the WebTransport API.

use js_sys::Promise;

/// This module contains the bindings to the WebTransport API.

Or actually the shortcut should go after the comment.

Co-authored-by: UkoeHB <37489173+UkoeHB@users.noreply.github.com>

…lse/renet into feature/webtransport

5tr1k3r

I'm sorry, I must have worded things poorly. I suggested the correct spelling now.

renet_webtransport/src/client.rs

renet_webtransport_server/src/lib.rs

Co-authored-by: Alex <noblemonitor@gmail.com>

TrustNoOneElse · 2023-10-13T19:35:18Z

I'm sorry, I must have worded things poorly. I suggested the correct spelling now.

ah thanks, i guess my brain autocompleted the sentence and filtered out "and not" lol, thanks for the suggestions :D!

… feature/webtransport

UkoeHB

Another review from me.

I now see two important issues with this PR.

Client ids are defined from an internal iterator. This means a server cannot directly map a pre-defined client list onto new connections (you would need a bespoke handshake after connecting to establish the client id mapping, which would be full of security challenges). With the netcode transport, client ids are assigned by the server when creating ConnectTokens, which allows the server to map 'internal user ids' onto 'renet client ids' up front. It also means the server doesn't have to re-map clients after a reconnect.
It seems that any client can connect to a server. Implementers must add a bespoke handshake on top of renet to validate new connections, which goes against the API expectation that you can safely send messages to clients once they are connected.

Both of these problems could be addressed by adopting the netcode ConnectToken workflow. Servers should issue WtConnectToken to clients. Then, the server should expect a valid WtConnectToken from connection requests, and should affiliate the client id embedded in that token with sent/received packets.

As a first-pass/proof-of-concept I would directly port ConnectToken logic into this PR, and then we can think about refinement. A big problem I have with the netcode protocol is it doesn't support reusing a ConnectToken for reconnects - the server needs to reissue tokens every time. It would be super nice to A) update the netcode protocol to enable token reuse (this is a project for me), B) for this webtransport PR to enable token reuse out of the box. EDIT: on further reflection, there are no good solutions for reconnects so I don't think the netcode protocol can be improved in this area.

UkoeHB · 2023-12-12T20:48:16Z

renet_webtransport_server/src/lib.rs

+    pub fn update(&mut self, renet_server: &mut RenetServer) {
+        let mut clients_added = 0;
+
+        if let Ok(session) = self.connection_receiver.try_recv() {


This needs to drain all new sessions, not just poll for one session.

renet_webtransport/src/client.rs

Co-authored-by: UkoeHB <37489173+UkoeHB@users.noreply.github.com>

…r update

…lse/renet into feature/webtransport

TrustNoOneElse · 2023-12-15T21:15:53Z

Another review from me.

I now see two important issues with this PR.

Client ids are defined from an internal iterator. This means a server cannot directly map a pre-defined client list onto new connections (you would need a bespoke handshake after connecting to establish the client id mapping, which would be full of security challenges). With the netcode transport, client ids are assigned by the server when creating ConnectTokens, which allows the server to map 'internal user ids' onto 'renet client ids' up front. It also means the server doesn't have to re-map clients after a reconnect.

It seems that any client can connect to a server. Implementers must add a bespoke handshake on top of renet to validate new connections, which goes against the API expectation that you can safely send messages to clients once they are connected.

Both of these problems could be addressed by adopting the netcode ConnectToken workflow. Servers should issue WtConnectToken to clients. Then, the server should expect a valid WtConnectToken from connection requests, and should affiliate the client id embedded in that token with sent/received packets.

As a first-pass/proof-of-concept I would directly port ConnectToken logic into this PR, and then we can think about refinement. A big problem I have with the netcode protocol is it doesn't support reusing a ConnectToken for reconnects - the server needs to reissue tokens every time. It would be super nice to A) update the netcode protocol to enable token reuse (this is a project for me), B) for this webtransport PR to enable token reuse out of the box. EDIT: on further reflection, there are no good solutions for reconnects so I don't think the netcode protocol can be improved in this area.

Okay after a lot of reading i get what this is about. Im not sure how complex it will be to get this mechanism into this. Thanks for the finding and i can understand the concern that it is not secure enough.

UkoeHB · 2023-12-15T21:48:42Z

Im not sure how complex it will be to get this mechanism into this.

I would look for the lowest common denominator between renetcode and webtransport, then port all the renetcode logic on top of webtransport. Renetcode has a lot of connection management logic that is quite useful for games.

It's possible that renetcode could be refactored into two crates: the netcode protocol pieces, and the integration with UDP sockets. Then the netcode protocol piece can be reused for the webtransport implementation. I haven't looked deeply into this, so I don't fully understand how much webtransport differs from UDP, or how much of the netcode protocol is not needed when using webtransport.

TrustNoOneElse · 2023-12-16T13:56:18Z

I would look for the lowest common denominator between renetcode and webtransport, then port all the renetcode logic on top of webtransport. Renetcode has a lot of connection management logic that is quite useful for games.

i will try my best, thanks for the tips! I guess we could refactor the token out like you say, @lucaspoffo, do you have any preferences on that?

UkoeHB · 2024-02-04T05:32:02Z

@TrustNoOneElse The issues I mentioned would be fixed by migrating your implementation to use the TransportSocket trait from #145, which should be very easy. Let's hope poffo merges that.

UkoeHB · 2024-04-08T21:14:52Z

@TrustNoOneElse can you give more detailed instructions for running the client example? I am having a lot of pain trying to run it. You reference ../pkg in package.json but idk what's supposed to be in there.

TrustNoOneElse · 2024-04-09T19:11:29Z

@TrustNoOneElse can you give more detailed instructions for running the client example? I am having a lot of pain trying to run it. You reference ../pkg in package.json but idk what's supposed to be in there.

Was answered on discord, but for everyone else wondering you need to build the example rust library with "wasm-pack build" :)

UkoeHB · 2024-04-20T00:35:37Z

This is now implemented in renet2, using this PR as a base.

TrustNoOneElse · 2024-04-20T09:48:11Z

Thanks @UkoeHB with that there is no reason to hold this PR any longer open :)

TrustNoOneElse added 5 commits September 4, 2023 18:56

First prototype client

2f77e10

Merge branch 'master' of https://github.com/TrustNoOneElse/renet into…

172bdaf

… feature/webtransport

Add server

960c2b7

Add example for server

dccef82

- started example for client - saving progress - fmt

First working example with server accepting client

ad4ccc8

Save Progress

1d03f81

- certificate generation reworked - non blocking read for server - non blocked read for client WIP

TrustNoOneElse added 12 commits October 1, 2023 15:23

Its finally working

08b737f

Add build script

7866855

Fmt and some clean up

d651125

Clippy part 1 of 4

155c95f

Clippy part 2 of 4

658fd40

Clippy part 3,4 of 4

9ed9006

License

ffaaa9b

fmt

0585364

fmt

c2940c8

Some extra bindings

91c6c0b

fmt

1a35cde

clippy fmt

bc8284d

TrustNoOneElse marked this pull request as ready for review October 2, 2023 13:17

TrustNoOneElse marked this pull request as draft October 2, 2023 16:05

TrustNoOneElse added 2 commits October 2, 2023 21:01

Fix writer problem

a2cf297

- Fix server too many lost clients - Client now doesn't need async

Example clean up

f2ff0c8

TrustNoOneElse marked this pull request as ready for review October 2, 2023 19:05

lucaspoffo self-requested a review October 4, 2023 01:10

TrustNoOneElse added 2 commits October 13, 2023 18:17

Worker feature added for read

586a7f9

Fix mistake

9e08196

TrustNoOneElse added 2 commits October 13, 2023 18:31

Fix mistakes

7b3bdd6

Fix spelling

fd4db7a

- imports

UkoeHB suggested changes Oct 13, 2023

View reviewed changes

TrustNoOneElse and others added 3 commits October 13, 2023 18:57

Apply suggestions from code review

cbdb90a

Co-authored-by: UkoeHB <37489173+UkoeHB@users.noreply.github.com>

Review

6994def

Merge branch 'feature/webtransport' of https://github.com/TrustNoOneE…

6275a1a

…lse/renet into feature/webtransport

5tr1k3r suggested changes Oct 13, 2023

View reviewed changes

Apply suggestions from code review

36bcb49

Co-authored-by: Alex <noblemonitor@gmail.com>

TrustNoOneElse added 5 commits November 22, 2023 21:53

Update dependencies

516fdb7

Merge branch 'master' of https://github.com/TrustNoOneElse/renet into…

5ed437c

… feature/webtransport

Merge branch 'master' of https://github.com/TrustNoOneElse/renet into…

607c91b

… feature/webtransport

Merge master

1f7af62

Fixed automatic dependency update

85d7702

UkoeHB suggested changes Dec 12, 2023

View reviewed changes

TrustNoOneElse and others added 3 commits December 15, 2023 19:55

Update renet_webtransport/src/client.rs

837524e

Co-authored-by: UkoeHB <37489173+UkoeHB@users.noreply.github.com>

Review - Server gets now all incomming connections, instead of one pe…

f0e47d5

…r update

Merge branch 'feature/webtransport' of https://github.com/TrustNoOneE…

3c3d039

…lse/renet into feature/webtransport

TrustNoOneElse closed this Apr 20, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Client and Server Transport Layers based on WebTransport #107

Add Client and Server Transport Layers based on WebTransport #107

TrustNoOneElse commented Sep 7, 2023 •

edited

Loading

TrustNoOneElse commented Sep 20, 2023

TrustNoOneElse commented Sep 23, 2023

TrustNoOneElse commented Sep 30, 2023

TrustNoOneElse commented Oct 2, 2023 •

edited

Loading

UkoeHB commented Oct 7, 2023 •

edited

Loading

TrustNoOneElse commented Oct 7, 2023

5tr1k3r commented Oct 13, 2023

UkoeHB left a comment

UkoeHB Oct 13, 2023

UkoeHB Oct 13, 2023 •

edited

Loading

5tr1k3r left a comment

TrustNoOneElse commented Oct 13, 2023

UkoeHB left a comment •

edited

Loading

UkoeHB Dec 12, 2023

TrustNoOneElse commented Dec 15, 2023

UkoeHB commented Dec 15, 2023 •

edited

Loading

TrustNoOneElse commented Dec 16, 2023

UkoeHB commented Feb 4, 2024

UkoeHB commented Apr 8, 2024

TrustNoOneElse commented Apr 9, 2024

UkoeHB commented Apr 20, 2024 •

edited

Loading

TrustNoOneElse commented Apr 20, 2024

		use js_sys::Promise;
		/// This module contains the bindings to the WebTransport API.

Add Client and Server Transport Layers based on WebTransport #107

Add Client and Server Transport Layers based on WebTransport #107

Conversation

TrustNoOneElse commented Sep 7, 2023 • edited Loading

Overview

Todos

Additional Notes

TrustNoOneElse commented Sep 20, 2023

TrustNoOneElse commented Sep 23, 2023

TrustNoOneElse commented Sep 30, 2023

TrustNoOneElse commented Oct 2, 2023 • edited Loading

UkoeHB commented Oct 7, 2023 • edited Loading

TrustNoOneElse commented Oct 7, 2023

5tr1k3r commented Oct 13, 2023

UkoeHB left a comment

Choose a reason for hiding this comment

UkoeHB Oct 13, 2023

Choose a reason for hiding this comment

UkoeHB Oct 13, 2023 • edited Loading

Choose a reason for hiding this comment

5tr1k3r left a comment

Choose a reason for hiding this comment

TrustNoOneElse commented Oct 13, 2023

UkoeHB left a comment • edited Loading

Choose a reason for hiding this comment

UkoeHB Dec 12, 2023

Choose a reason for hiding this comment

TrustNoOneElse commented Dec 15, 2023

UkoeHB commented Dec 15, 2023 • edited Loading

TrustNoOneElse commented Dec 16, 2023

UkoeHB commented Feb 4, 2024

UkoeHB commented Apr 8, 2024

TrustNoOneElse commented Apr 9, 2024

UkoeHB commented Apr 20, 2024 • edited Loading

TrustNoOneElse commented Apr 20, 2024

TrustNoOneElse commented Sep 7, 2023 •

edited

Loading

TrustNoOneElse commented Oct 2, 2023 •

edited

Loading

UkoeHB commented Oct 7, 2023 •

edited

Loading

UkoeHB Oct 13, 2023 •

edited

Loading

UkoeHB left a comment •

edited

Loading

UkoeHB commented Dec 15, 2023 •

edited

Loading

UkoeHB commented Apr 20, 2024 •

edited

Loading