From 31b56bfff08b6ca8f3fd582a77de24a070707892 Mon Sep 17 00:00:00 2001 From: vyzo Date: Mon, 29 Jul 2019 12:33:02 +0300 Subject: [PATCH 01/13] multistream simultaneous open protocol --- connections/simopen.md | 72 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 72 insertions(+) create mode 100644 connections/simopen.md diff --git a/connections/simopen.md b/connections/simopen.md new file mode 100644 index 000000000..b563a34bb --- /dev/null +++ b/connections/simopen.md @@ -0,0 +1,72 @@ +# Simultaneous Open for multistream-select + +| Lifecycle Stage | Maturity | Status | Latest Revision | +|-----------------|---------------|--------|-----------------| +| 1A | Working Draft | Active | DRAFT | + +Authors: [@vyzo] + +Interest Group: [@raulk], [@stebalien] + +[@vyzo]: https://github.com/vyzo +[@raulk]: https://github.com/raulk +[@stebalien]: https://github.com/stebalien + +See the [lifecycle document][lifecycle-spec] for context about maturity level +and spec status. + +[lifecycle-spec]: https://github.com/libp2p/specs/blob/master/00-framework-01-spec-lifecycle.md + + +## Introduction + +In order to support direct connections through NATs with hole +punching, we need to account for simultaneous open. In such cases, +there is no single initiator and responder, but instead both peers act +as initiators. This is breaks protocol negotiation in +multistream-select, which assumes a single initator. + +This draft proposes a simple extension to the multistream protocol +negotiation in order to select a single initator when both peers are +acting as such. + +## The Protocol + +When a peer acting as the initiator enters protocol negotiation, it +sends the string `iamclient` as first protocol selector. If the other +peers is a responder or doesn't support the extension, then it +responds with `na` and protocol negotiation continues as normal. + +If both peers believe they are the initiator, then they both send +`iamclient`. If this is the case, they enter an initiator selection +phase, where one of the peers is selected to act as the initiator. In +order to do so, they both generate a random 256-bit integer and send +it as response to the `iamclient` directive. The peer with the highest +integer is selected to act as the initator and sends an `initiator` +message. The peer with the lowest integer responds with `responder` +message and both peers transition to protocol negotiation with a +distinct initiator. + +The following schematic illustrates, for the case where A's integer is +higher than B's integer: + +``` +A ---> B: iamclient +B ---> A: iamclient +A: generate random integer IA +B: generate random integer IB +A ---> B: {IA} +B ---> A: {IB} +A ---> B: initiator +B ---> A: responder +``` + +In the unlikely case where both peers selected the same integer, they +generate a fresh one and enter another round of the protocol. + +## Implementation Considerations + +The protocol is simple to implement and is backwards compatible with +vanilla multistream-select. In the common case of a single initiator, +we can ensure that there there is no latency overhead by sending the +`iamclient` message together with the multistream header. From df1ad989e259d875d4c95d7510d81a5b78ed18ca Mon Sep 17 00:00:00 2001 From: vyzo Date: Tue, 30 Jul 2019 12:39:28 +0300 Subject: [PATCH 02/13] address security protocol pipelining concerns --- connections/simopen.md | 38 +++++++++++++++++++++++++++----------- 1 file changed, 27 insertions(+), 11 deletions(-) diff --git a/connections/simopen.md b/connections/simopen.md index b563a34bb..a2925f1ed 100644 --- a/connections/simopen.md +++ b/connections/simopen.md @@ -41,11 +41,15 @@ If both peers believe they are the initiator, then they both send `iamclient`. If this is the case, they enter an initiator selection phase, where one of the peers is selected to act as the initiator. In order to do so, they both generate a random 256-bit integer and send -it as response to the `iamclient` directive. The peer with the highest -integer is selected to act as the initator and sends an `initiator` -message. The peer with the lowest integer responds with `responder` -message and both peers transition to protocol negotiation with a -distinct initiator. +it as response to the `iamclient` directive, prefixed with the +`select:` string. The peer with the highest integer is selected to act +as the initator and sends an `initiator` message. The peer with the +lowest integer responds with `responder` message and both peers +transition to protocol negotiation with a distinct initiator. + +Note the importance of the prefix in the random integer, as it allows +peers to match the selection token and ignore potentially pipelined +security protocol negotiation messages. The following schematic illustrates, for the case where A's integer is higher than B's integer: @@ -55,18 +59,30 @@ A ---> B: iamclient B ---> A: iamclient A: generate random integer IA B: generate random integer IB -A ---> B: {IA} -B ---> A: {IB} +A ---> B: select:{IA} +B ---> A: select:{IB} A ---> B: initiator B ---> A: responder ``` In the unlikely case where both peers selected the same integer, they -generate a fresh one and enter another round of the protocol. +generate a fresh one and enter another round of the protocol. If +multiple rounds of the protocol result in the same integers, this is +indicative of a bug and both peers should abort the connection. ## Implementation Considerations The protocol is simple to implement and is backwards compatible with -vanilla multistream-select. In the common case of a single initiator, -we can ensure that there there is no latency overhead by sending the -`iamclient` message together with the multistream header. +vanilla multistream-select. An important consideration is avoiding RTT +overhead in the common case of a single initiator. In this case, the +initiator pipelines the security protocol negotiation together with the +selection, sending `multistream,iamclient,secproto`. If the receiving +peer is a responder, then it replies with `multistream,na,secproto`, +negotiating the security protocol without any overhead. + +If the peer is also a client, then it also sends +`multistream,iamclient,secproto`. On seeing the `iamclient` message, +both peers enter the initiator selection protocol and ignore the +`secproto` in the original packet. They can do so because the random +integer is prefixed with the `select:` string, allowing peers to match +the selection and ignore pipelined protocols. From 15eb9695486aa6aff37bce03f88a52a61a767ab3 Mon Sep 17 00:00:00 2001 From: vyzo Date: Tue, 30 Jul 2019 17:01:52 +0300 Subject: [PATCH 03/13] Update connections/simopen.md Co-Authored-By: Jacob Heun --- connections/simopen.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/connections/simopen.md b/connections/simopen.md index a2925f1ed..92c11cadc 100644 --- a/connections/simopen.md +++ b/connections/simopen.md @@ -23,7 +23,7 @@ and spec status. In order to support direct connections through NATs with hole punching, we need to account for simultaneous open. In such cases, there is no single initiator and responder, but instead both peers act -as initiators. This is breaks protocol negotiation in +as initiators. This breaks protocol negotiation in multistream-select, which assumes a single initator. This draft proposes a simple extension to the multistream protocol From 682894ec4a81128b5312b27436dfdd0c7fed7124 Mon Sep 17 00:00:00 2001 From: vyzo Date: Tue, 24 Sep 2019 12:40:15 +0300 Subject: [PATCH 04/13] Update title Co-Authored-By: bigs --- connections/simopen.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/connections/simopen.md b/connections/simopen.md index 92c11cadc..072e73e89 100644 --- a/connections/simopen.md +++ b/connections/simopen.md @@ -1,4 +1,4 @@ -# Simultaneous Open for multistream-select +# Simultaneous Open for bootstrapping connections in multistream-select | Lifecycle Stage | Maturity | Status | Latest Revision | |-----------------|---------------|--------|-----------------| From 21be08cec47b0a53d99c8401d10d40624c059b1d Mon Sep 17 00:00:00 2001 From: vyzo Date: Tue, 24 Sep 2019 12:44:08 +0300 Subject: [PATCH 05/13] mandate base64 encoding for initiator selection integers --- connections/simopen.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/connections/simopen.md b/connections/simopen.md index 072e73e89..8dbfe820a 100644 --- a/connections/simopen.md +++ b/connections/simopen.md @@ -42,7 +42,8 @@ If both peers believe they are the initiator, then they both send phase, where one of the peers is selected to act as the initiator. In order to do so, they both generate a random 256-bit integer and send it as response to the `iamclient` directive, prefixed with the -`select:` string. The peer with the highest integer is selected to act +`select:` string. The integer is in big-endian format, encoded in base64. +The peer with the highest integer is selected to act as the initator and sends an `initiator` message. The peer with the lowest integer responds with `responder` message and both peers transition to protocol negotiation with a distinct initiator. From 26774a2e31d125a1fc0e13bac884ed5cf8b50fb2 Mon Sep 17 00:00:00 2001 From: Max Inden Date: Tue, 27 Apr 2021 19:10:45 +0200 Subject: [PATCH 06/13] connections/simopen: Prefix fake protocol ID with a '/' In order to be backwards compatible with vanilla multistream-select the simultaneous open extension disguises its identifier as a protocol ID. According to the protocol negotiation specification protocol IDs can use any string, though by convention they are prefixed with a '/'. > Each protocol supported by a peer is identified using a unique string called a protocol id. While any string can be used, the conventional format is a path-like structure containing a short name and a version number, separated by / characters. For example: /mplex/1.0.0 identifies version 1.0.0 of the mplex stream multiplexing protocol. multistream-select itself has a protocol id of /multistream/1.0.0. https://github.com/libp2p/specs/tree/master/connections#protocol-negotiation This commit replace the `iamclient` identifier with `/libp2p/simultaneous-connect` to comply with the convention. --- connections/simopen.md | 54 +++++++++++++++++++++--------------------- 1 file changed, 27 insertions(+), 27 deletions(-) diff --git a/connections/simopen.md b/connections/simopen.md index 8dbfe820a..587d4da42 100644 --- a/connections/simopen.md +++ b/connections/simopen.md @@ -32,21 +32,21 @@ acting as such. ## The Protocol -When a peer acting as the initiator enters protocol negotiation, it -sends the string `iamclient` as first protocol selector. If the other -peers is a responder or doesn't support the extension, then it -responds with `na` and protocol negotiation continues as normal. +When a peer acting as the initiator enters protocol negotiation, it sends the +string `/libp2p/simultaneous-connect` as first protocol selector. If the other +peers is a responder or doesn't support the extension, then it responds with +`na` and protocol negotiation continues as normal. If both peers believe they are the initiator, then they both send -`iamclient`. If this is the case, they enter an initiator selection -phase, where one of the peers is selected to act as the initiator. In -order to do so, they both generate a random 256-bit integer and send -it as response to the `iamclient` directive, prefixed with the -`select:` string. The integer is in big-endian format, encoded in base64. -The peer with the highest integer is selected to act -as the initator and sends an `initiator` message. The peer with the -lowest integer responds with `responder` message and both peers -transition to protocol negotiation with a distinct initiator. +`/libp2p/simultaneous-connect`. If this is the case, they enter an initiator +selection phase, where one of the peers is selected to act as the initiator. In +order to do so, they both generate a random 256-bit integer and send it as +response to the `/libp2p/simultaneous-connect` directive, prefixed with the +`select:` string. The integer is in big-endian format, encoded in base64. The +peer with the highest integer is selected to act as the initator and sends an +`initiator` message. The peer with the lowest integer responds with `responder` +message and both peers transition to protocol negotiation with a distinct +initiator. Note the importance of the prefix in the random integer, as it allows peers to match the selection token and ignore potentially pipelined @@ -56,8 +56,8 @@ The following schematic illustrates, for the case where A's integer is higher than B's integer: ``` -A ---> B: iamclient -B ---> A: iamclient +A ---> B: /libp2p/simultaneous-connect +B ---> A: /libp2p/simultaneous-connect A: generate random integer IA B: generate random integer IB A ---> B: select:{IA} @@ -73,17 +73,17 @@ indicative of a bug and both peers should abort the connection. ## Implementation Considerations -The protocol is simple to implement and is backwards compatible with -vanilla multistream-select. An important consideration is avoiding RTT -overhead in the common case of a single initiator. In this case, the -initiator pipelines the security protocol negotiation together with the -selection, sending `multistream,iamclient,secproto`. If the receiving -peer is a responder, then it replies with `multistream,na,secproto`, -negotiating the security protocol without any overhead. +The protocol is simple to implement and is backwards compatible with vanilla +multistream-select. An important consideration is avoiding RTT overhead in the +common case of a single initiator. In this case, the initiator pipelines the +security protocol negotiation together with the selection, sending +`multistream,/libp2p/simultaneous-connect,secproto`. If the receiving peer is a +responder, then it replies with `multistream,na,secproto`, negotiating the +security protocol without any overhead. If the peer is also a client, then it also sends -`multistream,iamclient,secproto`. On seeing the `iamclient` message, -both peers enter the initiator selection protocol and ignore the -`secproto` in the original packet. They can do so because the random -integer is prefixed with the `select:` string, allowing peers to match -the selection and ignore pipelined protocols. +`multistream,/libp2p/simultaneous-connect,secproto`. On seeing the +`/libp2p/simultaneous-connect` message, both peers enter the initiator selection +protocol and ignore the `secproto` in the original packet. They can do so +because the random integer is prefixed with the `select:` string, allowing peers +to match the selection and ignore pipelined protocols. From 3dd887779117ab09445fa9d1342e3bcb4b7e8e6d Mon Sep 17 00:00:00 2001 From: Max Inden Date: Fri, 7 May 2021 10:01:27 +0200 Subject: [PATCH 07/13] connections/simopen: Add revision entry --- connections/simopen.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/connections/simopen.md b/connections/simopen.md index 587d4da42..e3f7e7c8f 100644 --- a/connections/simopen.md +++ b/connections/simopen.md @@ -2,7 +2,7 @@ | Lifecycle Stage | Maturity | Status | Latest Revision | |-----------------|---------------|--------|-----------------| -| 1A | Working Draft | Active | DRAFT | +| 1A | Working Draft | Active | r0, 2021-05-07 | Authors: [@vyzo] From debc8f557174a23b6ab64ab5e03ed9d4768237a5 Mon Sep 17 00:00:00 2001 From: Max Inden Date: Fri, 7 May 2021 10:02:50 +0200 Subject: [PATCH 08/13] connections/simopen: Use 64-bit instead of 256-bit nonce The Golang implementation is using a 64-bit integer: ``` Golang myNonce := binary.LittleEndian.Uint64(randBytes) ``` https://github.com/multiformats/go-multistream/blob/4e1d9a7e25b067b7527ca2af72ce9c06529ebfb7/client.go#L167 --- connections/simopen.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/connections/simopen.md b/connections/simopen.md index e3f7e7c8f..6f19578e4 100644 --- a/connections/simopen.md +++ b/connections/simopen.md @@ -40,7 +40,7 @@ peers is a responder or doesn't support the extension, then it responds with If both peers believe they are the initiator, then they both send `/libp2p/simultaneous-connect`. If this is the case, they enter an initiator selection phase, where one of the peers is selected to act as the initiator. In -order to do so, they both generate a random 256-bit integer and send it as +order to do so, they both generate a random 64-bit integer and send it as response to the `/libp2p/simultaneous-connect` directive, prefixed with the `select:` string. The integer is in big-endian format, encoded in base64. The peer with the highest integer is selected to act as the initator and sends an From dc663fec31b76d81f8ce3f7ec2a78e4b70663e1f Mon Sep 17 00:00:00 2001 From: Max Inden Date: Fri, 7 May 2021 10:11:13 +0200 Subject: [PATCH 09/13] connections/simopen: Send nonce in base-10 string representation The Golang implementation is sending the nonce in the base-10 string representation. ```Golang myselect := []byte(tieBreakerPrefix + strconv.FormatUint(myNonce, 10)) ``` https://github.com/multiformats/go-multistream/blob/4e1d9a7e25b067b7527ca2af72ce9c06529ebfb7/client.go#L171 --- connections/simopen.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/connections/simopen.md b/connections/simopen.md index 6f19578e4..0006cc90b 100644 --- a/connections/simopen.md +++ b/connections/simopen.md @@ -42,7 +42,7 @@ If both peers believe they are the initiator, then they both send selection phase, where one of the peers is selected to act as the initiator. In order to do so, they both generate a random 64-bit integer and send it as response to the `/libp2p/simultaneous-connect` directive, prefixed with the -`select:` string. The integer is in big-endian format, encoded in base64. The +`select:` string. The integer is sent in its base-10 string representation. The peer with the highest integer is selected to act as the initator and sends an `initiator` message. The peer with the lowest integer responds with `responder` message and both peers transition to protocol negotiation with a distinct From ae86e33f3131e0d8b23710ad2e1489feeead0c1c Mon Sep 17 00:00:00 2001 From: Max Inden Date: Fri, 7 May 2021 10:13:38 +0200 Subject: [PATCH 10/13] connections/simopen: Do not retry initiator selection The Golang implementation does not retry. See https://github.com/multiformats/go-multistream/pull/42#discussion_r558755480. --- connections/simopen.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/connections/simopen.md b/connections/simopen.md index 0006cc90b..7829532c8 100644 --- a/connections/simopen.md +++ b/connections/simopen.md @@ -66,10 +66,8 @@ A ---> B: initiator B ---> A: responder ``` -In the unlikely case where both peers selected the same integer, they -generate a fresh one and enter another round of the protocol. If -multiple rounds of the protocol result in the same integers, this is -indicative of a bug and both peers should abort the connection. +In the unlikely case where both peers selected the same integer, connection +establishment fails. ## Implementation Considerations From 13ea44ae091024970ea1ccd411b4bb11aca5bb4f Mon Sep 17 00:00:00 2001 From: Max Inden Date: Fri, 7 May 2021 10:15:24 +0200 Subject: [PATCH 11/13] connections/simopen: Clarify spec being for connection bootstrapping MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit As per Raúl's comment in https://github.com/libp2p/specs/pull/196#discussion_r308391270. --- connections/simopen.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/connections/simopen.md b/connections/simopen.md index 7829532c8..da164bf2e 100644 --- a/connections/simopen.md +++ b/connections/simopen.md @@ -1,4 +1,4 @@ -# Simultaneous Open for bootstrapping connections in multistream-select +# Connection bootstrapping: Handling simultaneous open in multistream-select | Lifecycle Stage | Maturity | Status | Latest Revision | |-----------------|---------------|--------|-----------------| From 41ce7790374bb3dba2e72248b3542de808cdd643 Mon Sep 17 00:00:00 2001 From: Max Inden Date: Fri, 7 May 2021 10:18:56 +0200 Subject: [PATCH 12/13] connections/simopen: Add mxinden to interest group --- connections/simopen.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/connections/simopen.md b/connections/simopen.md index da164bf2e..5979debab 100644 --- a/connections/simopen.md +++ b/connections/simopen.md @@ -6,11 +6,12 @@ Authors: [@vyzo] -Interest Group: [@raulk], [@stebalien] +Interest Group: [@raulk], [@stebalien], [@mxinden] [@vyzo]: https://github.com/vyzo [@raulk]: https://github.com/raulk [@stebalien]: https://github.com/stebalien +[@mxinden]: https://github.com/mxinden See the [lifecycle document][lifecycle-spec] for context about maturity level and spec status. From 77cbb43b1932a2189239ca805615ea7551e4bc3b Mon Sep 17 00:00:00 2001 From: Max Inden Date: Fri, 7 May 2021 10:31:45 +0200 Subject: [PATCH 13/13] connections/README: Reference simultaneous open extension --- connections/README.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/connections/README.md b/connections/README.md index 3d3d7ed9f..2fc160a0f 100644 --- a/connections/README.md +++ b/connections/README.md @@ -229,6 +229,10 @@ Once security and stream multiplexing are both established, the connection upgrade process is complete, and both peers are able to use the resulting libp2p connection to open new secure multiplexed streams. +Note: In the case where both peers initially act as initiators, e.g. during NAT +hole punching, tie-breaking is done via the [multistream-select simultaneous +open protocol extension][simopen]. + ## Opening New Streams Over a Connection @@ -404,3 +408,4 @@ updated to incorporate the changes. [go-eventbus]: https://gihub.com/libp2p/go-eventbus [go-net-notifee]: https://github.com/libp2p/go-libp2p-core/blob/master/network/notifee.go [identify/push]: ../identify/README.md#identify-push +[simopen]: ./simopen.md