Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gossipsub Protocol #898

Merged
merged 113 commits into from
Jan 24, 2020
Merged
Changes from 1 commit
Commits
Show all changes
113 commits
Select commit Hold shift + click to select a range
e9b7059
Create gossipsub crate - Basic template, borrowed from floodsub
AgeManning Jan 15, 2019
e09cb3f
Add a GossipsubConfig struct and set up basic structures in the Gossi…
AgeManning Jan 16, 2019
65f3211
Begin implementation of join. Adds get_random_peers helper function a…
AgeManning Jan 16, 2019
bcef5ff
Implements gossipsub leave()
AgeManning Jan 17, 2019
fb0160f
Update publishMany to incorporate gossipsub mesh and fanout logic
AgeManning Jan 17, 2019
e0933dd
Use the gossipsub mesh for determining peer subscription
AgeManning Jan 17, 2019
6de264e
Remove subscribed_topics field from the Gossipsub struct
AgeManning Jan 17, 2019
e3c3d0f
Rename gossipsubconfig to ProtocolConfig
AgeManning Jan 17, 2019
47dbb69
Implement the gossipsub control messages into the Codec's Encode/Deco…
AgeManning Jan 17, 2019
e05f363
Modify GossipsubActions to enums for succinctness.
AgeManning Jan 21, 2019
53c364b
Modify the memcache to store Gossipsub messages
AgeManning Jan 21, 2019
cb5e20b
Implement control message handling.
AgeManning Jan 21, 2019
e865ed5
Update control message handling to handle multiple messages.
AgeManning Jan 21, 2019
80673d1
Handle received gossipsub messages using pre-built handlers.
AgeManning Jan 21, 2019
1aa63ad
Remove excess connected peer hashmap
AgeManning Jan 21, 2019
db345bb
Add extra peer mapping and consistent topic naming.
AgeManning Jan 22, 2019
5531342
Implement heartbeat, emit_gossip and send_graft_prune.
AgeManning Jan 23, 2019
a29ae37
Group logic in forwarding messages. Add messages to memcache.
AgeManning Jan 23, 2019
a85ee9a
Add heartbeat timer and move location of helper function.
AgeManning Jan 23, 2019
e0142b6
Add gossipsub the libp2p workspace, makes layer structs public
AgeManning Jan 23, 2019
8cdaf6b
Add logging to gossipsub
AgeManning Jan 24, 2019
deccbe5
Add example chat for debugging purposes
AgeManning Jan 24, 2019
e69a772
Implement #868 for gossipsub.
AgeManning Jan 24, 2019
3dad3ea
Add rust documentation to gossipsub crate.
AgeManning Jan 28, 2019
77047c3
Re-introduce the initial heartbeat time config.
AgeManning Jan 28, 2019
c7b3ca5
Add subscribe tests.
AgeManning Jan 28, 2019
2b74f28
Add Bug fixes and further testing for gossipsub.
AgeManning Jan 29, 2019
0dd9cd2
Rename GossipsubMessage::msg_id -> id
AgeManning Jan 29, 2019
2442d50
Add bug fix for handling disconnected peers.
AgeManning Jan 29, 2019
abfa4da
Implements (partially) #889 for Gossipsub.
AgeManning Jan 29, 2019
f04d242
handle_iwant event count tests
g-r-a-n-t Jan 31, 2019
3c8ebe5
handle_ihave event count tests
g-r-a-n-t Feb 1, 2019
12e0703
Move layer.rs tests into separate file.
AgeManning Feb 1, 2019
6a126e6
Implement clippy suggestions for gossipsub.
AgeManning Feb 1, 2019
f8277fa
Modify control message tests for specific types.
AgeManning Feb 1, 2019
9165964
Implement builder pattern for GossipsubConfig.
AgeManning Feb 2, 2019
a46ecc7
Package version updates as suggested by @twittner.
AgeManning Feb 2, 2019
ec522f2
Correct line lengths in gossipsub.
AgeManning Feb 2, 2019
e5d85c9
Correct braces in found by @twittner.
AgeManning Feb 2, 2019
4a08459
Implement @twittner's suggestions.
AgeManning Feb 2, 2019
0a51e3f
Add NodeList struct to clarify topic_peers.
AgeManning Feb 2, 2019
2e855f4
Cleaner handling of messagelist
twittner Feb 2, 2019
8d32626
Cleaner handling of added peers.
twittner Feb 2, 2019
e78f417
handle_prune peer removed test
g-r-a-n-t Feb 4, 2019
30f98dc
basic grafting tests
g-r-a-n-t Feb 6, 2019
27414e3
multiple topic grafting test
g-r-a-n-t Feb 7, 2019
4ed865f
Convert &vec to slice.
twittner Feb 21, 2019
67ec8cc
Convert to lazy insert.
twittner Feb 21, 2019
1566043
Cleaner topic handling.
twittner Feb 21, 2019
dc5eddf
Merge branch 'gossipsub' of github.com:sigp/rust-libp2p into piggy-ba…
g-r-a-n-t Feb 22, 2019
866e507
control pool piggybacking
g-r-a-n-t Feb 22, 2019
faa7b03
Add Debug derives to gossipsub and correct tests.
AgeManning Feb 26, 2019
0bf1bcd
Merge master to gossipsub.
AgeManning Feb 26, 2019
e2a6c38
Merge branch 'gossipsub' of github.com:sigp/rust-libp2p into piggy-ba…
g-r-a-n-t Mar 2, 2019
d5459e7
changes from PR
g-r-a-n-t Mar 2, 2019
7ba4787
Merge pull request #3 from g-r-a-n-t/piggy-backing
AgeManning Mar 3, 2019
706c1bc
Merge latest master
AgeManning Mar 10, 2019
ed523b8
Implements Arc for GossipsubRpc events
AgeManning Mar 11, 2019
97e433e
Remove support for floodsub nodes
AgeManning Mar 11, 2019
2d9b125
Reconnected to disconnected peers, to mitigate timeout
AgeManning Mar 26, 2019
4d9cd66
Merge latest master
AgeManning Mar 26, 2019
987fec3
Use ReadOne WriteOne with configurable max gossip sizes
AgeManning Mar 26, 2019
b3c32d9
Remove length delimination from RPC encoding
AgeManning Mar 26, 2019
666d1f1
Prevent peer duplication in mesh
AgeManning Apr 2, 2019
fd6b1ab
Allow oneshot handler's inactivity_timeout to be configurable
AgeManning Apr 2, 2019
fb852bc
Implement inactivity timeout and update to latest master
AgeManning Apr 2, 2019
25ce0fa
Correct peer duplication in mesh bug
AgeManning Apr 30, 2019
9ccb186
Remove auto-reconnect to allow for user-level disconnects
AgeManning Apr 30, 2019
a6445a5
Update to latest master
AgeManning Apr 30, 2019
81e552d
Merge latest master
AgeManning Jun 18, 2019
5c612b6
Merge branch 'master' into gossipsub
AgeManning Jun 25, 2019
5284d01
Update to latest master
AgeManning Jul 26, 2019
9c56b84
Single long-lived inbound/outbound streams to match go implementation
AgeManning Jul 28, 2019
6169fba
Allow gossipsub topics to be optionally hashable
AgeManning Jul 29, 2019
7a9de4c
Improves gossipsub stream handling
AgeManning Aug 24, 2019
7680160
Update to latest master
AgeManning Aug 24, 2019
01632ef
Allows message validation in gossipsub
AgeManning Sep 3, 2019
697e46a
Replaces Cuckoofilter with LRUCache
AgeManning Sep 10, 2019
684956d
Merge latest master, updates protobuf
AgeManning Sep 29, 2019
264cf33
Renames configuration parameter and corrects logic
AgeManning Oct 24, 2019
bf7ad5c
Removes peer from fanout on disconnection
AgeManning Oct 29, 2019
eb1c5fc
Updates to the latest master
AgeManning Oct 29, 2019
6c5988f
Add publish and fanout tests
pawanjay176 Nov 6, 2019
2744b12
Merge pull request #17 from pawanjay176/gossipsub-tests
AgeManning Nov 10, 2019
3843c04
Apply @mxinden suggestions
AgeManning Nov 23, 2019
b805666
Merge latest master
AgeManning Nov 29, 2019
76b6e1f
Resend message if outbound stream negotiated
AgeManning Dec 9, 2019
8716930
Implement further reviewer suggestions
AgeManning Dec 9, 2019
82cf4dc
Add MessageId type and remove unnecessary comments
AgeManning Dec 13, 2019
9d69802
Add a return value to propagate_message function
AgeManning Dec 13, 2019
12226bc
Adds user-customised gossipsub message ids
AgeManning Dec 16, 2019
3af999a
Adds the message id to GossipsubEvent
AgeManning Dec 16, 2019
0bdf093
Implement Debug for GossipsubConfig
AgeManning Dec 16, 2019
6ac92bc
protocols/gossipsub: Add basic smoke test
mxinden Dec 23, 2019
a84d243
Merge latest master
AgeManning Jan 3, 2020
1774a76
Corrections pointed out by @mxinden
AgeManning Jan 3, 2020
776d13e
Add option to remove source id publishing
AgeManning Jan 3, 2020
ff9565b
protocols/gossipsub/tests/smoke: Remove unused variable
mxinden Jan 6, 2020
592cc9d
Merge pull request #21 from mxinden/gossipsub-smoke
AgeManning Jan 8, 2020
a255768
Merge latest master
AgeManning Jan 15, 2020
4d969ef
protocols/gossipsub: Move to stable futures
mxinden Jan 2, 2020
6375130
examples/gossipsub-chat.rs: Move to stable futures
mxinden Jan 2, 2020
1395205
protocols/gossipsub/src/behaviour/tests: Update to stable futures
mxinden Jan 2, 2020
542b100
protocols/gossipsub/tests: Update to stable futures
mxinden Jan 15, 2020
0c054df
protocols/gossipsub: Log substream errors
mxinden Jan 20, 2020
93b5bb2
protocols/gossipsub: Log outbound substream errors
mxinden Jan 20, 2020
a9e19a5
Merge pull request #22 from mxinden/gossipsub-stable-futures
AgeManning Jan 23, 2020
2b7bdbd
Merge in latest master
AgeManning Jan 23, 2020
684968c
Remove rust-fmt formatting
AgeManning Jan 23, 2020
1ca966b
Shift to prost for protobuf compiling
AgeManning Jan 24, 2020
a565efc
Merge branch 'master' into gossipsub
AgeManning Jan 24, 2020
25a9636
Use wasm_timer for wasm compatibility
AgeManning Jan 24, 2020
1033831
Merge branch 'master' into gossipsub
tomaka Jan 24, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Adds the message id to GossipsubEvent
  • Loading branch information
AgeManning committed Dec 16, 2019
commit 3af999aeb10ed8c1afdd5b9a18e1f1e92c21e39e
5 changes: 3 additions & 2 deletions examples/gossipsub-chat.rs
Original file line number Diff line number Diff line change
@@ -130,9 +130,10 @@ fn main() {
loop {
match swarm.poll().expect("Error while polling swarm") {
Async::Ready(Some(gossip_event)) => match gossip_event {
GossipsubEvent::Message(peer_id, message) => println!(
"Got message: {:?} from peer {:?}",
GossipsubEvent::Message(peer_id, id, message) => println!(
"Got message: {} with id: {} from peer: {:?}",
String::from_utf8_lossy(&message.data),
id,
peer_id
),
_ => {}
9 changes: 5 additions & 4 deletions protocols/gossipsub/src/behaviour.rs
Original file line number Diff line number Diff line change
@@ -553,7 +553,7 @@ impl<TSubstream> Gossipsub<TSubstream> {
if self.mesh.keys().any(|t| msg.topics.iter().any(|u| t == u)) {
debug!("Sending received message to user");
self.events.push_back(NetworkBehaviourAction::GenerateEvent(
GossipsubEvent::Message(propagation_source.clone(), msg.clone()),
GossipsubEvent::Message(propagation_source.clone(), msg_id, msg.clone()),
));
}

@@ -1178,9 +1178,10 @@ pub struct GossipsubRpc {
/// Event that can happen on the gossipsub behaviour.
#[derive(Debug)]
pub enum GossipsubEvent {
/// A message has been received. This contains the PeerId that we received the message from
/// and the actual message.
Message(PeerId, GossipsubMessage),
/// A message has been received. This contains the PeerId that we received the message from,
/// the message id (used if the application layer needs to propagate the message) and the
/// message itself.
Message(PeerId, MessageId, GossipsubMessage),

/// A remote subscribed to a topic.
Subscribed {
2 changes: 1 addition & 1 deletion protocols/gossipsub/src/lib.rs
Original file line number Diff line number Diff line change
@@ -146,5 +146,5 @@ mod topic;

pub use self::behaviour::{Gossipsub, GossipsubEvent, GossipsubRpc};
pub use self::config::{GossipsubConfig, GossipsubConfigBuilder};
pub use self::protocol::GossipsubMessage;
pub use self::protocol::{GossipsubMessage, MessageId};
pub use self::topic::{Topic, TopicHash};
12 changes: 12 additions & 0 deletions protocols/gossipsub/src/protocol.rs
Original file line number Diff line number Diff line change
@@ -276,6 +276,18 @@ impl Decoder for GossipsubCodec {
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct MessageId(pub String);

impl std::fmt::Display for MessageId {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{}", self.0)
}
}

impl Into<String> for MessageId {
fn into(self) -> String {
self.0.into()
}
}

/// A message received by the gossipsub system.
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct GossipsubMessage {