Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reconsider how we exchange routing table information #3838

Open
2 tasks
bowenwang1996 opened this issue Jan 21, 2021 · 6 comments
Open
2 tasks

Reconsider how we exchange routing table information #3838

bowenwang1996 opened this issue Jan 21, 2021 · 6 comments
Labels
A-network Area: Network T-core Team: issues relevant to the core team

Comments

@bowenwang1996
Copy link
Collaborator

bowenwang1996 commented Jan 21, 2021 •
edited by exalate-issue-sync bot
Loading

Currently we always exchange the entire routing table with peers when new connections are established and broadcast newly received routing table information. This is not efficient and leads to an increase of network traffic. We should consider optimizing this process with better strategies to synchronize routing table between nodes in the network.
@bowenwang1996 bowenwang1996 added the A-network Area: Network label Jan 21, 2021
@bowenwang1996
Copy link
Collaborator Author

@pmnoxx should we close this one in favor of #3934 ?

@pmnoxx
Copy link
Contributor

pmnoxx commented Mar 15, 2021

@bowenwang1996 I decided to split issues into two to make it more cleaner. #3934 focuses on theoretical work
This one on the actual implementation.

@pmnoxx pmnoxx mentioned this issue Mar 15, 2021
Closed
@pmnoxx
Copy link
Contributor

pmnoxx commented Mar 15, 2021 •
edited
Loading

I wrote a prototype: #4112
I'm going to test it and see if it works.

This was referenced Mar 15, 2021
Closed
Merged
@bowenwang1996 bowenwang1996 added the T-core Team: issues relevant to the core team label Jun 29, 2021
near-bulldozer bot pushed a commit that referenced this issue Sep 25, 2021
@pmnoxx
new routing exchange prototype using IBF (#4112)
51ab16a 
Implement new algorithm for exchanging routing tables using Inverse Bloom Filters.
#3838

Reposting #4108
mina86 added a commit to mina86/nearcore that referenced this issue Sep 27, 2021
@mina86
ibf: fix compilation error
1c57a9a 
Fixes the following compiler error:

    $ cargo build --release
    error[E0277]: `[usize; 3]` is not an iterator
       --> chain/network/src/ibf.rs:215:20
    215 |         for pos in pos_list {
        |                    ^^^^^^^^ borrow the array with `&` or call `.iter()` on it to iterate over it

Issue: near#3838
@mina86 mina86 mentioned this issue Sep 27, 2021
Merged
mina86 added a commit to mina86/nearcore that referenced this issue Sep 27, 2021
@mina86
ibf: fix adversarial build
3422568 
Fixes various compilation failures when building `cargo build --fetaures
adversarial`.

Issue: near#3838
mina86 added a commit to mina86/nearcore that referenced this issue Sep 27, 2021
@mina86
ibf: fix adversarial build
d9ca621 
Fixes various compilation failures when building `cargo build --fetaures
adversarial`.

Issue: near#3838
@mina86 mina86 mentioned this issue Sep 27, 2021
Merged
mina86 added a commit to mina86/nearcore that referenced this issue Sep 27, 2021
@mina86
nightly: remove unnecessary ‘adevrsarial’ feature
40b0eec 
NayDuck always enables the ‘adversarial’ feature when building ‘neard’.
There’s no need to explicitly enable it.  Doing so forces NayDuck to
unnecessarily create a separate build.

Issue: near#3838
@mina86 mina86 mentioned this issue Sep 27, 2021
Merged
near-bulldozer bot pushed a commit that referenced this issue Sep 27, 2021
@mina86
ibf: fix [usize; 3] not being an iterator compilation error (#4878)
e7f9998 
Fixes the following compiler error:

    $ cargo build --release
    error[E0277]: `[usize; 3]` is not an iterator
       --> chain/network/src/ibf.rs:215:20
    215 |         for pos in pos_list {
        |                    ^^^^^^^^ borrow the array with `&` or call `.iter()` on it to iterate over it

Issue: #3838
@mina86 mina86 mentioned this issue Sep 27, 2021
Merged
near-bulldozer bot pushed a commit that referenced this issue Sep 27, 2021
@mina86
nightly: remove unnecessary ‘adevrsarial’ feature (#4880)
78ecde2 
NayDuck always enables the ‘adversarial’ feature when building ‘neard’.
There’s no need to explicitly enable it.  Doing so forces NayDuck to
unnecessarily create a separate build.

Issue: #3838
near-bulldozer bot pushed a commit that referenced this issue Sep 27, 2021
@mina86
ibf: fix adversarial build (#4879)
a3a9d9b 
Fixes various compilation failures when building `cargo build --fetaures
adversarial`.

Issue: #3838
near-bulldozer bot pushed a commit that referenced this issue Sep 27, 2021
@mina86
Revert "ibf: fix [usize; 3] not being an iterator compilation error" (#…
4baa7ab 
…4881)

Turns out I was running an ‘ancient’ rustc (more specifically, for
some reason I had overrides configured in `~/.rustup/settings.toml`).
The original code compiles fine in stable Rust so my fix was
unnecessary and only adds unnecessary syntax.

This reverts commit e7f9998.

Issue: #3838
nikurt pushed a commit that referenced this issue Sep 30, 2021
@pmnoxx
new routing exchange prototype using IBF (#4112)
1bb07ac 
Implement new algorithm for exchanging routing tables using Inverse Bloom Filters.
#3838

Reposting #4108
nikurt pushed a commit that referenced this issue Sep 30, 2021
@mina86
ibf: fix [usize; 3] not being an iterator compilation error (#4878)
007d141 
Fixes the following compiler error:

    $ cargo build --release
    error[E0277]: `[usize; 3]` is not an iterator
       --> chain/network/src/ibf.rs:215:20
    215 |         for pos in pos_list {
        |                    ^^^^^^^^ borrow the array with `&` or call `.iter()` on it to iterate over it

Issue: #3838
nikurt pushed a commit that referenced this issue Sep 30, 2021
@mina86
nightly: remove unnecessary ‘adevrsarial’ feature (#4880)
f35ba23 
NayDuck always enables the ‘adversarial’ feature when building ‘neard’.
There’s no need to explicitly enable it.  Doing so forces NayDuck to
unnecessarily create a separate build.

Issue: #3838
nikurt pushed a commit that referenced this issue Sep 30, 2021
@mina86
ibf: fix adversarial build (#4879)
f4ad03f 
Fixes various compilation failures when building `cargo build --fetaures
adversarial`.

Issue: #3838
@stale
Copy link

stale bot commented Dec 16, 2021

This issue has been automatically marked as stale because it has not had recent activity in the last 2 months.
It will be closed in 7 days if no further activity occurs.
Thank you for your contributions.

@stale stale bot added the S-stale label Dec 16, 2021
@pmnoxx
Copy link
Contributor

pmnoxx commented Dec 16, 2021

There are still a few optimizations we can do to exchange routing tables:

  • Improve routing table exchange computation by factor of 4x.
    It used to take 1s to compute routing table computation for 100_100 test, now it takes 1ms, we can finish optimizing it, and make it 0.25ms.

@stale stale bot removed the S-stale label Dec 16, 2021
@pmnoxx pmnoxx added the S-stale label Dec 16, 2021
@stale stale bot removed the S-stale label Dec 16, 2021
@pmnoxx pmnoxx removed their assignment Mar 7, 2022
@stale
Copy link

stale bot commented Jun 5, 2022

This issue has been automatically marked as stale because it has not had recent activity in the last 2 months.
It will be closed in 7 days if no further activity occurs.
Thank you for your contributions.

@stale stale bot added the S-stale label Jun 5, 2022
@akhi3030 akhi3030 removed the S-stale label Jul 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
A-network Area: Network T-core Team: issues relevant to the core team
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@akhi3030 @pmnoxx @bowenwang1996