Propagate block after header validation #590

heifner · 2022-12-22T03:12:03Z

Validate block header and propagate blocks without touching the main thread. This should greatly improve block propagation times. This allows a block header to be validated and a block sent to peers without touching the main thread as long as the previous block has been processed. If the previous block has not been fully processed then it is queued for processing on the main thread.

Includes:

Removal of unused zipkin calls in net_plugin & removal of unused zipkin in libfc
Optimization of broadcast blocks behavior
Optimization of peer_block_state_index

Resolves #568
Built on #543

…tion-3.2

…uld restart block because speculative block may have been interrupted.

…tion-3.2

…tion-main

… full validation of the block.

…received so it can be processed when valid later.

…ck id.

…k received during block production

…ion-main

…read is doing block header validation at a time.

linh2931 · 2022-12-30T19:17:20Z

plugins/net_plugin/net_plugin.cpp

-            fc_dlog( logger, "signaled pre_accepted_block, blk num = ${num}, id = ${id}", ("num", block->block_num())("id", id) );
-
-            auto blk_trace = fc_create_trace_with_id("Block", id);
-            auto blk_span = fc_create_span(blk_trace, "PreAccepted");


Should FC zipkin related files be removed all together?

Now that fc is no longer a submodule, I would think yes.

…OGH-590-shutdown-race [3.1] Fix race condition on trace_api_plugin shutdown

…-590-into-main

…OGH-590-into-main [3.1 -> main] Fix race condition on trace_api_plugin shutdown

plugins/producer_plugin/producer_plugin.cpp

linh2931 · 2023-01-25T14:50:08Z

plugins/chain_interface/include/eosio/chain/plugin_interface.hpp

@@ -38,7 +38,7 @@ namespace eosio { namespace chain { namespace plugin_interface {

   namespace incoming {
      namespace methods {
-      // synchronously push a block/trx to a single provider, block_state_ptr may be null
+         // synchronously push a block/trx to a single provider, block_state_ptr may be null


Not sure if this comment is accurate about synchronously push a transaction, as its name is transaction_async. Maybe split block/trx

We have a somewhat confusing use of async used in our codebase. async here means that there is a next_function provided which will continue processing. The comment is meant to convey, that regardless of the name, these methods are called synchronously unlike channels which are implemented via post() calls.

linh2931 · 2023-01-25T14:55:13Z

plugins/producer_plugin/producer_plugin.cpp

@@ -1750,8 +1751,6 @@ producer_plugin_impl::start_block_result producer_plugin_impl::start_block() {
      }

      try {
-         _account_fails.clear();


Why is this removed?

It is called in abort_block(), no need to call it here.

plugins/net_plugin/net_plugin.cpp

greg7mdp · 2023-01-25T23:20:51Z

@heifner As much as I want to, I don't feel I understand enough to approve the PR, sorry about that!

dimas1185 · 2023-01-26T23:37:27Z

general comment for future:
it is better to make such changes granular: one PR for removing zipkin, another is for block propagation. In case of we will want to revert any it will be easier to manage.

plugins/net_plugin/net_plugin.cpp

…index has to be maintained.

…it can just be calculated quickly from id

heifner added 14 commits December 12, 2022 13:07

Merge branch 'GH-525-block-propagation-3.1' into GH-525-block-propaga…

4cdbfdc

…tion-3.2

GH-525 Make forkdb thread-safe

74366a2

GH-525 Remove unused block channel.

99533be

GH-525 Move scope_exit of schedule_production_loop since any call sho…

b408aa2

…uld restart block because speculative block may have been interrupted.

GH-525 Use shared_mutex to allow multiple readers

a143803

GH-525 Only interrupt start_block for validated block_header.

6f3ad96

Merge branch 'GH-525-block-propagation-3.1' into GH-525-block-propaga…

95d5620

…tion-3.2

Merge branch 'GH-525-block-propagation-3.2' into GH-525-block-propaga…

fcad3c3

…tion-main

GH-568 Remove unused zipkin calls.

f6caf63

GH-568 Cleanup and optimize peer block tracking

9de45ed

GH-568 Broadcast block after validating block header instead of after…

ffaf945

… full validation of the block.

GH-568 If after full validation block is unlinkable then mark as not …

3885747

…received so it can be processed when valid later.

GH-568 Handle case of close of connection after recording of peer blo…

42a5c96

…ck id.

GH-568 Optimize bcast_block by removing need for conn_mtx.

80aa64f

heifner added the OCI Work exclusive to OCI team label Dec 22, 2022

heifner added 4 commits December 22, 2022 13:03

Merge remote-tracking branch 'origin/main' into GH-568-block-propagate

a24ce77

GH-568 Do not send handshake to peer when dropping a block do to bloc…

d192253

…k received during block production

Merge remote-tracking branch 'origin/main' into GH-525-block-propagat…

fa178a9

…ion-main

Merge branch 'GH-525-block-propagation-main' into GH-568-block-propagate

d956b55

heifner marked this pull request as ready for review December 22, 2022 21:38

GH-568 Post block processing to dispatcher strand so that only one th…

1688568

…read is doing block header validation at a time.

linh2931 reviewed Dec 30, 2022

View reviewed changes

GH-568 Remove unused zipkin support

1bf42bc

cmadh pushed a commit to cmadh/leap that referenced this pull request Jan 2, 2023

AntelopeIOGH-590 Fix race condition on shutdown

0faf59b

cmadh pushed a commit to cmadh/leap that referenced this pull request Jan 2, 2023

Merge pull request AntelopeIO#591 from eosnetworkfoundation/AntelopeI…

6e17696

…OGH-590-shutdown-race [3.1] Fix race condition on trace_api_plugin shutdown

cmadh pushed a commit to cmadh/leap that referenced this pull request Jan 2, 2023

Merge remote-tracking branch 'origin/release/3.1.x' into AntelopeIOGH…

3ca2934

…-590-into-main

cmadh pushed a commit to cmadh/leap that referenced this pull request Jan 2, 2023

Merge pull request AntelopeIO#604 from eosnetworkfoundation/AntelopeI…

daa2eb0

…OGH-590-into-main [3.1 -> main] Fix race condition on trace_api_plugin shutdown

heifner changed the base branch from GH-525-block-propagation-main to main January 18, 2023 20:39

Merge remote-tracking branch 'origin/main' into GH-568-block-propagate

f43f040

heifner mentioned this pull request Jan 19, 2023

Parallelize Read-only Transaction Execution Design Document eosnetworkfoundation/product#130

Merged

greg7mdp reviewed Jan 20, 2023

View reviewed changes

plugins/producer_plugin/producer_plugin.cpp Show resolved Hide resolved

plugins/producer_plugin/producer_plugin.cpp Show resolved Hide resolved

linh2931 reviewed Jan 25, 2023

View reviewed changes

linh2931 approved these changes Jan 25, 2023

View reviewed changes

greg7mdp reviewed Jan 25, 2023

View reviewed changes

Merge remote-tracking branch 'origin/main' into GH-568-block-propagate

ec4a9af

heifner added 2 commits January 25, 2023 17:38

Empty commit to kick of ci/cd

c738ea7

GH-586 Fix merge issue

bcd4a8c

heifner requested a review from dimas1185 January 26, 2023 13:36

arhag requested review from arhag and removed request for dimas1185 January 26, 2023 22:17

dimas1185 approved these changes Jan 26, 2023

View reviewed changes

arhag reviewed Jan 27, 2023

View reviewed changes

plugins/net_plugin/net_plugin.cpp Show resolved Hide resolved

plugins/net_plugin/net_plugin.cpp Outdated Show resolved Hide resolved

plugins/net_plugin/net_plugin.cpp Outdated Show resolved Hide resolved

plugins/net_plugin/net_plugin.cpp Outdated Show resolved Hide resolved

heifner added 4 commits January 27, 2023 12:21

GH-568 Remove redundant bcast_block call for accepted_block signal.

1a24e95

GH-568 Combined by_block_num index into by_connection_id so only one …

8c37219

…index has to be maintained.

Merge remote-tracking branch 'origin/main' into GH-568-block-propagate

406b4eb

GH-568 Use block_header::num_from_id instead of storing block_num

30637af

heifner requested a review from arhag January 27, 2023 18:53

GH-568 Revert passing block_num to peer_block_state_index methods as …

f0ff749

…it can just be calculated quickly from id

arhag approved these changes Jan 27, 2023

View reviewed changes

Merge branch 'main' into GH-568-block-propagate

4916654

heifner merged commit a4c2960 into main Jan 27, 2023

heifner deleted the GH-568-block-propagate branch January 27, 2023 21:15

heifner mentioned this pull request Feb 3, 2023

net_plugin handshake storm #588

Closed

chillsauce mentioned this pull request Feb 8, 2023

February 8 - Special Purpose Nodes EOS-Nation/leap#115

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Propagate block after header validation #590

Propagate block after header validation #590

heifner commented Dec 22, 2022 •

edited

Loading

linh2931 Dec 30, 2022

heifner Dec 30, 2022

heifner Dec 30, 2022

linh2931 Jan 25, 2023

heifner Jan 25, 2023

linh2931 Jan 25, 2023

heifner Jan 25, 2023

greg7mdp commented Jan 25, 2023

dimas1185 commented Jan 26, 2023

Propagate block after header validation #590

Propagate block after header validation #590

Conversation

heifner commented Dec 22, 2022 • edited Loading

linh2931 Dec 30, 2022

Choose a reason for hiding this comment

heifner Dec 30, 2022

Choose a reason for hiding this comment

heifner Dec 30, 2022

Choose a reason for hiding this comment

linh2931 Jan 25, 2023

Choose a reason for hiding this comment

heifner Jan 25, 2023

Choose a reason for hiding this comment

linh2931 Jan 25, 2023

Choose a reason for hiding this comment

heifner Jan 25, 2023

Choose a reason for hiding this comment

greg7mdp commented Jan 25, 2023

dimas1185 commented Jan 26, 2023

heifner commented Dec 22, 2022 •

edited

Loading