IF: Implement proper IF transition

[arhag]

For some background on concepts related to IF see: #2080.

Carry out proper (non-atomic) transition from pre-IF to post-IF that was avoided as part of #2045.

After the IF hard fork, the old consensus algorithm rules are slightly modified to not allow dpos_irreversible_block_num to advance beyond an IF Genesis Block. This means the root of the legacy fork database can never be a descendant of the IF Genesis Block.

The root of the new fork database the first time it is used should always be an IF Genesis Block.

Do not vote on (or collect votes on) Transition IF Blocks. So, the first time finality advances past the IF Genesis block (according to the new IF algorithm) it will also advance past all Transition IF Blocks and clear out the old fork database.

For a branch of Transition IF Blocks to exist in the new fork database, it must contain an IF Critical Block that makes the IF Genesis Block irreversible (according to the old consensus algorithm). Furthermore, the fork-choice rule in the new fork database guarantees that a Proper IF Block will not be built on any of the Transition Blocks in that branch other than the IF Critical Block. However, we should still validate that when a Proper IF Block builds on top of a Transition IF Block that the Transition Block is an IF Critical Block (this is determined by look at the block_header_state_legacy of the corresponding Transition Legacy Block in the legacy fork database).

A Proper IF Block must set the schedule_version in the block header to 2^31. If the schedule_version is less than 2^31 then it is not a Proper IF Block.

As soon as there are blocks in the new fork database, the new fork database with its fork-choice rule should be used to determine the best head to build a new block on top of. All such blocks that are built would be Proper IF Blocks.

Transition Blocks should ignore any producer schedule changes.

Snapshots for Transition Blocks should include the data that would be included in the snapshot for both a Pre-IF Legacy Block and a Proper IF Block. This means including block_header_state_legacy but also including block_header_state plus the data in the valid structure of a block_state. When loading the snapshot for a Transition Block, both pieces of information should be used to construct a root for the legacy fork database and the new fork database.

Once the IF Genesis Block is identified (because it becomes irreversible according to the old algorithm), the currently uninitialized finalizer safety informations for any finalizers provided to nodeos should be initialized. Details for how this should be initialized are documented here.

Uncomment nodeos_read_terminate_at_block_if_lr_test in tests/CMakeLists.txt and verify it works as part of this effort.

[greg7mdp]

When in IF mode, we vote on proposals when we receive blocks from the network. When we transition, we probably have to send votes for the head branch of the blocks remaining in the new version of the fork_database ater the transition.
I'm thinking we may be missing votes if other nodes transition before us and send votes, as we don't have a pending_qc in the block_state_legacy to aggregate these votes.

[heifner]

When in IF mode, we vote on proposals when we receive blocks from the network. When we transition, we probably have to send votes for the head branch of the blocks remaining in the new version of the fork_database ater the transition. I'm thinking we may be missing votes if other nodes transition before us and send votes, as we don't have a pending_qc in the block_state_legacy to aggregate these votes.

Does it make any difference? Seems like voting can start happening and then LIB is recursively applied for the entire branch.

[heifner]

After transition to IF, newly created blocks should have zero action merkle roots in their block header block_header::action_mroot.

[greg7mdp]

Does it make any difference? Seems like voting can start happening and then LIB is recursively applied for the entire branch.

Yes, I think you're right we should be fine without doing anything special.

[arhag]

Still need to determine how to reconcile IF transition with changes in #2080 which added a tail and track the incremental Merkle tree for the finality tree on each validated block. In particular, we need to determine what state must be stored in a snapshot for legacy blocks.

[heifner]

b5   <-    b6   <-   b7  ..  <-   b100   <-   b101   <-  b102  <-  b103
         setfin                  b6-LIB                          b101-LIB
                                legacy-LIB                       savanna-LIB

PreIF   Transistion =>                   =X
Legacy   Legacy                 
Block    Block                 

                                              votes =>
                                               QCs  =>

          IF                      IF           IF =>
         Genesis                 Critical     Proper
         Block                   Block         Block

                                IF forkdb =>
legacy   legacy =>               legacy   =X
forkdb   forkdb                  forkdb     
 
                                               (*) =>
           IF      =>
         extension =>


(*) - New action Merkle
    - confirmed count is zero
    - Block Proposer signature on block_id
    - schedule_version == 2^31

[heifner]

To generate snapshot during transition blocks, need to create a corresponding block_state for the block using the same mechanism as the transition logic so the block_state can be stored in the snapshot along with the block_state_legacy.