test failure: nodeos_retry_transaction_if_lr_test

[greg7mdp]

See here

Total Test time (real) = 224.66 sec
1/1 Test #2162: nodeos_retry_transaction_if_lr_test ...***Failed  226.66 sec

[heifner]

error 2024-07-16T21:22:28.801 nodeos    main.cpp:217                  main                 ] 10 assert_exception: Assert Exception
g2: Invalid bls_signature
    {}
    nodeos  bls_signature.cpp:10 to_jacobian_montgomery_le
error unpacking NSt3__17variantIJN2fc6crypto6blslib23bls_aggregate_signatureEEEE
    {"type":"NSt3__17variantIJN2fc6crypto6blslib23bls_aggregate_signatureEEEE"}
    nodeos  raw.hpp:701 unpack
Error unpacking field sig
    {"field":"sig"}
    nodeos  raw.hpp:353 operator()
error unpacking eosio::chain::pending_quorum_certificate::votes_t
    {"type":"eosio::chain::pending_quorum_certificate::votes_t"}
    nodeos  raw.hpp:701 unpack
Error unpacking field strong_votes
    {"field":"strong_votes"}
    nodeos  raw.hpp:353 operator()
error unpacking eosio::chain::pending_quorum_certificate
    {"type":"eosio::chain::pending_quorum_certificate"}
    nodeos  raw.hpp:701 unpack
Error unpacking field pending_qc
    {"field":"pending_qc"}
    nodeos  raw.hpp:353 operator()
error unpacking eosio::chain::block_state
    {"type":"eosio::chain::block_state"}
    nodeos  raw.hpp:701 unpack
std::shared_ptr<T>
    {"type":"eosio::chain::block_state"}
    nodeos  raw.hpp:206 unpack
Error unpacking field _bsp
    {"field":"_bsp"}
    nodeos  raw.hpp:353 operator()
error unpacking eosio::chain::block_handle
    {"type":"eosio::chain::block_handle"}
    nodeos  raw.hpp:701 unpack

    {}
    nodeos  chain_plugin.cpp:1152 plugin_startup

I think the block_state was modified while it was being written out.

info  2024-07-16T21:22:27.720 nodeos    producer_plugin.cpp:1506      plugin_shutdown      ] exit shutdown
info  2024-07-16T21:22:27.720 nodeos    block_handle.cpp:11           write                ] Writing chain_head block 269 0000010d1f4e358cc067d89ce730ca31e599534897e080473a1161f85c3024b8
debug 2024-07-16T21:22:27.722 vote-0    quorum_certificate.cpp:162    add_vote             ] connection - 1 block_num: 269, vote strong: true, status: success, pre-state: unrestricted, post-state: unrestricted, quorum_met: false
debug 2024-07-16T21:22:27.723 vote-0    net_plugin.cpp:4066           broadcast_vote_messa ] connection - 1 on voted signal: success block #269 1f4e358cc067d89c.., strong, key SgoSky8C6d_8UPvg..
debug 2024-07-16T21:22:27.723 vote-0    net_plugin.cpp:4104           bcast_vote_message   ] bcast received vote: block #269 1f4e358cc067d89c.., strong, key SgoSky8C6d_8UPvg..
debug 2024-07-16T21:22:27.726 vote-2    quorum_certificate.cpp:162    add_vote             ] connection - 2 block_num: 269, vote strong: true, status: duplicate, pre-state: unrestricted, post-state: unrestricted, quorum_met: false
debug 2024-07-16T21:22:27.727 vote-3    quorum_certificate.cpp:162    add_vote             ] connection - 1 block_num: 269, vote strong: true, status: success, pre-state: unrestricted, post-state: strong, quorum_met: true
debug 2024-07-16T21:22:27.727 vote-3    net_plugin.cpp:4066           broadcast_vote_messa ] connection - 1 on voted signal: success block #269 1f4e358cc067d89c.., strong, key qVbh4IjYZpRGo8U_..
debug 2024-07-16T21:22:27.727 vote-3    net_plugin.cpp:4104           bcast_vote_message   ] bcast received vote: block #269 1f4e358cc067d89c.., strong, key qVbh4IjYZpRGo8U_..
info  2024-07-16T21:22:27.739 nodeos    fork_database.cpp:638         close                ] Persisting to fork_database file: /__w/spring/spring/build/TestLogs/nodeos_retry_transaction_test9/node_04/blocks/reversible/fork_db.dat

The fc::pack via FC_REFLECT does not grab the mutex on the pending_quorum_certificate.