[3.2] Backporting state history plugin improvements and fixes #340

vladtr · 2022-06-02T03:47:24Z

This PR is mainly targeting performance improvements with addition of threading support plus Unix socket support. A separate Issues / PR's for other backports that would be considered valuable will be created.

Backported in this PR:

Move all network operations to another thread (8f693d3ed720056867e2bb2864ee6bde83e2570c) and all dependencies
ported session_manager_t to make add/remove thread safe
Unix socket support
SHiP post send_update to main thread to avoid overwhelming the main
Added threaded trace history/chain state history writes
Make compatible with boost < 1.70
Add comment why include_delta of code_object just returns false
Pop back a delta with empty rows
Fixed support for boolean args in unit tests (as a side bug while bringing in ut for unix socket)
Included test for local socket connection and added support o ship_client to connect via domain socket

…hread

…ndation/mandel into state-history-improvements

spoonincode · 2022-06-21T19:34:43Z

The original PR adding ship unix socket support included adding unit test support for it. It seems like that might be something worthwhile keeping in at least one of the ship tests.

vladtr · 2022-06-22T18:44:19Z

The original PR adding ship unix socket support included adding unit test support for it. It seems like that might be something worthwhile keeping in at least one of the ship tests.

Added and it fails lol Debugging it now...

vladtr · 2022-06-23T00:37:08Z

The original PR adding ship unix socket support included adding unit test support for it. It seems like that might be something worthwhile keeping in at least one of the ship tests.

Added and it fails lol Debugging it now...

Looks like ship_client needs to support unix sockets for this test to work, This is added as a separate issue:

#508

currently there is a workaround to make UT passable in ship_client, that makes it switch to ws communication when unix+ws requested. This will be removed upon 508 resolution

vladtr · 2022-06-27T03:20:46Z

Proper unix_socket support implemented for ship_client, since it is closely connected to this issue, it is committed in this PR

…into state-history-improvements

libraries/state_history/include/eosio/state_history/log.hpp

plugins/state_history_plugin/state_history_plugin.cpp

tests/ship_client.cpp

tests/CMakeLists.txt

spoonincode · 2022-07-13T20:28:30Z

libraries/state_history/include/eosio/state_history/log.hpp

@@ -119,9 +132,37 @@ class state_history_log {
            vacuum();
         }
      }
+
+       thr = std::thread([this] {


An eosio::chain::named_thread_pool (even with a size of 1) may allow you to remove some of this boilerplate. Not critical, judgement up to you, just pointing it out.

eh the whole write_thread_has_exception thing may torpedo that, dunno. Still worth thinking over.

tests/ship_client.cpp

Backported threading, session manager, unix socket

5b65a5d

vladtr linked an issue Jun 2, 2022 that may be closed by this pull request

Backport eosio/develop-boxed state_history improvements #238

Closed

vladtr changed the title ~~Backported threading, session manager, unix socket~~ Backporting state history plugin improvements and fixes Jun 2, 2022

vladtr self-assigned this Jun 2, 2022

vladtr and others added 8 commits June 2, 2022 11:29

Merge branch 'main' into state-history-improvements

b58261f

Merge branch 'main' into state-history-improvements

341b507

Check and report error on socket close

02b7aba

Merge branch 'main' into state-history-improvements

f504eb5

SHiP post send_update to main thread to avoid overwhelming the main t…

08c814c

…hread

Merge branch 'state-history-improvements' of github.com:eosnetworkfou…

56726ff

…ndation/mandel into state-history-improvements

Added threaded write

364231c

Merge branch 'main' into state-history-improvements

d72fb56

tedcahalleos added 3.1 Candidates 3.2 Candidate and removed 3.1 Candidates labels Jun 14, 2022

vladtr and others added 3 commits June 15, 2022 22:07

Merge branch 'main' into state-history-improvements

cbdea59

Merge branch 'main' into state-history-improvements

d203c36

Make sure log gracefully destructs, fixes pruning unit test

b9f29c4

vladtr marked this pull request as ready for review June 19, 2022 19:17

vladtr requested review from arhag and heifner June 19, 2022 19:17

vladtr added 3 commits June 20, 2022 18:18

Make compatible with boost < 1.70

e1b9b18

Add comment why include_delta of code_object just returns false

a9a09b1

Pop back a delta with empty rows

23bf888

vladtr added 2 commits June 21, 2022 17:27

Added ut for unix socket / fixed boolean args support for tests

5c1ff95

Few more missed socket related things

365c1d8

Fixed typo

56ece06

vladtr added 2 commits June 22, 2022 23:27

pending ship_client socket implementation

0be1228

Implemented unix socket support for ship_client

acab5b3

vladtr linked an issue Jun 27, 2022 that may be closed by this pull request

[statehist] Improvements to ship_client and ship_test #508

Closed

Cleaned up and reformatted

1282a2c

vladtr requested review from spoonincode June 30, 2022 18:28

heifner changed the title ~~Backporting state history plugin improvements and fixes~~ [3.2] Backporting state history plugin improvements and fixes Jul 7, 2022

Merge branch 'main' of https://github.com/eosnetworkfoundation/mandel …

46f1d4c

…into state-history-improvements

jgiszczak suggested changes Jul 13, 2022

View reviewed changes

vladtr added 3 commits July 13, 2022 10:48

Stray whitespaces fixes

e1068c2

Fuxup fc_xlog vs xlog

6f520bf

Use single string for both sockets

2152cfa

spoonincode reviewed Jul 13, 2022

View reviewed changes

tests/CMakeLists.txt Outdated Show resolved Hide resolved

spoonincode reviewed Jul 13, 2022

View reviewed changes

vladtr added 4 commits July 13, 2022 17:44

One listen function, std::variant for acceptor, etc

9118617

Removed check for WIN32, windows should be supported now

8e26701

Un-clang-format

dc12109

Made both acceptors work at the same time

9e8fc55

jgiszczak reviewed Jul 18, 2022

View reviewed changes

tests/ship_client.cpp Show resolved Hide resolved

vladtr added 3 commits July 19, 2022 03:02

Do not unlock mutex in write_entry at all

987a85c

Adjusted clang-format

1113530

Helped clang-format manually a bit

8854dad

jgiszczak approved these changes Jul 22, 2022

View reviewed changes

vladtr merged commit 10b6513 into main Jul 22, 2022

vladtr deleted the state-history-improvements branch July 22, 2022 22:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[3.2] Backporting state history plugin improvements and fixes #340

[3.2] Backporting state history plugin improvements and fixes #340

vladtr commented Jun 2, 2022 •

edited

Loading

spoonincode commented Jun 21, 2022

vladtr commented Jun 22, 2022

vladtr commented Jun 23, 2022 •

edited

Loading

vladtr commented Jun 27, 2022

spoonincode Jul 13, 2022

spoonincode Jul 13, 2022

[3.2] Backporting state history plugin improvements and fixes #340

[3.2] Backporting state history plugin improvements and fixes #340

Conversation

vladtr commented Jun 2, 2022 • edited Loading

spoonincode commented Jun 21, 2022

vladtr commented Jun 22, 2022

vladtr commented Jun 23, 2022 • edited Loading

vladtr commented Jun 27, 2022

spoonincode Jul 13, 2022

Choose a reason for hiding this comment

spoonincode Jul 13, 2022

Choose a reason for hiding this comment

vladtr commented Jun 2, 2022 •

edited

Loading

vladtr commented Jun 23, 2022 •

edited

Loading