From b4c7787c70bba6c8f44371996fc1cd36b152b03f Mon Sep 17 00:00:00 2001 From: Lin Huang Date: Thu, 12 Jan 2023 12:07:40 -0500 Subject: [PATCH 01/14] initial version of parallel.md for parallelizing read-only transaction execution --- transactions/read-only/parallel.md | 182 +++++++++++++++++++++++++++++ 1 file changed, 182 insertions(+) create mode 100644 transactions/read-only/parallel.md diff --git a/transactions/read-only/parallel.md b/transactions/read-only/parallel.md new file mode 100644 index 0000000..a0f4e42 --- /dev/null +++ b/transactions/read-only/parallel.md @@ -0,0 +1,182 @@ +# Parallelize Read-only Transaction Execution + +Continuing PR (https://github.com/AntelopeIO/leap/pull/558) (on branch https://github.com/AntelopeIO/leap/tree/send_read_only_trx), this document describes an approach to parallelize read-only transaction execution. + +## Existing Requests Handling + +### HTTP RPC Requests + +RPC requests can be classified into synchronous reads and asynchronous read-writes. +- Synchronous reads are those whose names start with `get_`, like `get_info`, `get_block`, .... They do not modify states, and are executed immediately. +- Asynchronous read-writes are the rest, like `compute_transaction`, `send_transaction`, and `send_transaction2`. They may modify states and are executed asynchronously via producer thread scheduling. + +The diagram below depicts `get_info` and `send_transaction2` handling. + +```mermaid +flowchart TD + A>RPC requests] --> B["http thread -- beast_http_session::on_read() receives requests"] + B --> C["http thread -- beast_http_session::handle_request()"] + C --> D["http thread -- http_plugin::make_app_thread_url_handler() posts to main thread"] + D -->|get_info| E1["main thread -- chain_apis::get_info(); url_response_callback()"] + D -->|send_transaction2| E2["main thread -- chain_apis::send_transaction2() "] + E1 --> F["main thread -- make_http_response_handle()r posts response to http thread"] + F --> G["http thread -- beast_http_session::send_response()"] + E2 --> H["main thread -- incoming::methods::transaction_async>()"] + H --> I["main thread -- producer_puging::on_incoming_transaction()" post to producer thread] + I --> K["producer thread -- post to main thread"] + K --> L["main thread -- producer_puging::push_transaction() executes transaction"] + L --> F +``` + +Net requests can be classified into sync and non-sync: +- Non-sync requests are handshake_message, go_away_message, notice_message, time_message, request_message, sync_request_message. They do not modify states. +- Sync requests are signed_block, packed_transaction. They may modify states. + +#### handshake_message +```mermaid +flowchart TD + A>handshake_message] --> B["net thread -- msg_handler"] + B --> C["net thread -- connection::handle_message(handshake_message) posts to main thread"] + C --> D["main thread -- handshake check"] +``` + +#### go_away_message +```mermaid +flowchart TD + A>go_away_message] --> B["net thread -- msg_handler"] + B --> C["net thread -- connection::handle_message(go_away_message) closes connection"] +``` +#### time_message +```mermaid +flowchart TD + A>time_message] --> B["net thread -- msg_handler"] + B --> C["net thread -- connection::handle_message(time_message) sends time"] +``` +#### notice_message +```mermaid +flowchart TD + A>notice_message] --> B["net thread -- msg_handler"] + B --> C["net thread -- connection::handle_message(notice_message) requests next chunk"] +``` +#### request_message +```mermaid +flowchart TD + A>request_message] --> B["net thread -- msg_handler"] + B --> C["net thread -- connection::handle_message(request_message)"] + C --> D["net thread -- connection::blk_send() or connection::blk_send_branch()" post to main thread] + D --> E["main thread -- controller::fetch_block_by_id()"] + E --> F["net thread -- sends block"] +``` + +#### sync_request_message +```mermaid +flowchart TD + A>sync_request_message] --> B["net thread -- msg_handler"] + B --> C["net thread -- connection::handle_message(sync_request_message)"] + C --> D["net thread -- connection::enqueue_sync_block()" post to main thread] + D --> E["main thread -- controller::cc.fetch_block_by_number()"] + E --> F["net thread -- sends block"] +``` + +#### packed_transaction_ptr +```mermaid +flowchart TD + A>packed_transaction_ptr] --> B["net thread -- msg_handler"] + B --> C["net thread -- connection::handle_message(packed_transaction_ptr)"] + C --> D["net thread -- chain_plug::accept_transaction()"] + D --> D1["net thread -- chain_plug::incoming_transaction_async_method()"] + D1 --> E["net thread -- incoming::methods::transaction_async>()"] + E --> F["net thread -- producer_puging::on_incoming_transaction()" post to producer thread] + F --> G["producer thread -- post to main thread"] + G --> H["main thread -- producer_puging::push_transaction() executes transaction"] +``` +#### signed_block_ptr +```mermaid +flowchart TD + A>signed_block_ptr] --> B["net thread -- msg_handler"] + B --> C["net thread -- connection::handle_message(signed_block_ptr)" post to main thread] + C --> D["main thread -- connection::process_signed_block()"] +``` + +## Design Decisions +The API node toggles between `read-write` and `read-only` windows. In `read-write` window, the node handles requests normally, except queuing read-only transactions . In `read-only` window, the node runs read-only transactions in the read-only thread pool, and handles `get_` types of RPC requests and non-sync Net requests as they come in while hold on processing RPC write and Net sync requests. + +### When to Switch Windows? +Configurable options read-write window time, and read-only window time, number of read-only transaction threshold are provided. +- At the end of read-write window and if read-only transaction queue has entries, or the number of outstanding read-only transactions reaches the threshold, switch to read-only window. This ensures both low and high number of read-only transaction cases are handled. +- At the end of read-only window or read-only queue becomes empty, switch to read-write window. The threshold option helps to make sure not too many read-only transactions are held if last read-only window exits before its end time. + +### How to Handle Write Requests in `read-only` Window? +During read-only window, new write requests keep coming in. How to handle them? +- Drop write requests. This in not acceptable as it changes the behavior of the API node. +- Queue the requests. This is complex, considering different types of requests, how and where to re-process them. +- Repost to the main thread. All write requests are handled by the main thread for some period of time. In the functor of the post to the main thread, if node is in `read-only` window, re-post it to the main thread. Care must be taken to prevent infinite loops. Should the order of write requests be kept? + +## Call Flow + +### Read-write Window +```mermaid +flowchart TD + A(((read-write window))) --> B{read-only transaction?} + B -->|yes| C1[queue the transaction] + B -->|no| C2[normal handling] +``` + +### Read-only Window +```mermaid +flowchart TD + A(((read-only window))) --> B[main thread: post to read-only thread] + B --> C{{read-only-thread: process read-only transaction}} + C --> D{is queue empty or time to switch?} + D -->|yes| E[exits the processing task] + D -->|no| C + B --> F{{"main thread: process RPC read and Net non-sync requests; repost write and sync requests"}} + F --> G{all read-only thread tasks done?} + G -->|yes| H[exit read-only window] + G -->|no| F +``` + +### Read-only Transaction Queue +```c++ +struct read_only_transaction { + send_read_only_transaction_params params; + next_func_t next; +}; + +std::queue read_only_trx_queue; +``` +Protected by a mutex. A transaction failed due to read window deadline but not transaction deadline will be put back into the queue for next round. + +### Configuration Options + +- `read-only-transaction-num-threads`: the number of threads in read-only transaction thread pool. Default to `0`. If `0`, multi-threaded execution is not used; read-only transactions are executed single threaded +- `read-write-window-time`: time in milliseconds the read-write window runs. Default to 500 milliseconds +- `read-only-window-time`: time in milliseconds the read-only window runs. Must be equal to or greater than `max-read-only-transaction-time`. Default to 200 milliseconds +- `read-only-transaction-threshold`: when the number of queued read-only transactions reaches the threshold, node switches to read-only mode, even it is before the end of read-write window. Default to 0. If 0, this option is not used. +- `read-only-window-margin`: when the time remains in the read-only window is less than the margin, no new transactions are scheduled in read-only threads. Default to 5 milliseconds. +- `max-read-only-transaction-time`: time in milliseconds a read-only transaction can execute before being considered invalid. Default to 150 milliseconds. This option has already been implemented by #558 + + +## Thread Safety + +- Safety between read-only transaction threads and other `nodeos` threads + - _main_ thread: The `main` thread only performs read-only requests. It does not have any conflicts with read-only threads. + - _chain_ thread: `chain` threads are used in `apply_block`, `log_irreversible`, `finalize_block`, `create_block_state_future`. Those do not run while in read-only window. + - _net_ thread: Non-read requests are reposted to (held by) the main thread. No conflicts with read-only transaction execution. + - _http_ thread: It is used to receive requests and send back responses. No conflicts with read-only transaction execution. + - _prod_ thread: It is used in on_incoming_transaction_async, which is not running in read-only window + - _resource monitor_ thread: Resource monitor does not have any conflicts with any transaction execution. +- Safety between read-only transaction threads: no writes are made into `chainbase` and global states when a read-only transaction is executed. This is achieved by PR #558 + +## Tests + +- number of read-only threads is 0 +- number of read-only threads is 1 +- number of read-only transactions greater than number of threads +- `read-write-window-time` test +- `read-only-transaction-window-time` test +- `read-only-transaction-threshold` test +- `read-only-window-margin` test +- read-only transactions are processed within one read window +- read-only transactions are processed in multiple read windows +- make other RPC requests while read-only transactions are executed From 8d7e0c688b365cce8011d94e0b258f50c13bae0f Mon Sep 17 00:00:00 2001 From: Lin Huang <107445030+linh2931@users.noreply.github.com> Date: Thu, 12 Jan 2023 13:23:24 -0500 Subject: [PATCH 02/14] Minor editorial corrections. --- transactions/read-only/parallel.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/transactions/read-only/parallel.md b/transactions/read-only/parallel.md index a0f4e42..349caea 100644 --- a/transactions/read-only/parallel.md +++ b/transactions/read-only/parallel.md @@ -106,9 +106,9 @@ Configurable options read-write window time, and read-only window time, number o - At the end of read-write window and if read-only transaction queue has entries, or the number of outstanding read-only transactions reaches the threshold, switch to read-only window. This ensures both low and high number of read-only transaction cases are handled. - At the end of read-only window or read-only queue becomes empty, switch to read-write window. The threshold option helps to make sure not too many read-only transactions are held if last read-only window exits before its end time. -### How to Handle Write Requests in `read-only` Window? -During read-only window, new write requests keep coming in. How to handle them? -- Drop write requests. This in not acceptable as it changes the behavior of the API node. +### How to Handle Write and sync Requests in `read-only` Window? +During read-only window, new write and sync requests keep coming in. How to handle them? +- Drop the requests. This in not acceptable as it changes the behavior of the API node. - Queue the requests. This is complex, considering different types of requests, how and where to re-process them. - Repost to the main thread. All write requests are handled by the main thread for some period of time. In the functor of the post to the main thread, if node is in `read-only` window, re-post it to the main thread. Care must be taken to prevent infinite loops. Should the order of write requests be kept? @@ -130,7 +130,7 @@ flowchart TD C --> D{is queue empty or time to switch?} D -->|yes| E[exits the processing task] D -->|no| C - B --> F{{"main thread: process RPC read and Net non-sync requests; repost write and sync requests"}} + A --> F{{"main thread: process RPC read and Net non-sync requests; repost write and sync requests"}} F --> G{all read-only thread tasks done?} G -->|yes| H[exit read-only window] G -->|no| F From db5e9e9335a33fd3294705d5b0b9b76ef6a84944 Mon Sep 17 00:00:00 2001 From: Lin Huang <107445030+linh2931@users.noreply.github.com> Date: Fri, 13 Jan 2023 13:17:12 -0500 Subject: [PATCH 03/14] Restructure and complete all analysis Analyze all RPC APIs, use tables, elaborate how to queue non-thread safe requests, restructure document. --- transactions/read-only/parallel.md | 232 ++++++++++++++--------------- 1 file changed, 111 insertions(+), 121 deletions(-) diff --git a/transactions/read-only/parallel.md b/transactions/read-only/parallel.md index 349caea..5223c91 100644 --- a/transactions/read-only/parallel.md +++ b/transactions/read-only/parallel.md @@ -6,135 +6,125 @@ Continuing PR (https://github.com/AntelopeIO/leap/pull/558) (on branch https://g ### HTTP RPC Requests -RPC requests can be classified into synchronous reads and asynchronous read-writes. -- Synchronous reads are those whose names start with `get_`, like `get_info`, `get_block`, .... They do not modify states, and are executed immediately. -- Asynchronous read-writes are the rest, like `compute_transaction`, `send_transaction`, and `send_transaction2`. They may modify states and are executed asynchronously via producer thread scheduling. - -The diagram below depicts `get_info` and `send_transaction2` handling. - -```mermaid -flowchart TD - A>RPC requests] --> B["http thread -- beast_http_session::on_read() receives requests"] - B --> C["http thread -- beast_http_session::handle_request()"] - C --> D["http thread -- http_plugin::make_app_thread_url_handler() posts to main thread"] - D -->|get_info| E1["main thread -- chain_apis::get_info(); url_response_callback()"] - D -->|send_transaction2| E2["main thread -- chain_apis::send_transaction2() "] - E1 --> F["main thread -- make_http_response_handle()r posts response to http thread"] - F --> G["http thread -- beast_http_session::send_response()"] - E2 --> H["main thread -- incoming::methods::transaction_async>()"] - H --> I["main thread -- producer_puging::on_incoming_transaction()" post to producer thread] - I --> K["producer thread -- post to main thread"] - K --> L["main thread -- producer_puging::push_transaction() executes transaction"] - L --> F -``` - -Net requests can be classified into sync and non-sync: -- Non-sync requests are handshake_message, go_away_message, notice_message, time_message, request_message, sync_request_message. They do not modify states. -- Sync requests are signed_block, packed_transaction. They may modify states. - -#### handshake_message -```mermaid -flowchart TD - A>handshake_message] --> B["net thread -- msg_handler"] - B --> C["net thread -- connection::handle_message(handshake_message) posts to main thread"] - C --> D["main thread -- handshake check"] -``` - -#### go_away_message -```mermaid -flowchart TD - A>go_away_message] --> B["net thread -- msg_handler"] - B --> C["net thread -- connection::handle_message(go_away_message) closes connection"] -``` -#### time_message -```mermaid -flowchart TD - A>time_message] --> B["net thread -- msg_handler"] - B --> C["net thread -- connection::handle_message(time_message) sends time"] -``` -#### notice_message -```mermaid -flowchart TD - A>notice_message] --> B["net thread -- msg_handler"] - B --> C["net thread -- connection::handle_message(notice_message) requests next chunk"] -``` -#### request_message -```mermaid -flowchart TD - A>request_message] --> B["net thread -- msg_handler"] - B --> C["net thread -- connection::handle_message(request_message)"] - C --> D["net thread -- connection::blk_send() or connection::blk_send_branch()" post to main thread] - D --> E["main thread -- controller::fetch_block_by_id()"] - E --> F["net thread -- sends block"] -``` - -#### sync_request_message -```mermaid -flowchart TD - A>sync_request_message] --> B["net thread -- msg_handler"] - B --> C["net thread -- connection::handle_message(sync_request_message)"] - C --> D["net thread -- connection::enqueue_sync_block()" post to main thread] - D --> E["main thread -- controller::cc.fetch_block_by_number()"] - E --> F["net thread -- sends block"] -``` - -#### packed_transaction_ptr -```mermaid -flowchart TD - A>packed_transaction_ptr] --> B["net thread -- msg_handler"] - B --> C["net thread -- connection::handle_message(packed_transaction_ptr)"] - C --> D["net thread -- chain_plug::accept_transaction()"] - D --> D1["net thread -- chain_plug::incoming_transaction_async_method()"] - D1 --> E["net thread -- incoming::methods::transaction_async>()"] - E --> F["net thread -- producer_puging::on_incoming_transaction()" post to producer thread] - F --> G["producer thread -- post to main thread"] - G --> H["main thread -- producer_puging::push_transaction() executes transaction"] -``` -#### signed_block_ptr -```mermaid -flowchart TD - A>signed_block_ptr] --> B["net thread -- msg_handler"] - B --> C["net thread -- connection::handle_message(signed_block_ptr)" post to main thread] - C --> D["main thread -- connection::process_signed_block()"] -``` +#### Chain APIs +Chain APIs can be classified into reads and writes. +- Reads are those whose names start with `get_`, like `get_info`, `get_activated_protocol_features`, `get_block`, `get_block_info` ... They do not modify states. +- Writes are the rest of requests: `compute_transaction`, `push_transaction`, `push_transactions`, `send_transaction`, `send_transaction2`, and `push_block`. They may modify states. + +Chain APIs are received on the HTTP thread, processed on the main thread (and producer thread for non-get requests), and responses are sent on the HTTP thread. + +| API | Data modified | read-only thread safe | +|---------------------------------|-------------------------------|------------------------| +| get_info | none | yes | +| get_activated_protocol_features | none | yes | +| get_block | none | yes | +| get_block_info | none | yes | +| get_block_header_state | none | yes | +| get_account | none | yes | +| get_code | none | yes | +| get_code_hash | none | yes | +| get_abi | none | yes | +| get_raw_code_and_abi | none | yes | +| get_raw_abi | none | yes | +| get_table_rows | none | yes | +| get_table_by_scope | none | yes | +| get_currency_balance | none | yes | +| get_currency_stats | none | yes | +| get_producers | none | yes | +| get_producer_schedule | none | yes | +| get_scheduled_transactions | none | yes | +| abi_json_to_bin | none | yes | +| abi_bin_to_json | none | yes | +| get_required_keys | none | yes | +| get_transaction_id | none | yes | +| get_consensus_parameters | none | yes | +| get_accounts_by_authorizers | none | yes | +| get_transaction_status | none | yes | +| send_read_only_transaction | none | yes | +| compute_transaction | main thread: temp change chainbase | no | +| push_block | main thread: chainbase, forkdb, producer, controller | no | +| push_transaction | main thread: chainbase, forkdb, producer, controller | no | +| push_transactions | main thread: chainbase, forkdb, producer, controller | no | +| send_transaction | main thread: chainbase, forkdb, producer, controller | no | +| send_transaction2 | main thread: chainbase, forkdb, producer, controller | no | + + +#### Producer APIs +Producer APIs do not mutate states. They are received on the HTTP thread, processed on the main thread, and responses are sent on the HTTP thread. + +| API | Data modified | read-only thread safe | +|-----------------------------|-------------------------------|------------------------| +| pause | main thread: producer's \_pause_production | yes | +| resume | main thread: producer's \_pause_production | yes | +| paused | | yes | +| get_runtime_options | | yes | +| update_runtime_options | main thread: producer plugin and controller configs | no (configs used by trx processing| +| add_greylist_accounts | main thread: controller resource_greylist | yes (not used in read-only trx handling. only used by max_bandwidth_billed_accounts_can_pay) | +| remove_greylist_accounts | main thread: controller resource_greylist | yes | +| get_greylist | | yes | +| get_whitelist_blacklist | | yes | +| set_whitelist_blacklist | main thread:controller blacklist, whitelist| no | +| get_integrity_hash | | yes | +| create_snapshot | | yes | +| get_scheduled_protocol_feature_activations | | yes | +| schedule_protocol_feature_activations | | no | +| get_supported_protocol_features | | yes | +| get_account_ram_corrections | | yes | +| get_unapplied_transactions | | yes | + +#### Net APIs +Net APIs do not mutate states. They are received on the HTTP thread, processed on the main thread, and responses are sent on the HTTP thread. + +| API | Data modified | read-only thread safe | +|-----------------------------|---------------------------------|------------------------| +| connect | main thread: net::connections | yes | +| disconnect | main thread: net::connections | yes | +| status | none | yes | +| connections | none | yes | + + +#### Trace APIs + API | Data modified | read-only thread safe | +|-----------------------------|---------------------------------|------------------------| +| get_block | none | yes | +| get_transaction_trace | none | yes | + +#### DB Size API +| API | Data modified | read-only thread safe | +|-----------------------------|---------------------------------|------------------------| +| get | none | yes | | + +### Net Messages + +Net messages can be classified into sync and non-sync: +- Non-sync messages do not modify states. +- Sync messages are signed_block, packed_transaction. They may modify states. + +| Message | Data modified | threads involved | read-only thread safe | +|-----------------------------|---------------------------------|-----------------------------|-----------------------| +| handshake | none | net, main (handshake check) | yes | +| go_away | none | net | yes | +| time | none | net | yes | +| notice | none | net | yes | +| request | none | net, main (controller::fetch_block_by_id()) | yes | +| sync_request | none | net, main (controller::fetch_block_by_number()) | yes | +| packed_transaction | chainbase, forkdb, producer, net, controller | net, producer, main | no | +| signed_block | chainbase, forkdb, producer, net, controller | net, producer, main | no | ## Design Decisions -The API node toggles between `read-write` and `read-only` windows. In `read-write` window, the node handles requests normally, except queuing read-only transactions . In `read-only` window, the node runs read-only transactions in the read-only thread pool, and handles `get_` types of RPC requests and non-sync Net requests as they come in while hold on processing RPC write and Net sync requests. +The node toggles between `write` and `read` windows. In `write` window, the node handles requests normally, except queuing read-only transactions . In `read` window, the node runs read-only transactions in the read-only thread pool, and runs read-only thread safe requests in other threads. ### When to Switch Windows? -Configurable options read-write window time, and read-only window time, number of read-only transaction threshold are provided. -- At the end of read-write window and if read-only transaction queue has entries, or the number of outstanding read-only transactions reaches the threshold, switch to read-only window. This ensures both low and high number of read-only transaction cases are handled. -- At the end of read-only window or read-only queue becomes empty, switch to read-write window. The threshold option helps to make sure not too many read-only transactions are held if last read-only window exits before its end time. +To facilitate window switching, configurable options write window time, and read window time, and number of read-only transaction threshold are provided. +- From `write` to `read`: Switch at the end of the `write` window and read-only transaction queue has entries, or whenever the number of outstanding read-only transactions reaches the threshold. +- From `read` to `write`: Switch at the end of read-only window or whenever read-only queue becomes empty. The threshold option helps to make sure not too many read-only transactions are held if last read-only window exits before its end time. ### How to Handle Write and sync Requests in `read-only` Window? -During read-only window, new write and sync requests keep coming in. How to handle them? -- Drop the requests. This in not acceptable as it changes the behavior of the API node. -- Queue the requests. This is complex, considering different types of requests, how and where to re-process them. +During `read` window, new read-only thread non-safe requests keep coming in. How to handle them? +- Drop the requests. This in not acceptable as it changes the behavior of the node. +- Queue the requests. Modify appbase priority queue by adding a request type `thread-safe` and `not-thread-safe`. In `write` window, everything works as it is now. In `read` window, only `thread-safe` requests are dequeued and processed. This avoids introducing new queues and keeps the order of the request. - Repost to the main thread. All write requests are handled by the main thread for some period of time. In the functor of the post to the main thread, if node is in `read-only` window, re-post it to the main thread. Care must be taken to prevent infinite loops. Should the order of write requests be kept? -## Call Flow - -### Read-write Window -```mermaid -flowchart TD - A(((read-write window))) --> B{read-only transaction?} - B -->|yes| C1[queue the transaction] - B -->|no| C2[normal handling] -``` - -### Read-only Window -```mermaid -flowchart TD - A(((read-only window))) --> B[main thread: post to read-only thread] - B --> C{{read-only-thread: process read-only transaction}} - C --> D{is queue empty or time to switch?} - D -->|yes| E[exits the processing task] - D -->|no| C - A --> F{{"main thread: process RPC read and Net non-sync requests; repost write and sync requests"}} - F --> G{all read-only thread tasks done?} - G -->|yes| H[exit read-only window] - G -->|no| F -``` ### Read-only Transaction Queue ```c++ From 01b2e15db4613c89e2395fb2ec207534d786c6f3 Mon Sep 17 00:00:00 2001 From: Lin Huang <107445030+linh2931@users.noreply.github.com> Date: Tue, 17 Jan 2023 09:21:26 -0500 Subject: [PATCH 04/14] Incorporate Areg's comments, add more design --- transactions/read-only/parallel.md | 115 ++++++++++++++++++++--------- 1 file changed, 79 insertions(+), 36 deletions(-) diff --git a/transactions/read-only/parallel.md b/transactions/read-only/parallel.md index 5223c91..1df8bda 100644 --- a/transactions/read-only/parallel.md +++ b/transactions/read-only/parallel.md @@ -2,18 +2,20 @@ Continuing PR (https://github.com/AntelopeIO/leap/pull/558) (on branch https://github.com/AntelopeIO/leap/tree/send_read_only_trx), this document describes an approach to parallelize read-only transaction execution. -## Existing Requests Handling +## Main Ideas +The node toggles between `write` and `read` windows. In `write` window, the node operates normally, except queuing read-only transactions for later parallel execution. In `read` window, the node runs queued read-only transactions in a dedicated thread pool, while in parallel in the main and other threads runs operations which are safe to read-only transaction execution. -### HTTP RPC Requests +## Existing Operation Analysis +This section analyzes thread safety of existing operations. -#### Chain APIs +### Chain APIs Chain APIs can be classified into reads and writes. -- Reads are those whose names start with `get_`, like `get_info`, `get_activated_protocol_features`, `get_block`, `get_block_info` ... They do not modify states. +- Reads are those whose names start with `get_`, like `get_info`, `get_activated_protocol_features`, `get_block`, `get_block_info`. They do not modify states. - Writes are the rest of requests: `compute_transaction`, `push_transaction`, `push_transactions`, `send_transaction`, `send_transaction2`, and `push_block`. They may modify states. Chain APIs are received on the HTTP thread, processed on the main thread (and producer thread for non-get requests), and responses are sent on the HTTP thread. -| API | Data modified | read-only thread safe | +| API | Global data modified | Safe to read-only trx? | |---------------------------------|-------------------------------|------------------------| | get_info | none | yes | | get_activated_protocol_features | none | yes | @@ -41,7 +43,7 @@ Chain APIs are received on the HTTP thread, processed on the main thread (and pr | get_accounts_by_authorizers | none | yes | | get_transaction_status | none | yes | | send_read_only_transaction | none | yes | -| compute_transaction | main thread: temp change chainbase | no | +| compute_transaction | main thread: temporally change chainbase | no | | push_block | main thread: chainbase, forkdb, producer, controller | no | | push_transaction | main thread: chainbase, forkdb, producer, controller | no | | push_transactions | main thread: chainbase, forkdb, producer, controller | no | @@ -49,10 +51,10 @@ Chain APIs are received on the HTTP thread, processed on the main thread (and pr | send_transaction2 | main thread: chainbase, forkdb, producer, controller | no | -#### Producer APIs -Producer APIs do not mutate states. They are received on the HTTP thread, processed on the main thread, and responses are sent on the HTTP thread. +### Producer APIs +They are received on the HTTP thread, processed on the main thread, and responses are sent on the HTTP thread. -| API | Data modified | read-only thread safe | +| API | Global data modified | Safe to read-only trx? | |-----------------------------|-------------------------------|------------------------| | pause | main thread: producer's \_pause_production | yes | | resume | main thread: producer's \_pause_production | yes | @@ -72,10 +74,10 @@ Producer APIs do not mutate states. They are received on the HTTP thread, proces | get_account_ram_corrections | | yes | | get_unapplied_transactions | | yes | -#### Net APIs +### Net APIs Net APIs do not mutate states. They are received on the HTTP thread, processed on the main thread, and responses are sent on the HTTP thread. -| API | Data modified | read-only thread safe | +| API | Global data modified | Safe to read-only trx? | |-----------------------------|---------------------------------|------------------------| | connect | main thread: net::connections | yes | | disconnect | main thread: net::connections | yes | @@ -83,24 +85,23 @@ Net APIs do not mutate states. They are received on the HTTP thread, processed o | connections | none | yes | -#### Trace APIs - API | Data modified | read-only thread safe | +### Trace APIs +| API | Global data modified | Safe to read-only trx? | |-----------------------------|---------------------------------|------------------------| | get_block | none | yes | | get_transaction_trace | none | yes | -#### DB Size API -| API | Data modified | read-only thread safe | +### DB Size API +| API | Global data modified | read-only thread safe | |-----------------------------|---------------------------------|------------------------| | get | none | yes | | ### Net Messages - Net messages can be classified into sync and non-sync: - Non-sync messages do not modify states. - Sync messages are signed_block, packed_transaction. They may modify states. -| Message | Data modified | threads involved | read-only thread safe | +| Message | Global data modified | threads involved | Safe to read-only trx?| |-----------------------------|---------------------------------|-----------------------------|-----------------------| | handshake | none | net, main (handshake check) | yes | | go_away | none | net | yes | @@ -111,31 +112,73 @@ Net messages can be classified into sync and non-sync: | packed_transaction | chainbase, forkdb, producer, net, controller | net, producer, main | no | | signed_block | chainbase, forkdb, producer, net, controller | net, producer, main | no | -## Design Decisions -The node toggles between `write` and `read` windows. In `write` window, the node handles requests normally, except queuing read-only transactions . In `read` window, the node runs read-only transactions in the read-only thread pool, and runs read-only thread safe requests in other threads. +### SHiP +SHiP receives blockchain state data, saves it to files, and sends data to nodes who request it. -### When to Switch Windows? -To facilitate window switching, configurable options write window time, and read window time, and number of read-only transaction threshold are provided. -- From `write` to `read`: Switch at the end of the `write` window and read-only transaction queue has entries, or whenever the number of outstanding read-only transactions reaches the threshold. -- From `read` to `write`: Switch at the end of read-only window or whenever read-only queue becomes empty. The threshold option helps to make sure not too many read-only transactions are held if last read-only window exits before its end time. +| Request | Data modified | Safe to read-only trx? | +|-----------------------|---------------------------------|------------------------| +| get_status | internal | yes | +| get_blocks | internal | yes | +| get_blocks_ack | internal | yes | -### How to Handle Write and sync Requests in `read-only` Window? -During `read` window, new read-only thread non-safe requests keep coming in. How to handle them? -- Drop the requests. This in not acceptable as it changes the behavior of the node. -- Queue the requests. Modify appbase priority queue by adding a request type `thread-safe` and `not-thread-safe`. In `write` window, everything works as it is now. In `read` window, only `thread-safe` requests are dequeued and processed. This avoids introducing new queues and keeps the order of the request. -- Repost to the main thread. All write requests are handled by the main thread for some period of time. In the functor of the post to the main thread, if node is in `read-only` window, re-post it to the main thread. Care must be taken to prevent infinite loops. Should the order of write requests be kept? +## Design Decisions +### Window Toggling +To support toggling between `read` and `write` windows, configurable options `read-window-time`, `write-window-time`, and `read-only-max-queued-time` are provided. +- From `write` to `read`: When a new read-only transaction is queued, if the time the earliest transaction has been queued exceeds `read-only-max-queued-time`; or at the end of the `write` window and read-only transaction queue not empty. +- From `read` to `write`: when read-only transaction queue is emptied by the read-only threads; or at the end of `read` window. + +In the following state diagram, `longest_queued_time = now - the time when the oldest trx was queued`, `write_window_deadline = time when write window starts + write-window-time`, and `read_window_deadline = time when read window starts + read-window-time` + +```mermaid +flowchart TD + A(((write window))) -->|push a new trx| B[longest_queued_time < read-only-max-queued-time?] + B -->|yes| R(((read window))) + B -->|no| A + A --> D[write_window_deadline passed?] + D -->|yes| F[read-only trx queue empty?] + D -->|no| A + F -->|yes| A + F -->|no| R + + R --> S[read-only trx queue empty?] + S -->|yes| A + S -->|no| R + R --> T[read_window_deadline passed?] + T -->|yes| A + T -->|no| R +``` -### Read-only Transaction Queue +### Handling main thread functions that are not safe to read-only transaction execution in `read` Window +Several options are considered to handle +- Drop the functions. This in not acceptable as it changes the behavior. +- Re-post to the main thread. In `read` window, re-post a not-read-only-safe function back to the main thread. This is simple to implement but breaks the order of functions and waste time in popping and pushing. +- Modify appbase `execution_priority_queue` by adding a function type with value `read-only-safe` or `not-read-only-safe`. In `write` window, everything works as it is now. In `read` window, only `thread-safe` requests are dequeued and processed. This avoids introducing new queues and keeps the order of the request. But as the queue is a priority queue based on function priority, it is infeasible to incorporate the additional type into the priority in a single queue. +- Separate appbase queue into two queues: `read-only-safe` or `not-read-only-safe`. The function type is added when a function is posted. + - In `read` window, only functions in `read-only-safe` queue are executed. + - In `write` window, compare the priorities of the top functions in `read-only-safe` and `not-read-only-safe`. The one with higher priority is executed. If tied, three options are considered: + - `not-read-only-safe` function is favored + - randomly pick one + - add a time attribute to the functions and earlier one is picked. + +### Read-only Transaction Priority Queue +The queue is used to store read-only transactions during `write` window. To maintain the time order when a transaction is put back into the queue for the next round due to `red` window deadline, a priority queue based on the first time when a transaction is queued is used. ```c++ -struct read_only_transaction { - send_read_only_transaction_params params; - next_func_t next; +struct read_only_trx { + fc::time_point initial_queued_time; + packed_transaction_ptr trx; + next_func_t next; +}; + +struct read_only_trx_less { + bool operator() (const read_only_trx& a, const read_only_trx& b) { + // earlier queued time has higher priority + return a.initial_queued_time > b.initial_queued_time; + } }; -std::queue read_only_trx_queue; +std::priority_queue, read_only_trx_less> read_only_trx_queue; ``` -Protected by a mutex. A transaction failed due to read window deadline but not transaction deadline will be put back into the queue for next round. ### Configuration Options @@ -152,7 +195,7 @@ Protected by a mutex. A transaction failed due to read window deadline but not t - Safety between read-only transaction threads and other `nodeos` threads - _main_ thread: The `main` thread only performs read-only requests. It does not have any conflicts with read-only threads. - _chain_ thread: `chain` threads are used in `apply_block`, `log_irreversible`, `finalize_block`, `create_block_state_future`. Those do not run while in read-only window. - - _net_ thread: Non-read requests are reposted to (held by) the main thread. No conflicts with read-only transaction execution. + - _net_ thread: Non-read requests are reposted to the main thread. No conflicts with read-only transaction execution. - _http_ thread: It is used to receive requests and send back responses. No conflicts with read-only transaction execution. - _prod_ thread: It is used in on_incoming_transaction_async, which is not running in read-only window - _resource monitor_ thread: Resource monitor does not have any conflicts with any transaction execution. @@ -169,4 +212,4 @@ Protected by a mutex. A transaction failed due to read window deadline but not t - `read-only-window-margin` test - read-only transactions are processed within one read window - read-only transactions are processed in multiple read windows -- make other RPC requests while read-only transactions are executed +- initiate RPC requests while read-only transactions are executed From f728ebf8afc34ddd9a84f6884c7da2d0c05305e2 Mon Sep 17 00:00:00 2001 From: Lin Huang <107445030+linh2931@users.noreply.github.com> Date: Tue, 17 Jan 2023 11:22:00 -0500 Subject: [PATCH 05/14] Minor editorial changes --- transactions/read-only/parallel.md | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/transactions/read-only/parallel.md b/transactions/read-only/parallel.md index 1df8bda..2a09be8 100644 --- a/transactions/read-only/parallel.md +++ b/transactions/read-only/parallel.md @@ -3,10 +3,10 @@ Continuing PR (https://github.com/AntelopeIO/leap/pull/558) (on branch https://github.com/AntelopeIO/leap/tree/send_read_only_trx), this document describes an approach to parallelize read-only transaction execution. ## Main Ideas -The node toggles between `write` and `read` windows. In `write` window, the node operates normally, except queuing read-only transactions for later parallel execution. In `read` window, the node runs queued read-only transactions in a dedicated thread pool, while in parallel in the main and other threads runs operations which are safe to read-only transaction execution. +The node toggles between `write` and `read` windows. In `write` window, the node operates normally, except queuing read-only transactions for later parallel execution in `read` window. In `read` window, the node runs queued read-only transactions in a dedicated thread pool, while in the main and other threads runs operations which are safe to read-only transaction execution. ## Existing Operation Analysis -This section analyzes thread safety of existing operations. +This section analyzes existing operations' thread safety to read-only transaction execution. ### Chain APIs Chain APIs can be classified into reads and writes. @@ -182,22 +182,22 @@ std::priority_queue, read_only_trx_less ### Configuration Options -- `read-only-transaction-num-threads`: the number of threads in read-only transaction thread pool. Default to `0`. If `0`, multi-threaded execution is not used; read-only transactions are executed single threaded -- `read-write-window-time`: time in milliseconds the read-write window runs. Default to 500 milliseconds -- `read-only-window-time`: time in milliseconds the read-only window runs. Must be equal to or greater than `max-read-only-transaction-time`. Default to 200 milliseconds -- `read-only-transaction-threshold`: when the number of queued read-only transactions reaches the threshold, node switches to read-only mode, even it is before the end of read-write window. Default to 0. If 0, this option is not used. -- `read-only-window-margin`: when the time remains in the read-only window is less than the margin, no new transactions are scheduled in read-only threads. Default to 5 milliseconds. +- `read-only-num-threads`: the number of threads in read-only transaction execution thread pool. Default to `0`. If it is `0`, read-only transactions are executed on the main thread sequentially as they arrive +- `write-window-time`: time in milliseconds the `write` window lasts. Default to 500 milliseconds +- `read-window-time`: time in milliseconds the `read` window lasts. Must be equal to or greater than `max-read-only-transaction-time`. Default to 200 milliseconds +- `read-only-max-queued-time`: time in milliseconds when is exceeded by the time the earliest transaction, node switches to `read` window, even it is before the end of `write` window. +- `read-window-min-time`: time in milliseconds which must be remained in the `read` window when new transactions are scheduled for execution. Default to 5 milliseconds. This is to avoid unnecessary incomplete transaction execution. - `max-read-only-transaction-time`: time in milliseconds a read-only transaction can execute before being considered invalid. Default to 150 milliseconds. This option has already been implemented by #558 ## Thread Safety - Safety between read-only transaction threads and other `nodeos` threads - - _main_ thread: The `main` thread only performs read-only requests. It does not have any conflicts with read-only threads. - - _chain_ thread: `chain` threads are used in `apply_block`, `log_irreversible`, `finalize_block`, `create_block_state_future`. Those do not run while in read-only window. - - _net_ thread: Non-read requests are reposted to the main thread. No conflicts with read-only transaction execution. + - _main_ thread: The `main` thread only performs functions safe to read-only transaction execution. + - _chain_ thread: `chain` threads are used in `apply_block`, `log_irreversible`, `finalize_block`, `create_block_state_future`. Those do not run while in `read` window. + - _net_ thread: It is used for low-level networking. No conflicts with read-only transaction execution. - _http_ thread: It is used to receive requests and send back responses. No conflicts with read-only transaction execution. - - _prod_ thread: It is used in on_incoming_transaction_async, which is not running in read-only window + - _prod_ thread: It is used in `on_incoming_transaction_async`, which is not running in `read` window - _resource monitor_ thread: Resource monitor does not have any conflicts with any transaction execution. - Safety between read-only transaction threads: no writes are made into `chainbase` and global states when a read-only transaction is executed. This is achieved by PR #558 From e3149f49a274c2ad81b792679a9e8d77c4a885f5 Mon Sep 17 00:00:00 2001 From: Lin Huang <107445030+linh2931@users.noreply.github.com> Date: Tue, 17 Jan 2023 14:07:39 -0500 Subject: [PATCH 06/14] Update tests --- transactions/read-only/parallel.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/transactions/read-only/parallel.md b/transactions/read-only/parallel.md index 2a09be8..5523c72 100644 --- a/transactions/read-only/parallel.md +++ b/transactions/read-only/parallel.md @@ -206,10 +206,10 @@ std::priority_queue, read_only_trx_less - number of read-only threads is 0 - number of read-only threads is 1 - number of read-only transactions greater than number of threads -- `read-write-window-time` test -- `read-only-transaction-window-time` test -- `read-only-transaction-threshold` test -- `read-only-window-margin` test +- `write-window-time` test +- `read-window-time` test +- `read-only-max-queued-time` test +- `read-window-min-time` test - read-only transactions are processed within one read window - read-only transactions are processed in multiple read windows - initiate RPC requests while read-only transactions are executed From c90d46ded39780524018466aeb25bab48d30b7a5 Mon Sep 17 00:00:00 2001 From: Lin Huang <107445030+linh2931@users.noreply.github.com> Date: Wed, 18 Jan 2023 09:38:54 -0500 Subject: [PATCH 07/14] More appbase priority queue option discussion Discuss further the options when priorities are tied. --- transactions/read-only/parallel.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/transactions/read-only/parallel.md b/transactions/read-only/parallel.md index 5523c72..bd35091 100644 --- a/transactions/read-only/parallel.md +++ b/transactions/read-only/parallel.md @@ -159,7 +159,7 @@ Several options are considered to handle - In `write` window, compare the priorities of the top functions in `read-only-safe` and `not-read-only-safe`. The one with higher priority is executed. If tied, three options are considered: - `not-read-only-safe` function is favored - randomly pick one - - add a time attribute to the functions and earlier one is picked. + - add a time attribute to the functions and the older one is picked. This keeps the original behavior. Even though at a cost of the extra time field and an extra comparison, this option seems best. ### Read-only Transaction Priority Queue The queue is used to store read-only transactions during `write` window. To maintain the time order when a transaction is put back into the queue for the next round due to `red` window deadline, a priority queue based on the first time when a transaction is queued is used. From cf6f2ed07bd1552b45b0dd3de4df662417cb5137 Mon Sep 17 00:00:00 2001 From: Lin Huang <107445030+linh2931@users.noreply.github.com> Date: Sun, 22 Jan 2023 22:14:17 -0500 Subject: [PATCH 08/14] Incorporate review comments --- transactions/read-only/parallel.md | 285 ++++++++++++++--------------- 1 file changed, 141 insertions(+), 144 deletions(-) diff --git a/transactions/read-only/parallel.md b/transactions/read-only/parallel.md index bd35091..61ca4c4 100644 --- a/transactions/read-only/parallel.md +++ b/transactions/read-only/parallel.md @@ -1,12 +1,20 @@ # Parallelize Read-only Transaction Execution -Continuing PR (https://github.com/AntelopeIO/leap/pull/558) (on branch https://github.com/AntelopeIO/leap/tree/send_read_only_trx), this document describes an approach to parallelize read-only transaction execution. +Currently all transactions are executed in the main thread. To take advantage of multi-core architecture, executing read-only transactions in a separate thread pool would make the main thread have more time to process other operations and improve performance. + +Continuing the work to support read-only transaction RPC by PR (https://github.com/AntelopeIO/leap/pull/558) (on branch https://github.com/AntelopeIO/leap/tree/send_read_only_trx), this document describes an approach to parallelize read-only transaction execution. ## Main Ideas -The node toggles between `write` and `read` windows. In `write` window, the node operates normally, except queuing read-only transactions for later parallel execution in `read` window. In `read` window, the node runs queued read-only transactions in a dedicated thread pool, while in the main and other threads runs operations which are safe to read-only transaction execution. +The node toggles between `write` and `read` windows. In `write` window, the node operates normally, except read-only transactions are queued for later parallel execution in `read` window. In `read` window, the queued read-only transactions are executed in a dedicated thread pool, while operations which are safe to read-only transaction execution are executed in the main thread. + +Several terms are defined to help understand the design. + +- *read-only transaction* - an Antelope transaction that does not modify state +- *read operation* - a task (like `get_info`) that does not modify state and any global data used during read-only transaction execution. It is safe to run in parallel with read-only transaction. +- *write operation* - a task (like `send_transacttion2`) that modifies state or any global data used during read-only transaction execution. It is not safe to run in parallel with read-only transaction. ## Existing Operation Analysis -This section analyzes existing operations' thread safety to read-only transaction execution. +This section analyzes thread safety of existing operations to read-only transaction execution. ### Chain APIs Chain APIs can be classified into reads and writes. @@ -15,201 +23,190 @@ Chain APIs can be classified into reads and writes. Chain APIs are received on the HTTP thread, processed on the main thread (and producer thread for non-get requests), and responses are sent on the HTTP thread. -| API | Global data modified | Safe to read-only trx? | -|---------------------------------|-------------------------------|------------------------| -| get_info | none | yes | -| get_activated_protocol_features | none | yes | -| get_block | none | yes | -| get_block_info | none | yes | -| get_block_header_state | none | yes | -| get_account | none | yes | -| get_code | none | yes | -| get_code_hash | none | yes | -| get_abi | none | yes | -| get_raw_code_and_abi | none | yes | -| get_raw_abi | none | yes | -| get_table_rows | none | yes | -| get_table_by_scope | none | yes | -| get_currency_balance | none | yes | -| get_currency_stats | none | yes | -| get_producers | none | yes | -| get_producer_schedule | none | yes | -| get_scheduled_transactions | none | yes | -| abi_json_to_bin | none | yes | -| abi_bin_to_json | none | yes | -| get_required_keys | none | yes | -| get_transaction_id | none | yes | -| get_consensus_parameters | none | yes | -| get_accounts_by_authorizers | none | yes | -| get_transaction_status | none | yes | -| send_read_only_transaction | none | yes | -| compute_transaction | main thread: temporally change chainbase | no | -| push_block | main thread: chainbase, forkdb, producer, controller | no | -| push_transaction | main thread: chainbase, forkdb, producer, controller | no | -| push_transactions | main thread: chainbase, forkdb, producer, controller | no | -| send_transaction | main thread: chainbase, forkdb, producer, controller | no | -| send_transaction2 | main thread: chainbase, forkdb, producer, controller | no | +| API | priority | Global data modified | Safe to read-only trx? | +|---------------------------------|-------------|------------------------------------------------------|------------------------| +| get_info | medium_high | none | yes | +| get_activated_protocol_features | medium_low | none | yes | +| get_block | medium_low | none | yes | +| get_block_info | medium_low | none | yes | +| get_block_header_state | medium_low | none | yes | +| get_account | medium_low | none | yes | +| get_code | medium_low | none | yes | +| get_code_hash | medium_low | none | yes | +| get_abi | medium_low | none | yes | +| get_raw_code_and_abi | medium_low | none | yes | +| get_raw_abi | medium_low | none | yes | +| get_table_rows | medium_low | none | yes | +| get_table_by_scope | medium_low | none | yes | +| get_currency_balance | medium_low | none | yes | +| get_currency_stats | medium_low | none | yes | +| get_producers | medium_low | none | yes | +| get_producer_schedule | medium_low | none | yes | +| get_scheduled_transactions | medium_low | none | yes | +| abi_json_to_bin | medium_low | none | yes | +| abi_bin_to_json | medium_low | none | yes | +| get_required_keys | medium_low | none | yes | +| get_transaction_id | medium_low | none | yes | +| get_consensus_parameters | medium_low | none | yes | +| get_accounts_by_authorizers | medium_low | none | yes | +| get_transaction_status | medium_low | none | yes | +| send_read_only_transaction | medium_low | none | yes | +| compute_transaction | low | main thread: temporally change chainbase | no | +| push_block | medium_low | main thread: chainbase, forkdb, producer, controller | no | +| push_transaction | low | main thread: chainbase, forkdb, producer, controller | no | +| push_transactions | low | main thread: chainbase, forkdb, producer, controller | no | +| send_transaction | low | main thread: chainbase, forkdb, producer, controller | no | +| send_transaction2 | low | main thread: chainbase, forkdb, producer, controller | no | ### Producer APIs They are received on the HTTP thread, processed on the main thread, and responses are sent on the HTTP thread. -| API | Global data modified | Safe to read-only trx? | -|-----------------------------|-------------------------------|------------------------| -| pause | main thread: producer's \_pause_production | yes | -| resume | main thread: producer's \_pause_production | yes | -| paused | | yes | -| get_runtime_options | | yes | -| update_runtime_options | main thread: producer plugin and controller configs | no (configs used by trx processing| -| add_greylist_accounts | main thread: controller resource_greylist | yes (not used in read-only trx handling. only used by max_bandwidth_billed_accounts_can_pay) | -| remove_greylist_accounts | main thread: controller resource_greylist | yes | -| get_greylist | | yes | -| get_whitelist_blacklist | | yes | -| set_whitelist_blacklist | main thread:controller blacklist, whitelist| no | -| get_integrity_hash | | yes | -| create_snapshot | | yes | -| get_scheduled_protocol_feature_activations | | yes | -| schedule_protocol_feature_activations | | no | -| get_supported_protocol_features | | yes | -| get_account_ram_corrections | | yes | -| get_unapplied_transactions | | yes | +| API | priority | Global data modified | Safe to read-only trx? | +|--------------------------------------------|-------------|------------------------------|------------------------| +| pause | medium_high | main thread: producer's \_pause_production | yes | +| resume | medium_high | main thread: producer's \_pause_production | yes | +| paused | medium_high | | yes | +| get_runtime_options | medium_high | | yes | +| update_runtime_options | medium_high | main thread: producer plugin and controller configs | no (configs used by trx processing| +| add_greylist_accounts | medium_high | main thread: controller resource_greylist | yes (not used in read-only trx handling. only used by max_bandwidth_billed_accounts_can_pay) | +| remove_greylist_accounts | medium_high | main thread: controller resource_greylist | yes | +| get_greylist | medium_high | | yes | +| get_whitelist_blacklist | medium_high | | yes | +| set_whitelist_blacklist | medium_high | main thread:controller blacklist, whitelist| no | +| get_integrity_hash | medium_high | | yes | +| create_snapshot | medium_high | | yes | +| get_scheduled_protocol_feature_activations | medium_high | | yes | +| schedule_protocol_feature_activations | medium_high | | no | +| get_supported_protocol_features | medium_high | | yes | +| get_account_ram_corrections | medium_high | | yes | +| get_unapplied_transactions | medium_high | | yes | ### Net APIs Net APIs do not mutate states. They are received on the HTTP thread, processed on the main thread, and responses are sent on the HTTP thread. -| API | Global data modified | Safe to read-only trx? | -|-----------------------------|---------------------------------|------------------------| -| connect | main thread: net::connections | yes | -| disconnect | main thread: net::connections | yes | -| status | none | yes | -| connections | none | yes | +| API | priority | Global data modified | Safe to read-only trx? | +|-----------------------------|-------------|--------------------------------|------------------------| +| connect | medium_high | main thread: net::connections | yes | +| disconnect | medium_high | main thread: net::connections | yes | +| status | medium_high | none | yes | +| connections | medium_high | none | yes | ### Trace APIs -| API | Global data modified | Safe to read-only trx? | -|-----------------------------|---------------------------------|------------------------| -| get_block | none | yes | -| get_transaction_trace | none | yes | +Trace APIs do not go through main thread. They are executed only in http thread. +| API | priority | Global data modified | Safe to read-only trx? | +|-----------------------------|----------|------------------------|------------------------| +| get_block | NA | none | yes | +| get_transaction_trace | NA | none | yes | ### DB Size API -| API | Global data modified | read-only thread safe | -|-----------------------------|---------------------------------|------------------------| -| get | none | yes | | +| API | priority | Global data modified | read-only thread safe | +|-----------------------------|------------|------------------------|------------------------| +| get | medium_low | none | yes | ### Net Messages Net messages can be classified into sync and non-sync: - Non-sync messages do not modify states. - Sync messages are signed_block, packed_transaction. They may modify states. -| Message | Global data modified | threads involved | Safe to read-only trx?| -|-----------------------------|---------------------------------|-----------------------------|-----------------------| -| handshake | none | net, main (handshake check) | yes | -| go_away | none | net | yes | -| time | none | net | yes | -| notice | none | net | yes | -| request | none | net, main (controller::fetch_block_by_id()) | yes | -| sync_request | none | net, main (controller::fetch_block_by_number()) | yes | -| packed_transaction | chainbase, forkdb, producer, net, controller | net, producer, main | no | -| signed_block | chainbase, forkdb, producer, net, controller | net, producer, main | no | +| Message | priority | Global data modified | threads involved | Safe to read-only trx? | +|--------------------|----------|---------------------------------|-----------------------------|-------------------------| +| handshake | medium | none | net, main (handshake check) | yes | +| go_away | NA | none | net | yes | +| time | NA | none | net | yes | +| notice | NA | none | net | yes | +| request | medium | none | net, main (controller::fetch_block_by_id()) | yes | +| sync_request | medium | none | net, main (controller::fetch_block_by_number()) | yes | +| packed_transaction | low | chainbase, forkdb, producer, net, controller | net, producer, main | no | +| signed_block | medium | chainbase, forkdb, producer, net, controller | net, producer, main | no | ### SHiP SHiP receives blockchain state data, saves it to files, and sends data to nodes who request it. -| Request | Data modified | Safe to read-only trx? | -|-----------------------|---------------------------------|------------------------| -| get_status | internal | yes | -| get_blocks | internal | yes | -| get_blocks_ack | internal | yes | +| Request | priority | Data modified | Safe to read-only trx? | +|------------------------|----------|----------------------|------------------------| +| get_status_request | medium | internal | yes | +| get_blocks_request | medium | internal | yes | +| get_blocks_ack_request | medium | internal | yes | ## Design Decisions +### Configuration Options + +Attempt is made to only introduce a minimum set of options. Other options might be added in the future if needed. +- `read-only-transaction-threads`: number of worker threads in read-only transaction thread pool. If it is `0`, read-only transactions are executed on the main thread sequentially. Default to `0`. The number of the threads will be limited by the virtual memory size the system has (see https://github.com/AntelopeIO/leap/issues/645). +- `read-only-write-window-time-ms`: time in milliseconds the `write` window lasts. For this option to take effect, `read-only-transaction-threads` must be set greater than 0. Default to 300 milliseconds. +- `read-only-read-window-time-ms`: time in milliseconds the `read` window lasts. For this option to take effect, `read-only-transaction-threads` must be set greater than 0. Default to 100 milliseconds. + +Note: `read-only-max-transaction-time-ms` (time in milliseconds a read-only transaction can execute before being considered invalid) was considered. But to keep new options minimal, for now, use the existing `max-transaction-time` for read-only transactions. + +### Data Structures + +Three queues are proposed: `appbase`'s `execution_priority_queue` is replaced with read operation queue and write operation queue, and a new read-only transaction queue is added. +- `read operation queue` stores read operations which are safe to read-only transaction execution. It is a priority queue sorted by operation priority. When an operation is inserted into `read operation queue` or `write operation queue`, a global sequence number is incremented. This sequence number is stored together with each operation. +- `write operation queue` stores write operations which are not safe to read-only transaction execution. It uses the same type of priority queue as in `read operation queue`. +- `read-only transaction queue` stores read-only transactions. It is a deque. + +### Window Processing + +In `write window`, operations in `write queue` and `read queue` are executed sequentially in the main thread, while read-only transactions are queued in the read-only transaction queue for later execution. To select next operation to be executed, front entries in `write operation queue` and `read operation queue` are compared; the one with higher priority is chosen. If the priorities are tied, the sequence numbers are compared; the one with smaller number (earlier) is selected. This keeps the new behavior as close as to the existing behavior. + +In `read window`, read-only transactions are executed in the read-only thread pool, while operations in `read operation queue` are executed in the main thread. Design details are: +- At the start of `read-window`, all threads in the pool are signaled to start process read-only transactions. Each thread enters a loop of pulling one transaction a time from the front of the read-only transaction queue and executing it. New incoming read-only transactions are placed at the back of the queue. The queue is protected by mutex. +- If less than 5% (hardcoded for now) of `read-only-read-window-time-ms` time remains in the `read window`, a thread will stop scheduling new transaction for execution and exit the execution loop. +- At the end of the window, non-finished transactions are put to the front of the read-only transaction queue so that they can be executed the first in the next round. +- Transactions exceeding `max-transaction-time` will be discarded and a proper response will be sent back. +- Before a transaction is selected for execution, it is desirable to check if the HTTP connection is still up. If not, simply discard the transaction. A warning log or a stats should be generated so the node operator can adjust the configuration options or take any other actions like throttling read-only transactions. + + ### Window Toggling -To support toggling between `read` and `write` windows, configurable options `read-window-time`, `write-window-time`, and `read-only-max-queued-time` are provided. -- From `write` to `read`: When a new read-only transaction is queued, if the time the earliest transaction has been queued exceeds `read-only-max-queued-time`; or at the end of the `write` window and read-only transaction queue not empty. -- From `read` to `write`: when read-only transaction queue is emptied by the read-only threads; or at the end of `read` window. + +- From `write window` to `read window`: at the end of the `write window` and read-only transaction queue not empty. +- From `read window` to `write window`: when read-only transaction queue is empty; or at the end of `read window`. -In the following state diagram, `longest_queued_time = now - the time when the oldest trx was queued`, `write_window_deadline = time when write window starts + write-window-time`, and `read_window_deadline = time when read window starts + read-window-time` +In the following state diagram, `write_window_deadline = time when write window starts + write-window-time`, and `read_window_deadline = time when read window starts + read-window-time` ```mermaid flowchart TD - A(((write window))) -->|push a new trx| B[longest_queued_time < read-only-max-queued-time?] - B -->|yes| R(((read window))) + A(((write window))) -->|timer progressing| B[write_window_deadline passed?] + B -->|yes| C[read-only trx queue empty?] B -->|no| A - A --> D[write_window_deadline passed?] - D -->|yes| F[read-only trx queue empty?] - D -->|no| A - F -->|yes| A - F -->|no| R + C -->|yes| A + C -->|no| R(((read window))) - R --> S[read-only trx queue empty?] + R -->|transaction execution| S[read-only trx queue empty?] S -->|yes| A S -->|no| R - R --> T[read_window_deadline passed?] + R -->|timer progressing| T[read_window_deadline passed?] T -->|yes| A T -->|no| R ``` -### Handling main thread functions that are not safe to read-only transaction execution in `read` Window -Several options are considered to handle -- Drop the functions. This in not acceptable as it changes the behavior. -- Re-post to the main thread. In `read` window, re-post a not-read-only-safe function back to the main thread. This is simple to implement but breaks the order of functions and waste time in popping and pushing. -- Modify appbase `execution_priority_queue` by adding a function type with value `read-only-safe` or `not-read-only-safe`. In `write` window, everything works as it is now. In `read` window, only `thread-safe` requests are dequeued and processed. This avoids introducing new queues and keeps the order of the request. But as the queue is a priority queue based on function priority, it is infeasible to incorporate the additional type into the priority in a single queue. -- Separate appbase queue into two queues: `read-only-safe` or `not-read-only-safe`. The function type is added when a function is posted. - - In `read` window, only functions in `read-only-safe` queue are executed. - - In `write` window, compare the priorities of the top functions in `read-only-safe` and `not-read-only-safe`. The one with higher priority is executed. If tied, three options are considered: - - `not-read-only-safe` function is favored - - randomly pick one - - add a time attribute to the functions and the older one is picked. This keeps the original behavior. Even though at a cost of the extra time field and an extra comparison, this option seems best. - -### Read-only Transaction Priority Queue -The queue is used to store read-only transactions during `write` window. To maintain the time order when a transaction is put back into the queue for the next round due to `red` window deadline, a priority queue based on the first time when a transaction is queued is used. -```c++ -struct read_only_trx { - fc::time_point initial_queued_time; - packed_transaction_ptr trx; - next_func_t next; -}; - -struct read_only_trx_less { - bool operator() (const read_only_trx& a, const read_only_trx& b) { - // earlier queued time has higher priority - return a.initial_queued_time > b.initial_queued_time; - } -}; - -std::priority_queue, read_only_trx_less> read_only_trx_queue; -``` - -### Configuration Options - -- `read-only-num-threads`: the number of threads in read-only transaction execution thread pool. Default to `0`. If it is `0`, read-only transactions are executed on the main thread sequentially as they arrive -- `write-window-time`: time in milliseconds the `write` window lasts. Default to 500 milliseconds -- `read-window-time`: time in milliseconds the `read` window lasts. Must be equal to or greater than `max-read-only-transaction-time`. Default to 200 milliseconds -- `read-only-max-queued-time`: time in milliseconds when is exceeded by the time the earliest transaction, node switches to `read` window, even it is before the end of `write` window. -- `read-window-min-time`: time in milliseconds which must be remained in the `read` window when new transactions are scheduled for execution. Default to 5 milliseconds. This is to avoid unnecessary incomplete transaction execution. -- `max-read-only-transaction-time`: time in milliseconds a read-only transaction can execute before being considered invalid. Default to 150 milliseconds. This option has already been implemented by #558 +### Requests Starvation Discussion +- Comparing with executing all requests in the single main thread, offloading read-only transactions to a separate thread pool makes the time needed to process read-only transactions available for `write operations` and `read operations`, resulting more operations are processed. +- We are most concerned about starvation of block sync operation. Its priority is medium, which is higher than the majority of read operations. When `read window` is switched to `write window`, if the front operation in the write window queue is block sync, it is likely to be selected for execution. The worst case is a block sync at the front of the queue right before the execution window is switched to read-window. Configuring a small `read-only-read-window-ms` would reduce the waiting time for block sync. Another point is `write window` is switched to `read window` if there are read-only transactions in the read-only transaction queue; this means the main thread has already saves the time requried to executing those read-only transactions. The block sync is processed earlier than it would be in the existing single thread approach otherwise. +- In `read window`, whenever read-only transaction queue becomes empty, `read window` is switched to `write window`. This helps to process any write operations earlier if they are available. -## Thread Safety +## Thread Safety In Read Window - Safety between read-only transaction threads and other `nodeos` threads - - _main_ thread: The `main` thread only performs functions safe to read-only transaction execution. + - _main_ thread: See above sections. - _chain_ thread: `chain` threads are used in `apply_block`, `log_irreversible`, `finalize_block`, `create_block_state_future`. Those do not run while in `read` window. - _net_ thread: It is used for low-level networking. No conflicts with read-only transaction execution. - _http_ thread: It is used to receive requests and send back responses. No conflicts with read-only transaction execution. - - _prod_ thread: It is used in `on_incoming_transaction_async`, which is not running in `read` window + - _prod_ thread: _prod_ thread is used by `on_incoming_transaction_async` in `producer_plugin.cpp`. This method will be modified such that it is not running in `read` window. - _resource monitor_ thread: Resource monitor does not have any conflicts with any transaction execution. -- Safety between read-only transaction threads: no writes are made into `chainbase` and global states when a read-only transaction is executed. This is achieved by PR #558 +- Safety between read-only transaction threads: no writes are made into `chainbase` and global states when a read-only transaction is executed. This is achieved by PR [#558](https://github.com/AntelopeIO/leap/pull/558) ## Tests - number of read-only threads is 0 - number of read-only threads is 1 - number of read-only transactions greater than number of threads -- `write-window-time` test -- `read-window-time` test -- `read-only-max-queued-time` test -- `read-window-min-time` test +- `read-only-write-window-time-ms` test +- `read-only-read-window-time-ms` test - read-only transactions are processed within one read window - read-only transactions are processed in multiple read windows - initiate RPC requests while read-only transactions are executed From ea6202c3a211f737582c70e78da1cfb78621243e Mon Sep 17 00:00:00 2001 From: Lin Huang <107445030+linh2931@users.noreply.github.com> Date: Mon, 23 Jan 2023 16:20:38 -0500 Subject: [PATCH 09/14] Add windows size discussion. --- transactions/read-only/parallel.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/transactions/read-only/parallel.md b/transactions/read-only/parallel.md index 61ca4c4..76b6ed9 100644 --- a/transactions/read-only/parallel.md +++ b/transactions/read-only/parallel.md @@ -136,8 +136,8 @@ SHiP receives blockchain state data, saves it to files, and sends data to nodes Attempt is made to only introduce a minimum set of options. Other options might be added in the future if needed. - `read-only-transaction-threads`: number of worker threads in read-only transaction thread pool. If it is `0`, read-only transactions are executed on the main thread sequentially. Default to `0`. The number of the threads will be limited by the virtual memory size the system has (see https://github.com/AntelopeIO/leap/issues/645). -- `read-only-write-window-time-ms`: time in milliseconds the `write` window lasts. For this option to take effect, `read-only-transaction-threads` must be set greater than 0. Default to 300 milliseconds. -- `read-only-read-window-time-ms`: time in milliseconds the `read` window lasts. For this option to take effect, `read-only-transaction-threads` must be set greater than 0. Default to 100 milliseconds. +- `read-only-write-window-time-ms`: time in milliseconds the `write` window lasts. For this option to take effect, `read-only-transaction-threads` must be set greater than 0. Default to 200 milliseconds. +- `read-only-read-window-time-ms`: time in milliseconds the `read` window lasts. For this option to take effect, `read-only-transaction-threads` must be set greater than 0. Default to 60 milliseconds. Note: `read-only-max-transaction-time-ms` (time in milliseconds a read-only transaction can execute before being considered invalid) was considered. But to keep new options minimal, for now, use the existing `max-transaction-time` for read-only transactions. @@ -187,6 +187,7 @@ flowchart TD - Comparing with executing all requests in the single main thread, offloading read-only transactions to a separate thread pool makes the time needed to process read-only transactions available for `write operations` and `read operations`, resulting more operations are processed. - We are most concerned about starvation of block sync operation. Its priority is medium, which is higher than the majority of read operations. When `read window` is switched to `write window`, if the front operation in the write window queue is block sync, it is likely to be selected for execution. The worst case is a block sync at the front of the queue right before the execution window is switched to read-window. Configuring a small `read-only-read-window-ms` would reduce the waiting time for block sync. Another point is `write window` is switched to `read window` if there are read-only transactions in the read-only transaction queue; this means the main thread has already saves the time requried to executing those read-only transactions. The block sync is processed earlier than it would be in the existing single thread approach otherwise. - In `read window`, whenever read-only transaction queue becomes empty, `read window` is switched to `write window`. This helps to process any write operations earlier if they are available. +- To prevent read-only transactions from stuck in `read window` and `write operations` from stuck in `write window` for too long, `read-only-write-window-ms` and `read-only-read-window-ms` should be configured to small numbers, especially when `read-only-transaction-threads` is set to be big. This promotes rapid toggling of the two windows and reducing starvation. ## Thread Safety In Read Window From d5823a6543723cbad7edd662aec4adfec145792b Mon Sep 17 00:00:00 2001 From: Lin Huang <107445030+linh2931@users.noreply.github.com> Date: Mon, 23 Jan 2023 18:17:22 -0500 Subject: [PATCH 10/14] Use microseconds for read and write window time --- transactions/read-only/parallel.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/transactions/read-only/parallel.md b/transactions/read-only/parallel.md index 76b6ed9..e96709e 100644 --- a/transactions/read-only/parallel.md +++ b/transactions/read-only/parallel.md @@ -136,10 +136,10 @@ SHiP receives blockchain state data, saves it to files, and sends data to nodes Attempt is made to only introduce a minimum set of options. Other options might be added in the future if needed. - `read-only-transaction-threads`: number of worker threads in read-only transaction thread pool. If it is `0`, read-only transactions are executed on the main thread sequentially. Default to `0`. The number of the threads will be limited by the virtual memory size the system has (see https://github.com/AntelopeIO/leap/issues/645). -- `read-only-write-window-time-ms`: time in milliseconds the `write` window lasts. For this option to take effect, `read-only-transaction-threads` must be set greater than 0. Default to 200 milliseconds. -- `read-only-read-window-time-ms`: time in milliseconds the `read` window lasts. For this option to take effect, `read-only-transaction-threads` must be set greater than 0. Default to 60 milliseconds. +- `read-only-write-window-time-us`: time in microseconds the `write` window lasts. For this option to take effect, `read-only-transaction-threads` must be set greater than 0. Default to `200,000` microseconds. +- `read-only-read-window-time-us`: time in microseconds the `read` window lasts. For this option to take effect, `read-only-transaction-threads` must be set greater than 0. Default to `60,000` microseconds. -Note: `read-only-max-transaction-time-ms` (time in milliseconds a read-only transaction can execute before being considered invalid) was considered. But to keep new options minimal, for now, use the existing `max-transaction-time` for read-only transactions. +Note: `read-only-max-transaction-ms` (time in milliseconds a read-only transaction can execute before being considered invalid) was considered. But to keep new options minimal, for now, use the existing `max-transaction-time` for read-only transactions. ### Data Structures @@ -154,7 +154,7 @@ In `write window`, operations in `write queue` and `read queue` are executed seq In `read window`, read-only transactions are executed in the read-only thread pool, while operations in `read operation queue` are executed in the main thread. Design details are: - At the start of `read-window`, all threads in the pool are signaled to start process read-only transactions. Each thread enters a loop of pulling one transaction a time from the front of the read-only transaction queue and executing it. New incoming read-only transactions are placed at the back of the queue. The queue is protected by mutex. -- If less than 5% (hardcoded for now) of `read-only-read-window-time-ms` time remains in the `read window`, a thread will stop scheduling new transaction for execution and exit the execution loop. +- If less than 5% (hardcoded for now) of `read-only-read-window-time-us` time remains in the `read window`, a thread will stop scheduling new transaction for execution and exit the execution loop. - At the end of the window, non-finished transactions are put to the front of the read-only transaction queue so that they can be executed the first in the next round. - Transactions exceeding `max-transaction-time` will be discarded and a proper response will be sent back. - Before a transaction is selected for execution, it is desirable to check if the HTTP connection is still up. If not, simply discard the transaction. A warning log or a stats should be generated so the node operator can adjust the configuration options or take any other actions like throttling read-only transactions. @@ -206,8 +206,8 @@ flowchart TD - number of read-only threads is 0 - number of read-only threads is 1 - number of read-only transactions greater than number of threads -- `read-only-write-window-time-ms` test -- `read-only-read-window-time-ms` test +- `read-only-write-window-time-us` test +- `read-only-read-window-time-us` test - read-only transactions are processed within one read window - read-only transactions are processed in multiple read windows - initiate RPC requests while read-only transactions are executed From bed3a081edd936c6035c4fd366eb6776d39459e1 Mon Sep 17 00:00:00 2001 From: Lin Huang <107445030+linh2931@users.noreply.github.com> Date: Tue, 24 Jan 2023 12:45:05 -0500 Subject: [PATCH 11/14] Clarify write and read queues description --- transactions/read-only/parallel.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/transactions/read-only/parallel.md b/transactions/read-only/parallel.md index e96709e..79f006e 100644 --- a/transactions/read-only/parallel.md +++ b/transactions/read-only/parallel.md @@ -143,16 +143,16 @@ Note: `read-only-max-transaction-ms` (time in milliseconds a read-only transacti ### Data Structures -Three queues are proposed: `appbase`'s `execution_priority_queue` is replaced with read operation queue and write operation queue, and a new read-only transaction queue is added. -- `read operation queue` stores read operations which are safe to read-only transaction execution. It is a priority queue sorted by operation priority. When an operation is inserted into `read operation queue` or `write operation queue`, a global sequence number is incremented. This sequence number is stored together with each operation. -- `write operation queue` stores write operations which are not safe to read-only transaction execution. It uses the same type of priority queue as in `read operation queue`. +Three queues are proposed: the existing internal priority queue in `execution_priority_queue` of appbase is replaced with read operation queue and write operation queue, and a new read-only transaction queue is added. +- `read operation queue` stores read operations which are safe to read-only transaction execution. `appbase`'s `post` method is enhanced to have a new parameter indicating opertion type. +- `write operation queue` stores write operations which are not safe to read-only transaction execution. - `read-only transaction queue` stores read-only transactions. It is a deque. ### Window Processing -In `write window`, operations in `write queue` and `read queue` are executed sequentially in the main thread, while read-only transactions are queued in the read-only transaction queue for later execution. To select next operation to be executed, front entries in `write operation queue` and `read operation queue` are compared; the one with higher priority is chosen. If the priorities are tied, the sequence numbers are compared; the one with smaller number (earlier) is selected. This keeps the new behavior as close as to the existing behavior. +In `write window`, operations in `write operation queue` and `read operation queue` are executed by the main thread, while read-only transactions are queued in the read-only transaction queue for later execution in the `read window`. To select next operation to execute, priority and order of front entries in `write operation queue` and `read operation queue` are compared; the one with higher priority or higher order (older) is chosen. This is the existing behavior. -In `read window`, read-only transactions are executed in the read-only thread pool, while operations in `read operation queue` are executed in the main thread. Design details are: +In `read window`, only operations in `read operation queue` are executed by the main thread, and transactions in read-only queue are executed by the read-only. Design details are: - At the start of `read-window`, all threads in the pool are signaled to start process read-only transactions. Each thread enters a loop of pulling one transaction a time from the front of the read-only transaction queue and executing it. New incoming read-only transactions are placed at the back of the queue. The queue is protected by mutex. - If less than 5% (hardcoded for now) of `read-only-read-window-time-us` time remains in the `read window`, a thread will stop scheduling new transaction for execution and exit the execution loop. - At the end of the window, non-finished transactions are put to the front of the read-only transaction queue so that they can be executed the first in the next round. From 954b15a96fdaa9e06e807c8593652f621b344c88 Mon Sep 17 00:00:00 2001 From: Lin Huang <107445030+linh2931@users.noreply.github.com> Date: Thu, 26 Jan 2023 16:59:41 -0500 Subject: [PATCH 12/14] Update with further discussion with Areg --- transactions/read-only/parallel.md | 33 +++++++++++++++--------------- 1 file changed, 17 insertions(+), 16 deletions(-) diff --git a/transactions/read-only/parallel.md b/transactions/read-only/parallel.md index 79f006e..5d53063 100644 --- a/transactions/read-only/parallel.md +++ b/transactions/read-only/parallel.md @@ -5,7 +5,7 @@ Currently all transactions are executed in the main thread. To take advantage of Continuing the work to support read-only transaction RPC by PR (https://github.com/AntelopeIO/leap/pull/558) (on branch https://github.com/AntelopeIO/leap/tree/send_read_only_trx), this document describes an approach to parallelize read-only transaction execution. ## Main Ideas -The node toggles between `write` and `read` windows. In `write` window, the node operates normally, except read-only transactions are queued for later parallel execution in `read` window. In `read` window, the queued read-only transactions are executed in a dedicated thread pool, while operations which are safe to read-only transaction execution are executed in the main thread. +The node toggles between `write` and `read` windows. In `write` window, the node operates normally, except read-only transactions are queued for later parallel execution in `read` window. In `read` window, the queued read-only transactions are executed in a dedicated thread pool, while operations safe to read-only transaction execution are executed by the main thread. Several terms are defined to help understand the design. @@ -51,12 +51,12 @@ Chain APIs are received on the HTTP thread, processed on the main thread (and pr | get_accounts_by_authorizers | medium_low | none | yes | | get_transaction_status | medium_low | none | yes | | send_read_only_transaction | medium_low | none | yes | -| compute_transaction | low | main thread: temporally change chainbase | no | -| push_block | medium_low | main thread: chainbase, forkdb, producer, controller | no | -| push_transaction | low | main thread: chainbase, forkdb, producer, controller | no | -| push_transactions | low | main thread: chainbase, forkdb, producer, controller | no | -| send_transaction | low | main thread: chainbase, forkdb, producer, controller | no | -| send_transaction2 | low | main thread: chainbase, forkdb, producer, controller | no | +| compute_transaction | low | main thread: temporally change chainbase | no | +| push_block | medium_low | main thread: chainbase, forkdb, producer, controller | no | +| push_transaction | low | main thread: chainbase, forkdb, producer, controller | no | +| push_transactions | low | main thread: chainbase, forkdb, producer, controller | no | +| send_transaction | low | main thread: chainbase, forkdb, producer, controller | no | +| send_transaction2 | low | main thread: chainbase, forkdb, producer, controller | no | ### Producer APIs @@ -136,21 +136,21 @@ SHiP receives blockchain state data, saves it to files, and sends data to nodes Attempt is made to only introduce a minimum set of options. Other options might be added in the future if needed. - `read-only-transaction-threads`: number of worker threads in read-only transaction thread pool. If it is `0`, read-only transactions are executed on the main thread sequentially. Default to `0`. The number of the threads will be limited by the virtual memory size the system has (see https://github.com/AntelopeIO/leap/issues/645). -- `read-only-write-window-time-us`: time in microseconds the `write` window lasts. For this option to take effect, `read-only-transaction-threads` must be set greater than 0. Default to `200,000` microseconds. -- `read-only-read-window-time-us`: time in microseconds the `read` window lasts. For this option to take effect, `read-only-transaction-threads` must be set greater than 0. Default to `60,000` microseconds. +- `read-only-write-window-time-us`: time in microseconds the `write` window lasts. For this option to take effect, `read-only-transaction-threads` must be set to greater than 0. Default to `200,000` microseconds. +- `read-only-read-window-time-us`: time in microseconds the `read` window lasts. For this option to take effect, `read-only-transaction-threads` must be set to greater than 0. Default to `60,000` microseconds. Note: `read-only-max-transaction-ms` (time in milliseconds a read-only transaction can execute before being considered invalid) was considered. But to keep new options minimal, for now, use the existing `max-transaction-time` for read-only transactions. ### Data Structures Three queues are proposed: the existing internal priority queue in `execution_priority_queue` of appbase is replaced with read operation queue and write operation queue, and a new read-only transaction queue is added. -- `read operation queue` stores read operations which are safe to read-only transaction execution. `appbase`'s `post` method is enhanced to have a new parameter indicating opertion type. -- `write operation queue` stores write operations which are not safe to read-only transaction execution. +- `read operation queue` stores read operations safe to read-only transaction execution. Those operations are executed in both `read window` and `write window`. `appbase`'s `post` method is enhanced to have a new parameter indicating operation type. +- `write operation queue` stores write operations not safe to read-only transaction execution. Those operations are only executed in `write` window. - `read-only transaction queue` stores read-only transactions. It is a deque. ### Window Processing -In `write window`, operations in `write operation queue` and `read operation queue` are executed by the main thread, while read-only transactions are queued in the read-only transaction queue for later execution in the `read window`. To select next operation to execute, priority and order of front entries in `write operation queue` and `read operation queue` are compared; the one with higher priority or higher order (older) is chosen. This is the existing behavior. +In `write window`, operations in `write operation queue` and `read operation queue` are executed by the main thread, while read-only transactions are queued in the read-only transaction queue for later execution in the `read window`. To select next operation to execute, priorities of the front operation in `write operation queue` and the front operation in `read operation queue` are compared; the one with higher priority is chosen. If the priorities are tied, the sequence numbers (`order` field in the existing code base) are compared; the one scheduled earlier is selected. This is the existing behavior. In `read window`, only operations in `read operation queue` are executed by the main thread, and transactions in read-only queue are executed by the read-only. Design details are: - At the start of `read-window`, all threads in the pool are signaled to start process read-only transactions. Each thread enters a loop of pulling one transaction a time from the front of the read-only transaction queue and executing it. New incoming read-only transactions are placed at the back of the queue. The queue is protected by mutex. @@ -175,7 +175,7 @@ flowchart TD C -->|yes| A C -->|no| R(((read window))) - R -->|transaction execution| S[read-only trx queue empty?] + R -->|read-only trx executed| S[read-only trx queue empty agreed by all the threads?] S -->|yes| A S -->|no| R R -->|timer progressing| T[read_window_deadline passed?] @@ -184,10 +184,11 @@ flowchart TD ``` ### Requests Starvation Discussion -- Comparing with executing all requests in the single main thread, offloading read-only transactions to a separate thread pool makes the time needed to process read-only transactions available for `write operations` and `read operations`, resulting more operations are processed. -- We are most concerned about starvation of block sync operation. Its priority is medium, which is higher than the majority of read operations. When `read window` is switched to `write window`, if the front operation in the write window queue is block sync, it is likely to be selected for execution. The worst case is a block sync at the front of the queue right before the execution window is switched to read-window. Configuring a small `read-only-read-window-ms` would reduce the waiting time for block sync. Another point is `write window` is switched to `read window` if there are read-only transactions in the read-only transaction queue; this means the main thread has already saves the time requried to executing those read-only transactions. The block sync is processed earlier than it would be in the existing single thread approach otherwise. +- Comparing with executing all requests in the single main thread, offloading read-only transactions to a separate thread pool makes the time used to execute them available for other operations, resulting more operations are processed. +- We are most concerned about starvation of NET `signed_block` message. Its priority is medium, which is higher than the majority of read operations. When `read window` is switched to `write window`, if the front operation in the write window queue is block sync, it is likely to be selected for execution. The worst case is a `signed_block` at the front of the queue right after the execution window is switched to `read window`, resulting in the `signed_block` message waits for the whole duration of `read window`. Configuring a small `read-only-read-window-ms` would reduce the waiting time for `signed_block`. Furthermore, `write window` is switched to `read window` only when there are read-only transactions in the read-only transaction queue; this means the main thread has already saved the time required to executing those read-only transactions. The `signed_block` is processed earlier than it would be in the existing single thread approach otherwise. +- Other operations not safe to read-only transactions are `compute_transaction` (low priority), `push_transaction` (low), `push_transactions` (low), `send_transaction` (low), `send_transaction2` (low), `push_block` (medium_low), `set_whitelist_blacklist` (medium_high), `schedule_protocol_feature_activations` (medium_high), `packed_transaction` (low). Those operations with low priority would not be scheduled even allowed to, if operations safe to read-only transactions are queued. `set_whitelist_blacklist` and `schedule_protocol_feature_activations` are not used frequently, `send_block` mostly not used (can be deprecated). - In `read window`, whenever read-only transaction queue becomes empty, `read window` is switched to `write window`. This helps to process any write operations earlier if they are available. -- To prevent read-only transactions from stuck in `read window` and `write operations` from stuck in `write window` for too long, `read-only-write-window-ms` and `read-only-read-window-ms` should be configured to small numbers, especially when `read-only-transaction-threads` is set to be big. This promotes rapid toggling of the two windows and reducing starvation. +- To prevent read-only transactions from stuck in `read window` and prevent `write operations` from stuck in `write window` for too long, `read-only-write-window-ms` and `read-only-read-window-ms` should be configured to be small numbers, especially when `read-only-transaction-threads` is set to a big number (which results in more read-only transactions can be executed in shorter time). This promotes rapid toggling of the two windows and reducing starvation. ## Thread Safety In Read Window From ae6297047fd8488c215ad7bc910f4611a168b85a Mon Sep 17 00:00:00 2001 From: Lin Huang <107445030+linh2931@users.noreply.github.com> Date: Fri, 27 Jan 2023 18:08:31 -0500 Subject: [PATCH 13/14] Incorporate Areg's review comments. --- transactions/read-only/parallel.md | 35 ++++++++++++++++-------------- 1 file changed, 19 insertions(+), 16 deletions(-) diff --git a/transactions/read-only/parallel.md b/transactions/read-only/parallel.md index 5d53063..a8b24e7 100644 --- a/transactions/read-only/parallel.md +++ b/transactions/read-only/parallel.md @@ -9,9 +9,9 @@ The node toggles between `write` and `read` windows. In `write` window, the node Several terms are defined to help understand the design. -- *read-only transaction* - an Antelope transaction that does not modify state -- *read operation* - a task (like `get_info`) that does not modify state and any global data used during read-only transaction execution. It is safe to run in parallel with read-only transaction. -- *write operation* - a task (like `send_transacttion2`) that modifies state or any global data used during read-only transaction execution. It is not safe to run in parallel with read-only transaction. +- *read-only transaction* - an Antelope transaction that does not modify any state. +- *read operation* - a task that does not modify any state that is accessible (even indirectly) by read-only transaction execution. It is safe to execute these on the main thread in parallel with read-only transactions. As the name suggests, these include pure read operations, e.g. `get_info`, but it can also include other operations executing on the main thread that may mutate state that does not impact read-only transactions executing on separate threads, e.g. `pause` and `resume` from the producer API. +- *write operation* - a task that may modify state that can be accessible by read-only transaction execution, e.g. `send_transaction2` which can modify Chainbase state. It is not safe to execute these operations in parallel with read-only transactions. ## Existing Operation Analysis This section analyzes thread safety of existing operations to read-only transaction execution. @@ -50,7 +50,7 @@ Chain APIs are received on the HTTP thread, processed on the main thread (and pr | get_consensus_parameters | medium_low | none | yes | | get_accounts_by_authorizers | medium_low | none | yes | | get_transaction_status | medium_low | none | yes | -| send_read_only_transaction | medium_low | none | yes | +| send_read_only_transaction | low | none | yes | | compute_transaction | low | main thread: temporally change chainbase | no | | push_block | medium_low | main thread: chainbase, forkdb, producer, controller | no | | push_transaction | low | main thread: chainbase, forkdb, producer, controller | no | @@ -83,7 +83,7 @@ They are received on the HTTP thread, processed on the main thread, and response | get_unapplied_transactions | medium_high | | yes | ### Net APIs -Net APIs do not mutate states. They are received on the HTTP thread, processed on the main thread, and responses are sent on the HTTP thread. +Net APIs do modify state but do not modify the state accessible to read-only transaction execution. They are received on the HTTP thread, processed on the main thread, and responses are sent on the HTTP thread. | API | priority | Global data modified | Safe to read-only trx? | |-----------------------------|-------------|--------------------------------|------------------------| @@ -139,24 +139,24 @@ Attempt is made to only introduce a minimum set of options. Other options might - `read-only-write-window-time-us`: time in microseconds the `write` window lasts. For this option to take effect, `read-only-transaction-threads` must be set to greater than 0. Default to `200,000` microseconds. - `read-only-read-window-time-us`: time in microseconds the `read` window lasts. For this option to take effect, `read-only-transaction-threads` must be set to greater than 0. Default to `60,000` microseconds. -Note: `read-only-max-transaction-ms` (time in milliseconds a read-only transaction can execute before being considered invalid) was considered. But to keep new options minimal, for now, use the existing `max-transaction-time` for read-only transactions. +Note: `read-only-max-transaction-ms` (time in milliseconds a read-only transaction can execute before being considered invalid) was considered. But to keep new options minimal, for now, read-only transactions will have an enforced wall-clock deadline limit automatically determined from `read-only-read-window-time-us` and `max-transaction-time` (see "Window Processing" section for more details). ### Data Structures Three queues are proposed: the existing internal priority queue in `execution_priority_queue` of appbase is replaced with read operation queue and write operation queue, and a new read-only transaction queue is added. -- `read operation queue` stores read operations safe to read-only transaction execution. Those operations are executed in both `read window` and `write window`. `appbase`'s `post` method is enhanced to have a new parameter indicating operation type. -- `write operation queue` stores write operations not safe to read-only transaction execution. Those operations are only executed in `write` window. +- `read operation queue` stores read operations which are safe to execute on the main thread in parallel with read-only transactions executing on separate threads. Those operation are executed during both the `read window` and `write window`. +- `write operation queue` stores write operations which are not safe to execute concurrently with read-only transaction execution. Those operations are only executed during the `write window`. - `read-only transaction queue` stores read-only transactions. It is a deque. ### Window Processing In `write window`, operations in `write operation queue` and `read operation queue` are executed by the main thread, while read-only transactions are queued in the read-only transaction queue for later execution in the `read window`. To select next operation to execute, priorities of the front operation in `write operation queue` and the front operation in `read operation queue` are compared; the one with higher priority is chosen. If the priorities are tied, the sequence numbers (`order` field in the existing code base) are compared; the one scheduled earlier is selected. This is the existing behavior. -In `read window`, only operations in `read operation queue` are executed by the main thread, and transactions in read-only queue are executed by the read-only. Design details are: -- At the start of `read-window`, all threads in the pool are signaled to start process read-only transactions. Each thread enters a loop of pulling one transaction a time from the front of the read-only transaction queue and executing it. New incoming read-only transactions are placed at the back of the queue. The queue is protected by mutex. +In `read window`, operations only in `read operation queue` are executed by the main thread, and transactions in `read-only transaction queue` are executed under the restrictive context of read-only transactions in parallel within the read-only threads. +- At the start of `read window`, all threads in the pool are signaled to start processing read-only transactions. Each thread enters a loop of pulling one transaction a time from the front of the read-only transaction queue and executing it. New incoming read-only transactions are placed at the back of the queue. The queue is protected by mutex. - If less than 5% (hardcoded for now) of `read-only-read-window-time-us` time remains in the `read window`, a thread will stop scheduling new transaction for execution and exit the execution loop. -- At the end of the window, non-finished transactions are put to the front of the read-only transaction queue so that they can be executed the first in the next round. -- Transactions exceeding `max-transaction-time` will be discarded and a proper response will be sent back. +- At the end of the window, unfinished transactions are put to the front of the `read-only transaction queue` (a deque) so that they can be executed the first in the next round. For implementation simplicity and execution efficiency, the original order of unfinished transactions are not preserved. Unfinished transactions are likely to have been scheduled in a close range of time period and they are all read-only. Their execution order might not matter much. +- Transaction exceeding wall-clock deadline for read-only transactions will be discarded and a proper response will be sent back. The wall-clock deadline for a read-only transaction is calculated by adding a duration (in microseconds) to the start time of its execution. The duration value will initially be set to the lesser of `read-only-write-read-window-time-us * 95 / 100` and the existing `max-transaction-time * 1000`. Note that the 95 in the formula is assuming the 5% hardcoded value for when to stop scheduling new transactions in the execution window. The code will only have one source of truth for that hardcoded 5% number and compute the other derived numbers appropriately to remain consistent. - Before a transaction is selected for execution, it is desirable to check if the HTTP connection is still up. If not, simply discard the transaction. A warning log or a stats should be generated so the node operator can adjust the configuration options or take any other actions like throttling read-only transactions. @@ -184,11 +184,14 @@ flowchart TD ``` ### Requests Starvation Discussion -- Comparing with executing all requests in the single main thread, offloading read-only transactions to a separate thread pool makes the time used to execute them available for other operations, resulting more operations are processed. -- We are most concerned about starvation of NET `signed_block` message. Its priority is medium, which is higher than the majority of read operations. When `read window` is switched to `write window`, if the front operation in the write window queue is block sync, it is likely to be selected for execution. The worst case is a `signed_block` at the front of the queue right after the execution window is switched to `read window`, resulting in the `signed_block` message waits for the whole duration of `read window`. Configuring a small `read-only-read-window-ms` would reduce the waiting time for `signed_block`. Furthermore, `write window` is switched to `read window` only when there are read-only transactions in the read-only transaction queue; this means the main thread has already saved the time required to executing those read-only transactions. The `signed_block` is processed earlier than it would be in the existing single thread approach otherwise. -- Other operations not safe to read-only transactions are `compute_transaction` (low priority), `push_transaction` (low), `push_transactions` (low), `send_transaction` (low), `send_transaction2` (low), `push_block` (medium_low), `set_whitelist_blacklist` (medium_high), `schedule_protocol_feature_activations` (medium_high), `packed_transaction` (low). Those operations with low priority would not be scheduled even allowed to, if operations safe to read-only transactions are queued. `set_whitelist_blacklist` and `schedule_protocol_feature_activations` are not used frequently, `send_block` mostly not used (can be deprecated). +- Comparing with executing all requests in the single main thread, offloading read-only transactions to a separate thread pool makes the time used to execute them available for other operations, result in more operations processed. +- We are most concerned about starvation and delays of processing Net `signed_block` messages. + + Its priority is medium, which is higher than the majority of read operations. With the existing behavior in the single-threaded version of the read-only transaction feature (see PR https://github.com/AntelopeIO/leap/pull/558), `send_read_only_transaction` cannot starve `signed_block` messages since `send_read_only_transaction` has a low priority (same as `compute_transaction`). The same behavior holds in this design if `read-only-transaction-threads` is set to 0. + + When `read-only-transaction-threads` is greater than 0, the priority of selecting the `signed_block` message for processing during `write window` from the combined `write operation queue` and `read operation queue` is no different than with the existing behavior. However, it also does introduce the `read window` which can add additional latency before a `signed_block` message can be processed. The worst case is a `signed_block` message at the front of the queue right after the execution window switches to `read window`, resulting in the `signed_block` message waiting for the whole duration of `read window` before beginning processing. + + The `read-only-read-window-time-us` parameter bounds the additional waiting time added for processing `signed_block` messages due to this design. The `read-only-read-window-time-us` can be lowered to reduce the delay in processing `signed_block` messages, but it comes with the trade-off of lower deadline limit for read-only transactions. +- Other operations not safe to read-only transactions are `compute_transaction` (low priority), `push_transaction` (low), `push_transactions` (low), `send_transaction` (low), `send_transaction2` (low), `push_block` (medium_low), `update_runtime_options` (medium_high), `set_whitelist_blacklist` (medium_high), `schedule_protocol_feature_activations` (medium_high), `packed_transaction` (low). Under behavior with `read-only-transaction-threads` set to 0, the operations in the prior list with low priority would be competing for processing time with read-only transactions based on the order the request came in. So there is not much concern about increased risk of starvation or delays of such messages due to the introduction of the `read window` by setting `read-only-transaction-threads` to greater than 0. And again, concerns of added delays for some of these operations can be mitigated by setting a lower value for `read-only-read-window-time-us`. Among the other operations in the list (`update_runtime_options`, `set_whitelist_blacklist`, `schedule_protocol_feature_activations`, and `push_block`), they are very infrequently used in practice, not very timing sensitive, and `push_block` could even be deprecated. - In `read window`, whenever read-only transaction queue becomes empty, `read window` is switched to `write window`. This helps to process any write operations earlier if they are available. -- To prevent read-only transactions from stuck in `read window` and prevent `write operations` from stuck in `write window` for too long, `read-only-write-window-ms` and `read-only-read-window-ms` should be configured to be small numbers, especially when `read-only-transaction-threads` is set to a big number (which results in more read-only transactions can be executed in shorter time). This promotes rapid toggling of the two windows and reducing starvation. +- To prevent read-only transactions from getting stuck in the `read-only transaction queue` for too long and to prevent `write operations` from getting stuck in `write operation queue` for too long, `read-only-read-window-time-us` and `read-only-write-window-us` should be configured to be reasonably small numbers. This promotes rapid toggling between the two windows and reduces delays and starvation. However, `read-only-read-window-time-us` should not be too low to give enough time for a reasonable read-only transaction to successfully execute. Additionally, the ratio between `read-only-read-window-time-us` and `read-only-write-window-time-us` should be set to an appropriate amount to avoid starvation of either type of operations; when `read-only-transaction-threads` is set to a large number (and there are enough cores available on the machine to actually justify that number), that ratio can be made smaller. ## Thread Safety In Read Window From bd9ea267a666032026dfcaaab6b65a5db22f250d Mon Sep 17 00:00:00 2001 From: Lin Huang <107445030+linh2931@users.noreply.github.com> Date: Fri, 27 Jan 2023 19:54:31 -0500 Subject: [PATCH 14/14] Minor spelling corrections --- transactions/read-only/parallel.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/transactions/read-only/parallel.md b/transactions/read-only/parallel.md index a8b24e7..ad48a1f 100644 --- a/transactions/read-only/parallel.md +++ b/transactions/read-only/parallel.md @@ -144,7 +144,7 @@ Note: `read-only-max-transaction-ms` (time in milliseconds a read-only transacti ### Data Structures Three queues are proposed: the existing internal priority queue in `execution_priority_queue` of appbase is replaced with read operation queue and write operation queue, and a new read-only transaction queue is added. -- `read operation queue` stores read operations which are safe to execute on the main thread in parallel with read-only transactions executing on separate threads. Those operation are executed during both the `read window` and `write window`. +- `read operation queue` stores read operations which are safe to execute on the main thread in parallel with read-only transactions executing on separate threads. Those operations are executed during both the `read window` and `write window`. - `write operation queue` stores write operations which are not safe to execute concurrently with read-only transaction execution. Those operations are only executed during the `write window`. - `read-only transaction queue` stores read-only transactions. It is a deque. @@ -155,8 +155,8 @@ In `write window`, operations in `write operation queue` and `read operation que In `read window`, operations only in `read operation queue` are executed by the main thread, and transactions in `read-only transaction queue` are executed under the restrictive context of read-only transactions in parallel within the read-only threads. - At the start of `read window`, all threads in the pool are signaled to start processing read-only transactions. Each thread enters a loop of pulling one transaction a time from the front of the read-only transaction queue and executing it. New incoming read-only transactions are placed at the back of the queue. The queue is protected by mutex. - If less than 5% (hardcoded for now) of `read-only-read-window-time-us` time remains in the `read window`, a thread will stop scheduling new transaction for execution and exit the execution loop. -- At the end of the window, unfinished transactions are put to the front of the `read-only transaction queue` (a deque) so that they can be executed the first in the next round. For implementation simplicity and execution efficiency, the original order of unfinished transactions are not preserved. Unfinished transactions are likely to have been scheduled in a close range of time period and they are all read-only. Their execution order might not matter much. -- Transaction exceeding wall-clock deadline for read-only transactions will be discarded and a proper response will be sent back. The wall-clock deadline for a read-only transaction is calculated by adding a duration (in microseconds) to the start time of its execution. The duration value will initially be set to the lesser of `read-only-write-read-window-time-us * 95 / 100` and the existing `max-transaction-time * 1000`. Note that the 95 in the formula is assuming the 5% hardcoded value for when to stop scheduling new transactions in the execution window. The code will only have one source of truth for that hardcoded 5% number and compute the other derived numbers appropriately to remain consistent. +- At the end of the window, unfinished transactions are put to the front of the `read-only transaction queue` (a deque) so that they can be executed the first in the next round. For implementation simplicity and execution efficiency, the original order of unfinished transactions is not preserved. Unfinished transactions are likely to have been scheduled in a close range of time period and they are all read-only. Their execution order might not matter much. +- A transaction exceeding wall-clock deadline for read-only transactions will be discarded and a proper response will be sent back. The wall-clock deadline for a read-only transaction is calculated by adding a duration (in microseconds) to the start time of its execution. The duration value will initially be set to the lesser of `read-only-write-read-window-time-us * 95 / 100` and the existing `max-transaction-time * 1000`. Note that the 95 in the formula is assuming the 5% hardcoded value for when to stop scheduling new transactions in the execution window. The code will only have one source of truth for that hardcoded 5% number and compute the other derived numbers appropriately to remain consistent. - Before a transaction is selected for execution, it is desirable to check if the HTTP connection is still up. If not, simply discard the transaction. A warning log or a stats should be generated so the node operator can adjust the configuration options or take any other actions like throttling read-only transactions. @@ -184,7 +184,7 @@ flowchart TD ``` ### Requests Starvation Discussion -- Comparing with executing all requests in the single main thread, offloading read-only transactions to a separate thread pool makes the time used to execute them available for other operations, result in more operations processed. +- Comparing with executing all requests in the single main thread, offloading read-only transactions to a separate thread pool makes the time used to execute them available for other operations, resulting in more operations processed. - We are most concerned about starvation and delays of processing Net `signed_block` messages. + Its priority is medium, which is higher than the majority of read operations. With the existing behavior in the single-threaded version of the read-only transaction feature (see PR https://github.com/AntelopeIO/leap/pull/558), `send_read_only_transaction` cannot starve `signed_block` messages since `send_read_only_transaction` has a low priority (same as `compute_transaction`). The same behavior holds in this design if `read-only-transaction-threads` is set to 0. + When `read-only-transaction-threads` is greater than 0, the priority of selecting the `signed_block` message for processing during `write window` from the combined `write operation queue` and `read operation queue` is no different than with the existing behavior. However, it also does introduce the `read window` which can add additional latency before a `signed_block` message can be processed. The worst case is a `signed_block` message at the front of the queue right after the execution window switches to `read window`, resulting in the `signed_block` message waiting for the whole duration of `read window` before beginning processing.