Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PH] argparse.ArgumentParser usage updates #518

Merged
merged 4 commits into from
Nov 30, 2022

Conversation

oschwaldp-oci
Copy link
Contributor

@oschwaldp-oci oschwaldp-oci commented Nov 29, 2022

Make use of ArgumentParser parents and groups.

This will greatly reduce work to bubble up arguments between scripts/modules. Also brings clarity and grouping of arguments by use with group titles and descriptions.

Test Helper arguments can now be bubbled up to user scripts via direct access to the ArgumentParser. Performance Test Basic makes use of Test Helper arguments. Perfomance Test makes use of both Performance Test Basic and Test Helper arguments as well.

./build/tests/performance_tests/performance_test.py -? now prints:

usage: performance_test.py [-?] [-p P] [-n N] [-d D] [--nodes-file NODES_FILE]
                           [-s {mesh}] [--dump-error-details] [-v]
                           [--leave-running] [--clean-run]
                           [--tps-limit-per-generator TPS_LIMIT_PER_GENERATOR]
                           [--genesis GENESIS]
                           [--num-blocks-to-prune NUM_BLOCKS_TO_PRUNE]
                           [--signature-cpu-billable-pct SIGNATURE_CPU_BILLABLE_PCT]
                           [--chain-state-db-size-mb CHAIN_STATE_DB_SIZE_MB]
                           [--chain-threads CHAIN_THREADS]
                           [--database-map-mode {mapped,heap,locked}]
                           [--net-threads NET_THREADS]
                           [--disable-subjective-billing DISABLE_SUBJECTIVE_BILLING]
                           [--last-block-time-offset-us LAST_BLOCK_TIME_OFFSET_US]
                           [--produce-time-offset-us PRODUCE_TIME_OFFSET_US]
                           [--cpu-effort-percent CPU_EFFORT_PERCENT]
                           [--last-block-cpu-effort-percent LAST_BLOCK_CPU_EFFORT_PERCENT]
                           [--producer-threads PRODUCER_THREADS]
                           [--http-max-response-time-ms HTTP_MAX_RESPONSE_TIME_MS]
                           [--del-perf-logs] [--del-report] [--quiet]
                           [--prods-enable-trace-api] [--skip-tps-test]
                           [--calc-producer-threads {none,lmax,full}]
                           [--calc-chain-threads {none,lmax,full}]
                           [--calc-net-threads {none,lmax,full}]
                           [--del-test-report]
                           [--max-tps-to-test MAX_TPS_TO_TEST]
                           [--test-iteration-duration-sec TEST_ITERATION_DURATION_SEC]
                           [--test-iteration-min-step TEST_ITERATION_MIN_STEP]
                           [--final-iterations-duration-sec FINAL_ITERATIONS_DURATION_SEC]

Test Helper Arguments:
  Test Helper configuration items used to configure and spin up the
  regression test framework and blockchain environment.

  -?                    show this help message and exit
  -p P                  producing nodes count (default: 1)
  -n N                  total nodes (default: 0)
  -d D                  delay between nodes startup (default: 1)
  --nodes-file NODES_FILE
                        File containing nodes info in JSON format. (default:
                        None)
  -s {mesh}             topology (default: mesh)
  --dump-error-details  Upon error print etc/eosio/node_*/config.ini and
                        var/lib/node_*/stderr.log to stdout (default: False)
  -v                    verbose logging (default: False)
  --leave-running       Leave cluster running after test finishes (default:
                        False)
  --clean-run           Kill all nodeos and keosd instances (default: False)

Performance Test Basic Base:
  Performance Test Basic base configuration items.

  --tps-limit-per-generator TPS_LIMIT_PER_GENERATOR
                        Maximum amount of transactions per second a single
                        generator can have. (default: 4000)
  --genesis GENESIS     Path to genesis.json (default:
                        tests/performance_tests/genesis.json)
  --num-blocks-to-prune NUM_BLOCKS_TO_PRUNE
                        The number of potentially non-empty blocks, in
                        addition to leading and trailing size 0 blocks, to
                        prune from the beginning and end of the range of
                        blocks of interest for evaluation. (default: 2)
  --signature-cpu-billable-pct SIGNATURE_CPU_BILLABLE_PCT
                        Percentage of actual signature recovery cpu to bill.
                        Whole number percentages, e.g. 50 for 50% (default: 0)
  --chain-state-db-size-mb CHAIN_STATE_DB_SIZE_MB
                        Maximum size (in MiB) of the chain state database
                        (default: 10240)
  --chain-threads CHAIN_THREADS
                        Number of worker threads in controller thread pool
                        (default: 2)
  --database-map-mode {mapped,heap,locked}
                        Database map mode ("mapped", "heap", or "locked"). In
                        "mapped" mode database is memory mapped as a file. In
                        "heap" mode database is preloaded in to swappable
                        memory and will use huge pages if available. In
                        "locked" mode database is preloaded, locked in to
                        memory, and will use huge pages if available.
                        (default: mapped)
  --net-threads NET_THREADS
                        Number of worker threads in net_plugin thread pool
                        (default: 2)
  --disable-subjective-billing DISABLE_SUBJECTIVE_BILLING
                        Disable subjective CPU billing for API/P2P
                        transactions (default: True)
  --last-block-time-offset-us LAST_BLOCK_TIME_OFFSET_US
                        Offset of last block producing time in microseconds.
                        Valid range 0 .. -block_time_interval. (default: 0)
  --produce-time-offset-us PRODUCE_TIME_OFFSET_US
                        Offset of non last block producing time in
                        microseconds. Valid range 0 .. -block_time_interval.
                        (default: 0)
  --cpu-effort-percent CPU_EFFORT_PERCENT
                        Percentage of cpu block production time used to
                        produce block. Whole number percentages, e.g. 80 for
                        80% (default: 100)
  --last-block-cpu-effort-percent LAST_BLOCK_CPU_EFFORT_PERCENT
                        Percentage of cpu block production time used to
                        produce last block. Whole number percentages, e.g. 80
                        for 80% (default: 100)
  --producer-threads PRODUCER_THREADS
                        Number of worker threads in producer thread pool
                        (default: 2)
  --http-max-response-time-ms HTTP_MAX_RESPONSE_TIME_MS
                        Maximum time for processing a request, -1 for
                        unlimited (default: 990000)
  --del-perf-logs       Whether to delete performance test specific logs.
                        (default: False)
  --del-report          Whether to delete overarching performance run report.
                        (default: False)
  --quiet               Whether to quiet printing intermediate results and
                        reports to stdout (default: False)
  --prods-enable-trace-api
                        Determines whether producer nodes should have
                        eosio::trace_api_plugin enabled (default: False)

Performance Harness:
  Performance Harness testing configuration items.

  --skip-tps-test       Determines whether to skip the max TPS measurement
                        tests (default: False)
  --calc-producer-threads {none,lmax,full}
                        Determines whether to calculate number of worker
                        threads to use in producer thread pool ("none",
                        "lmax", or "full"). In "none" mode, the default, no
                        calculation will be attempted and default configured
                        --producer-threads value will be used. In "lmax" mode,
                        producer threads will incrementally be tested until
                        the performance rate ceases to increase with the
                        addition of additional threads. In "full" mode
                        producer threads will incrementally be tested from
                        2..num logical processors, recording each performance
                        and choosing the local max performance (same value as
                        would be discovered in "lmax" mode). Useful for
                        graphing the full performance impact of each available
                        thread. (default: none)
  --calc-chain-threads {none,lmax,full}
                        Determines whether to calculate number of worker
                        threads to use in chain thread pool ("none", "lmax",
                        or "full"). In "none" mode, the default, no
                        calculation will be attempted and default configured
                        --chain-threads value will be used. In "lmax" mode,
                        producer threads will incrementally be tested until
                        the performance rate ceases to increase with the
                        addition of additional threads. In "full" mode
                        producer threads will incrementally be tested from
                        2..num logical processors, recording each performance
                        and choosing the local max performance (same value as
                        would be discovered in "lmax" mode). Useful for
                        graphing the full performance impact of each available
                        thread. (default: none)
  --calc-net-threads {none,lmax,full}
                        Determines whether to calculate number of worker
                        threads to use in net thread pool ("none", "lmax", or
                        "full"). In "none" mode, the default, no calculation
                        will be attempted and default configured --net-threads
                        value will be used. In "lmax" mode, producer threads
                        will incrementally be tested until the performance
                        rate ceases to increase with the addition of
                        additional threads. In "full" mode producer threads
                        will incrementally be tested from 2..num logical
                        processors, recording each performance and choosing
                        the local max performance (same value as would be
                        discovered in "lmax" mode). Useful for graphing the
                        full performance impact of each available thread.
                        (default: none)
  --del-test-report     Whether to save json reports from each test scenario.
                        (default: False)

Performance Harness - TPS Test Config:
  TPS Performance Test configuration items.

  --max-tps-to-test MAX_TPS_TO_TEST
                        The max target transfers realistic as ceiling of test
                        range (default: 50000)
  --test-iteration-duration-sec TEST_ITERATION_DURATION_SEC
                        The duration of transfer trx generation for each
                        iteration of the test during the initial search
                        (seconds) (default: 150)
  --test-iteration-min-step TEST_ITERATION_MIN_STEP
                        The step size determining granularity of tps result
                        during initial search (default: 500)
  --final-iterations-duration-sec FINAL_ITERATIONS_DURATION_SEC
                        The duration of transfer trx generation for each final
                        longer run iteration of the test during the final
                        search (seconds) (default: 300)

This will greatly reduce work to bubble up arguments between scripts/modules.  Also brings clarity and grouping of arguments by use with group titles and descriptions.

Test Helper arguments can now be bubbled up to user scripts via direct access to the ArgumentParser.  Performance Test Basic makes use of Test Helper arguments.  Perfomance Test makes use of both Performance Test Basic and Test Helper arguments as well.
Base automatically changed from ph-thread-count-tests to feature_performance_harness_stage_2 November 30, 2022 21:11
@oschwaldp-oci oschwaldp-oci merged commit 118a2a0 into feature_performance_harness_stage_2 Nov 30, 2022
@oschwaldp-oci oschwaldp-oci deleted the ph-argparse branch November 30, 2022 21:11
@oschwaldp-oci oschwaldp-oci self-assigned this Apr 14, 2023
@oschwaldp-oci oschwaldp-oci added the OCI Work exclusive to OCI team label Apr 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
OCI Work exclusive to OCI team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants