[Das Benchmarks]: Stress test 1 Full or Bridge Node against x Light Nodes

[derrandz]

Important

Merge when celestiaorg/celestia-node#1376 is merged and go.mod is corrected to point to celestia-node main

Overview

We are interested in benchmarking the bridge node against a multitude of light node groups, starting from (say) a 100 up to 100K

To do so, this PR adds test cases and local telemetry to enable the benchmark alongside with metrics collection for benchmarking results visualization.

More details on #79

Changes

• Adds documentation for the new test plan 002

• Adds first test case

• Adds local compositions

• Display how many light clients are currently connected to the bridge node (concurrency level)

• And finally, since the charts are rough, we will work on smoothening them out (cleaning labels, improving readability and so on)

• Adds k8s compositions?

• Add matrix for 64/128 square sizes (Check comment here)

  • 64
  • 128

• Add resource consumption metrics for the bridge node to analyze how # of light nodes impacts resource consumption

• Add comparative charts for results across multiple concurrency levels:
  example: DASing time for (100 light clients, 500 light clients, 1000 light clients) all in the same chart for comparison.

How to run:

1. Start testground:

$ make tg-start

2. In another terminal start the telemetry infrastructure

$ make telemetry-infra-up

3. Run the test-case

$ make tg-run-composition RUNNER=local-docker TESTPLAN=das-benchmarks COMPOSITION=001-lights-dasing-latest-from-bridge-16-50-28

4. Go to http://localhost:3000 to access Grafana

5. Add prometheus as a data source
   5.1 For the URL, if you are running in a droplet, use your instance's IP instead of localhost

6. Import the dashboard under ./build/grafana/dashboards/benchmarks.json

Dependencies

This PR depends on #132

Owners: @derrandz

[Bidon15]

It will be awesome to have 32/64/128 matrix as we are having an assumption that it works already here #96 😅

[derrandz]

To configure the celestia node with the benchmark parameters, these two PRs have to go in;

• refactor(availability): use functional options pattern to configure availability implementations celestia-node#1224: Makes SampleAmount configurable
• refactor(daser): use functional options pattern to configure daser celestia-node#1225: Makes SampleFrom configurable

[derrandz]

The DASer PR is not required to tell configure the light node to start dasing from a specific height. We changed the logic of the test to unblock this PR, so that's no longer required.

The other one that's mentioned in the comment might be required down the line for different sample amounts.

[derrandz]

Define the Non Functional Requirements that have to be met for this test plan alongside the metrics to collect and their thresholds.

Referencing #108

[derrandz]

Update

The course of this PR is changing to include test isolation from test setup, metrics' collection and the infrastructure to support the metrics collection.

Although hacky, it's convenient to take this route. After getting a fully working version, we will rewrite the history of this PR to clean this up.

More context in here

The required infrastructure efforts are document in #109

[derrandz]

Ongoing work to enable blackbox telemetry is in:

• feat(telemetry): add blackbox instrumentation to the header module + share module + p2p bandwidth metrics celestia-node#1376

[derrandz]

Progress update regarding the blackbox telemetry efforts:

We managed to get a few charts to look at, by which we measure the performance of the bridge node in terms of how well it’s serving the DASing process. At the moment, since we would benchmark a bridge node against a multitude of light nodes, we will go with the option of displaying charts per light node instance (you can choose a random instance from the drop down in the screenshot)

Aggregate charts that display the overall state of the DASing process across all light nodes is the next step. (Check the PR's TODOs)

(Screenshots from a local run with 1 validator, 1 bridge node and 28 light nodes)

The Selection of Light Nodes from the dropdown

[derrandz]

Some improvements to charting:

• Added the native influx-db metrics for testground to track # of alive light nodes (a way to experiment with testground’s influxdb)

• switched histogram chartings to use tabular like logic (see now DASing time and block time charts looking more clear per height)

[Bidon15]

let's get his home run and nits figure out later

[derrandz]

Final nits tracked in an issue for future resolution #167