Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

subsystem-bench: add regression tests for availability read and write #3311

Merged
merged 56 commits into from
Mar 1, 2024
Merged
Changes from 1 commit
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
c01894b
Split subsystem-bench to lib and cli tool
AndreiEres Feb 13, 2024
f71c502
Remove unused values from BenchCli
AndreiEres Feb 13, 2024
733f0eb
Move usage display to tests
AndreiEres Feb 13, 2024
fdc1392
Update displaying of the configuration
AndreiEres Feb 13, 2024
815c023
fixup: add log target
AndreiEres Feb 13, 2024
0a65126
Remove toml fmt
AndreiEres Feb 13, 2024
42963a9
fixup: remove toml fmt
AndreiEres Feb 13, 2024
98224fd
Move approval to lib
AndreiEres Feb 13, 2024
d98b899
Move availability to lib
AndreiEres Feb 13, 2024
5175897
Return usage
AndreiEres Feb 14, 2024
a721bca
Add skeleton tests
AndreiEres Feb 14, 2024
b27edaa
Add approval tests
AndreiEres Feb 15, 2024
9136955
Add regression tests
AndreiEres Feb 15, 2024
18a0c7e
Address clippy warnings
AndreiEres Feb 16, 2024
d72a035
Address clippy warnings
AndreiEres Feb 16, 2024
d161de1
Address clippy warnings
AndreiEres Feb 16, 2024
ab2e179
Update
AndreiEres Feb 16, 2024
1e59be1
Add draft ci run
AndreiEres Feb 16, 2024
b54b85d
Filter regression tests
AndreiEres Feb 16, 2024
0e4432a
Update nexttest config
AndreiEres Feb 16, 2024
3b6a3e9
Skip benches
AndreiEres Feb 16, 2024
2fec1c1
Skip benches
AndreiEres Feb 16, 2024
2657432
Rename test job
AndreiEres Feb 16, 2024
c037885
Fix label
AndreiEres Feb 16, 2024
d46e9fa
Update values
AndreiEres Feb 16, 2024
708e002
Update availability recovery bench values
AndreiEres Feb 19, 2024
d1b3631
Use average run
AndreiEres Feb 19, 2024
282745a
Address clippy errors
AndreiEres Feb 19, 2024
f6df907
Update README
AndreiEres Feb 19, 2024
f0aea37
Update values
AndreiEres Feb 19, 2024
ff7dac3
Update values
AndreiEres Feb 19, 2024
e1e6f2d
Fix md line length
AndreiEres Feb 20, 2024
68ec964
Update a wrong range
AndreiEres Feb 20, 2024
184eee1
Merge branch 'master' into AndreiEres/subsystem-bench-lib
AndreiEres Feb 20, 2024
f03f4a7
Disable linting in code block
AndreiEres Feb 20, 2024
4abf5d8
Update .gitlab/pipeline/test.yml
alvicsam Feb 20, 2024
9df54ec
Update values
AndreiEres Feb 20, 2024
3acfe18
Update values
AndreiEres Feb 20, 2024
3868962
Try warming up
AndreiEres Feb 22, 2024
e4f3c87
Fix format
AndreiEres Feb 22, 2024
f55ae88
Address clippy errors
AndreiEres Feb 22, 2024
222f791
Warm up availability-distribution-regression-bench
AndreiEres Feb 22, 2024
ddf4408
Run benches
AndreiEres Feb 22, 2024
4f40d1f
Update warm up
AndreiEres Feb 22, 2024
0e1b59e
Run benchmarks
AndreiEres Feb 22, 2024
31a5090
Run benchmarks
AndreiEres Feb 22, 2024
732b4ba
Disable approval-voting-regression-bench
AndreiEres Feb 22, 2024
c0e6f31
Update values
AndreiEres Feb 22, 2024
02d63b8
Update benches
AndreiEres Feb 22, 2024
341c8b7
Disable approval-voting tests
AndreiEres Feb 23, 2024
a03fd98
Merge branch 'master' into AndreiEres/subsystem-bench-lib
AndreiEres Feb 23, 2024
77f3da5
Remove approval-voting tests
AndreiEres Feb 29, 2024
36b3f6c
Remove regression tests from CI
AndreiEres Feb 29, 2024
ff91819
Add comments
AndreiEres Feb 29, 2024
46ec768
Merge branch 'master' into AndreiEres/subsystem-bench-lib
AndreiEres Mar 1, 2024
07b4662
Add comments
AndreiEres Mar 1, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Disable linting in code block
  • Loading branch information
AndreiEres committed Feb 20, 2024
commit f03f4a742fd63d7c88823bc62283b7312a93349a
5 changes: 5 additions & 0 deletions polkadot/node/subsystem-bench/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,8 @@ the test is started.
Let's run an availabilty read test which will recover availability for 200 cores with max PoV size on a 1000
node validator network.

<!-- markdownlint-disable line-length -->

```
target/testnet/subsystem-bench polkadot/node/subsystem-bench/examples/availability_write.yaml
[2024-02-19T14:10:32.981Z INFO subsystem_bench] Sequence contains 1 step(s)
Expand Down Expand Up @@ -197,6 +199,8 @@ availability-store 0.304 0.101
Test environment 3.213 1.071
```

<!-- markdownlint-enable line-length -->

`Block time` in the current context has a different meaning. It measures the amount of time it
took the subsystem to finish processing all of the messages sent in the context of the current test block.

Expand Down Expand Up @@ -226,6 +230,7 @@ Since the execution will be very slow, it's recommended not to run it together w
benchmark results into account. A report is saved in a file `cachegrind_report.txt`.

Example run results:

```
$ target/testnet/subsystem-bench --cache-misses cache-misses-data-availability-read.yaml
$ cat cachegrind_report.txt
Expand Down