fix: benchmark dead channels

[fraidev]

Should fix #4360

[sehz]

it's bit weird, last iteration seems to slow down:

36344 records sent, 41584 records/sec: (208.9 MB/sec), 0.72ms avg latency, 2.69ms max latency
42168 records sent, 41895 records/sec: (210.4 MB/sec), 0.71ms avg latency, 3.55ms max latency
21460 records sent, 34783 records/sec: (174.7 MB/sec), 0.72ms avg latency, 2.22ms max latency

0.72ms avg latency, 3.55ms max latency, 0.69ms p0.50, 0.81ms p0.95, 1.93ms p0.99
100000 total records sent, 34783 records/sec: (174.7 MB/sec) 
Benchmark completed

It only uses numbers from last iteration. not from e2e.

Here's suggested strategy:

Suppose you have window of 1 seconds. compute stat for that window (min/max latencies), number of events, total count. when you event has exceed window, compute rate then send them to aggregate stat

[fraidev]

it's bit weird, last iteration seems to slow down:

36344 records sent, 41584 records/sec: (208.9 MB/sec), 0.72ms avg latency, 2.69ms max latency
42168 records sent, 41895 records/sec: (210.4 MB/sec), 0.71ms avg latency, 3.55ms max latency
21460 records sent, 34783 records/sec: (174.7 MB/sec), 0.72ms avg latency, 2.22ms max latency

0.72ms avg latency, 3.55ms max latency, 0.69ms p0.50, 0.81ms p0.95, 1.93ms p0.99
100000 total records sent, 34783 records/sec: (174.7 MB/sec) 
Benchmark completed

It only uses numbers from last iteration. not from e2e.

Here's suggested strategy:

Suppose you have window of 1 seconds. compute stat for that window (min/max latencies), number of events, total count. when you event has exceed window, compute rate then send them to aggregate stat

Good catch! I added a sleep to send the end command. That was decreasing performance. I remade it without that.

[sehz]

last number doesn't show up:

(base) $> target/release/fluvio-benchmark   producer --num-records 100000 --record-size 5012 
Benchmark started
35896 records sent, 40287 records/sec: (202.3 MB/sec), 0.74ms avg latency, 2.19ms max latency
42392 records sent, 41400 records/sec: (207.9 MB/sec), 0.71ms avg latency, 3.54ms max latency

0.73ms avg latency, 7.03ms max latency, 0.70ms p0.50, 0.82ms p0.95, 1.85ms p0.99
100000 total records sent, 41374 records/sec: (207.8 MB/sec) 
Benchmark completed

[fraidev]

last number doesn't show up:

(base) $> target/release/fluvio-benchmark   producer --num-records 100000 --record-size 5012 
Benchmark started
35896 records sent, 40287 records/sec: (202.3 MB/sec), 0.74ms avg latency, 2.19ms max latency
42392 records sent, 41400 records/sec: (207.9 MB/sec), 0.71ms avg latency, 3.54ms max latency

0.73ms avg latency, 7.03ms max latency, 0.70ms p0.50, 0.82ms p0.95, 1.85ms p0.99
100000 total records sent, 41374 records/sec: (207.8 MB/sec) 
Benchmark completed

Do you mean the blank line? I added it before in another PR to separate the progress of the total result.

Maybe change it for a separator like "---------------"? 😅

Got it, it's not showing all the results if sum all records. I'll fix it.

[fraidev]

last number doesn't show up:

(base) $> target/release/fluvio-benchmark   producer --num-records 100000 --record-size 5012 
Benchmark started
35896 records sent, 40287 records/sec: (202.3 MB/sec), 0.74ms avg latency, 2.19ms max latency
42392 records sent, 41400 records/sec: (207.9 MB/sec), 0.71ms avg latency, 3.54ms max latency

0.73ms avg latency, 7.03ms max latency, 0.70ms p0.50, 0.82ms p0.95, 1.85ms p0.99
100000 total records sent, 41374 records/sec: (207.8 MB/sec) 
Benchmark completed

I added a fix for it!

[sehz]

Benchmark started
40528 records sent, 46584 records/sec: (234.0 MB/sec), 6.78ms avg latency, 36.93ms max latency
50145 records sent, 48578 records/sec: (244.0 MB/sec), 6.40ms avg latency, 11.45ms max latency
9110 records sent, 34819 records/sec: (174.9 MB/sec), 6.49ms avg latency, 9.61ms max latency

6.56ms avg latency, 36.93ms max latency, 6.25ms p0.50, 8.04ms p0.95, 11.08ms p0.99
100000 total records sent, 48757 records/sec: (244.9 MB/sec) 

can you add total elapsed time as well? summary time 244.9 MB/Sec doesn't seem to be align with numbers from 3 rows.

[fraidev]

Benchmark started
40528 records sent, 46584 records/sec: (234.0 MB/sec), 6.78ms avg latency, 36.93ms max latency
50145 records sent, 48578 records/sec: (244.0 MB/sec), 6.40ms avg latency, 11.45ms max latency
9110 records sent, 34819 records/sec: (174.9 MB/sec), 6.49ms avg latency, 9.61ms max latency

6.56ms avg latency, 36.93ms max latency, 6.25ms p0.50, 8.04ms p0.95, 11.08ms p0.99
100000 total records sent, 48757 records/sec: (244.9 MB/sec) 

can you add total elapsed time as well? summary time 244.9 MB/Sec doesn't seem to be align with numbers from 3 rows.

Sure, also fixed that last item slow.