How to improve the performance of high-level-producer

[JohnnyZHH]

Hi

We tried performance testing using script high-level-producer.js under kafka-node-master/example, we set interval of sending message as the smallest value (1 or 0 ms), and each message size is 1k, then we found the fastest speed is only 651.7 messages/sec (0.6MB/sec). Seems it is too slow for KAFKA Node.js client, and seems the bottleneck is function send (), is there any method could improve the performance? Thanks!

[JohnnyZHH]

Anybody could help to answer it? Thanks!

[hyperlink]

Any change with requireAcks set to 0 ?

[JohnnyZHH]

Hi,

Thanks for your reply!
We tried requireAcks with 0 or 1, the performance dose not improve anymore.

[JohnnyZHH]

Is there a benchmark for KAFKA-node high-level-producer?

[hyperlink]

@crzidea was working on some performance improvements in protocol code that could improve performance in this area. Some of this work is already merged into master.

I have benchmark code in a local branch for the consumer but nothing for the producer. I would be glad to review If you wanted to take a stab at creating one.

[JohnnyZHH]

Hi,

Could you please provide the benchmark about consumer for reference? Thanks!

[crzidea]

@JohnnyZHH I have some production performance results here #654 .