Buffer batch for async producers #262
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
9ebbb42 to
7d4ada2
Compare
Other kafka connectors such as the Java one have the possibility to configurate an optional batch (client-side) for async producer mode, in order to send batches of messages instead of single messages at time. This increases performance because the kafka server doesnt need to answer ACK at every message. The options are: - Batch size in bytes - Batch age in miliseconds
7d4ada2 to
fcce260
Compare
|
+1 |
|
@carlessistare could you also update the document for this change. |
|
Sorry @haio I'm not sure of which document you want me to update, could you precise? I'll do it with pleasure though |
| * `connectionString`: Zookeeper connection string, default `localhost:2181/` | ||
| * `clientId`: This is a user-supplied identifier for the client application, default `kafka-node-client` | ||
| * `zkOptions`: **Object**, Zookeeper options, see [node-zookeeper-client](https://github.com/alexguan/node-zookeeper-client#client-createclientconnectionstring-options) | ||
| * `noAckBatchOptions`: **Object**, when requireAcks is disabled on Producer side we can define the batch properties, 'noAckBatchSize' in bytes and 'noAckBatchAge' in miliseconds. The default value is `{ noAckBatchSize: 500, noAckBatchAge: 300 }` |
c981598 to
c314a79
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Other kafka connectors such as the Java one have the possibility to configurate an optional batch (client-side) for async producer mode, in order to send batches of messages instead of single messages at time. This increases performance because the kafka server doesnt need to answer ACK at every message.
The options are:
This will resolve #214