Kafka streams binder creates a -repartition topic even though no key modifying operations are done by client code

[danishgarg]

The kafka streams documentation states that the serialization/deserialization of kafka message keys is not handled by the framework and it is left to Kafka:

"It is worth to mention that Kafka Streams binder does not serialize the keys on outbound - it simply relies on Kafka itself"

It is worth to mention that Kafka Streams binder does not deserialize the keys on inbound - it simply relies on Kafka itself

However, when using kafka streams binder, the binder automatically creates a -repartition topic because the serialization/deserialization code uses the map function on the stream. This marks the stream for re-partitioning and any subsequent group by operation done by client code results in creation of a repartition topic.
The use of Spring cloud stream should ideally be transparent and side effect free. Since there is no serialization/deserialization done by the framework when it comes to keys, there is a case to use mapValues instead of map when doing message value serialization/deserialization. This will prevent creation of any extra topics.
The following SO has the details:
(https://stackoverflow.com/questions/49704688/groupbykey-creates-repartition-topic-even-though-there-is-no-key-change?answertab=active#tab-top)

[sobychacko]

@danishgarg Thank you for reporting the issue. We will look into it soon. In the meantime, contributions are welcomed!

[sabbyanandan]

Hi, @danishgarg. Good write-up! Thanks for bringing it to our attention. Let us know if you have any other suggestions for improvements.