[FEATURE REQ] Support Spring Cloud Stream Batch Consumer function in Spring Cloud Azure for Service Bus

[z1069867141]

Context

Refer to #20080

According to Spring Cloud Stream's documentation, a binder implementation may support batch consumers. Currently Azure Spring Cloud Service Bus binder does not support it.

For Kafka binder:

• Referring to Kafka binder's batch-consumer feature, it supports configuration to specify the batch-message polling size, measurement covers 3 types: message number/ bytes/time, and its configuration is as max.poll.records, fetch.min.bytes, fetch.max.wait.ms.
• Kafka binder supports acking batched messages by batch. Ref: https://cloud.spring.io/spring-cloud-stream-binder-kafka/spring-cloud-stream-binder-kafka.html#kafka-consumer-properties

Goal

Support the batch-consuming function for Service Bus binder as described in the Spirng Cloud Stream documentation.

[michaelqi793]

Currently, after some codes are modified, the spring cloud stream application can work by providing a list Consumer. The following is just my draft ideas and hypothesis:
In the top-down way, there are 2 steps needed:

2, Push a batch of records as a single Spring Message into Spring framework
1, Get the batch of records from the azure-messaging-servicebus sdk

There are household chores for step 2 even though the pushing way is straightforward: Just make a list of records the payload of Spring Message, and push the Message into Spring framework by fundamentally invoking sendMessage(msg) method, because there is Json format required for the element in the list. For example, object need start with { and end with }, but string need start with " and end with ". In which layer to deal with format problem is still a question.
More efforts are needed for step 1, because azure-messaging-servicebus sdk need provide a method that returns a batch of records and properly expose it via maybe ServiceBusProcessorClient, so that azure-spring-integration-servicebus as well as the higher level azure-spring-cloud-stream can benefit from it. The question is, does the ServiceBusProcessorClient support to register a Consumer that consumes a list of ServiceBusReceivedMessageContext?

[michaelqi793]

The draft design picture:

[michaelqi793]

#23039
#23040
#23292
#23155

[yiliuTo]

The feature now is blocked due to below SDK related issues. And we need to wait for those issues get resolved for moving forward.

[stliu]

category to spring feature parity

1. understand the spring feature
2. design for the implementation

[github-actions]

Hi @z1069867141, we deeply appreciate your input into this project. Regrettably, this issue has remained inactive for over 2 years, leading us to the decision to close it. We've implemented this policy to maintain the relevance of our issue queue and facilitate easier navigation for new contributors. If you still believe this topic requires attention, please feel free to create a new issue, referencing this one. Thank you for your understanding and ongoing support.