Investigate excessive flushing in Netty

[ejona86]

Determine ways to reduce number of flushes we perform. For example, if the header frame is being sent, flushing afterward is generally a waste since a DATA frame typically follows. We want methods that allow smart semi-automatic flushing or using knowledge from application layer.

[buchgr]

this sounds like a good first project for me to get to know the whole stack?

[ejona86]

Yes and no. @louiscryan has already delved into this one some and came up empty-handed. Reducing the number of flushes did possibly have an improvement, but not enough to really care about (<5µs latency decrease) given our current performance. Although combining with a throughput benchmark may show it to be more benefit.

[rschildmeijer]

@ejona86 would something like a AutoFlushingWriteBatcher[1] be too generic and would apply "smart batching" too often?

[1] https://github.com/spotify/netty-zmtp/blob/master/src/main/java/com/spotify/netty/handler/queue/AutoFlushingWriteBatcher.java

[ejona86]

@rschildmeijer, it looks like that one just does regular flushes based on a timer. That wouldn't work for us because it would introduce latency.

Instead, we looked at something more akin to BetterWriteWithAtomic. As I mentioned though, performance may have improved, but it was mostly lost in the noise. Thus, we are going to focus on other things and then revisit the flushing.

[rschildmeijer]

@ejona86 it is regular flusher based on a timer on steroids. If writes are sparse no additional latency will be added.

The functionality is based on the following statement: if mean_time_between_sends < mean_time_to_send, then applying smart batching will improve throughput and reduce netty flush calls.

So consider it and benchmark. Might be worth it in your case. Start with a non default setting(?)

Based on Todds work: https://qconsf.com/sf2012/dl/qcon-sanfran-2012/slides/ToddMontgomery_RunningWithTheDevilMechanicalSympatheticNetworking.pdf

[nmittler]

@ejona86 @louiscryan I think this is a dup of #331, #332, and #305. Can we close this?

[ejona86]

I'd like to leave this open until we integrate fixes for netty/netty#3670, which currently doesn't have any other associated issue in gRPC.

[louiscryan]

Issue in Netty has now been resolved and with the introduction of our own write queue

#431

or flush behavior is pretty good