Investigate breakage of the http3 server demo after a few iterations of client connections

[t8m]

Copying comment from @vdukhovni openssl/openssl#24749 (comment)

@vdukhovni wrote:

Did you mean "was"? I see you added some code for that, but I still the client reach an error state after a few tries:

$ while : ; do ./ossl-nghttp3-demo server.example:12345; sleep 1; done
:status: 200
content-length: 20

12345678901234567890:status: 200
content-length: 20

12345678901234567890:status: 200
content-length: 20

12345678901234567890:status: 200
content-length: 20

12345678901234567890<added new line for clarity>
4060A4C15F7F0000:error:8000006F:system library:dgram_recvmmsg:Connection refused:crypto/bio/bss_dgram.c:1589:
4060A4C15F7F0000:error:0A000183:SSL routines:ossl_quic_port_raise_net_error:quic network error:ssl/quic/quic_port.c:717:port failed due to network BIO I/O error
4060A4C15F7F0000:error:0A0000CF:SSL routines:quic_read_again:protocol is shutdown:ssl/quic/quic_impl.c:2890:
4060A4C15F7F0000:error:400C0107:lib(128):main:operation fail:ossl-nghttp3-demo.c:141:cannot handle events
40000530937F0000:error:8000006F:system library:dgram_recvmmsg:Connection refused:crypto/bio/bss_dgram.c:1589:
40000530937F0000:error:0A000183:SSL routines:ossl_quic_port_raise_net_error:quic network error:ssl/quic/quic_port.c:717:port failed due to network BIO I/O error
40000530937F0000:error:0A0000CF:SSL routines:quic_do_handshake:protocol is shutdown:ssl/quic/quic_impl.c:1878:
40000530937F0000:error:400C0103:lib(128):OSSL_DEMO_H3_CONN_handle_events:internal error:ossl-nghttp3.c:650:writing HTTP/3 data to network failed
40000530937F0000:error:400C0107:lib(128):main:operation fail:ossl-nghttp3-demo.c:141:cannot handle events
40A042A74F7F0000:error:8000006F:system library:dgram_recvmmsg:Connection refused:crypto/bio/bss_dgram.c:1589:
40A042A74F7F0000:error:0A000183:SSL routines:ossl_quic_port_raise_net_error:quic network error:ssl/quic/quic_port.c:717:port failed due to network BIO I/O error
40A042A74F7F0000:error:0A0000CF:SSL routines:quic_do_handshake:protocol is shutdown:ssl/quic/quic_impl.c:1878:
40A042A74F7F0000:error:400C0103:lib(128):OSSL_DEMO_H3_CONN_handle_events:internal error:ossl-nghttp3.c:650:writing HTTP/3 data to network failed
40A042A74F7F0000:error:400C0107:lib(128):main:operation fail:ossl-nghttp3-demo.c:141:cannot handle events
^C

The last few messages from the server were:

...
=> Received connection on 2
read_from_ssl_ids 3 events
Create bidi?
Create uni
revent 128 (128) on 3
revent 128 (128) on 7
read_from_ssl_ids hassomething nothing...
waitsocket for 5
read_from_ssl_ids timeout

[jfclere]

I have reproduced the issue... it takes several minutes on my box.
Either the client forgets to send a packet and the server misses something in the poll logic and does a timeout after 5 seconds.

[t8m]

I have reproduced the issue... it takes several minutes on my box. Either the client forgets to send a packet and the server misses something in the poll logic and does a timeout after 5 seconds.

This really sounds like something in the server code in the library. Even if the client was doing something wrong, it should never cause the server to break on the subsequent connections.

[jfclere]

Yes the error in ossl-nghttp3-demo-server.c
https://github.com/openssl/openssl/blob/feature/quic-server/demos/http3/ossl-nghttp3-demo-server.c#L179
It seems that reading from the new stream and processing the data (if any) fixes the issue.

[jfclere]

=> Received connection on 2
There is a setting from the client but the server doesn't process it and a timeout occurs

I should have a patch soon, the code needs some refactoring :-(

[vdukhovni]

@t8m, the above proposed fix to the demo server code appears to be effective for this issue.

[nhorman]

we have a linked pr, need to review