tokio hangs up with hyper 0.12

[nyuichi]

Hi, I've been using hyper 0.12 to talk to my local docker daemon and observed my program gets stuck during start up at a very low probability. I managed to minimize the reproducing code to the following piece but had no idea at all why this happens. The phenomenon is that running the following code after setting up the docker daemon to talk via tcp instead of unix socket, which is default, eventually makes the whole program stuck after a certain period of time. (In my Ubuntu on VMware on MacBookPro 2020 it takes less than a minute) I know that my software stack is quite outdated, but to make sure this problem is entirely fixed I think I need find out the actual cause. Is this a bug in hyper or is my usage simply incorrect?

How to reproduce

Before running the code one needs to set up the docker daemon to talk tcp. (I found this instruction https://gist.github.com/styblope/dc55e0ad2a9848f2cc3307d4819d819f helpful).

// [dependencies]
// futures = "0.1"
// hyper = "0.12"
// tokio = "0.1"

use futures::{Future, Stream};

fn main() {
    let client = hyper::Client::new();

    let mut runtime = tokio::runtime::Runtime::new().unwrap();

    for i in 0.. {
        println!("{}", i);

        let client = client.clone();

        let future = client
            .get("http://localhost:2375//events".parse().unwrap()) // get 301 response and discard it; appearently meaningless code but removing this line changes the behavior.
            .and_then(move |_resp| client.get("http://localhost:2375/events".parse().unwrap()));

        let res = runtime.block_on(future).unwrap(); // this doesn't return at a low probability

        let future = res.into_body().for_each(|_| Ok(()));

        runtime.spawn(future.map_err(|_| ()));
    }
}

[seanmonstar]

What do the logs look like, if you enable them? Are you able to upgrade to 0.13?

[nyuichi]

Here are the logs. In this run the above program stopped emitting logs at the 19th iteration.
https://gist.github.com/nyuichi/3bda7ee7a95b94e0214a04fbd6dd4db5

Upgrading the above particular code to 0.13 seems get rid of the hangup, but I was unable to figure out whether this means the issue was gone by upgrading hyper (and tokio) or it just lowered the probability the bug appears.

[seanmonstar]

The logs do seem to show that right before the 19th iteration, it stores a new idle connection, whereas previous iterations it marked the connection dead since its chunked-encoding and you don't consume the body. I can't say why exactly it does that, but if it's fixed in 0.13, I wouldn't spend time trying to debug 0.12.

[nyuichi]

Well, I am using 0.12 in an embedded system and its codebase is not that small. What is worse is it is already deployed in production and this may affect many running devices. In this kind of situation merely upgrading dependencies can be a risk, so, if possible, I would first like to understand the exact reason this is caused and then wish to evaluate how to fix (of course the best option at the moment is to migrate the codebase to 0.13, unless the cause gets revealed).

[seanmonstar]

So many smaller details changed, I don't know what would be the issue. Is it simply just the usage pattern of calling block_on on individual futures instead of letting the runtime constantly run?

Since this is fixed in 0.13, I'm going to close the issue at least.