fix: avoid setting content-length before middleware

[SabrinaJewson]

Motivation

Axum currently sets content-length automatically inside Route. However, if this occurs before middleware is run then should the middleware add a body to the request, Axum will avoid overwriting the content-length and so the user is stuck with an incorrect content-length, leading to panics in Hyper.

Solution

Only set content-length for top-level Routes.

(there are a couple miscellaneous changes as well; one is a one-line change in route.rs to avoid double-boxing the body for middleware, another removes the unused enum RouteFutureKind)

[jplatte]

I didn't dive into the details here, but nothing stands out as wrong and I'm pretty sure we have sufficient tests to cover this change (especially with you adding another one for the previous bug).

[jplatte]

Do you want to add a changelog entry though?

[SabrinaJewson]

Yup, good point – added.

[jplatte]

Are you interested in backporting this to v0.7? I've tried, but failed. Maybe it makes sense to first backport #2894, which touched the same code 🤷🏼

[SabrinaJewson]

I can try. I’ll be able to get to it hopefully in a couple days.

[jplatte]

Thanks! There's a v0.7.x branch you can make a PR against.

[Turbo87]

Before

After

was this change intended? 🤔

(this is from rust-lang/crates.io#9969)

[yanns]

Chunked response bodies must not have any content length, right?

"Messages MUST NOT include both a Content-Length header field and a non-identity transfer-coding. If the message does include a non-identity transfer-coding, the Content-Length MUST be ignored." (RFC 2616, Section 4.4)

[mladedav]

Yes, it's either content length or chunked bodies (or closing the connection after the request).

But if the length is known we should use that so if we started using chunked transfer encoding instead of content length for a given request after the update, then that was not an intentional change.

I would guess we're not setting the length properly in some case.

[blind-oracle]

Are we sure this needs to be in a minor release? We got some tests failing after moving to axum 0.7.9 that were dependant on having the Content-Length in the middleware. Nothing serious but still I think this breaks backwards compatibility.

Also it seems that the Content-Length header is also not added to the responses (not requests) that are returned from the handler.

E.g. the following (pseudo-)code fails for me:

async fn handler() -> impl IntoResponse {
    "a".repeat(1000)
}

fn test() {
    let mut app = Router::new()
        .route("/", post(handler))
        .layer(middleware::from_fn(|req: Request, next: Next| -> impl IntoResponse
          let resp = next.run(req);
          assert!(resp.headers().get(CONTENT_LENGTH).is_some());
        ));

    let req = Request::new(...);
    app.call(req).await.unwrap();
}

If I inject a Content-Length in the handler() it works. I think it shouldn't be like that?

[SabrinaJewson]

I’m not goïng to comment on whether it’s a breaking change or not; I don’t have that much of an opinion.

I think it shouldn't be like that?

This is the intended behaviour of the change. First of all, axum never changes Content-Length of requests I don’t believe; the difference is that the Content-Length header is added after middleware runs, and not before. So if you checked for the header after calling call, it would be there.

My idea is that if you want to check the content length from middleware, what you should do is call response.body().size_hint().exact() (see Body::size_hint), which also eliminates the need for parsing header values.

[blind-oracle]

Thanks. Yes, I agree that the new approach is more logical. But anyway I'd vote for such changes not to be backported between "major" versions.

[jplatte]

I get that. If it makes you feel better, this is something I considered as well. But given that this fixed a bug that could pretty badly affect services after a change like adding compression support, I decided that backporting it to v0.7 would be better, rather than rush out v0.8 and tell everybody to upgrade (there's still a bunch of things I'd like to get done before releasing v0.8).

[blind-oracle]

@jplatte Thanks, yes, probably that was worth some breakage :)

[blind-oracle]

I finally managed to get my hands on this and it seems that using response.body().size_hint().exact() isn't working for me. The size_hint when called inside a middleware always returns SizeHint { lower: 0, upper: None }

Middleware is simply something like:

pub async fn middleware(
    request: Request,
    next: Next,
) -> impl IntoResponse {
  let mut response = next.run(request).await;
  let hint = response.body().size_hint();
  tracing::warn!("size_hint: {:?}", hint);
  response.extensions_mut().insert(hint);
  response
}

{"timestamp":"2024-12-12T09:58:15.247561Z","level":"WARN","message":"size_hint: SizeHint { lower: 0, upper: None }"

Before this PR extracting the Content-Length from the same response worked fine. And in the HTTP response in CURL I now get a correct Content-Length so the Response in middleware isn't streaming or chunked:

< content-length: 132

In Tower-like tests this works fine e.g.

async fn handler() -> impl IntoResponse {
    "a".repeat(100)
}

fn test() {
    let mut app = Router::new()
        .route("/", post(handler))
        .layer(middleware::from_fn(
            middleware,
        ));

    let request = Request::new(...);
    let response = router.call(request).await.unwrap();
    assert!(response.extensions().get::<SizeHint>().exact().is_some())
}

Any idea what I did wrong here? Thanks in advance.

[jplatte]

Could you please open a new discussion and post a full reproducing example (repo including a cargo lockfile)?