Next 13.4 API routes gzip already gzipped content

[tkmcmaster]

Verify canary release

• I verified that the issue exists in the latest Next.js canary release

Provide environment information

Operating System:
      Platform: linux
      Arch: x64
      Version: #1 SMP Fri Jan 27 02:56:13 UTC 2023
    Binaries:
      Node: 18.16.0
      npm: 9.5.1
      Yarn: N/A
      pnpm: N/A
    Relevant packages:
      next: 13.4.2
      eslint-config-next: 13.4.2
      react: 18.2.0
      react-dom: 18.2.0
      typescript: 5.0.4

Which area(s) of Next.js are affected? (leave empty if unsure)

Middleware / Edge (API routes, runtime)

Link to the code that reproduces this issue

https://github.com/tkmcmaster/next13gzip

To Reproduce

export default function handler(
  req: NextApiRequest,
  res: NextApiResponse<Readable>
) {
  res.status(200);
  const json: Data = { name: 'John Doe' };
  const content = Readable.from(JSON.stringify(json)).pipe(createGzip());
  res.setHeader("Content-Type", "application/json");
  res.setHeader("Content-Encoding", "gzip");
  res.send(content);
}

$ npm ci
$ npm run build
$ npm start

Browser:
http://localhost:3000/api/hello
http://localhost:3000/api/gzip

Describe the Bug

Next.js API layer appears to be gzipping already gzipped data starting in Next.13.4.x. In Next 13.2 and 13.3 an already gzipped readable will be correctly read by the receiving browser. But starting in Next 13.4 that is no longer the case. The browser gets what appears to be binary data.

The example above uses a pure Readable. The real data I'm using comes from an already Gzipped AWS S3 Object Body.

res.send(s3Object.Body);

If I downgrade to Next 13.2.4 on the example above, loading those URLs in Chrome both work.

Expected Behavior

If the Content-Encoding header is already set to gzip, the API layer shouldn't add it's own gzip.

Which browser are you using? (if relevant)

Chrome 113

How are you deploying your application? (if relevant)

AWS

[tkmcmaster]

Any updates on this? This is a regression from everything before Next.js 13.4.x. This code has been working since Next.js 8.x

[tkmcmaster]

Just tested and 13.4.9 still has the issue.

[tkmcmaster]

Still broken in 13.4.12

[tkmcmaster]

Interestingly it's semi-fixed in 13.4.13, we're just getting truncated results on long responses. Possibly fixed by #53004? The truncated could be related to #53737

[tkmcmaster]

Looks like this is fixed in 13.4.14-canary.1. Closing issue.

[tkmcmaster]

Verified fixed in next: 13.4.16

[github-actions]

This closed issue has been automatically locked because it had no new activity for 2 weeks. If you are running into a similar issue, please create a new issue with the steps to reproduce. Thank you.