Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Oddities rendering ReadableStream #4828

Closed
1 task
thepassle opened this issue Sep 21, 2022 · 2 comments · Fixed by #4832
Closed
1 task

Oddities rendering ReadableStream #4828

thepassle opened this issue Sep 21, 2022 · 2 comments · Fixed by #4832
Assignees
@matthewp
Labels
- P2: nice to have Not breaking anything but nice to have (priority)

Comments

@thepassle
Copy link
Contributor

What version of astro are you using?

1.2.6

Are you using an SSR adapter? If so, which one?

Netlify edge

What package manager are you using?

npm

What operating system are you using?

mac

Describe the Bug

We (@matthewp and I) already had a twitter thread going on about this issue here, as well as some discussion on the Discord #netlify channel, but stuff tends to easily get lost in those channels, and I wanted to make a more complete/detailed summary of the issue.

Usecase

When using Astro in SSR mode on Netlify Edge, I'd like to fetch HTML, and stream the result of the HTML to the browser. My code looks as follows:

---
---
<html lang="en">
	<head>
		<meta charset="utf-8" />
		<link rel="icon" type="image/svg+xml" href="/favicon.svg" />
		<meta name="viewport" content="width=device-width" />
		<meta name="generator" content={Astro.generator} />
		<title>Astro</title>
	</head>
	<body>
		<h1>Astro</h1>
                {fetch(import.meta.env.SITE + '/foo.html').then(r => r.body)}
	</body>
</html>

During local development using Node v16, if I refresh the page, I can clearly see that the HTML is correctly being streamed to the browser.

Once deployed on Netlify Edge, however, it seems to output/stringify a Uint8Array instead:
image

Investigation

Since Netlify Edge runs on Deno, and locally I'm running Astro with Node, I initially thought this might be a difference between the environments. So I tried to create a minimal reproduction of the rendering logic found here.

My reproduction looks like this:

async function* renderChild(child) {
	child = await child;
	if (child instanceof HTMLString) {
		yield child;
	} else if (typeof child === 'string') {
		yield markHTMLString(child);
	} else if (typeof child === 'object' && Symbol.asyncIterator in child) {
		yield* child;
	} else {
		yield child;
	}
}

for await (const chunk of renderChild(fetch('https://astro-blog-2.netlify.app/404.html').then(r => r.body))) {
	console.log(chunk);
}

However, this outputs (in both Deno and Node):

Uint8Array(27) [
   60, 112,  62, 79, 111, 112, 115,
   33,  32,  80, 97, 103, 101,  32,
  110, 111, 116, 32, 102, 111, 117,
  110, 100,  60, 47, 112,  62
]

If I add the following code (taken from @natemoo-re 's Microstream here):

+const decoder = new TextDecoder();

async function* renderChild(child) {
	child = await child;
	if (child instanceof HTMLString) {
		yield child;
+	} else if (child instanceof ReadableStream) {
+		const reader = child.getReader();
+		let res = await reader.read();
+		while (!res.done) {
+			yield decoder.decode(res.value);
+			res = await reader.read()
+		}
	} else if (typeof child === 'string') {
		yield markHTMLString(child);
	} else if (typeof child === 'object' && Symbol.asyncIterator in child) {
		yield* child;
	} else {
		yield child;
	}
}

for await (const chunk of renderChild(fetch('https://astro-blog-2.netlify.app/404.html').then(r => r.body))) {
	console.log(chunk);
}

The output I get is (as expected):

<p>Oops! Page not found.</p>

It's super confusing however that in the first example, the output is a Uint8Array, since during local development I definitely see the HTML being streamed:

stream.mov

So the problem is:
Why does Astro output a Uint8Array on Netlify Edge?

Bonus question:
How is it possible that in my minimal reproduction I'm seeing the Uint8Array being output on Node, whereas in local development with Astro on Node, I'm seeing the HTML being streamed?

I'll continue to dig into this/debug this some more as well in the meantime, and will update this issue if I've made any progress.

Links

Link to Minimal Reproducible Example

see post

Participation

  • I am willing to submit a pull request for this issue.
@matthewp
Copy link
Contributor

Thanks for doing the research! We discussed this more on Discord

Some conclusions we drew:

  1. Unsure about support Uint8Array in the template unescaped, could be a security issue.
  2. Supporting Response objects that are of type text/html seems like a win! This would enable: {fetch('/some.html')}
  • It still might be good to escape these by default. Maybe we can add Astro.unescape so you would do {Asro.unescape(fetch('/some.html'))}

@matthewp matthewp added the - P2: nice to have Not breaking anything but nice to have (priority) label Sep 21, 2022
@matthewp matthewp self-assigned this Sep 21, 2022
@matthewp
Copy link
Contributor

Actually, this can go through regular set:html, don't need a new Astro.unescape.

@matthewp matthewp mentioned this issue Sep 21, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@matthewp matthewp

Labels
- P2: nice to have Not breaking anything but nice to have (priority)
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Allow passing fetch() Response to set:html withastro/astro
2 participants
@matthewp @thepassle