How should Pushup components work?

[paulsmith]

It's always been my intention that Pushup pages be components, in the sense that they have come to mean in the modern web dev era: self-contained classes, or units of encapsulation of markup that can be included in other pages or components themselves. They should take "arguments" somehow from the inclusion site.

Having everything be a component simplifies Pushup and its implementation. Currently, layouts are a separate concept from pages, but they are both .up files and parsed identically (they differ in code generation mainly). If we had true components, layouts could just be regular components and sit alongside pages. We could also then remove the "^layout blah" syntax from the language.

There are a couple of design challenges to work out before we can claim that Pushup pages are components.

The first is what does the syntax look like? This may seem relatively minor but there is substantial practical implications. Because we are intermingling Go and HTML in the Pushup template language, the syntax of how components are referenced and how arguments are passed to them (and how, on the component implementation side, parameters are defined) is important to get right.

The possibly most obvious thing to do would be to adopt a JSX-like approach here, which is just a syntactical transformation from angle brackets-like markup to a function call. This appeals because much of the front-end dev world is very familiar with JSX at this point. The challenge is that it probably means inverting the current sense of Pushup pages, from one as fundamentally HTML with additional syntax to switch to Go code mode, to one that is fundamentally Go code source files with embedded JSX that is transformed to function calls (including what would be ultimately treated as plain HTML).

Another idea would be to adopt a web components-like approach, as most recently seen by the WebC plugin of the static-site generator 11ty, and also from Svelte. If we treat Pushup pages as essentially web components (which has implications especially with respect to scoped CSS and JS, which we can punt on for now), then referencing them and composing them becomes a matter of mapping the page name to the element name. The design challenge here is how to refer to pages that are down in some directory - some kind of import statement or clause might be necessary.

There are two kinds of values that can be passed to a component: attributes or "props", like <my-comp bar="baz">, and child nodes, like <my-comp><p>This</p></p>That</p></my-comp>. Any approach will need to at least provide an API for Pushup authors to access these values in their pages, and at best type-check them (ultimately they will fall-through to the Go compiler but that's maybe not the best developer experience). It also needs to allow for dynamic values, which should be straightforward since we already have implicit and explicit Go expressions in Pushup syntax.

Another design consideration: file-based routing. Currently, pages are automatically routable by virtue of being placed in the app/pages directory. If everything is just a component, how do we distinguish between components that want to be pages (and hence exposed to be routed to by the app) and components that want to play other non-route support roles?

In the spirit of rough-consensus-and-working-code, I don't expect for this to be a lengthy discussion before making a decision decision and starting an implementation, but I did want to solicit some feedback before getting too far down the road.

[earthboundkid]

The other night I was thinking it would be neat if this syntax worked:

<^wrapper>
   <^mycomponent size="large" ^value="x" />
</^wrapper>

Basically just Angular/Vue/Alpine syntax-ish except that caret is the magic character that signals value interpolation instead of colon. Since it uses caret, you could include JS components that use colon and @ without conflicting. You could reuse the x/net/html parser, at least as a bootstrap.

[paulsmith]

For attributes this works today:

^layout !
^{ name := "class"; value := "myclass" }
<p ^name="^value"></p>

$ pushup run -page foo.up &
$ curl localhost:8080/foo
<p class="myclass"></p>

So I think extending this to tag names is a natural change.

[paulsmith]

I've been sketching out an alternative compilation strategy and project layout structure so that we can have Pushup components. I think in the course of doing this, Pushup also gets simpler and easier to use and even more Go-like.

The main idea is to get closer to Go by just having .up files live alongside regular .go files. Directories as Go packages. Compiled .up files would just be importable via a package like any other Go code. This eliminates the awkward build package that all .up files get compiled to currently.

.
├── go.mod
└── mypkg
    ├── something.up
    ├── something.up.go
    └── other.go

For these examples let's assume the Go module is example/myapp. In this case mypkg is a directory that has .up and .go files, and one of them is the result of a Pushup compilation (something.up.go). This is then just a Go package and so it can be imported as normal, like import "example/myapp/mypkg".

The next change is how .up files get compiled. First, Pushup pages (that is, .up files that get routed to per file-based routing) must live in a package named pages (which might eventually be configurable/overridable), but this can be in the top-level of the module, and doesn't need to be a directory under app, which is how Pushup currently works.

.
├── go.mod
├── mypkg
│   └── ...
└── pages
    ├── index.up
    ├── index.up.go
    └── products
        ├── index.up
        └── index.up.go

The same file-based routing strategy as current Pushup works here, nothing about that part of Pushup changes. So pages/index.up maps to the URL path / and pages/products/index.up to /products/. These are also simply Go packages, in this case the import paths would be example/myapp/pages and example/myapp/pages/products

Now we can introduce components. Components are .up files that are reusable, either in pages or in other components. Pages and components differ because pages are routable from the web app, and components are not. They are all .up files, but if a .up file is in a pages package (or child package of as in the case of products above), it is treated as a page from a compilation and routing point of view, and if it lives anywhere else, it is a component. This means that the package/directory names that components live in are entirely up to the user. As long as the Pushup compiler can find them, they can be placed in the project wherever the user prefers. There will be some defaults, such as looking for .up files in a components directory. But the goal is for this to be configurable and flexible.

Components, like pages, are .up files that are compiled to .go, and live in Go packages. This means they can be in other modules that you go get and add to your go.mod file. When a component is used, in another component or in a page, it is imported into the caller, using a regular import declaration (^import in Pushup syntax).

To use a component in another .up file, use HTML-like tag syntax, with the element name being a construction of the package name and a derivative of the component's filesystem name. For example, let's say we have a card.up component in components:

.
├── components
│   ├── card.up
│   └── card.up.go
├── go.mod
├── mypkg
│   └── ...
└── pages
    └── ...

For now let's say our card component is entirely static, but we'll soon add a way to make it dynamic and be able to use data passed into it.

<div class="card">
    <h2>Card title</h2>
    <p>Card body</p>
</div>

Again, .up files start in HTML parsing mode, so this is a valid, complete component.

We would like to use the card component in one of our pages. Our pages/index.up might look like this:

^import "example/myapp/components"

<h1>Hello, Pushup</h1>
<components.Card />

Notice that the element name of the component is the concatenation of the package name, a dot, and the uppercase name of the .up file minus the extension. (See generatedTypename() for the basic logic used here.) It needs to be uppercase in order to be exported from Go package per naming rules. The golang.org/x/net/html tokenizer handles the unusual name just fine, although it does lower-case the name so care will be taken to properly recover the name that is also the generated and exported name.

The generated code for the component might look like this:

package components

import (
	"io"
	"net/http"

	"github.com/adhocteam/pushup"
)

func Card(w http.ResponseWriter, req *http.Request) error {
	io.WriteString(w, `<div class="card">\n`)
	io.WriteString(w, `\t<h2>`)
	io.WriteString(w, `Card title`)
	io.WriteString(w, `</h2>\n`)
	io.WriteString(w, `\t<p>`)
	io.WriteString(w, `Card body`)
	io.WriteString(w, `</p>\n`)
	io.WriteString(w, `</div>\n`)
        return nil
}

Very straightforward and consistent with the Responder interface in current Pushup.

The codegen for the index page looks like:

package pages

import (
	"fmt"
	"io"
	"net/http"

	"example/myapp/components"
)

func Index(w http.ResponseWriter, req *http.Request) error {
	io.WriteString(w, "<h1>")
	io.WriteString(w, "Hello, Pushup")
	io.WriteString(w, "</h1>\n")
	if err := components.Card(w, req); err != nil {
		return err
	}
	return nil
}

So using a component in a .up file translates into calling a function in Go code. Very simple.

(Note that the codegen for pages in this revised compilation strategy would include information about URL paths for the later "linking" step where all the routes of the Pushup app are wired up together, but I'm eliding that for now.)

But static components wouldn't be very useful, so let's allow them to be passed in data from their callers. Since we're reusing HTML-ish syntax for Pushup components let's just use element attributes and properties for this.

Let's say a card takes in a title and body from it's caller. The revised card.up would be:

^declare title string
^declare body string

<div class="card">
    <h2>^prop(title)</h2>
    <p>^prop(body)</p>
</div>

The new ^declare keyword would be used to declare what data can be passed in to the components, this is sort of equivalent to a function/method signature with the declaration of the formal parameters. Types would be Go types.

To access this data in a component, the new ^prop special Pushup function would be used, taking the name of the previously declared names. We're using "prop" in the React-ish sense here, though I'm open to a different name (as well as for ^declare).

I'm on the fence about whether ^declare is really needed, or if it would simply be enough to build the set of passed-in props from the occurences of ^prop in the .up file.

In any case, from the caller side, passing data to a component would look like this (revised index.up):

^import "example/myapp/components"

^{ title := "My card title" }

<h1>Hello, Pushup</h1>
<components.Card title=^title body="My card body" />

This is just to demonstrate two different ways to pass data, either a Pushup expression or as a regular attribute value string.

To accommodate the passed-in data, we need to change the generated Go code, both the signature of the functions that correspond to components, and the callsite.

Our card component would now look like:

package components

import (
	"io"
	"net/http"

	"github.com/adhocteam/pushup"
)

func Card(w http.ResponseWriter, req *http.Request, props map[string]any) error {
	io.WriteString(w, `<div class="card">\n`)
	io.WriteString(w, `\t<h2>`)
	pushup.EscapeHTML(w, pushup.Prop(props, "title"))
	io.WriteString(w, `</h2>\n`)
	io.WriteString(w, `\t<p>`)
	pushup.EscapeHTML(w, pushup.Prop(props, "body"))
	io.WriteString(w, `</p>\n`)
	io.WriteString(w, `</div>\n`)
	return nil
}

Note the addition of the props map[string]any parameter. This would be accompanied by a Pushup API, Prop(), that would make thread-safe access to the map. (There may be other concurrency issues I haven't considered yet.)

The updated codegen for our index page would be:

package pages

import (
	"fmt"
	"io"
	"net/http"

	"example/myapp/components"
)

func Index(w http.ResponseWriter, req *http.Request) error {
	title := "My card title"
	io.WriteString(w, "<h1>")
	io.WriteString(w, "Hello, Pushup")
	io.WriteString(w, "</h1>\n")
	if err := components.Card(w, req, map[string]any {
		"title": title,
		"body": "My card body",
	}); err != nil {
		return err
	}
	return nil
}

Now the map that represents the props is constructed and used as an argument to the component function when called.

There's one final major piece of the component puzzle, which is that HTML elements can have children. My proposed solution is a simple special <children /> element, which is translated to a function closure in the codegen.

With child elements and allowing non-page .up files to live in any package, we can dispense with the special handling of layouts in the current version of Pushup, and simply let layouts be components.

.
├── components
│   └── ...
├── go.mod
├── layouts
│   ├── base.up
│   └── base.up.go
├── mypkg
│   └── ...
└── pages
    └── ...

We add a layouts directory, but just to emphasize the point, this is not a special or blessed directory/package in Pushup, components can live anywhere.

Our layouts/base.up looks like:

^declare title string
<!DOCTYPE html>
<title>^prop(title)</title>
<main>
	<children />
</main>

And we use it in our page as so:

^import "example/myapp/components"
^import "example/myapp/layouts"

^{ title := "My card title" }

<layouts.Base title="Pushup components example">
	<h1>Hello, Pushup</h1>
	<components.Card title=^title body="My card body" />
</layouts.Base>

Note the import of the layouts package. Again, our layout component can live in any Go package.

To the codegen. The layout base component would look like:

package layouts

import (
	"io"
	"net/http"
)

func Base(w http.ResponseWriter, req *http.Request, props map[string]any, children func()) {
	io.WriteString(w, `<!DOCTYPE html>\n`)
	io.WriteString(w, `<title>`)
	io.WriteString(w, pushup.Escape(pushup.Prop(props, "title")))
	io.WriteString(w, `</title>\n`)
	io.WriteString(w, `<main>\n`)
	children()
	io.WriteString(w, `</main>\n`)
}

Note the addition of the children func() parameter. This is a function closure that closes over the http.ResponseWriter so anything written to it would be streamed out when called.

Let's see what that looks like in the caller. The updated page codegen:

package pages

import (
	"fmt"
	"io"
	"net/http"

	"example/myapp/components"
	"example/myapp/layouts"
)

func Index(w http.ResponseWriter, req *http.Request) error {
	title := "My card title"
	if err := layouts.Base(w, req, map[string]any {
		"title": "Pushup components example",
	}, func() {
		io.WriteString(w, "<h1>")
		io.WriteString(w, "Hello, Pushup")
		io.WriteString(w, "</h1>\n")
		if err := components.Card(w, req, map[string]any {
			"title": title,
			"body": "My card body",
		}, nil); err != nil {
			return err
		}
	}); err != nil {
		 return err
	}
	return nil
}

Our previous content is now encapsulated in a function closure handed to the component to be executed at the appropriate time. Note that we also update the callsite for components.Card() to take a nil final argument, since we gave it no child elements when we used it in our index page.

The function signature of codegen pages should not change to accommodate components, it's still func(http.ResponseWriter, *http.Request) error, but since components are the units of reuse and composability, they need additional parameters than pages do.

This does draw a hard line in Pushup that pages cannot be components. If there is something in a page that wants to be reused across pages, extract it into a component.

There are additional design questions, such as passing any additional unused attributes through to the component and the component having a way to access them (this is commonly done to provide wrappers around base HTML elements like <img>). But that can hopefully be addressed incrementally after this change.

This was a long-winded post but I treated it like a think-aloud, because I wanted to make sure we get the composability of components right, and that only works if the syntax and the codegen go hand in hand.

[earthboundkid]

I like this a lot. First thoughts:

• Can ^declare be ^var? I do like having it at the top of the file just for the clarity it brings, vs. having it implicit. Or maybe if we can't use var, props with an S:

  ^props (
       title string
       body string
   )
  

• You could unify the signatures of pages and components by passing props as the http.Request context.Context (yuck, I know). For routing a page like /pages/:id, it would be convenient to be able to treat :id as a prop. The helper could look like prop(r, "name") or prop(r.Context(), "name").

[paulsmith]

Can ^declare be ^var? I do like having it at the top of the file just for the clarity it brings, vs. having it implicit. Or maybe if we can't use var, props with an S:

I like ^var. It wouldn't be a big lift to add syntax for a parenthesized list of var declarations ala Go, and if we did that I would extend that to ^import too.

You could unify the signatures of pages and components by passing props as the http.Request context.Context (yuck, I know).

I'm open to using context.Context since passing request-scoped values is part of its intended purpose. That would simplify the thread-safety question as well. I'll explore it. The main thing I don't like is that the ABI of Pushup functions is now not plainly visible. It would be harder to describe an interface that plugins and extensions could use because the simple type system wouldn't be sufficient.

For routing a page like /pages/:id, it would be convenient to be able to treat :id as a prop. The helper could look like prop(r, "name") or prop(r.Context(), "name").

Yes, currently this is the magical ^getParam() function but unifying props and URL params behind a single Pushup API should be a goal here.

[paulsmith]

I like ^var.

On second thought I think I prefer ^param.

^param title string
^param done bool

or

^param (
    title string
    done bool
)

They aren't really variables, and they are more like function parameters (although at the ABI level they aren't really).

[earthboundkid]

Are they params or args?

[earthboundkid]

Stack Overflow says params are definitions and args are actual values.

[earthboundkid]

Another project in the space: https://templ.guide