[templates] A declarative JavaScript templating API

[justinfagnani]

Related to many template, template instantiation, and DOM parts discussions (but especially #777, #682, and #704) I wonder if we should add a JavaScript-based templating API?

I think JS-based templating would be extremely useful and help with a lot of issues and questions:

• Ergonomics and convenience: Templating in JS is an incredibly common thing to do on the web, yet there's no built-in API for it. Developers have often asked for a built-in declarative way to build and update DOM.
• Safety: We can provide a safe way of interpolating untrusted values into the DOM protecting against XSS.
• Clarify the MVP for DOM parts: Templating in JS can cover a lot of open questions for template instantiation, like those brought up in [templates] How featureful should the default processor be? #682, like what the expression syntax is, how to do conditionals and looping, how scoping works, etc.
• Performance: As shown by several JS libraries, the DOM Parts model can be used to construct very fast DOM renderers. Fast, stable DOM updates as a platform feature would be a performance feature.
• Improve the initial value delivered by DOM Parts: The DOM Parts model can be used for fast templating, but given some Chrome prototypes a native implementation might not be a large performance win over similar userland implementations. There is still a lot of value in offering built-in templating, however. As @rniwa said about template instantation it "provides a mechanism to easily create a template instance without having to import a third party library"
• Code size: It would also reduce code size for those that use the native feature, and possible higher-level libraries that build on top of it. The tagged template literal syntax can be a compile target for other popular template syntaxes like JSX, Mustache, and HTML-like template like Vue, Angular, Svelte, etc.
• Interoperability: If multiple user-facing template systems use this API, they could interop, ie render a JSX template into a tagged-template literal, a Mustache template into JSX, etc.

Idea

Add an HTMLElement.html template tag and an HTMLElement.render() method that would allow describing the desired DOM structure in JS, along with bound data for interpolation and updating.

Templates would be written as tagged template literals using the HTMLElement.html tag:

const {html} = HTMLElement;

const renderPost = (title, summary) = html`
  <h1>${title}</h1>
  <p>${summary}</p>
`;

Templates would be rendered to an element with the HTMLElement.render() method:

document.body.render(renderPost('Hello World', 'This is a template'));

DOM rendered from a template can be updated by re-rendering with the same template, preserving the static DOM and only updating the bindings that changed. This would be done with DOM Parts.

// Only updates the <h1> element created from the template. No other DOM updates.
document.body.render(renderPost('Hello Templates', 'This is a template'));

Features

Detailed features can be worked out later, but we know there are some basics that need to be covered to make a useful DOM templating system:

• Various binding types: child, attribute, property, event listener, element.
• Auto-escaping of binding values
• Fast updates with minimal DOM mutations: re-rendering a template should only update bindings that change.
• Composition: the ability to render a child template or array of child templates into a child binding
• Extensibility: Many use cases require some customization of how a binding is processed
• Hydration: There needs to be a way to update DOM that has been pre-rendered with SSR, SSG, etc.

Prior art

The general shape of this API has been popularized by several userland libraries including hyperHTML, lit-html, and FAST.

[EisenbergEffect]

Immediate questions I have:

• How does this integrate with signals?
• Can fine-grained updates be accomplished or is re-rendering required?
• How do we expect a server DSD version to be generated? How do we expect it to resume on the client?
• Can we have an in-HTML equivalent that doesn't require JS? Pretty please.

These days, I'm highly focused on declarative patterns that don't require JS for initial render. I know we want to move the ball forward for templating, but I'd still prefer to move it forward on the HTML side rather than in JS. For most of the clients I'm consulting with, this will not help their key scenarios at all. More progress on declarative approaches is what they need.

---

Note that in FAST, things work a good bit differently, even though the tagged template literal part looks the same. In FAST, state is backed by signals and bindings are live. So, there is no re-render step. Rather, when a signal value changes, FAST makes a fine-grained update to precisely the DOM node that has changed (batched with any other changes that happened in the same micro task).

[justinfagnani]

I was just about to add a note about signals. I think this is a great place to start adding signals integration to the DOM.

If you use a signal in a binding, that binding should be bale to update without a full re-render.:

const count = new Signal.State(0);

document.body.render(HTMLElement.html`<p>count: ${count}</p>`);

count.set(count.get() + 1);

But not every bit of state in a page is or ever will be signals, so I think the ability to have a fast re-render is necessary.

[justinfagnani]

How do we expect a server DSD version to be generated? How do we expect it to resume on the client?

I think the key there is defining the hydration protocol - which is very closely related to SSR'ed DOM Parts. Once you have an HTML syntax that can be hydrated, then servers can generate that HTML. JavaScript servers can interpret the template literals to do so.

[justinfagnani]

Can we have an in-HTML equivalent that doesn't require JS? Pretty please.

I just think that this is a lot harder to do with all the features that real world templates require. The nice thing about starting in JS for full-feature templates is that the underlying DOM Parts / Template Instantiation layer can be much simpler.

Otherwise will have to figure out expression syntax and semantics, scopes, control flow, within the context of HTML up front. With a JS layer, JavaScript takes care a lot of that for us. I think then we could continue with the HTML syntax.

[JRJurman]

These days, I'm highly focused on declarative patterns that don't require JS for initial render. I know we want to move the ball forward for templating, but I'd still prefer to move it forward on the HTML side rather than in JS.

Agreed on this point. While I think this is common enough that it probably is worth baking into the platform, I do feel like I'd rather see us make strides to do HTML building in HTML.

On a separate point, I wonder how much we would want to lean on existing Template parsing functions, like parseHTMLUnsafe or setHTMLUnsafe. While potentially not as feature rich as we might want, it probably isn't hard or terribly contentious to bake in a cleaner interface for these functions (even if that means making it called unsafeHTML).

If this should be very different from those functions, it might be worth calling out in the beginning of the proposal, and in what ways. I worry about the number of ways we have to create DOM from string templates, and it's worth calling out how these might be different (rather than adding to an existing one).

[justinfagnani]

While having the full initial HTML for a page is great, the DOM includes a lot of imperative, dynamic mutation APIs and is still sorely missing a safe, fast, declarative DOM creation/update API.

This is a natural compliment and alternative to innerHTML, createElement(), etc. And it does not preclude features to make server-rendered HTML more useful.

[rictic]

I know that Google's internal security team has replaced a bunch of innerHTML calls in our internal codebase with Lit to improve security. Putting this in the platform seems like a solid thing to point people to when they'd otherwise reach for innerHTML, would greatly improve security and (for updates) performance too.

[NullVoxPopuli]

I have a concern, but I'm not sure where it belongs -- but with reactive rendering w/ Signals -- and maybe this is something that a library/framework would use to solve my concern, but -- given this example:

const count = new Signal.State(0);

document.body.render(HTMLElement.html`<p>count: ${count}</p>`);

count.set(count.get() + 1);

count is rendered reactively, inherently -- which is great! This isn't where my concern is tho. Say you have a situation like this:

class Animal {
  givenName = new Signal.State();
} 

class Cat {
  get name() {
    return `${this.givenName}, the cat`;
  }
}

class Dog {
  name = () => {
    return `${this.givenName}, the dog`;
  }
}

Assuming we use the same rendering function from the example, we can't reliably or consistently render reactive data, unless we pass on the responsibility to handle all situations to the developer:

let cat = new Cat();
let dog = new Dog();

document.body.render(HTMLElement.html`
  <details><summary>Your Cat</summary>
    ${() => cat.name}
    If we don't specify a function, ${cat.name} is burned in and non-reactive, 
    as the value of name is eagerly evaluated.
    This would require that the processing of HTMLElement.html call functions, 
    if encountered.
  </details>
  
   <details><summary>Your Dog</summary>
    ${dog.name()}
    Not everyone is on board with getters when it comes to reactivity, and that's fine,
    but the other way of accessing *maybe* reactive values is a function,
    this also burns in the value, and would require non-invocation, e.g.: 
    ${dog.name}.

    This reveals in an inconsistency and footgun between:
      burned in:
         ${cat.name}
         ${dog.name()}
      reactive, provided HTMLEelment.html calls all functions:
         ${() => cat.name}
         ${dog.name}
  </details>
`);

// some time later
cat.givenName = 'Alexander';
dog.givenName = 'Sir"

Now, what I would prefer, is taking a templating system further, and not using tagged template literals at all -- (though, I don't immediately see a reason why what I'm about to show couldn't be implemented with the above).

let cat = new Cat();
let dog = new Dog();

document.body.render(<template>
  <details><summary>Your Cat</summary>
    {{cat.name}}
  </details>
  
   <details><summary>Your Dog</summary>
    {{dog.name()}}
  </details>
</template>);

// some time later
cat.givenName = 'Alexander';
dog.givenName = 'Sir"

In this example, both ways you'd access the value in JS would inherently be reactive -- there is no need to know about any footguns, because there would be none (unless someone points out something to me haha). This removes the need to worry about what is reactive and what is not, no need to ${() => value} everything due to not knowing if something could be reactive (A library may always be able to make something reactive in the future, and you wouldn't know!)

This uses https://wycats.github.io/polaris-sketchwork/content-tag.html which is very pre-proposal, but is a variant on something that the corner of the JS ecosystem I'm in has decided to use for its rendering.
(But again, I guess it could be built on tagged-template-literals, and be more of a sugar)

anywho, my 2c

[au5ton]

class Cat {
  get name() {
    return `${this.givenName}, the cat`;
  }
}

class Dog {
  name() => {
    return `${this.givenName}, the dog`;
  }
}

@NullVoxPopuli From what I understand, this would be an incorrect usage of Signals. I believe you're supposed to use new Signal.Computed(...) for such a scenario.

See: https://github.com/tc39/proposal-signals?tab=readme-ov-file#example---a-signals-counter

[brunnerh]

Usage of Computed with signals is not strictly necessary.
The main advantage they provide is memoization of expensive calculations.

[justinfagnani]

@o-t-w JSX would require major changes to JavaScript via TC39. A tagged template literal based system wouldn't.

But we've seen compilers that transform JSX to tagged template literals before, so frameworks that use JSX could target this API, or if JavaScript ever does add a JSX-like syntax, it could be made to work with this template system.

[justinfagnani]

@NullVoxPopuli if there's any appetite for this idea, we can go into a lot of detail about how signals integration would work.

At a high level for now, I think that there are a few good spots for integration:

1. At the DOM Part level: Individual parts accept signals and update when the signals change. To feed in a computation that uses signals, you'd first wrap it in a computed to tell the DOM Part to watch it.
2. By wrapping a template expression + .render() call in an effect so that any signal access is tracked and updates cause a re-render. By doing the effect outside/around of the .render() call, it lets the effect setup use the scheduling it needs - microtask, framework scheduler, etc.
3. An integration of (1) + (2) where a DOM Part holding a signal defers its updates to an external scheduler, so that they can use the same scheduler as in (2). This could let pin-point updates happen without a full template re-render, or be batched with a full re-render if one is already scheduled. This kind of deferring scheduling would also work with a component system that controls the effect and scheduler.

An API to pass in the external scheduler could take the form of extra options to .render(), though the shape would have to be figured out - is the option a schedule callback, or a Watcher, or is there an event?

We do something similar to this with lit-html where a RenderOptions (maybe it would be better named a RenderContext though) object is passed down the tree of templates as they are rendered.

Re the second syntax you show, I'm trying to propose something here that doesn't require any changes to JavaScript, but could use them if they ever happen in the future.

[EisenbergEffect]

Getting back to the issue that @NullVoxPopuli raises...what if we could introduce the idea of lazy evaluation to tagged template literals? For example, ${value} is the same eager evaluation that we have today, but {value} could be lazily evaluated. Essentially, it would get converted to something like ${() => value} under the hood. It's maybe too late to change tagged template literals, as that would be a break, but maybe a new literal type with three backticks or similar could enable this. Or what if the tagged template function itself could opt into this interpretation somehow?

Obviously, this then becomes a TC39 thing, which complicates matters. But it seems to me that tagged template literals could be a lot more powerful if that part of the language were explored more.