Dynamic Partials and Partial Collections through Coercion

[ghost]

I've just created a pull request over at janl/mustache.js#242

I am using this solution in a couple of projects and it addresses a number of concerns I often hear with regards dynamic partials in Mustache, please see ticket (janl/mustache.js#242) and documentation here:

• https://github.com/thelucid/mustache.js#dynamic-partials
• https://github.com/thelucid/mustache.js#partial-collections

Really, I urge everyone to have a play with it as it has cleaned up my templates no end.

Kind regards,

Jamie

[devinrhode2]

The proposed syntax:

base.mustache
{{#items}}
<p>{{>.}}</p>
{{^items}}

and

base.mustache
{{@items}}

Assume there is a key named 'partial' and then looks up that partial. The idea of dynamic partials is great, but I don't know about assuming a certain key to be present is the most developer friendly approach and the easiest to understand quickly. It could instead by {{>.partialName}} which, when the lookup for the literal key '.partialName' fails, it resolves .partialName to 'text' and looks that up.

It's clearly valuable, but I don't think it fits in the spec for 2.0

[ghost]

See my response here: janl/mustache.js#242 (comment)

As for it not fitting the 2.0 spec, there is currently no way to render partials dynamically which I see as a significant drawback. This is fine for smaller projects as you just add a boolean to each object in a collection such as is_text or is_image and have conditions in the template:

View

{
  items: [
    { type: 'image', url: 'Some URL', is_image: true },
    { type: 'text', content: 'Some text', is_text: true }
  ]
}

{{#items}}
  {{#is_text}}
    <p>{{content}}</p>
  {{/is_text}}
  {{#is_image}}
    <p><img src="{{url}}"/></p>
  {{/is_image?}}
{{/items}}

But... as soon as this goes above a couple of types, things get hard to manage. The whole premise of Mustache is to keep logic out of the template and this reads to me as a bunch of if statements.

Wouldn't it be nicer to be able to do:

base.mustache
{{@items}}

text.mustache
<p>{{content}}</p>

image.mustache
<p><img src="{{url}}"/></p>

So much cleaner, don't you think?

Sorry to labour my point, but I'm using this in production and it's changed my life :)

[groue]

@thelucid As usual, when one's code contains too many if, it means that you need a to inject a little object-oriented design. Let's replace your is_text, is_image "queriable" attributes by some "performing" attributes, and use Mustache lambdas:

{
  items: [
    { url:     'Some URL',  html: function() { return '<p><img src="{{url}}"/></p>'; } },
    { content: 'Some text', html: function() { return '<p>{{content}}</p>'; } }
  ]
}

base.mustache:
{{#items}}
  <p>{{{html}}}<p>
{{/items}}

And, if you absolutely want to use partials:

{
  items: [
    { url: 'Some URL',      html: function() { return '{{>image}}'; } },
    { content: 'Some text', html: function() { return '{{>text}}'; } },
  ]
}

base.mustache
{{#items}}
  {{{html}}}
{{/items}}

text.mustache
<p>{{content}}</p>

image.mustache
<p><img src="{{url}}"/></p>

And voilà, no need for a spec update!

[ghost]

@groue Ew, I'm sorry but this is exactly why the functionality is needed. Returning chunks of html from the view is in no way a good solution to the problem... I would hope I'm not alone on this.

This is in fact a perfect illustration as to why we need a clean solution to this problem, if hacks like @groue is suggesting are being suggested then something is fundamentally wrong.

[groue]

@thelucid You call it a "hack" because you see data as a model model object, not as a view model. I can understand you: Mustache has always been ambiguous on that subject: advocating simplicity, saying "just give me your model objects, I'll render them", while defending itself against missing features with arguments like "just add the missing keys to your view model, you shouldn't have provided a model in the first place anyway".

Mustache spec issues history is full on this kind of situations, if you read them carefully.

So, in a way, I gave an answer that is typical of the "Mustache 1" spirit. Be free to suggest improvements for Mustache 2. But dont' call "hacks" ways to use Mustache that you just don't happen to like.

[ghost]

@groue I'm sorry, I just don't see that returning a chunk of html from a view object (be it a view model or a data model) is a maintainable solution. Where do you draw the line, you may as well bypass Mustache completely and just interpolate strings.

I would much rather do:

base.mustache
{{@items}}

text.mustache
<p>{{content}}</p>

image.mustache
<p><img src="{{url}}"/></p>

[groue]

@thelucid Have you read my message? Check after the "And, if you absolutely want to use partials:" sentence.

[ghost]

@groue Yes, I have read the message and there are a few problems with the suggested partial solution:

1. It's ugly but that's just my opinion.
2. It's not DRY, you have two near-identical functions.
3. It's unmaintainable, i.e you will have to add a function for every new partial.
4. It's open to injection attacks since you are using the {{{html}}} approach.

Injection example:

{
  items: [
    { url: 'Some URL',      html: function() { return '{{>image}}'; } },
    { content: 'Some text', html: function() { return '{{>text}}'; } },
    { html: '<script>alert("I am hijacking your page")</script>' }
  ]
}

[groue]

It's ugly but that's just my opinion.

As much as the is_text and type attributes, and more OO.

It's not DRY

Single interface + Liskov substitution principle is no DRY for you? Gosh, I can't see how more DRY it could be.

It's unmaintainable, i.e you will have to add a function for every new partial.

Your solution must keep synced the type value and the partial name, which is just the same chore.

It's open to injection attacks since you are using the {{{html}}} approach.

Is it? open a new issue, and ask for a removal of the triple mustache.

Listen: I won't defend my proposal more than necessary. Actually, I'm quite sure you didn't know about the lambda before I introduced them to you. Think more about the tool, make sure you know every tiny details: Mustache is more rich that it looks. Maybe you'll find a solution that conforms to your standards with the spec as it is.

[ghost]

I'm at a loss as to why you're against a clean solution to dynamic partials.

As much as the is_text and type attributes, and more OO.

That is exactly why I am suggesting such a feature, it is dry and concise.

Single interface + Liskov substitution principle is no DRY for you? Gosh, I can't see how more DRY it could be.

You are still duplicating a function which is not DRY.

Your solution must keep synced the type value and the partial name, which is just the same chore.

Yes, but you are just specifying the partial to render, not a function that actually renders a chunk of html.

Is it? open a new issue, and ask for a removal of the triple mustache.

The triple mustache doesn't escape the value by design, that's what it's for and how it works. So yes, your example is open to injection attacks... try it if you don't believe me:

View

{
  html: '<script>alert("See, I have control");</script>'
}

Template

{{{html}}}

I did know about lambdas before you mentioned them (that's a little patronising, especially given your lack of understanding when it comes to the triple mustache), I just don't see them as a good fit for this particular problem.

We'll just have to agree to disagree on this one.

Maybe you'll find a solution that conforms to your standards with the spec as it is.

There is currently no way to render dynamic partials other than the suggestions we have already exhausted here, hence this ticket.

[groue]

I'm not against your solution. I just wanted to make sure everybody knows that there are already existing solutions. Usually, people ask for a feature because they don't know their tool enough. Sorry if I have patronized.

About the triple mustache: they allow injection in all situations, not only in the one we are talking about here. What I meant was that the topic of injection should hence not pollute this topic, and that you can ask for the removal of the triple mustache in some other issue.

[devinrhode2]

I think @groue has a great point. This is already possible, therefore it's best not to add a feature for it.

I vote no.

[ghost]

It's possible to drive a nail into a block of wood with a mallet but it's not necessarily the best tool for the job.

[devinrhode2]

I think letting{{>string}}first lookup the partial named 'string' - and then fallback to looking up the value of the string would certainly be acceptable. This would work perfectly without changing anything, only at a point of failure does it kick in.

And with @bobthecow's suggestion of moar dot notation hotness, you could do{{>.string}}

[ghost]

Yes, that's not too bad, what about the {{@collection}} shorthand?

[ghost]

The only potential problem with that solution is that it's a little ambiguous i.e. the same syntax does two different things. Having a separate syntax makes it easier to see what is going on.

[bobthecow]

That should be explicit, if that's what you want. Otherwise you're opening yourself up for exploits. Maybe use a "variable dereferencing" type token? {{> *string }}

[ghost]

What about using the @ symbol for everything related to dynamic partials e.g.

{{#items}}
{{>@partial}}
{{/items}}

or shorthand that assumes a partial key:

{{@items}}

I would personally prefer it being implicit, going with a convention whereby a partial key is always assumed when using dynamic partials i.e.

{{#items}}
{{@}}
{{/items}}

and shorthand:

{{@items}}

The @ symbol then means "render an object or collection of objects using their partial keys". Of course, the key doesn't necessarily have to be partial, perhaps > would be a better fit to denote a partial? ...hmm, maybe not, partial does what it says on the tin.

Has anyone tried a few examples using my patch to mustache.js? The {{@collection}} shorthand really is a joy to use, maybe that's all that's needed and we abandon the longhand.

[devinrhode2]

Not a fan of using@ :(

@bobthecow Could you give an example of being exploited?

We could also use another interesting syntax:

{{> {{string}} }}

It seems the only way we dodge ambiguity is by having a special char or marker that means lookup the value of this string not the name 'string'

[ghost]

%? Looks a bit like repeating partials if you tilt your head to the left :) {{%items}}.

In response to the dereferenceing thing, & is commonly used for this in programming languages e.g. {{> &thing}}.

[devinrhode2]

{{> {{string}} }}is kinda interesting because it presents the idea that anything can be dynamic lookup. *could be just as universal.

[devinrhode2]

Essentially we're talking about the same thing - some symbol to 'dereference' the variable. I really like @bobthecow's* because that's exactly what it means in any non-garbage collected language. (C, C++, etc..)

[ghost]

Not keen on the taches in taches thing, wouldn't that choke the parser?

I'm currently leaning toward:

{{#items}}
{{> &partial}}
{{/items}}

with a shortcut of:

{{%items}}

or

{{@items}}

[devinrhode2]

The{{> {{string}} }}syntax would require a more intelligent parser... compiling it would also be a bit more complicated too, I'm not very big on it.

I wonder how a lambda/filter/helper could assist you with a shorthand version...

What does everyone think of having a deference operator, like*?

[ghost]

I'm not against using * although prefer & as it's more Rubyish and reminds me of things.map(&:name) which does exactly what we want.

With regards the shorthand version, in practice this is more useful than the longhand. In fact, I'm using this setup in a couple of projects and have not actually had a need for the longhand version over the shorthand. The resulting templates are super clean and concice.

Let's say you have a page with main content and sub content, and each contains a collection of objects, this becomes (using @ here but could use % instead):

<html>
<body>

<div class="main">
{{@main}}
</div>

<div class="sub">
{{@sub}}
</div>

</body>
</html>

I would be more interested in the shorthand making it in over the longhand, be it {{%items}} or {{@items}}. It's a super simple patch, only requiring a few lines of code in return for powerful dynamic partial functionality.

[groue]

@thelucid : are you proposing to reserve a special key partial for your shorthand notation?

[ghost]

Not to necessarily reserve it but the convention would be that when rendering a collection of partials, use the partial key. Have a play with my fork of mustache.js, you'll see how nice it is to use, it just works.

I do quite like the {{> *partial}} or ideally {{> &partial}} notation too, however in practice the shorthand gets far more use.

[groue]

@thelucid : Don't play on words: you are indeed "blessing" the partial key it for some special purpose of your own convenience.

Of course, you are feeling it is a bad idea. And you are right: it's a big threat on compatibility. Each time a special key is added to the spec, one has to bump the Mustache version by a major increment (Mustache 2.0, 3.0, etc.) since it could break existng code.

You can give up this idea right away - it will never enter the spec.

[ghost]

I'm not playing on words, this functionality is backwards compatible, if you took the time to think about it or try out the patch before mouthing off you would see that. I'm just throwing some ideas out there that I have found to be hugely beneficial in my own projects and that I thought others could benefit from.

This ticket has thrown up some interesting discussion around the {{> *partial}}/{{> &partial}} syntax so I believe it to have been worthwhile. Far more worthwhile than your suggestion of returning a chunk of HTML.

Your negativity is getting on my nerves and don't tell me what or what not to do. Go find another ticket to troll.

[dasilvacontin]

It was interesting reading this. Almost two years later, heh.

[spiffytech]

bump

Any movement on a v2 spec that includes this feature?

[hydrastro]

bump

Any movement on a v2 spec that includes this feature?

^

[jgonggrijp]

To anyone still hoping for a feature that involves detecting a hard-coded name such as partial in the context: I don't see that ever happening. Not that I'm the one to decide, but I think too many people are against it (for reasons that I agree with).

To anyone still looking for a way to dynamically select a partial based on context, I suggest the following approach based on feature detection.

Data:

{
  items: [
    { content: 'Some text' },
    { url: 'Some URL' }
  ]
}

Templates:

{{!base.mustache}}
{{#items}}
<p>{{>polymorphic}}</p>
{{^items}}

{{!polymorphic.mustache}}
{{#content}}
    {{>text}}
{{/content}}
{{#url}}
    {{>image}}
{{/url}}

{{!text.mustache}}
{{content}}

{{!image.mustache}}
<img src="{{url}}"/>

I know, I know. This looks suspiciously like the code that @thelucid was trying to avoid. The base template is not dynamically selecting a partial at all, to the contrary. It's expanding the same one every time, polymorphic, and that just contains the if-else again that @thelucid wanted to get rid of. You might think that I just moved the problem. But consider the following.

• polymorphic.mustache is performing dynamic partial selection. And nothing else. So we have a clean separation of concerns.
• base.mustache no longer contains the if-else; it is as clean as the "fine-grained control" version of what @thelucid was suggesting.
• With this approach, you don't need to tag your data in any way. No hard-coded partial property that names a partial, no html property with a lambda. Just the actual data.
• By extension, the code that prepares the data no longer needs to contain an if-else construction for injecting the tags. So I have actually moved the solution. Arguably to a better place, since Mustache templates are portable while your data-preparing code is not.

Last but not least: this already works, even without the lamdas extension, so you don't need to wait for anyone to change a specification in highly controversial ways.

[ghost]

• polymorphic.mustache is performing dynamic partial selection. And nothing else. So we have a clean separation of concerns.
  Yes, it is performing dynamic partial selection, but it's not an ideal solution for all cases: in your example your dynamic loading is bounded to your data structure / template, which is fairly limited

In my template I want to dynamically load contents of a page, following your example this is what my code would look like:

{{!template.mustache}}
{{>header}}
{{>content}}
{{>footer}}

{{!content.mustache}}
{{#mainpage}}
  {{>mainpage}}
{{/mainpage}}
{{#aboutpage}}
  {{>aboutpage}}
{{/aboutpage}}
{{#foobarpage}}
  {{>foobarpage}}
{{/foobarpage}}
{{!and so on... for all my pages in the database}}

I could use lambdas to solve this but I find it kinda tedious (not to mention the fact that this way I'm still tagging the data).
(Also I have no idea how to do it).
Or I could dynamically generate templates with a wrapper language
Wrapper:

<?php
$content_file = "foobar.mustache";
ob_start();
include("template.mustache.php");
$content = ob_get_clean();
// (mustache rendering here)
?>

template.mustache.php:

{{>header}}
{{><?=$content_file;?>}}
{{>footer}}

Pretty bad right?

you don't need to wait for anyone to change a specification in highly controversial ways.

We don't want to change a specification, we just want to add a feature, a new token.
Here's the proposal:
Data:

{
  content: "mainpage"
}

Templates:

{{!template.mustache}}
{{>header}}
{{*content}}
{{>footer}}

Pretty simple, right?
This would make life easier for a lot of people.
PS. I didn't know which operator to choose, so I just picked up *, we can change it.

[gasche]

I'm new to this discussion and the top post has mostly dead links. Can proponents of the change point to a post that clearly explains the feature being proposed?

I have the impression that some use-cases are in common with "template inheritance" which was finally accepted as an optionally-supported feature (see #125):

base.mustache:

header...
{{ $content }}
footer...

main.mustache:

{{< base }}{{$content}}
main page content...
{{/content}}{{/base}} 

aboutpage.mustache:

{{< base }}{{$content}}
about page content...
{{/content}}{{/base}} 

(This can be extended to pass a title to the base template, etc.)

[ghost]

some use-cases are in common with "template inheritance" which was finally accepted as an optionally-supported feature (see #125)

Yes, you're right, some use cases are common.
What we're proposing is to solve the problem the other way, "forward" (with "includes"/dynamic partials) rather than "backwards" (with inheritance).
It's way more clean.

Data:

{
  content: "aboutpage"
}

Templates:

{{!base.mustache}}
{{>header}}
{{*content}}
{{>footer}}

{{!main.mustache}}
main page content...

{{!aboutpage.mustache}}
about page content...

[jgonggrijp]

@gasche The originally proposed feature is described in detail in janl/mustache.js#242.

@anomal00us You are proposing a different feature, although it serves the same purpose and the suggested syntax may have been inspired on the discussion. To be honest, I find your proposal more sensible than the original. The syntax is less surprising and it doesn't involve a hard-coded special name.

I must point out that this is the spec repository. If you only want to add a feature to one particular implementation, you should probably take the discussion to that implementation. There is nothing wrong with discussing it here (again, as I said, I actually like what you're describing), but anything discussed here is about changing the specification, by definition.

[jgonggrijp]

Following up on what I just wrote: I realized that the syntax {{*key}} could theoretically be generalized to also supporting similar constructions with parents or even sections. This is why the discussion contained more radical suggestions such as {{> &key}} and {{> "name"}}. I pondered for a moment whether dynamic parents should be possible, and if so, how.

I ended up deciding that I like the language more in the minimalist form that it has today. As far as I'm concerned, there is still room for dynamic partials with a {{*key}} syntax, but we shouldn't take the dereferencing any further than that. There doesn't need to be a dynamic parent for parity, because as @anomal00us pointed out, inheritance and dynamic partials are to some extent alternatives. I certainly wouldn't want to complicate the internals of tags in order to support key dereferencing more generally.

[ghost]

I must point out that this is the spec repository. If you only want to add a feature to one particular implementation, you should probably take the discussion to that implementation.

Yes, I know. I already added this into my implementation, that's also why I am here.
It would be cool if this will be merged into the spec!

I realized that the syntax {{*key}} could theoretically be generalized to also supporting similar constructions with parents or even sections.

Can you make an example?
Do you mean something like this: (?)
Data:

{
  items: [
    { content: 'header' },
    { content: 'mainpage' },
    { content: 'footer' },
  ]
}

Template:

{{!template.mustache}}
{{#items}}{{*content}}{{/items}}

but we shouldn't take the dereferencing any further than that.

I totally agree.

There doesn't need to be a dynamic parent for parity, because as @anomal00us pointed out, inheritance and dynamic partials are to some extent alternatives. I certainly wouldn't want to complicate the internals of tags in order to support key dereferencing more generally.

So if we're writing a pull request for this we're going to keep things simple and allow deferencing only on simple data?

[ghost]

Can you make an example?

Ah I think I got it. Making * a deference operator and allowing to combine it with all the other tags, right?

{{#*deferencedarray}}
  {{&*deferencedunescaped}}
{{/*deferencedarray}}
{{**doubledeference}}
{{<*deferencedparent}}

Well, I don't feel like this is a good idea.. it's probably better to keep things simple.
I may be wrong but I can't come up with ideas where you could possibly need to deference a section.
Also implementing this would imply to change a lot of things, first of all the tokenizer, which will need to detect the * operator, then the abstract syntax tree generated by the parsers: we need an extra field to tell whether something is deferenced or not, and at last the compiler, which will need to "deference" the stuff.
Not to mention all the existing specifications.

So yeah, my proposal is to have * as dynamic partial only (stand alone) operator.

[jgonggrijp]

@anomal00us

So if we're writing a pull request for this we're going to keep things simple and allow deferencing only on simple data?

Honestly, I am probably not going to write a pull request for this, at least not soon, because I haven't implemented the feature myself (yet). But if you were to write one, I would definitely be in favor of keeping it simple, allowing only single-level dereferencing specifically for the purpose of interpolating plain partials (i.e., not parents). Basically, {{*key}} being a composition of {{key}} and {{>value}}. Also, I would be happy to review it.

[ghost]

Great, I'm planning to write it soon.

@jgonggrijp do you know if there is a tool for getting the json from the yml, or did people here write everything by hand?

[ghost]

Okay I made PR #134

[jgonggrijp]

@anomal00us There is a command for it, which is mentioned in the Readme if I'm right. I suppose you already found it.

[bobthecow]

Probably a bad idea, but what if we spec general purpose dereferencing instead?

{{ *variable }}
{{# *list }}
{{> *template }}
{{/ *list }}

{
  variable: "a",
  list: "b",
  template: "c",
  a: "foo",
  b: [
    {name: "bar"},
    {name: "baz"},
  ],
  c: "somepartial"
}

{{! somepartial.mustache }}
{{ name }}

Results in…

foo
bar
baz

[jgonggrijp]

@bobthecow

Probably a bad idea, but what if we spec general purpose dereferencing instead?

It crossed other people's minds before, so at least it's not stupid. It was first discussed in 2012, from #54 (comment) onwards. That never led to anything (at least not in the spec). It was again discussed by me and @anomal00us just a few days ago, from #54 (comment) onwards. Bottom line of the latter discussion is that we both preferred not complicating the tag contents. See also #134.

For me personally, Mustache's syntactic simplicity is one of its main selling points. A general purpose dereference operator would arguably make the language even more powerful than it already is, but I think the added value is questionable for all tags except partials (where it enables a template equivalent of polymorphism through tagged unions, similar to how parents and blocks enable polymorphism through inheritance). I feel it would also start a slippery slope towards a bloated language like Handlebars or Jinja. Besides dereferencing, can we apply other general purpose operators within tag contents? Can we dereference multiple levels? Etcetera... it's a direction I rather not explore.

On another note, I would be highly interested in your opinion on #131.

[bobthecow]

ahahahahah i clicked through from #134 and didn't expand all the collapsed past conversation here and missed that I proposed this exact thing a decade ago when it first came up 😛

#54 (comment)

[bobthecow]

The question of dynamic tag names has come up for more than just partials in the past. To me it feels far more consistent to add a general purpose dereference operator (while I do acknowledge the slope we start slipping down) than to introduce a special form of partial that pulls the template name from the view model.

[jgonggrijp]

My question is this: can't we make a hard choice to not start slipping down that slope? We could have dereferencing in other ways. For example, we could give lambdas (read-only) access to the context stack through an additional argument.

[agentgt]

I agree with @bobthecow a general purpose dereference operator might be useful when trying to reference things that would normal iterate or be conditional especially in the context of #135 .

That is the only need of dereferencing in my implementation since we don't support dynamic partials (or templates in general) is to pass lists and booleans around to lambdas.

Currently the easiest solution for my case is to use virtual keys instead of a new sigil. e.g. to reference a list instead of iterating I do the following:

{{#someList.this}}
{{#someLambdaExpectsListOnTopOfStack}}
{{/someLambdaExpectsListOnTopOfStack}}
{{/someList.this}}

this being the special virtual key that would kind of be equivalent to {{#*someList}}.

I am starting to get the idea that a large amount of problems that should be easy in mustache can be solved with virtual keys notable index information (e.g. -last, @last a boolean virtual binding that indicates if you are on the last item in a list).

Virtual keys are much easier to add to existing implementations than a new sigil albeit I'm not sure if that holds up to dynamic partial needs (to be frank I'm like @gasche in that I'm not sure if I even follow all the ideas in this thread).

Otherwise I think this can quickly become what Handlebars is which I don't think we should go down that path.

[jgonggrijp]

@agentgt In my particular implementation, adding virtual keys like @last would actually be much harder than adding new sigils. That's because my tokenizer and parser already have generic facilities for sigils, but there is nothing yet to handle special reserved key names (other than the entire key being the implicit iterator .).

Power lambdas would make it possible to add your own virtual keys, but we already discussed that in #135. I understand it doesn't help in your case, and I figure that your implementation probably does already inspect key names inside the template, since it aims to typecheck everything. Mustache was clearly conceived as a dynamically typed language.

For context, dynamic partials did make it into the spec eventually, in #134.