Tower Web templating engine

[carllerche]

Tower Web is a web framework built on top of the Tokio stack. It provides a
macro based API. It currently supports HTML templates using an existing
community handlebars library. However, better HTML templating, both in terms of
features and performance, can be achieved by implementing a custom library.

Expected outcomes

• A handlebars based templating library is implemented.
• Templates are compiled ahead of time using a procedural macro.
• Templates are rendered into a rope data structure.
• A rope data structure is implemented.

Skills

• Rust
• Parsing

Difficulty level

Medium

[BlinfoldKing]

Hi, i really interested in this project, i really love to learn about parsing, and also currently learning rust, so seeing this project really excite me, but i don't really have any experience in low level and networking stuff, do i need to learn those beforehand to apply for this project?

and is there anything that i should know or do?

[carllerche]

@BlinfoldKing Thanks for the interest.

I think the details of this project are a bit in flux. The question is now whether Askama can be adapted to fit the requirements or whether a new templating library should be developed from scratch.

The requirements of a templating library for tower-web are as follows:

Convention over configuration

As much as possible, things should just work "out of the box" using a set of conventions. When using templates from the context of tower-web, there is more available information than using the library standalone. This is a design goal that should influence the other requirements.

Templates are compiled at project build time.

This is already supported by askama. However, Askama currently requires each template struct to specify the path to the template file. Instead, tower-web's templating should be conventional.

For example, lets assume that the "web" code lives in a web module in the crate. The crate layout would be:

+ src
|- web
| |- mod.rs
| |- users.rs
|- main.rs

Say that in mod.rs there is:

#[derive(Response)]
struct Index { ... }

and in users.rs there is:

#[derive(Response)]
struct Get { ... }

#[derive(Response)]
struct Update { ... }

Given this, the crate layout would be:

+ src
| ...
+ templates
| |- users
| | |- get.html
| | |- update.html
| |- index.html

Template inheritance

Askama already supports this. However, in the spirit of convention over configuration, I would prefer if the parent template does not need to be specified in the child.

Performance

Askama is already pretty fast, but one thing that I would like to experiment is avoiding having to copy any template data around. Currently, I believe Askama renders using std::fmt::Display? However, doing this requires a lot of copying.

I would like to see the ability to render into a T: BufStream. This would allow rendering the template as "chunks" so that any static template components (generated during template compilation) can remain static.

By rendering into a T: BufStream the actual copying can be punted until the data is consumed. It might even be possible to avoid all copying at all by writing to the socket using writev. Or, if the response must be deflated, the deflate step can deal with copying.

Asynchronous rendering

This goal is a bit tricky, but it would be worth exploring.

Is it possible to render the template without all the data needed for the full template to be available yet?

For example, as a simple example: A user has many blog posts. To render this page, you only need to know if the user ID exists (to decide if the HTTP response should be 404 or not). The template can start rendering before all the blog posts have been loaded. The advantage of doing this is that assets like CSS are included in the head and can start being loaded by the browser.

This ties into rendering into a T: BufStream as that type is asynchronous. The data struct passed in would need to be able to include futures / streams and the templating lib would need to know how to wait on them.

cc/ @djc

[carllerche]

@BlinfoldKing To answer your actual questions :)

but i don't really have any experience in low level and networking stuff, do i need to learn those beforehand to apply for this project?

There is no networking component to this project. As for "low level", that probably depends on your definition of low level. This project would require byte manipulation and being sensitive to avoiding copying data.

and is there anything that i should know or do?

Probably a good idea would be to review Askama and how tower-web currently does templating.

[BlinfoldKing]

okey i will check on Askama, and maybe will make some kind of Proof of Concept on my take of the template engine.

is there anything that i could contribute to learn on how currently handlebars works on tower?

[carllerche]

@BlinfoldKing one initial step could be to add support for askama in tower-web via a feature flag. So, follow the same strategy as done w/ handlebars.

[djc]

@carllerche it's still a little unclear to me how inheritance would be defined by "convention".

Also, it's not clear to me whether it's even possible to support the idea that a struct called ::users::Get could generate code to automatically find the users/get.html template. Procedural macros do not have access to the file name or module name of the incoming token stream.

[BlinfoldKing]

would it be possible to use this? @djc

[djc]

I don't think it would work. That path would get expanded late during compilation, whereas we'd need it in the early phases of compilation (when the procedural macro code is running).

[carllerche]

Also, it's not clear to me whether it's even possible to support the idea that a struct
called ::users::Get could generate code to automatically find the users/get.html
template. Procedural macros do not have access to the file name or module name
of the incoming token stream.

Well, never say never... the macro could scan the entire source directory 😆

I guess Askama's feature is type full type safety at compile time, but if I had to pick, I would opt for less boilerplate at the expense of more dynamism.

[djc]

Your boilerplate is my "explicit is better than implicit". 😛 But I think you're also looking for great performance, which may be somewhat at odds with dynamism? Maybe you should investigate Tera or Ramhorns in more detail and see if they fit better with your ideal design. You did write "Templates are compiled ahead of time using a procedural macro." as one of your very first design points, for which Askama is by the most popular option by far AFAICT.

[BlinfoldKing]

sorry for no update in a while, having my mid-term test :/

Well, never say never... the macro could scan the entire source directory 😆

will it be really slow when there are too many template? and can you give me any refrence to this, i must've miss this
i still researching about what kind on convention to use,
so far i like the way askama handle things by using Template traits, maybe being explicit ain't that bad

But, i do have an idea, a standalone template engine could be implemented explicitly, but the integration with Tower should be seamless, for example we could bind a template by it's route on tower, a /login route should automatically render /login.html

and i currently trying to implement askama on tower web, a bit stuck stuck in here too tho, i can't really use the handlebar strategy, handlebar use a registry on the other hand askama using trait. still trying to find a way around

[BlinfoldKing]

maybe i'm gonna check on Tera and Ramhorns too

[djc]

Did you check the existing integrations that Askama has? For example, see:

https://github.com/djc/askama/blob/master/askama/src/lib.rs#L468
https://github.com/djc/askama/blob/master/askama_derive/src/generator.rs#L203