RFC - cargo templates

[jsjoeio]

🖼️ Rendered

📝 Summary

RFC for the following new feature:

Add the ability to reference a template when running cargo new or cargo init

Example using a template from the rocket crate:

cargo new my-rocket-app --template rocket/hello-world

💁 Additional Notes

This RFC went through two Pre-RFCs, which can be viewed here:

• v1
• v2

[jsjoeio]

Thanks for the fixes @XAMPPRocky ❤️ I copied this from the internals forum (hence why it had some funky markdown syntax)

[ckaran]

@jsjoeio It looks like a good start! I'm assuming that you're planning on fleshing out the details before the RFC is accepted, correct? E.g., a more precise description of what a template is, and how it works.

[jsjoeio]

(Moving our discussion to the main comment thread so that it's more visible).

My apologies @ckaran - now I see the hole you're pointing out. I think this RFC does not sufficiently answer this requirement from the template:

It is reasonably clear how the feature would be implemented.

Which seems to be around what you're encouraging we do. Thanks for calling that out. (I naively overlooked that thinking the implementation details could be resolved after acceptance of the RFC).

How would this be implemented

I think I'd like to import a comment from the Pre-RFC by @ckaran which answers the question, "but how do we implement this?"

My goal: import it here to surface it and start a discussion around what we agree about this proposed implementation and what we would like to further discuss.

Once we reach an agreement, I'll add it to the RFC so that it is part of the final proposal.

Implementation Solution 1

This solution was proposed by @ckaran in the Pre-RFC in the comments here:

"I want to turn attention to how templates are supposed to function for a moment. Basically, what is the underlying engine going to do, and going to be able to do. Is the engine going to be allowed to do arbitrary things? Or is it going to be sandboxed? How will it be sandboxed? Are we going to develop a new domain specific language (DSL) for templates alone? Can we use a language that is already available?

My vote is to use the absolute simplest method I can think of; copy/paste/text substitution. This would involve making the convention that each child of the template directory is a complete template. Thus:

- mycrate
    - templates
        - A 
            - Template.toml
            - src
                - Cargo.toml
                - src
                    - file1.rs
                    - file2.rs
        - B
            - Template.toml
            - src
                - file1.rs

cargo --template would interpret this as there being two templates, A and B. Within a given template is the Template.toml file. This file enumerates all of the keys that are available within that template, along with any help information that is needed. E.g. A/Template.toml might contain:

[package]
name = "A"
version = "0.1.0"
authors = ["Alice B Caring <abc@place.com>"]
edition = "2018"
description = """A small, complete template for the [foo-bar-baz](https://crates.io/crates/foo-bar-baz) crate.

This template will create a small, but complete, project that uses the [foo-bar-baz](https://crates.io/crates/foo-bar-baz) crate.  
"""
keywords = ["foo", "bar", "baz"]
categories = ["template"]
license = "MIT OR Apache-2.0" # It would probably be better to require that templates use the same license as the original crate.

[badges]
maintenance = {status = "experimental"}

[dependencies]
foo-bar-baz = {version = ">= 0.4, <= 0.6", features = ["serde"], uuid = "B23901D91F984C39888E468F531E0F97"}

[keys]
authors = {description = "A list of strings of authors like 'Alice B Caring <abc@place.com>'", default = ""}
date = {description = "The date that the file was generated.  Can be any form you want, it will be directly copied into the template.", default = ""}

Most of those keys are directly copied from the Cargo docs, and so should already be familiar to users of the tool. The only new keys are uuid, which contains a uuid generated by the trick I mentioned above (by the way, uuid would have to become a new key for Cargo.toml files), and the whole keys section, which (I hope) should be relatively obvious.

When using cargo --template, you might have subcommands like:

• cargo --template list, which would return A and B to you.
• cargo --template show A would give you a pretty-printed version of A/Template.toml.
• cargo --template generate --authors "Alice B Caring <abc@place.com>, Donald E Francis <def@place.com>" --date "Thursday, 29 February 2024" A ~/Documents/repositories/my_project/ which actually generates your file(s) from the template.

The generate command would have two required arguments; the first is the name of the template to use (A) in the example above, and the location where you want the template to be copied to (~/Documents/repositories/my_project/) above. The complete contents of a template's src directory would be copied directly to that path; thus, when using template A I would expect to see a directory structure like:

- ~/Documents/repositories/my_project
    - Cargo.toml
        - src
            - file1.rs
            - file2.rs

but if I used template B, I'd see something like this:

- ~/Documents/repositories/my_project
    - file1.rs

This actually solves the 'what is a template?' question; to use template B correctly, I would need to use cargo --template generate --authors "Alice B Caring <abc@place.com>, Donald E Francis <def@place.com>" --date "Thursday, 29 February 2024" A ~/Documents/repositories/my_project/src/, which would place file1.rs where it belongs.

The engine would be a simple text substitution engine. It would search for text strings of the form {{key}} and replace them with either what is on the command line, or the specified default in the Template.toml file. If a key doesn't have a default and is not specified on the command line, then the engine would issue an error about the missing key (maybe with the key's description copied from the Template.toml file), and require that the template user fix the errors before continuing.

This solves the major security problems because the template isn't being executed, and you can't really do a DOS attack on text substitution (notice how the method above doesn't permit re-evaluation of the generated text, which means you don't get a billion-laughs style attack).

This leaves two issues; how do template authors indicate that they really want the string {{ or }}, and how do you reuse a template for the same destination path multiple times in a row?

• How do template authors indicate that they really want to have {{ or }} in the source text? A method that could work is to pre-define the keys {{left double curly brackets}} and {{right double curly brackets}}, which template authors could then put into their templates as needed. The engine wouldn't need any modification as it would substitute the appropriate strings in directly whenever it encountered those keys. The only tricky part would be binary blobs (e.g., audio or image files that were part of the template). My suggestion is to have a cargo --template from command. It does the reverse of the template engine, copying everything from to almost verbatim; the difference obviously being that whenever the engine encountered {{ it would substitute {{left double curly brackets}} in, and similarly for }}.
• What happens when a user has the same destination path multiple times in a row? The trivial answer is that the engine simply and blindly overwrites the contents of the destination path. This would be a horrible user experience, so let's not do that. Instead, cargo --template generate could have additional switches, maybe -f, --force and -i, --interactive, which would be mutually exclusive. If the user selected -f, then the destination path would be overwritten. If -i were selected, then the engine would ask what to do for each destination file that was going to be overwritten (probably overwrite, don't overwrite, compare with the user's diff tool, etc.; whatever you'd expect from a good VCS when merging in conflicts). If neither switch is selected, then the engine would quit with an error warning about the conflict. Note that it would probably be nice to have a key argument to rename the file to make this easier to use.

I think that something like the above would be sufficient for most templates; can anyone think of something where it wouldn't work?"

[l4l]

Should urls might be used for sources as well? I think it's quite handy to fetch template directly from git source (e.g. from yet unpublished crate or private repo). Example usage might be cargo new example-crate --template ssh://example.com/repository.

[ckaran]

Should urls might be used for sources as well? I think it's quite handy to fetch template directly from git source (e.g. from yet unpublished crate or private repo). Example usage might be cargo new example-crate --template ssh://example.com/repository.

I think it could be, but I would strongly prefer it not to be. The issue was covered in the RFC; your specific example means that we're tied to both ssh and git. That dramatically expands the scope of the proposal.

That said, I agree that this would be a useful ability. Quickly scanning the cargo subcommands wiki page, it looks like cargo clone would be a good starting point for this. Is there a way of tying the commands together? E.g., if you have cargo clone installed already, could cargo template use it for your purposes?

[matu3ba]

@jsjoeio
I like the overall idea to have a fast start, but I am unsure about practical examples in Rust, where these are needed.
Probably do a survey asking what type of code, in contrast to fixed examples, would be useful.

The motivation should contain a background on how granular you want to make things on practical examples ie 1. what is the difference to example code 2. why are automatic substitutions/code production useful.
Ideally you explain what kind of problems you want to solve to estimate the cost/gain tradeoff in contrast to prior art.
I know that a generalization is hard there, but since this creates maintainance cost necessary.

The main points of disagreemant between community and cargo team should be listed and resolved in the rfc.

Personally I dont like template as name due to the usage in c++, where code is derived from code (vs here code is created as starting point), but I think it is okayish.

[ckaran]

@matu3ba I can't answer for @jsjoeio , but I can describe what I see this being useful for; perhaps @jsjoeio can provide further explanations.

The main use I see for such a feature is for creating templates for large frameworks where the example requires a significant amount of customization before it is ready to use. For example, amethyst templates might require the same key in numerous places (e.g., Cargo.toml, every source file, a change log, etc). This requires a more structured approach than just providing an example in the examples directory for all of the following reasons:

• Examples are not complete; they rely on the Cargo.toml file of the project itself, which means that you need to figure out which parts of that file are relevant. A template is complete, and can be minimal. This makes it a good starting point.
• Templates are structured. While it isn't explicitly stated in the RFC (and it does need to be @jsjoeio!), since all of the keys are specified in the Template.toml file for a particular template, the engine is able to verify that all and only those keys that are used within the template are listed within the Template.toml file. This helps protect against stale documentation which can easily happen if someone updates a template by hand but forgets to update a README with the updates.
• Templates do not need to be a part of the same repository as the the project they are provided for. This isn't possible for examples as examples are not complete. As communities grow around different frameworks, you can expect to see a large number of community-maintained templates being created which are entirely separate from the project itself.
• Templates are required to specify the range of releases that they work with. Examples don't; they are always assumed to work with the latest release only (verifiable via cargo run --example). That means that if you cannot upgrade to the latest major release of a project for some reason, you can still search for templates that work with older releases. Searching for examples that work with a given range of releases will require poking around in the history of the repository, which can be slow, or involve lots and lots of web searches where you have to manually test and reject examples that are either too old or too new to use.

As for using text substitution, I'll admit that I came up with that idea. Other methods are possible, but they are more difficult to implement, and could leave the end user open to attacks by bad actors. Text substitution seemed like a good balance between expressive power, danger, and ease of integrating with cargo. Basically, I see this as a mashup of @BurntSushi 's ripgrep and sed, or maybe something like a version of sd, and as such, something that is relatively easy to implement.

[ssokolow]

Regarding literal {{ and }}, I'd much prefer to have something like a template.toml which specifies a whitelist of file patterns to not pass through verbatim.

In my view, anything not built around whitelisting file patterns is a footgun of a nature comparable to the C preprocessor's approach to macros.

(I say this as someone who had a justfile mangled by a previous attempt at cargo templating.)

I also find it a little odd to push everything except pulling from crates.io off to a later date when it seems most sensible to test with local files and/or git repos so you're not encouraging people to pollute crates.io with crates, just to test an experimental feature.

[jsjoeio]

Should urls might be used for sources as well?

As @ckaran said, we covered this in the Pre-RFC. I think we should should leave it out and keep this MVP as small as possible.

[jsjoeio]

I like the overall idea to have a fast start, but I am unsure about practical examples in Rust, where these are needed.
Probably do a survey asking what type of code, in contrast to fixed examples, would be useful

@matu3ba those are fair points! I was hoping the Pre-RFC + RFC would serve as discussions places for the community to decide "yes, we want this" vs. "no, we don't want this" rather than using a survey (unless that's how other RFCs do it?)

@ckaran gave some thorough and detailed examples so I won't repeat that here.

[jsjoeio]

The main points of disagreemant between community and cargo team should be listed and resolved in the rfc.

As far as I can tell, I don't see any comments on here yet from members of the Cargo team. Would love to get their feedback and see if they think this is a worthy addition to cargo and its usefulness for the community.

[jsjoeio]

it seems most sensible to test with local files and/or git repos so you're not encouraging people to pollute crates.io with crates, just to test an experimental feature

This is a really good point. You should be able to test it somehow locally so that you don't have to push to crates.io.

@ckaran thoughts?

[jsjoeio]

I want to redirect the conversation to the "Implementation Solution 1" by @ckaran.

I feel like we can simplify it even more.

• use everything you mention there up until "When using cargo --template, you might have subcommands like:"
• remove all the subcommands
• declare the last two issues in your message out of scope

And for this "how do you reuse a template for the same destination path multiple times in a row?" maybe the v1 doesn't account for this. If we implement it and people request this feature, then we add it in.

I feel like those last two issues can be considered out of scope. Trying to keep this as small as possible.

@ckaran thoughts?

[ckaran]

it seems most sensible to test with local files and/or git repos so you're not encouraging people to pollute crates.io with crates, just to test an experimental feature

This is a really good point. You should be able to test it somehow locally so that you don't have to push to crates.io.

@ckaran thoughts?

I think that testing locally is going to be a must-have feature; without it, crates.io will rapidly fill up with terrible half-baked templates.

It would also make automation a little bit easier, especially when the template is in a different crate than the crate the template is for; you can run a cron job that pulls the latest crate, then tests the templates locally against the latest pull. If all tests of the template pass, then the script can bump up the range of versions that the template works with automatically, and publish that to crates.io. If any tests fail, then the template will not have been published (leaving crates.io in a clean state), and the template author(s) can correct as needed.

[ckaran]

I want to redirect the conversation to the "Implementation Solution 1" by @ckaran.

I feel like we can simplify it even more.

* use everything you mention there up until "When using cargo --template, you might have subcommands like:"

* remove all the subcommands

* declare the last two issues in your message out of scope

And for this "how do you reuse a template for the same destination path multiple times in a row?" maybe the v1 doesn't account for this. If we implement it and people request this feature, then we add it in.

I feel like those last two issues can be considered out of scope. Trying to keep this as small as possible.

@ckaran thoughts?

I think that doing this would oversimplify the cargo --template command. The issue as I see it is that this command is there to make programmers more efficient; anything that can be added with minimal effort that improves ergonomics and efficiency should be kept. cargo --template list and cargo --template show aren't needed, but they do make it faster and easier to figure out what is available. cargo --template generate could be replaced with something like cargo new --template ... where the ... are all the arguments you'd pass to generate, but that would require separation of the template type into multiple area. Finally cargo --template from is going to be really useful to quickly create templates from already working minimal projects; converting projects without it is going to be annoying (but doable, assuming sed can be used on binary files).

An alternative to creating an RFC that includes both the command and the template directory structure/substitution strings might be to specify the template directory structure only. Other tools could then be created as cargo extensions that are aware of and understand the directory structure/substitution strings. Once the community decides which tool is the best one to use, it could be incorporated into core cargo.

[pickfire]

How do we show list of templates for a crate like cargo --example? cargo --template rocket?

[ckaran]

How do we show list of templates for a crate like cargo --example? cargo --template rocket?

The command will likely need to be run within a project to get a listing. That would mean that you would only see the templates defined within that particular project. To get more templates, you'd need to search crates.io for template crates specific to your needs, download them, and then use them.

It would be better to have the tool do this for you, but I'm starting to lean away from an RFC that specifies the command, and lean towards an RFC that specifies the directory structure and precise format of a template (e.g., Template.toml plus the string substitution mechanism mentioned earlier). The reason I'm leaning more towards this is that it would allow others to build their own tools on top of the convention, e.g., any of the major IDE vendors. @jsjoeio, would you be willing to refocus this RFC to define the conventions for what a template is instead of making it about the commands themselves? E.g., how an author creates a template, where they are stored, how they are identified so that search engines know that this is a template and not a project, etc., etc., etc. We can come up with hypothetical tool designs to test out ideas, with the understanding that eventually there will be a real cargo command of some kind that works as a reference example.

[pickfire]

Ah, so it is something like a protocol. Sounds like the ability to support cargo-<command>. I think it might be nice but I do think having a cargo command to solve this may be better off at this moment.

[ehuss]

I want to thank @jsjoeio for putting this together, and for everyone's comments. Per the comment at #2922 (comment), the author has expressed that they will not be able to continue with it, so I am going to move to close this as postponed. There is still interest in a feature like this, so if someone is willing to pick it up again, starting with this and trying to address the comments here might be a good starting point. Also, such a feature can be developed as a plugin, so working in that direction is always a possibility.

@rfcbot fcp postpone

[rfcbot]

Team member @ehuss has proposed to postpone this. The next step is review by the rest of the tagged team members:

• @Eh2406
• @alexcrichton
• @ehuss
• @joshtriplett

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[tbelaire]

I have one question which I haven't seen addressed yet (with just a quick search through the comments.)

When looking at the rocket example, for a while rocket had to main branches, master and async, which included a large refactoring to work with the async/await apis. (I think async has landed now as v0.4, but I'm sure similar examples will show up again).

If rocket's async branch also updates the templates, would there be a way to use the new templates before they get published?

So on rocket's readme, they can say: "Hey, you can try the async branch. cargo new my-rocket-app --template rocket@async/helloworld" and it'll create a Cargo.toml which points at the rocket

[dependancies.rocket]
git = ...
branch = "async"

Ah, I see it's likely to be postponed, so well, nvm I guess.

[ckaran]

@tbelaire Yes it's been postponed because nobody has time to finish it up yet (are you volunteering? 😉), but the likely process would be fairly simple:

1. Clone the repository in question, which should have templates appropriate to each branch within each branch.
2. Switch to the branch you're interested in.
3. Use the templates as needed.

As a summary, the discussion moved away from creating tools to creating a standard directory structure & format for templates. The last one discussed wouldn't care about branches at all, so the above would probably be sufficient.

[rfcbot]

The final comment period, with a disposition to postpone, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

The RFC is now postponed.

[thor314]

@ckaran, It looks like this RFC nearly made it to the finish line and sat down.

It was mentioned above that "nobody has time to finish it". If someone would point me at the next steps to push it over the finish line, I'd love to contribute if possible. I'm...shudders...volunteering.

[ckaran]

@thor314 Honestly, it's been so long since I looked at this that I completely forgot about it. Skimming it, it looks like what needs to be done is to write an RFC outlining the directory structure, what templating is (e.g., the use of {{ and }}) including how to use it in binary files that are being templated, and what is not allowed to be templated. The only security concern that I could think of was templating file names that let you prepend absolute file paths such that you overwrite user files/directories. I think that it's sufficient to require that template engines only emit output to specified directories, emitting an error if they have to cross outside of the given directory, but I'm not 100% sure.

I can give you more hints here and there, but I'm kind of swamped at work right now, so my responses will be late and sporadic. Sorry! :(

[ssokolow]

...and I would like to re-raise my concern about blacklisting vs. whitelisting which files get templated rather than just passed through verbatim.

I would consider it a dereliction of my duty as a responsible package maintainer to use the standard templating tool rather than providing my own if the standard templating tool uses blacklist-based templating prone to what I experienced with the previous attempt. Too much of a footgun to be consistent with the rest of a Rust codebase.

(I misunderstood the docs and wound up mangling my justfile because it was blacklist-based and both the templating and just used {{ and }} as part of their syntax. I didn't realize it had happened until just taskname errored out part-way through actually running a task because {{some variable}} had become an empty string in the justfile and the command being run by just returned failure. It was pure luck that the task didn't leave the codebase in a state that required Git's help to revert.)

[ckaran]

@ssokolow I understand your concerns, but I think this would be an issue regardless of the templating scheme used. Blacklisting/whitelisting require your to maintain the lists appropriately. Using external tools that convert all files in a directory to a template format (whatever format might be chosen) might not be used, or might be inappropriately used¹.

We might be able to add some metadata to a .template file that we store at the root of the template directory, but there could be issues that way too. In short, the world is full of tradeoffs, and we need to pick our poison. Which dangerous path would you like to take? 😉

Footnotes

1. A danger of using an external tool is ensuring that operations are always idempotent. If a tool blindly replaces {{ with {{left double curly brackets}} every time it's run, then it's possible to have something like {{left double curly brackets}}left double curly brackets{{right double curly brackets}} after the second run. Subsequent runs will do even more damage. This is even more damaging if the template author adds new files after having run the tool as some files will have the correct {{left double curly brackets}}, and other files will have {{left double curly brackets}}left double curly brackets{{right double curly brackets}} (or worse). The simple solution of refusing to process {{left double curly brackets}} a second time feels right, but can be wrong; for example, if I have a documentation file explaining the correct use of {{left double curly brackets}} that I want to process into my template directory, the correct output of the tool will be {{left double curly brackets}}left double curly brackets{{right double curly brackets}}. But only I as a human can know which is the correct operation. ↩

[ssokolow]

I'd need to see the fine details of a given approach to discuss them but, generally, I think the approach most in line with Rust's philosophy elsewhere is to have something like a template.toml or, if you're unconditionally special-casing Cargo.toml and reprocessing it anyway, a [[template]] section in Cargo.toml which must list a set of globs to be processed as more than a literal copy.

That way, you don't wind up accidentally corrupting an un-checksummed file that happens to contain the relevant metacharacters. (eg. a .bmp file with pixels that happen to be interpretable as valid 7-bit ASCII or UTF-8 containing {{...}})

As for defaults, there could be discussion about whether things like Cargo.toml and *.rs should be provided as default entries, but I think it'd be most reasonable to say that a template.toml or a [[template]] section in Cargo.toml or some other equivalent must exist to indicate that the template is to be processed as more than just an ordinary folder or git repository, similar to how cargo build will refuse to do target autodiscovery if it doesn't find a Cargo.toml.

It would also resolve the idempotency issue by providing a signifier that the output of a template is not a valid template. (Whether it's possible to specify that a template generates a template.toml can be discussed, but feels like it'd be scope creep akin to arguing that Cargo should subsume the functionality of something like cargo make or just so it can be a proper competitor to GNU Make or NPM as a means to run arbitrary tasks.)

We might be able to add some metadata to a .template file that we store at the root of the template directory, but there could be issues that way too. In short, the world is full of tradeoffs, and we need to pick our poison. Which dangerous path would you like to take? 😉

I'm curious what issues you foresee with that approach. To me, that feels like pushing back against mandatory seatbelt-wearing in vehicles because an improperly used shoulder belt can cause injury to a child in a crash.

[ckaran]

I'd need to see the fine details of a given approach to discuss them but, generally, I think the approach most in line with Rust's philosophy elsewhere is to have something like a template.toml or, if you're unconditionally special-casing Cargo.toml and reprocessing it anyway, a [[template]] section in Cargo.toml which must list a set of globs to be processed as more than a literal copy.

Sorry, I wasn't being clear enough. template.toml would be for user-visible configuration of how templates are processed into the final files. .template is more like a database. When a template engine is run, it can store information about what actions its performed in there. The idea is that if you run some kind of external processing tool, it can add in information about which files it processed into the .template file so that it doesn't process them a second time.

That way, you don't wind up accidentally corrupting an un-checksummed file that happens to contain the relevant metacharacters. (eg. a .bmp file with pixels that happen to be interpretable as valid 7-bit ASCII or UTF-8 containing {{...}})

Strangely enough, I was coming at it from the opposite direction; such files ARE processed by the external tool to turn them into templates, even if they contain those sequences. The templatized equivalents are no longer valid files, but when the template engine is run to create the template instances, all of the {{left double curly brackets}} will be replaced by {{ anyways. Basically, it reduces a template engine to the following two steps:

1. Copy the entire directory to wherever the template is supposed to be instantiated.
2. Run a sed or awk script to replace {{left double curly brackets}} with {{ in-place. Other template keys are handled similarly.

I'm curious what issues you foresee with that approach. To me, that feels like pushing back against mandatory seatbelt-wearing in vehicles because an improperly used shoulder belt can cause injury to a child in a crash.

I only see the following tradeoffs, but I haven't thought deeply about this:

1. Someone deletes the .template file, or forgets to include it in the repository.
2. Multiple template engines are involved, and they don't agree on the format for the .template file.

Neither of these should be used as an excuse NOT to do something smart though! Seat belts are a GOOD thing! We just need to be aware that they aren't perfect, and try to minimize the possibility of errors, that's all.

[ssokolow]

This is sounding less like you want a Cargo template system MVP that actually stands a chance of getting approved and more like you want to jump right to something straight out of CMake, to be honest.

Sorry, I wasn't being clear enough. template.toml would be for user-visible configuration of how templates are processed into the final files. .template is more like a database. When a template engine is run, it can store information about what actions its performed in there. The idea is that if you run some kind of external processing tool, it can add in information about which files it processed into the .template file so that it doesn't process them a second time.

I still don't see why .template would be necessary at this stage, and it feels like an opportunity for a mysterious source of statefulness that will prompt people to get frustrated and pull out the nuclear cargo clean to try to get things to behave as they expect.

I like that, as long as I don't touch the contents of target, Cargo is more resistant to getting into a broken state than GNU Make.

For a templating MVP:

1. If the external processing is on the input site, have template.toml set to ignore the analogues to things like Makefile.in. TOML is a standard with plenty of implementations so, in the worst case, have the input stage have a template.toml.in which it amends with a machine-generated list of files to ignore.
2. If the external processing is on the output side, then template.toml being missing from the output is sufficient to indicate that it's gone through the templating process.

This approach should be trivial to make forward-compatible with something that requires a .template later if and when it's accepted in a separate RFC.

This "make an MVP and then incrementally add stuff as proven necessary, rather than trying to compete with every feature of a competing build system" approach is characteristic of Cargo.

[ckaran]

OK, so if I understand your goals correctly, you assume that the engine runs exactly once; nobody tries to pull in updated templates and rerun an engine, which might mung things up on the output side, correct? If that's so, then your design is not only fine, it's preferable. I agree with you that .template could end up with you in a broken state. We can see if using template.toml alone is sufficient for 'in the wild' usage. If for some reason state is required, we can revisit .template, which will likely require some pretty heavyweight specifications to ensure that all engines interoperate properly with each other.

[ssokolow]

Exactly. I think interoperability dependent on buy-in from other tools and support for re-running the engine is something that shouldn't be spec'd out now when we don't even have a simple "like cargo new, but with more than just the built-in --bin and --lib" solution yet.

[ckaran]

Works for me.