Reduce template dangers

[yajo]

The problem is, templates just need these features. Writing good migrations
takes time, but they provide smooth updates; but what if users are able to
turn them off? Then you can't predict the output. Tasks also are needed to
replay last render when updating. And jinja would die if you use it without
the required extensions.

Good point!

So, although it's OK to warn the user, the features are needed for copier
to do its work. Basically, running Copier without --UNSAFE would only work
with templates without these features. At least, we shouldn't require that
flag when the operation won't involve any of those. Also we should provide
a different exit code for this exit path.

👌

think it'd be OK to present a question before starting, which tells all
dangers at once, so user allows them all or exits. In the case of updates,
we need to include replay dangers and migrations if there are any between
the two versions involved.

👍

Copier is a development tool, so it will usually need access to other
development tools expected in that environment.

Also good point! I hadn't considered that.

I don't recommend running malicious templates in a container. I just
recommend not running them 😆.

However, it would be good to have a page in the docs dedicated to our
threat model and possible workarounds. And that could happen even without
implementing any other fix. It could even be "the fix". 🤔

Yes, I think docs plus the additional question when tasks/migrations/extensions are present or the --UNSAFE flag in case of non-interactive usage, as you described above.

I think this sounds like a good actions plan to raise awareness of the dangers and let users make conscious decisions. 👍

Originally posted by @sisp in #1132 (reply in thread)

[yajo]

@sisp would you like to tackle this issue?

[sisp]

Yes, feel free to assign me to this issue. 👍

[fcollonval]

Hey, I understand all the security concerns raised here. But I feel like the --UNSAFE flag is a bit scary for newbies that don't understand all the good reasons for it. I'm wondering if copier could store a list of trusted template source in config file in the user home directory. The idea is to avoid having to set UNSAFE every time you use a trusted template.

We could then add a flag --trust to copier copy that implies --UNSAFE and add the template to the trusted list;

# First time use
copier copy --trust https://github.com/org/trusted-template.git .

# Update does not require the flag
copier update

# Next time use, does not raise
copier copy https://github.com/org/trusted-template.git .

[pawamoy]

I feel the same as @fcollonval. I'm exclusively using my own Copier template (I don't think I've ever generated a project using someone else's template), so it feels a bit tedious to add --UNSAFE (ugh, capital letters 😅 I'm such a pedant) each time. I'll probably set an alias in the meantime: alias cc='copier copy --UNSAFE'. Sorry for voicing this concern only afterwards! And thanks for the amazing work on Copier ❤️

[yajo]

Hi! I was expecting something like this 😄

I use fish instead of bash, so adding flags is much cheaper. That can help you.

The solution of having a registry of trusted templates was among the ones we considered, but it didn't land because it was Harper. However feel free to open a PR, as it's still good. You can open an issue first if you want to discuss the implementation.

[carlsylvia]

The addition of this flag is very problematic and I recommend you consider rolling this back. My guess is the majority of your users are building templates for in-house use, and your attempt to implement some form of "safeness" is not appropriate. How are you determining/defining what is "safe"?

[pawamoy]

@carlsylvia you can stay on v7 in the meantime. Unless Copier's updates are automatic within your system?

[carlsylvia]

A simple change in the semantics would fix the issue immediately. If you simply default to not trusting a template and require the user to provide a --trust flag so that they are explicitly granting trust to the source then all would be well. That simple logic inversion from --UNSAFE to --trust would make all the difference.

[carlsylvia]

If you want to implement some trusted source repository in the future that is fine, but sometimes the simplest possible solution is the best.

[carlsylvia]

For the time being we will pin our users to version 7.x until this issue is resolved.

[sisp]

A simple change in the semantics would fix the issue immediately. If you simply default to not trusting a template and require the user to provide a --trust flag so that they are explicitly granting trust to the source then all would be well. That simple logic inversion from --UNSAFE to --trust would make all the difference.

--UNSAFE and --trust could even coexist as mutually exclusive flags where --UNSAFE is a one-time opt-in to a template with unsafe features and --trust makes it permanent.

I like the trust store idea. And it seems it would be a solution for everybody who has raised concerns about the new --UNSAFE flag until now.

[carlsylvia]

The --trust flag could record the repo in a local copier metadata file that maintains a list of "trusted" repositories.
Similar to .ssh/known_hosts or authorized_keys
Could even support "trusting" an entire repo and all of the templates contained within.

[sisp]

Could even support "trusting" an entire repo and all of the templates contained within.

By "entire repo", do you mean a namespace (user namespace or organization/group) on GitHub/GitLab/...?

[carlsylvia]

Could even support "trusting" an entire repo and all of the templates contained within.

By "entire repo", do you mean a namespace (user namespace or organization/group) on GitHub/GitLab/...?

I was referring to a GitLab group and/or GitHub org, sorry I was not clear. For example if the copier group contained a collection of template projects then all of projects within https://github.com/copier-org could be trusted, or a single project within the group, depending on the user preference. Not exactly sure how they would specify that preference, but could be convenient to flag the entire group rather than individual template projects.

[yajo]

Hi folks, I've got the MVP in #1179. I'll wait for your reviews before the merge.

For now, it just trusts the current execution, just like --UNSAFE. If somebody wants to contribute a more advanced trusting system, like the ones discussed above in #1137 (comment) or #1137 (comment), I think that should be a separate subcommand:

copier copy --trust git+https://example.com/tpl1 ./here # One-time trust

copier trust git+https://example.com/tpl1 # Permanent trust

copier copy git+https://example.com/tpl1 ./here # Now it works because it's trusted

I don't think it's a good idea to trust an entire organization because one could do copier trust https://github.com and shoot your own foot. But trusting once per template doesn't seem that problematic IMHO.

[fcollonval]

copier trust git+https://example.com/tpl1 # Permanent trust

I would personally prefer to keep the ability to do everything in a single command aka copier copy --trust <src> <dest>. Although I agree that having a new subcommand will be needed if a trusted list is stored to be able to list trusted source, remove some,... . But rather than having a dedicated subcommand, I would suggest going for a config subcommand as storing settings at the user level may be useful for much more things in the future.

[pawamoy]

I don't think it's a good idea to trust an entire organization because one could do copier trust https://github.com and shoot your own foot. But trusting once per template doesn't seem that problematic IMHO.

Trusting the entire site is preventable with code: just make sure the argument points to an org and not just github.com.

[yajo]

Yes, but then we have ssh connections to other git providers, non-git templates... well, I'm not telling this isn't feasible, but it's... complex. And the main issue here AFAICS is that some people don't like UNSAFE because it is upper case and scary; nothing more.

[pawamoy]

True, not every Git platform stores things under orgs. Git has no concept of organization so this would indeed be complex to support in Copier.