feat: Add support for Doppler

[equals03]

fixes #3132

Should be good for a review.

[halostatue]

This looks like it would work, but I think that there are two design choices that should be changed.

[twpayne]

Thank you @equals03, this is fantastic! Thank you for such an excellent pull request!

+1 on @halostatue's comments. For info, the passwordManagerJson template functions were either added before the fromJson template function was added, or still make sense when the JSON output contains significantly different data to the plain text output. If my understanding is correct, fromJson does the job of dopplerJson here. Please correct me if this is not the case.

We can also drop the dopplerMinVersion variable for now and re-add it later if needed.

[halostatue]

If my understanding is correct, fromJson does the job of dopplerJson here. Please correct me if this is not the case.

That's my understanding of the implementation, too. When I was looking at the Doppler documentation, it appears that unlike vlt, there is a way to get all of the secrets for a given project / config at once, so {{ dopplerJson [project [config]] }} could be used for a more efficient project configuration retrieval as well.

It might be better to call that dopplerProjectJson [project [config]], but I think that it is potentially very useful. Using a JSON-shaped value inside that JSON will require the application of | fromJson, but I think that’s fine.

And just to be clear, I really appreciate the work that @equals03 put into this and it’s really good work.

[equals03]

Thanks for all the positive feedback @halostatue @twpayne! 👍

Loving the idea of switching dopplerJson to dopplerProjectJson [project [config]]. I'll make it happen.
As for the minVersion I did notice there was a breaking change in release 3.25.0 that I thought we may have to peg it to, but I don't think that will be required. I'll remove and we can always add it back later if needs be.

Also, side note: I wrote the windows tests blind as I don't have access to a windows machine atm. I'm going to try a subtle change (when I update for the new template funcs) and see if this one passes. If not, I may need someone to help on those. 😄

[halostatue]

This looks very good overall and is I think very close to complete.

[equals03]

@halostatue I've made some updates 😄

Most notably, while implementing the feedback, I did a bit of performance testing and realised there was no good reason not to "unify" the pipelines for both commands. The now use the same mechanism to get the secrets and can make use of the output cache.

In my testing this cuts down on individual calls, dramatically, has quite a performance boost and ensures that the results from both functions are consistent.

As such, I took another stab at rewriting the docs for the user guide.

Also, I have removed that "magic" as you suggested.

[halostatue]

Most notably, while implementing the feedback, I did a bit of performance testing and realised there was no good reason not to "unify" the pipelines for both commands. The now use the same mechanism to get the secrets and can make use of the output cache.

In my testing this cuts down on individual calls, dramatically, has quite a performance boost and ensures that the results from both functions are consistent.

I think that this is OK, but it is worth noting that from a "least privilege" security stance, you are now downloading and caching in memory secrets that may not be used in the chezmoi process by using download all the time. I’m not sure that this requires changing anything, but it is worth noting.

As such, I took another stab at rewriting the docs for the user guide.

I didn’t comment on the lines, but there is an extra comma after doppler.project in both command documents; it’s a small thing.

[twpayne]

Thank you, this is excellent.

A small change is needed for the tests to pass on Windows.

Once you've fixed that, please could you squash all the commits together. Then it's ready to ship :)

[equals03]

@twpayne should be fixed now and commits squashed.

@halostatue

I think that this is OK, but it is worth noting that from a "least privilege" security stance, you are now downloading and caching in memory secrets that may not be used in the chezmoi process by using download all the time. I’m not sure that this requires changing anything, but it is worth noting.

Honestly, I went through the same thought process BUT I landed on: "this is OK" because of the following:

1. we already allow the download of all the secrets through dopplerProjectJson
2. as far as "attack vectors" go, the CLI is already available on the machine. It would probably just be easier to exploit it via the CLI
3. from a more practical point of view, you should probably only keep the secrets you are wanting to use in chezmoi in the same project - no more & no less. 😄 essentially, they should be there because you are going to use them.

[halostatue]

Thank you very much for your contribution, @equals03!

[twpayne]

And thank you for the excellent reviews @halostatue!