feat(cli): add cmd to preview generated apps of appsets (#10895)

[agaudreault]

Closes #10895

The goal of this PR is to update the argocd appset create command to add a --dry-run argument. When it is specified, the server will call the applicationSet templating and app generation logic to evaluate and transform all the generator. It will use the resulting applications to generate a list of "generated" application.

To avoid creating a hard dependency between the server and applicationSet, which is not a default component of ArgoCD, the server does the templating based on the same configuration the ApplicationSet adds. If not configured the same, the generation may be different. Once ApplicationSet is a native components of Argo, the templating could be exposed as an api call, similar to how it is done with the repo-server component.

./dist/argocd appset create --dry-run ./appset.yaml -o json | jq -r '.status.resources[].name' 
ApplicationSet 'tests.simple-application' unchanged (dry-run)
tests.simple-application.dev
tests.simple-application.production-1
tests.simple-application.production-2
tests.simple-application.staging

The output of this command will allow to compare the list of resulting applications with the output of argocd appset get tests.simple-application -o json | jq -r '.status.resources[].name to know which app will be added or removed once the manifest is applied.

Suggestions

I think another CLI command could be argocd appset template ./appset.yaml -o json and return the list of templated applications with the full object and not only their name. Some people expressed the need to have the whole applications so they could validate the logic client-side.

With the current implementation, only the app name is returned in create and the get.

An implementation to be able to do client-side diff would be to have the template command return the current AppSet with all the applications templated, and modify the get to return the full templated application as well (this would avoid an argocd app get for each generated apps).

[codecov]

Codecov Report

Attention: Patch coverage is 62.45955% with 116 lines in your changes missing coverage. Please review.

Project coverage is 50.25%. Comparing base (4c6ad9d) to head (02bb52a).
Report is 489 commits behind head on master.

Files with missing lines	Patch %	Lines
cmd/argocd-server/commands/argocd_server.go	0.00%	46 Missing ⚠️
cmd/argocd/commands/applicationset.go	0.00%	27 Missing ⚠️
cmd/argocd/commands/headless/headless.go	0.00%	22 Missing ⚠️
applicationset/controllers/template/template.go	83.01%	7 Missing and 2 partials ⚠️
server/applicationset/applicationset.go	82.22%	4 Missing and 4 partials ⚠️
applicationset/generators/scm_provider.go	70.00%	3 Missing ⚠️
applicationset/generators/pull_request.go	75.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #16781      +/-   ##
==========================================
- Coverage   50.28%   50.25%   -0.04%     
==========================================
  Files         312      315       +3     
  Lines       43020    43125     +105     
==========================================
+ Hits        21633    21671      +38     
- Misses      18905    18971      +66     
- Partials     2482     2483       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[haiwu]

This feature is really needed in order to do any meaningful dryrun if using applicationset.

In production environment, we have to review the dryrun changes before any updates to applicationset could be landed. With this PR in place, we could get a list of applications, and then we could iterate the list of applications to get its diff via "argocd app diff" command at least.

Without this PR in place, there's no way to do any meaningful dryrun for applicationset.

This is the most needed feature for argocd applicationset.

[agaudreault]

@argoproj/argocd-approvers what do you think if we merge this as an "alpha" feature, and when the dependency is merged, we can change the output of the command to use the new fields?
Maybe with a warning in the CLI doc? Never mind, I forgot that the current get command does not return the list of applications when progressive sync is not present. So it does need the dependency.

The dependency seems to be coupled with a few other PRs and may take some time to merge.
@alexymantha @csantanapr @reggie-k

[cheskayang]

👍 Long-waited feature to allow simple and reliable validations on any non-trivial manifests repo that uses appset

[Jack-R-lantern]

I think --name=generator or make codegen again
Currently, the go generate command generates Generator.go, not generator.go.

[Jack-R-lantern]

I think --name=renderer or make codegen again
Currently, the go generate command generates Renderer.go, not renderer.go.

[jakedgy]

This feature was marked as having been completed in 2.12, but it is not anywhere on the 2.12 branch. I think we need to move it to 2.13 now.

[mikesac-aru]

has this been included into 2.13? cannot see it in changelog. Any estimation on when could be available ?

[skydontdribble]

when is this going to be released?