templatize ev2 gap

[geoberle]

What this PR does

This PR tries to close the gap between templatize current haviour and EV2s mode of operations.

Name generation

This PR removes the name generator functions e.g. azureKeyVaultName since for EV2 the real names need to be determined at runtime not at buildtime. so we restrict ourselves to only use ctx.region ctx.regionShort and ctx.stamp
this sadly has the impact that certain names for resources will change and that means it is disruptive for DEVs and our permantent DEV envs. after merging this, ENVs need to be recreated.

Generate with EV2 placeholders

generate an output file and replaces all placeholders with EV2 double-underscore ones when --ev2-placeholders is specified. this supports delayed evaluation of variables to distinguish between the EV2 build and run stage. this also means that the naming functions like azureKeyVaultName have been removed. Instead region-short has been introduced to have a short region name (as defined by EV2) available for resource naming that will not exceed lenght limitations.

templatize generate ... --ev2-placeholders --input xx --output xx`

SERVICE_KV ?= __SERVICEKEYVAULTNAME__
OIDC_STORAGE_ACCOUNT ?= __OIDCSTORAGEACCOUNTNAME__
IMAGE_REPO ?= __CLUSTERSERVICEIMAGEREPO__

EV2 ServiceConfig helper

in order to support the generation of a ServiceConfig.json, the pkg.ev2.GetNonRegionalServiceConfigVariables function can be used to fetch the variables from config.yaml that should go into a service configs global section. all variable values are formatted in a way to support EV2 system variables, e.g. $location(), $stamp(), $(someOtherConfigValue)

the pkg.ev2.GetNonRegionalServiceConfigVariables function can be used to fetch the regional overrides for a deploy env, again formatted for EV2 system variable usage.

EV2 ScopeBinding helper

the pkg.ev2.ScopeBindingVariables function can be used to extract a find/replace suitable variable list for an EV2 ScopeBinding. the result is a dictionary with entries like __VAR__ = $config(var) or __PARENT_CHILD__ = $config(parent.child) for nested configs.

EV2 file preprocess helper

the pkg.ev2.PreprocessFileForEV2 function can be used to process a file with {{}} placeholders into one that uses EV2 system variables like $location() etc.

Pipeline.yaml runner

the EV2 generator from the ADO repository will use a pipeline.yaml file to generate EV2 artifacts (see example in this PR).
it is important that we use this pipeline.yaml declaration also in DEV to drive deployments. this way such declarations remain well maintained.

templatize run-pipeline --config-file config/config.yaml --cloud public --deploy-env personal-dev -p maestro/server/pipeline.yaml --region westus3 --region-short usw3gobe --stamp 1

Right now, only Shell runner is supported. the idea is to revamp the Makefiles so they don't require the config.tmpl.mk files anymore but rely exclusively on ENV variables to be injected as defined in the pipeline.yaml. in DEV, the run-pipeline command will ensure these env vars are set. in higher envs, EV2 will take care of it.

depending on the subscription and RG, the run-pipeline command will provide a suitable kubeconfig. in higher envs EV2 will take care of this.

Jira: https://issues.redhat.com/browse/ARO-12087
Link to demo recording:

Special notes for your reviewer

[github-actions]

Please rebase pull request.

[github-actions]

Please rebase pull request.

[tony-schndr]

lgtm