Question - Policy modification clarification

[cspring86]

Let us know the feedback or general question

Following on from #859, I've had a look at the documentation, but I have a number of questions and clarifications.

Context

My use case is changing a policy (or short while) from DENY to AUDIT, but I want all changes made through version control; I don't want any manual changes made through, for example, the Azure portal or CLI.

I've run the Bicep Accelerator (new version) which has set up a new repository called "alz-mgmt" in my Azure DevOps project.

That repository contains CI/CD pipelines and configuration files:

alz-mgmt
├── .pipelines/
├── accelerator/
├── config/
├── parameters.json
└── README.md

My plan is as follows:

• Add the existing DENY policy to the parExcludedPolicyAssignments parameter to exclude the policy from being assigned.
• Create and assign the new policy, taking a copy of the existing policy and changing the effect from "deny" to "audit".

Ideal Solution

As the "alz-mgmt" repository created by the Accelerator is a configuration repository, the ideal solution would be to expose another set of configuration to override existing policies. This would meet my use case perfectly.

For use cases around adding new policies, perhaps offer an empty directory structure that users can just drop in new policies that will be automatically picked up by the pipelines.

However, I don't think either is supported currently, which leads me onto the existing solutions.

Existing Solutions (and questions about them)

The main question I have is that it looks like all the steps are executed within the ALZ-Bicep repository, but that's not a repository I own or maintain; that's the "alz-mgmt" repository.

It might become clearer if I run through option 1 and make comments under each step.

Option 1

1. Extend the ALZ Default Policy Assignments module from ALZ-Bicep

• (optional) Adding additional assignments as a .json file to the lib and add additional variables like varPolicyAssignmentDenyIPForwarding etc.
  • Can use Invoke-PolicyToBicep.ps1 script as explained in How Does ALZ-Bicep Implement Azure Policies? to generate variables for assignments once assignment .json file in the lib.

Steps:

1. Navigate to the Policy Assignments lib directory: infra-as-code\bicep\modules\policy\assignments\lib\policy_assignments

Ok, so this is definitely to be performed within the ALZ-Bicep repository. So I've cloned the repository onto my machine and I navigate to the location.

2. Copy/clone an existing .json file and rename it to something appropriate

• Try to copy a policy with the same effect as the policy you are wanting to add
• Important: The file name of the .json file is not important. It can be anything you like as long as it ends .json

Now this is a bit weird because I'm being asked to change code within a repository that I don't manage or maintain, but fine, I'll go with it.

3. Amend contents of new file to values for the new policy assignment
   Common properties to change: name, displayName, description, metadata, parameters, policyDefinitionId, enforcementMode, identity

I change the name, description, displayName and effect from "deny" to "audit".

Run the Invoke-PolicyToBicep.ps1 script to update the _policyAssignmentsBicepInput.txt file in the lib folder
i. Copy the entire contents of the relevant _policyAssignmentsBicepInput.txt file and replace the variables for the policy assignments metadata (lines 78 to 202 today in the alzDefaultPolicyAssignments.bicep module)

Again, strange that I'm being asked to modify code in a repository I don't own or maintain.

Also, note that the indentation in the _policyAssignmentsBicepInput.txt file is tabs, not spaces, and that the order doesn't seem to be maintained across runs because a git diff showed many more changes than just the addition of my new policy.

5. Define a new module declaration using the Policy Assignments module in the alzDefaultPolicyAssignments.bicep module

• You could also copy an existing alzDefaultPolicyAssignments.bicep module policy assignment module declaration that uses the same effect as you require in your new assignment
  • DeployIfNotExists effect example - Deploy-MDFC-Config policy assignment - modPolicyAssignmentIntRootDeployMDFCConfig (lines 242 to 269)
    • Note the use of parPolicyAssignmentParameterOverrides
  • Deny effect example - Deny-Subnet-Without-Nsg policy assignment - modPolicyAssignmentIdentDenySubnetWithoutNSG (lines 445 to 458)
• Change the parameter input values to the desired values to assign your new policy
  • Refer to the Policy Assignments module documentation for parameter information
  • These can be "hardcoded" or from the associated .json file in the lib, referenced via a variable generated in step 4, that you have created in the previous steps

I added a new module declaration.

I also needed to update varModuleDeploymentNames with a new entry, but this isn't stated in the docs for option 1.

6. Redeploy the updated ALZ Default Policy Assignments module via your configured method (locally via Azure CLI or PowerShell or via Azure DevOps pipeline or GitHub action)

Redeploy how?

All these code changes are in the "ALZ-Bicep" repository and my pipelines are in the "alz-mgmt" repository.

I think @oZakari's comment suggests the solution:

You can override the templateFilePath directory property within the bicep-templates.yaml file for the ALZ Default Policy Assignments module with a custom module as well as a custom lib directory as well.

So I've got to copy the alzDefaultPolicyAssignments.bicep module over to my "alz-mgmt" repository.

I've copied it over to the same location as @oZakari states: config/custom-modules/policy/assignments/alzDefaults/alzDefaultPolicyAssignments.bicep.

I then need to copy over the new policy json to my "alz-mgmt" repository. I've chosen the following location mirroring the ALZ-Bicep repository: config/custom-modules/policy/assignments/lib/policy_assignments

However, as a result, none of the relative paths work in the alzDefaultPolicyAssignments.bicep module, so I need to update all the relative paths.

I tried updating them to point to the ALZ-Bicep files cloned as part of the pipelines e.g. ../../../policy/assignments/lib/policy_assignments/policy_assignment_es_audit_appgw_waf.tmpl.json => ../../../../../infra-as-code/bicep/modules/policy/assignments/lib/policy_assignments/policy_assignment_es_audit_appgw_waf.tmpl.json. However, this fails the now activated "Bicep Build & Lint All Custom Modules" step in the ci-template.yaml file because that job doesn't clone the ALZ-Bicep repository.

What should I do now?

I could either copy the entire "policy" directory, including the module and json files, over to the "alz-mgmt" repository or update the ci-template.yaml to clone the ALZ-Bicep repository. Which is better?

I've then got to keep all this in sync with any upstream changes to the alzDefaultPolicyAssignments.bicep module and policy assignment changes in the ALZ-Bicep repository.

This is a lot of manual work and ongoing maintenance for a relatively simple use case.

Am I doing things as expected or am I completely off track?

Please advise.

Code of Conduct

• I agree to follow this project's Code of Conduct

[cspring86]

I could either copy the entire "policy" directory, including the module and json files, over to the "alz-mgmt" repository or update the ci-template.yaml to clone the ALZ-Bicep repository. Which is better?

I've tried going down the pipeline modification route, but it's proving very awkward.

I've updated the ci-template.yaml file in my PR, but because the templates are tied to main, the updated template isn't used so the pipeline always fails.

I've tried adding a ref such that it uses the current branch, but now there's a "Required YAML template" check that's failing on the sc-alz-mgmt-plan service connection because it requires the ref to be main.

I'm doubting that this is a good path to go down.

I can obviously work around this by first merging the update to the template and then re-running the CI pipeline on the original PR, but there's no way to test pipeline template changes then.

[cspring86]

Ok, I've got it working by updating the ci-template.yaml file in a separate PR, merging that, rebasing my original PR and then it passes.

The change I made to the ci-template.yaml file is as follows:

diff --git a/.pipelines/ci-template.yaml b/.pipelines/ci-template.yaml
index 6075158..53f6162 100644
--- a/.pipelines/ci-template.yaml
+++ b/.pipelines/ci-template.yaml
@@ -20,6 +20,18 @@ stages:
             parameters:
               serviceConnection: 'sc-alz-mgmt-plan'
 
+          - template: helpers/bicep-variables.yaml
+            parameters:
+              parametersFileName: $(parametersFileName)
+
+          - template: helpers/bicep-on-demand-folder.yaml
+            parameters:
+              repository: "https://github.com/Azure/ALZ-Bicep"
+              releaseArtifactName: "accelerator.zip"
+              releaseVersion: "$(RELEASE_VERSION)"
+              sourcePath: "infra-as-code"
+              targetPath: "infra-as-code"
+
           - pwsh: |
               if (Test-Path -Path ./custom-modules/*)
               {

I'm still not sure if this is a good solution though?

[oZakari]

Hi @cspring86, I see your concerns. Let me work through this tomorrow a bit more and get back to you to see how to best structure the directory. The toughest part as you mentioned will be staying in sync with the upstream policies. However, we are working on refactoring ALZ-Bicep, so the Accelerator should be a bit more flexible in the future. Will get back to you shortly👍🏼

[cspring86]

Thanks @oZakari !

No rush though. This solution, although the initial changes were a bit painful and have ended up with me modifying the ci-template.yaml file (owned by you guys and may be overwritten every time I re-run the Accelerator), does make modifying existing policies or adding new policies pretty simple now and has kept the amount of copying from the ALZ-Bicep module down to a minimum.

For completion, the solution I've gone with for now is to copy the Bicep module across (to config/custom-modules/policy/assignments/alzDefaults/alzDefaultPolicyAssignments.bicep) and to update it's file paths to reference the ALZ-Bicep files when fetched within the pipeline. This means I don't have to copy across potentially hundreds of files from the ALZ-Bicep repo, but has resulted in an update to the ci-template.yaml file such that it also fetches the ALZ-Bicep files like the other pipeline jobs. Any policies I need to modify or add new can then be placed into config/custom-modules/policy/assignments/lib/policy_assignments with the Bicep module updated to reference my policy, not the ALZ-Bicep policy. Keeping in sync with upstream changes will be a pain, mainly around the Bicep module tbh, but it's working well for now.

As stated, the ideal solution would be to expose configuration in the "alz-mgmt" repository to achieve the same effect i.e. an "overrides" section for policies where you can override any part of a policies parameters as well as a known directory in "alz-mgmt" where additional policies can be included.

[oZakari]

Ok, thank you @cspring86 for letting me know your solution is working for you at the moment. Appreciate the feedback and will get back to you soon.