Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support to specify tags in the config.yaml that will be applied to all resources deployed by TRE #3623

Closed
wants to merge 18 commits into from
Closed
Show file tree
Hide file tree
Changes from 15 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .github/actions/devcontainer_run_command/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,9 @@ inputs:
description: "JSON string containing key/value pairs to injet into the Resource Processor as ENV vars"
required: false
default: ""
TAGS:
description: "Tags to apply to all Azure resources"
required: false

runs:
using: composite
Expand Down Expand Up @@ -229,6 +232,10 @@ runs:
&& inputs.RP_BUNDLE_VALUES) || '{}' }}' \
-e TF_VAR_resource_processor_number_processes_per_instance="${{ (inputs.RESOURCE_PROCESSOR_NUMBER_PROCESSES_PER_INSTANCE != ''
&& inputs.RESOURCE_PROCESSOR_NUMBER_PROCESSES_PER_INSTANCE) || 5 }}" \
-e TAGS='${{ (toJson(inputs.TAGS) != '""'
&& inputs.TAGS) || '{}' }}' \
-e TF_VAR_tags='${{ (toJson(inputs.TAGS) != '""'
&& inputs.TAGS) || '{}' }}' \
-e E2E_TESTS_NUMBER_PROCESSES="${{ inputs.E2E_TESTS_NUMBER_PROCESSES }}" \
'${{ inputs.CI_CACHE_ACR_NAME }}${{ env.ACR_DOMAIN_SUFFIX }}/tredev:${{ inputs.DEVCONTAINER_TAG }}' \
bash -c "${{ inputs.COMMAND }}"
10 changes: 10 additions & 0 deletions .github/workflows/deploy_tre_reusable.yml
Original file line number Diff line number Diff line change
Expand Up @@ -243,6 +243,7 @@ jobs:
TERRAFORM_STATE_CONTAINER_NAME: ${{ vars.TERRAFORM_STATE_CONTAINER_NAME }}
MGMT_RESOURCE_GROUP_NAME: ${{ secrets.MGMT_RESOURCE_GROUP_NAME }}
MGMT_STORAGE_ACCOUNT_NAME: ${{ secrets.MGMT_STORAGE_ACCOUNT_NAME }}
TAGS: ${{ vars.TAGS }}

- name: ACR Login
# failure in the first attempt indicates a new ACR, so we need to try again after it's been created
Expand Down Expand Up @@ -353,6 +354,7 @@ jobs:
CORE_APP_SERVICE_PLAN_SKU: ${{ vars.CORE_APP_SERVICE_PLAN_SKU }}
RESOURCE_PROCESSOR_NUMBER_PROCESSES_PER_INSTANCE: ${{ vars.RESOURCE_PROCESSOR_NUMBER_PROCESSES_PER_INSTANCE }}
RP_BUNDLE_VALUES: ${{ vars.RP_BUNDLE_VALUES }}
TAGS: ${{ vars.TAGS }}

- name: API Healthcheck
uses: ./.github/actions/devcontainer_run_command
Expand Down Expand Up @@ -522,6 +524,7 @@ jobs:
TRE_ID: ${{ secrets.TRE_ID }}
LOCATION: ${{ vars.LOCATION }}
BUNDLE_TYPE: ${{ matrix.BUNDLE_TYPE }}
TAGS: ${{ vars.TAGS }}

register_bundles:
name: Register Bundles
Expand Down Expand Up @@ -582,6 +585,7 @@ jobs:
TRE_ID: ${{ secrets.TRE_ID }}
LOCATION: ${{ vars.LOCATION }}
BUNDLE_TYPE: ${{ matrix.BUNDLE_TYPE }}
TAGS: ${{ vars.TAGS }}

register_user_resource_bundles:
name: Register User Resource Bundles
Expand Down Expand Up @@ -632,6 +636,7 @@ jobs:
LOCATION: ${{ vars.LOCATION }}
BUNDLE_TYPE: ${{ matrix.BUNDLE_TYPE }}
WORKSPACE_SERVICE_NAME: ${{ matrix.WORKSPACE_SERVICE_NAME }}
TAGS: ${{ vars.TAGS }}

deploy_shared_services:
name: Deploy shared services
Expand Down Expand Up @@ -662,6 +667,7 @@ jobs:
TEST_ACCOUNT_CLIENT_SECRET: "${{ secrets.TEST_ACCOUNT_CLIENT_SECRET }}"
TRE_ID: ${{ secrets.TRE_ID }}
LOCATION: ${{ vars.LOCATION }}
TAGS: ${{ vars.TAGS }}

- name: State Store Migrations
uses: ./.github/actions/devcontainer_run_command
Expand All @@ -681,6 +687,7 @@ jobs:
TERRAFORM_STATE_CONTAINER_NAME: ${{ vars.TERRAFORM_STATE_CONTAINER_NAME }}
MGMT_RESOURCE_GROUP_NAME: ${{ secrets.MGMT_RESOURCE_GROUP_NAME }}
MGMT_STORAGE_ACCOUNT_NAME: ${{ secrets.MGMT_STORAGE_ACCOUNT_NAME }}
TAGS: ${{ vars.TAGS }}

deploy_ui:
name: Deploy UI
Expand Down Expand Up @@ -711,6 +718,7 @@ jobs:
MGMT_RESOURCE_GROUP_NAME: ${{ secrets.MGMT_RESOURCE_GROUP_NAME }}
MGMT_STORAGE_ACCOUNT_NAME: ${{ secrets.MGMT_STORAGE_ACCOUNT_NAME }}
SWAGGER_UI_CLIENT_ID: "${{ secrets.SWAGGER_UI_CLIENT_ID }}"
TAGS: ${{ vars.TAGS }}

e2e_tests_smoke:
name: "Run E2E Tests (Smoke)"
Expand Down Expand Up @@ -746,6 +754,7 @@ jobs:
TRE_ID: ${{ secrets.TRE_ID }}
IS_API_SECURED: false
WORKSPACE_APP_SERVICE_PLAN_SKU: ${{ vars.WORKSPACE_APP_SERVICE_PLAN_SKU }}
TAGS: ${{ vars.TAGS }}

- name: Upload Test Results
if: always()
Expand Down Expand Up @@ -790,6 +799,7 @@ jobs:
IS_API_SECURED: false
WORKSPACE_APP_SERVICE_PLAN_SKU: ${{ vars.WORKSPACE_APP_SERVICE_PLAN_SKU }}
E2E_TESTS_NUMBER_PROCESSES: ${{ inputs.E2E_TESTS_NUMBER_PROCESSES }}
TAGS: ${{ vars.TAGS }}

- name: Upload Test Results
if: always()
Expand Down
3 changes: 3 additions & 0 deletions config.sample.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -76,3 +76,6 @@ developer_settings:

# Used by the API and Resource processor application to change log level
# debug: true

# Specify here tags that should be applied to all resources deployed by the TRE in JSON string format
# tags: '{"tag_key":"tag_value"}'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add this to the docs, https://microsoft.github.io/AzureTRE/v0.13.0/tre-admins/environment-variables/,

Can you also update the CHANGELOG.md file please.

Noticed this page needs updating to explain how config.yaml maps to ENV vars etc. Will create a separate issue for that.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added this to the docs, can you verify it's correct. Thanks.

10 changes: 6 additions & 4 deletions core/terraform/appgateway/locals.tf
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,12 @@ locals {
redirect_configuration_name = "rdrcfg-tosecure"

certificate_name = "cert-primary"
tre_core_tags = {
tre_id = var.tre_id
tre_core_service_id = var.tre_id
}
tre_core_tags = merge(
var.tre_core_tags, {
tre_id = var.tre_id
tre_core_service_id = var.tre_id
}
)

appgateway_diagnostic_categories_enabled = ["ApplicationGatewayAccessLog", "ApplicationGatewayPerformanceLog", "ApplicationGatewayFirewallLog"]
}
5 changes: 5 additions & 0 deletions core/terraform/appgateway/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,8 @@ variable "api_fqdn" {}
variable "keyvault_id" {}
variable "static_web_dns_zone_id" {}
variable "log_analytics_workspace_id" {}
variable "tre_core_tags" {
type = map(string)
description = "Tags to be applied to all resources"
default = {}
}
10 changes: 6 additions & 4 deletions core/terraform/locals.tf
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
locals {
myip = var.public_deployment_ip_address != "" ? var.public_deployment_ip_address : chomp(data.http.myip[0].response_body)
tre_core_tags = {
tre_id = var.tre_id
tre_core_service_id = var.tre_id
}
tre_core_tags = merge(
var.tags, {
tre_id = var.tre_id
tre_core_service_id = var.tre_id
}
)

api_diagnostic_categories_enabled = [
"AppServiceHTTPLogs", "AppServiceConsoleLogs", "AppServiceAppLogs", "AppServiceFileAuditLogs",
Expand Down
16 changes: 10 additions & 6 deletions core/terraform/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -42,12 +42,13 @@ provider "azurerm" {
resource "azurerm_resource_group" "core" {
location = var.location
name = "rg-${var.tre_id}"
tags = {
project = "Azure Trusted Research Environment"
tre_id = var.tre_id
source = "https://github.com/microsoft/AzureTRE/"
ci_git_ref = var.ci_git_ref # TODO: not include if empty
}
tags = merge(
local.tre_core_tags, {
project = "Azure Trusted Research Environment"
tre_id = var.tre_id
source = "https://github.com/microsoft/AzureTRE/"
ci_git_ref = var.ci_git_ref # TODO: not include if empty
})

lifecycle { ignore_changes = [tags] }
}
Expand Down Expand Up @@ -78,6 +79,7 @@ module "network" {
resource_group_name = azurerm_resource_group.core.name
core_address_space = var.core_address_space
arm_environment = var.arm_environment
tre_core_tags = local.tre_core_tags
}

module "appgateway" {
Expand All @@ -91,6 +93,7 @@ module "appgateway" {
keyvault_id = azurerm_key_vault.kv.id
static_web_dns_zone_id = module.network.static_web_dns_zone_id
log_analytics_workspace_id = module.azure_monitor.log_analytics_workspace_id
tre_core_tags = local.tre_core_tags

depends_on = [
module.network,
Expand Down Expand Up @@ -159,6 +162,7 @@ module "resource_processor_vmss_porter" {
resource_processor_vmss_sku = var.resource_processor_vmss_sku
arm_environment = var.arm_environment
rp_bundle_values = var.rp_bundle_values
tre_core_tags = local.tre_core_tags

depends_on = [
module.network,
Expand Down
10 changes: 6 additions & 4 deletions core/terraform/network/locals.tf
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,12 @@ locals {
# FREE = local.core_services_vnet_subnets[11] # .128 - .191
# FREE = local.core_services_vnet_subnets[12] # .192 - .254

tre_core_tags = {
tre_id = var.tre_id
tre_core_service_id = var.tre_id
}
tre_core_tags = merge(
var.tre_core_tags, {
tre_id = var.tre_id
tre_core_service_id = var.tre_id
}
)


private_dns_zone_names = toset([
Expand Down
5 changes: 5 additions & 0 deletions core/terraform/network/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,8 @@ variable "location" {}
variable "resource_group_name" {}
variable "core_address_space" {}
variable "arm_environment" {}
variable "tre_core_tags" {
type = map(string)
description = "Tags to be applied to all resources"
default = {}
}
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ write_files:
AZURE_ENVIRONMENT=${azure_environment}
AAD_AUTHORITY_URL=${aad_authority_url}
MICROSOFT_GRAPH_FQDN=${microsoft_graph_fqdn}
RP_BUNDLE_tags=${tags}
${rp_bundle_values}
- path: /etc/cron.hourly/docker-prune
# An hourly cron job to have docker free disk space. Running this frquently
Expand Down
1 change: 1 addition & 0 deletions core/terraform/resource_processor/vmss_porter/data.tf
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ data "template_file" "cloudconfig" {
aad_authority_url = module.terraform_azurerm_environment_configuration.active_directory_endpoint
microsoft_graph_fqdn = regex("(?:(?P<scheme>[^:/?#]+):)?(?://(?P<fqdn>[^/?#:]*))?", module.terraform_azurerm_environment_configuration.microsoft_graph_endpoint).fqdn
rp_bundle_values = local.rp_bundle_values_formatted
tags = replace(jsonencode(local.tre_core_tags), "\"", "'")
}
}

Expand Down
10 changes: 6 additions & 4 deletions core/terraform/resource_processor/vmss_porter/locals.tf
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
locals {
version = replace(replace(replace(data.local_file.version.content, "__version__ = \"", ""), "\"", ""), "\n", "")
tre_core_tags = {
tre_id = var.tre_id
tre_core_service_id = var.tre_id
}
tre_core_tags = merge(
var.tre_core_tags, {
tre_id = var.tre_id
tre_core_service_id = var.tre_id
}
)

azure_environment = lookup({
"public" = "AzureCloud"
Expand Down
6 changes: 6 additions & 0 deletions core/terraform/resource_processor/vmss_porter/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -32,3 +32,9 @@ variable "rp_bundle_values" {
locals {
rp_bundle_values_formatted = join("\n ", [for key in keys(var.rp_bundle_values) : "RP_BUNDLE_${key}=${var.rp_bundle_values[key]}"])
}

variable "tre_core_tags" {
type = map(string)
description = "Tags to be applied to all resources"
default = {}
}
6 changes: 6 additions & 0 deletions core/terraform/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -180,3 +180,9 @@ variable "is_cosmos_defined_throughput" {
type = bool
default = false
}

variable "tags" {
type = map(string)
description = "Tags to be applied to all resources"
default = {}
}
12 changes: 8 additions & 4 deletions devops/terraform/bootstrap.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,17 @@ set -o nounset

# Baseline Azure resources
echo -e "\n\e[34m»»» 🤖 \e[96mCreating resource group and storage account\e[0m..."
# shellcheck disable=SC2154
az group create --resource-group "$TF_VAR_mgmt_resource_group_name" --location "$LOCATION" -o table
# shellcheck disable=SC2154
# shellcheck disable=SC2154,SC2046
az group create --resource-group "$TF_VAR_mgmt_resource_group_name" \
--location "$LOCATION" \
-o table \
--tags $(echo "$TAGS" | jq -r 'to_entries | map("\(.key)=\(.value)")| join(" ")')
# shellcheck disable=SC2154,SC2046
az storage account create --resource-group "$TF_VAR_mgmt_resource_group_name" \
--name "$TF_VAR_mgmt_storage_account_name" --location "$LOCATION" \
--allow-blob-public-access false \
--kind StorageV2 --sku Standard_LRS -o table
--kind StorageV2 --sku Standard_LRS -o table \
--tags $(echo "$TAGS" | jq -r 'to_entries | map("\(.key)=\(.value)")| join(" ")')

# Blob container
# shellcheck disable=SC2154
Expand Down
15 changes: 13 additions & 2 deletions devops/terraform/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,11 @@ resource "azurerm_resource_group" "mgmt" {
name = var.mgmt_resource_group_name
location = var.location

tags = {
# tflint-ignore: azurerm_resource_missing_tags
tags = merge(var.tags, {
project = "Azure Trusted Research Environment"
source = "https://github.com/microsoft/AzureTRE/"
}
})

lifecycle { ignore_changes = [tags] }
}
Expand All @@ -25,6 +26,9 @@ resource "azurerm_storage_account" "state_storage" {
account_replication_type = "LRS"
allow_nested_items_to_be_public = false

# tflint-ignore: azurerm_resource_missing_tags
tags = var.tags

lifecycle { ignore_changes = [tags] }
}

Expand All @@ -36,6 +40,9 @@ resource "azurerm_container_registry" "shared_acr" {
sku = var.acr_sku
admin_enabled = true

# tflint-ignore: azurerm_resource_missing_tags
tags = var.tags

lifecycle { ignore_changes = [tags] }
}

Expand All @@ -44,6 +51,10 @@ resource "azurerm_container_registry" "shared_acr" {
resource "azurerm_container_registry_task" "tredev_purge" {
name = "tredev_purge"
container_registry_id = azurerm_container_registry.shared_acr.id

# tflint-ignore: azurerm_resource_missing_tags
tags = var.tags

platform {
os = "Linux"
architecture = "amd64"
Expand Down
7 changes: 7 additions & 0 deletions devops/terraform/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -23,3 +23,10 @@ variable "acr_name" {
type = string
description = "Name of ACR"
}


variable "tags" {
type = map(string)
description = "Tags to be applied to all resources"
default = {}
}
6 changes: 6 additions & 0 deletions templates/shared_services/admin-vm/parameters.json
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,12 @@
"source": {
"env": "ARM_ENVIRONMENT"
}
},
{
"name": "tags",
"source": {
"env": "TAGS"
}
}
]
}
Loading