Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Version 0.2 cleanup #549

Merged
merged 1 commit into from
Jan 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 33 additions & 35 deletions docs/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,41 +70,23 @@ Legend:
> </blockquote>
> 2. Return a single boolean value from the Remove-FinOpsHub command.

🖥️ PowerShell
{: .fs-5 .fw-500 .mt-4 mb-0 }

> ➕ Added:
>
> 1. New-FinOpsCostExport
> 2. Remove-FinOpsHubScope


📒 Cost Optimization Workbook

> ➕ Added:
>
> 1. Storage: Identify Idle Backups: Review protected items' backup activity to spot items not backed up in the last 90 days.
> 2. Storage: Review Replication Settings: Evaluate and improve your backup strategy by identifying resources with default geo-redundant storage (GRS) replication.
> 3. Networking: Azure Firewall Premium Features: Identify Azure Firewalls with Premium SKU and ensure associated policies leverage premium-only features.
> 4. Networking: Firewall Optimization: Streamline Azure Firewall usage by centralizing instances in the hub virtual network or Virtual WAN secure hub.
> 5. Top 10 services: Improved Monitoring Tabs: Enhance your monitoring experience with updated Azure Advisor recommendations for Log Analytics.
>
> 🛠️ Fixed:
>
> 1. AHB: Fixed AHB to support Windows 10/Windows 11

<br>

## 🚚 v0.2

🌐 Open data
🏦 FinOps hubs
{: .fs-5 .fw-500 .mt-4 mb-0 }

> ➕ Added:
> 🛠️ Fixed:
>
> 1. [Resource types](./open-data/README.md#-resource-types) to map Azure resource types to friendly display names.
> 2. [Get-FinOpsResourceType](./powershell/data/Get-FinOpsResourceType.md) PowerShell command to support resource type to display name mapping.
> 3. [Sample exports](./open-data/README.md#-sample-data) for each of the datasets that can be exported from Cost Management.
> 1. Fixed error in some China regions where deployment scripts were not supported ([#259](https://github.com/microsoft/finops-toolkit/issues/259)).
>
> ✏️ Changed:
>
> 1. Switch from amortized cost exports to FOCUS cost exports.
> 2. Updated ingestion container month folders from `yyyyMMdd-yyyyMMdd` to `yyyyMM`.
> 3. Renamed **msexports_extract** pipeline to **msexports_ExecuteETL**.
> 4. Renamed **msexports_transform** pipeline to **msexports_ETL_ingestion**.

📊 Power BI reports
{: .fs-5 .fw-500 .mt-4 mb-0 }
Expand All @@ -130,19 +112,26 @@ Legend:
> 4. Fixed null error when Billing Account ID is empty ([#473](https://github.com/microsoft/finops-toolkit/issues/473)).
> 5. Added missing commitment discount refunds to the actual cost data ([#447](https://github.com/microsoft/finops-toolkit/issues/447)).

🏦 FinOps hubs
📒 Azure Monitor workbooks
{: .fs-5 .fw-500 .mt-4 mb-0 }

> 🛠️ Fixed:
> ➕ Added:
>
> 1. Fixed error in some China regions where deployment scripts were not supported ([#259](https://github.com/microsoft/finops-toolkit/issues/259)).
> - [Optimization workbook](./optimization-workbook/README.md):
> 1. Storage: Identify Idle Backups: Review protected items' backup activity to spot items not backed up in the last 90 days.
> 2. Storage: Review Replication Settings: Evaluate and improve your backup strategy by identifying resources with default geo-redundant storage (GRS) replication.
> 3. Networking: Azure Firewall Premium Features: Identify Azure Firewalls with Premium SKU and ensure associated policies leverage premium-only features.
> 4. Networking: Firewall Optimization: Streamline Azure Firewall usage by centralizing instances in the hub virtual network or Virtual WAN secure hub.
>
> ✏️ Changed:
>
> 1. Switch from amortized cost exports to FOCUS cost exports.
> 2. Updated ingestion container month folders from `yyyyMMdd-yyyyMMdd` to `yyyyMM`.
> 3. Renamed **msexports_extract** pipeline to **msexports_ExecuteETL**.
> 4. Renamed **msexports_transform** pipeline to **msexports_ETL_ingestion**.
> - [Optimization workbook](./optimization-workbook/README.md):
> 1. Top 10 services: Improved Monitoring tabs: Enhance your monitoring experience with updated Azure Advisor recommendations for Log Analytics.
>
> 🛠️ Fixed:
>
> - [Optimization workbook](./optimization-workbook/README.md):
> 1. AHB: Fixed AHB to support Windows 10/Windows 11

🖥️ PowerShell
{: .fs-5 .fw-500 .mt-4 mb-0 }
Expand All @@ -168,6 +157,15 @@ Legend:
> - With native support for FOCUS 1.0 preview in Cost Management, we are deprecating both commands, which only support FOCUS 0.5.
> - If you would like to see the PowerShell commands updated to 1.0 preview, please let us know in discussions or via a GitHub issue.

🌐 Open data
{: .fs-5 .fw-500 .mt-4 mb-0 }

> ➕ Added:
>
> 1. [Resource types](./open-data/README.md#-resource-types) to map Azure resource types to friendly display names.
> 2. [Get-FinOpsResourceType](./powershell/data/Get-FinOpsResourceType.md) PowerShell command to support resource type to display name mapping.
> 3. [Sample exports](./open-data/README.md#-sample-data) for each of the datasets that can be exported from Cost Management.

<br>

## 🛠️ v0.1.1
Expand Down
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "ftk",
"version": "0.1.2",
"version": "0.2.0",
"description": "Starter kits, scripts, and advanced solutions to accelerate your FinOps journey in the Microsoft Cloud.",
"main": "index.js",
"directories": {
Expand Down
101 changes: 62 additions & 39 deletions src/scripts/Publish-Toolkit.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@
.PARAMETER Build
Optional. Indicates whether the the Build-Toolkit command should be executed first. Default = false.

.PARAMETER Commit
Optional. Indicates whether the changes should be committed to the Git repo. Default = false.
.PARAMETER Branch
Optional. Indicates whether the changes should be committed to a new branch in the Git repo. Default = false.

.EXAMPLE
./Publish-Toolkit "finops-hub"
Expand All @@ -39,47 +39,50 @@ Param(
[string]$RegistryRepo = "bicep-registry-modules",
[string]$appInsightsRepo = "Application-Insights-Workbooks",
[switch]$Build,
[switch]$Commit
[Alias("Commit")][switch]$Branch
)

# Use the debug flag from common parameters to determine whether to run in debug mode
$Debug = $DebugPreference -eq "Continue"

# Repo config
$repoConfig = @{
aqt = @{
aqt = @{
mainBranch = 'master'
possibleNames = @($QuickstartRepo, 'azure-quickstart-templates', 'aqt')
relativePath = "quickstarts/microsoft.costmanagement"
requiredFiles = @("main.bicep", "metadata.json", "README.md", "azuredeploy.parameters.json")
}
brm = @{
brm = @{
mainBranch = 'main'
possibleNames = @($RegistryRepo, 'bicep-registry-modules', 'brm', 'br')
relativePath = "modules/cost"
requiredFiles = @("main.bicep", "main.json", "metadata.json", "README.md", "version.json")
}
appInsights = @{
mainBranch = 'master'
possibleNames = @($appInsightsRepo, 'Application-Insights-Workbooks')
relativePath = "Workbooks/Azure Advisor/Cost Optimization"
requiredFiles = @("CostOptimization.workbook","Storage.workbook","Networking.workbook","Compute.workbook","AHB.workbook","Reservations.workbook")
mainBranch = 'master'
possibleNames = @($appInsightsRepo, 'Application-Insights-Workbooks')
relativePath = "Workbooks/Azure Advisor/Cost Optimization"
requiredFiles = @("CostOptimization.workbook", "Storage.workbook", "Networking.workbook", "Compute.workbook", "AHB.workbook", "Reservations.workbook")
}
}

# Build toolkit if requested
if ($Build) {
if ($Build)
{
./Build-Toolkit $Template
}

$relDir = "../../release"

# Find the local repo folder
function Find-Repo($config, [string]$templateName) {
function Find-Repo($config, [string]$templateName)
{
Write-Debug "Verifying repo..."
return $config.possibleNames | ForEach-Object {
$path = "../../../$_"
if (Test-Path "$path/$($config.relativePath)") {
if (Test-Path "$path/$($config.relativePath)")
{
Write-Debug " Found @ $path"
$config | Add-Member path "$path/$($config.relativePath)/$templateName" -Force
return $config
Expand All @@ -88,6 +91,9 @@ function Find-Repo($config, [string]$templateName) {
}
}

# Get version for branch name and commit message
$ver = & "$PSScriptRoot/Get-Version.ps1"

# Loop thru templates
Get-ChildItem "$relDir/$Template*" -Directory `
| ForEach-Object {
Expand All @@ -98,26 +104,34 @@ Get-ChildItem "$relDir/$Template*" -Directory `
Write-Host "Publishing template $templateName..."

# Confirm metadata.json exists
if (-not (Test-Path "$templateDir/metadata.json")) {
if (-not (Test-Path "$templateDir/metadata.json"))
{
Write-Error "Template folder invalid. metadata.json required. Please ensure all required files are present. See src/<type>/README.md for details."
return
}

# Find target repo
$schema = (Get-Content "$templateDir/metadata.json" -Raw | ConvertFrom-Json).PSObject.Properties['$schema'].Value
if ($schema.Contains('azure-quickstart-templates')) {
if ($schema.Contains('azure-quickstart-templates'))
{
$repo = Find-Repo $repoConfig.aqt $templateName
if (-not $repo) {
if (-not $repo)
{
Write-Error "Azure Quickstart Templates repo not found. Please close the repo locally or specify the QuickstartRepo parameter."
return
}
} elseif ($schema.Contains('bicep-registry-module')) {
}
elseif ($schema.Contains('bicep-registry-module'))
{
$repo = Find-Repo $repoConfig.brm $templateName
if (-not $repo) {
if (-not $repo)
{
Write-Error "Bicep Registry repo not found. Please close the repo locally or specify the RegistryRepo parameter."
return
}
} else {
}
else
{
Write-Error "Template schema not recognized: $schema"
return
}
Expand All @@ -126,76 +140,85 @@ Get-ChildItem "$relDir/$Template*" -Directory `
# Validate release requirements
Write-Debug "Verifying required files..."
$repo.requiredFiles | ForEach-Object {
if (-not (Test-Path "$relDir/$templateName/$_")) {
if (-not (Test-Path "$relDir/$templateName/$_"))
{
Write-Error "$_ required. Please add $_ to the template folder."
return
}
}

# Switch to main branch in local fork
if ($Commit) {
if ($Branch)
{
Push-Location
if (-not (Test-Path ($repo.path))) {
if (-not (Test-Path ($repo.path)))
{
./New-Directory $repo.path
}
Set-Location $repo.path

# Validate local repo is clean
if (-not (git status | Select-String 'working tree clean')) {
if (-not (git status | Select-String 'working tree clean'))
{
Write-Error 'Local repo has uncommitted changes. Please commit or stash changes and try again.'
Pop-Location
return
}

# Switch to master branch
if (-not (git rev-parse --abbrev-ref HEAD) -eq 'master') {
if (-not (git rev-parse --abbrev-ref HEAD) -eq 'master')
{
Write-Host " Switching to $($repo.mainBranch) branch..."
git checkout $repo.mainBranch --quiet
}

# Pull latest changes
if (-not (git status | Select-String 'Your branch is behind')) {
if (-not (git status | Select-String 'Your branch is behind'))
{
Write-Host ' Pulling latest changes...'
git pull --rebase --quiet
}

# Create new branch if needed
if (-not (git status | Select-String 'Your branch is up to date')) {
$branch = "$($templateName)_$(Get-Date -Format yyMMddHHmm)"
Write-Host " Creating new $branch..."
git checkout -b $branch --quiet
git branch --set-upstream-to="origin/$($repo.mainBranch)" --quiet
git pull --rebase --quiet
}
# Create new branch
$branchName = "$($templateName)-$($ver)_$(Get-Date -Format yyMMdd)"
Write-Host " Creating new $branchName branch..."
git checkout -b $branchName --quiet
git branch --set-upstream-to="origin/$($repo.mainBranch)" --quiet
git pull --rebase --quiet

Pop-Location
}

# Copy files
Write-Host ' Copying release files...'
if (Test-Path $repo.path) {
if (Test-Path $repo.path)
{
Remove-Item $repo.path -Recurse -Force
}
./New-Directory $repo.path
Get-ChildItem "$relDir/$templateName" -Exclude .buildignore | Copy-Item -Destination $repo.path -Recurse

# Commit changes
if ($Commit) {
if ($Branch)
{
Push-Location
Set-Location $repo.path
Write-Host ' Committing updates...'
git add .
$isNew = ((git status) | Select-String "new file: +$($repo.relativePath)/$templateName/main.bicep").length -eq 1
if ($isNew) {
if ($isNew)
{
$commitMessage = "New FinOps toolkit template - $templateName"
} else {
}
else
{
$commitMessage = "FinOps toolkit $ver - $templateName update"
}
git commit --message $commitMessage --quiet
$branch = git rev-parse --abbrev-ref HEAD
git push origin $branch --quiet
$branchName = git rev-parse --abbrev-ref HEAD
git push origin $branchName --quiet
$fork = git remote get-url origin | Select-String "github.com/([^/]+/[^/\.]+)" | % { $_.Matches[0].Groups[1].Value.Replace('/', ':') }
Write-Host " Create PR @ https://github.com/Azure/$($repo.possibleNames[1])/compare/$($repo.mainBranch)...$($fork + ':' + $branch)?expand=1"
Write-Host " Create PR @ https://github.com/Azure/$($repo.possibleNames[1])/compare/$($repo.mainBranch)...$($fork + ':' + $branchName)?expand=1"
Pop-Location
}

Expand Down
14 changes: 7 additions & 7 deletions src/scripts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -222,17 +222,17 @@ Examples:

[Publish-Toolkit.ps1](./Publish-Toolkit.ps1) publishes a template to the Azure Quickstart Templates repository.

| Parameter | Description |
| -------------- | ------------------------------------------------------------------------------------------------------------------------ |
| `‑Template` | Required. Name of the template or module to deploy. |
| `‑Destination` | Required. Path to the local clone of the Azure Quickstart Templates repository. |
| `‑Build` | Optional. Indicates whether the the `Build-Toolkit` command should be executed first. Default = `false`. |
| `‑Commit` | Optional. Indicates whether to commit the changes and start a pull request in the Azure Quickstart Templates repository. |
| Parameter | Description |
| -------------- | -------------------------------------------------------------------------------------------------------- |
| `‑Template` | Required. Name of the template or module to deploy. |
| `‑Destination` | Required. Path to the local clone of the Azure Quickstart Templates repository. |
| `‑Build` | Optional. Indicates whether the the `Build-Toolkit` command should be executed first. Default = `false`. |
| `‑Branch` | Optional. Indicates whether to commit the changes to a new branch in the Git repo. Default = `false`. |

Example:

```powershell
./Publish-Toolkit "finops-hub" "../../../aqt" -Build -Commit
./Publish-Toolkit "finops-hub" "../../../aqt" -Build -Branch
```

<br>
Expand Down
Loading