Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

move to new signing process for release build and prep for 1.20.0 #1625

Merged
merged 36 commits into from
Apr 22, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
150d81e
First cut at new release/signing build yml
JamesWTruher Jan 6, 2021
7943547
remove display name from incorrect location
JamesWTruher Jan 6, 2021
232cb28
change paths for changes in build system
JamesWTruher Jan 6, 2021
21a029a
add debugging info
JamesWTruher Jan 6, 2021
daadefc
Be sure to build all versions
JamesWTruher Jan 7, 2021
1faf27b
change debugging code for 'Copy signed files'
JamesWTruher Jan 7, 2021
d47c013
be sure to be in the correct location for creating the nupkg
JamesWTruher Jan 7, 2021
fd6d5c2
Change nupkg building logic
JamesWTruher Jan 7, 2021
57d4635
make not finding dotnet.exe a non fatal error for loading the module
JamesWTruher Jan 7, 2021
308f923
fix path to suppression file
JamesWTruher Jan 7, 2021
2d1f8f4
debugging statements for compliance
JamesWTruher Jan 7, 2021
5495b95
fix typo for credscan suppressions
JamesWTruher Jan 8, 2021
b535c72
Update to publish build directory
JamesWTruher Jan 8, 2021
caf23fa
Be sure to publish the build directory
JamesWTruher Jan 8, 2021
f9231e5
published signed files need their own name
JamesWTruher Jan 8, 2021
91e0921
add exception for README.md for cred scanner
JamesWTruher Jan 8, 2021
7fafc46
try to get binskim to work with more direct path to assemblies
JamesWTruher Jan 8, 2021
484c051
debugging to find issues with signing files
JamesWTruher Jan 8, 2021
8a6aeb2
fix typo
JamesWTruher Jan 8, 2021
0794d19
more debugging
JamesWTruher Jan 8, 2021
39edbf6
remove debugging
JamesWTruher Jan 8, 2021
a55814b
remove publish of nupkg
JamesWTruher Jan 8, 2021
275cbf2
remove newtonsoft signing stage
JamesWTruher Jan 8, 2021
3dab1f9
try the newton soft signing again
JamesWTruher Jan 8, 2021
29e870d
disable newtonsoft signing
JamesWTruher Jan 8, 2021
5e0a1de
try a 2 step process for signing
JamesWTruher Jan 11, 2021
f0d5e1c
remove most of the debugging code
JamesWTruher Jan 11, 2021
bb976f7
calculate version and publish nupkg
JamesWTruher Jan 12, 2021
7f55032
fix 1 for module version
JamesWTruher Jan 12, 2021
a66cb27
typo for module version
JamesWTruher Jan 12, 2021
2f1cf42
Add a lot of comments to the yaml
JamesWTruher Jan 12, 2021
b6415c3
suppress warnings for using PSRepository cmdlets
JamesWTruher Jan 13, 2021
43935ad
bump version to 1.20.0
JamesWTruher Feb 10, 2021
debd5fc
change to use allowlist in documentation and tests
JamesWTruher Feb 11, 2021
c0714ab
Update .ci/releaseBuild.yml
JamesWTruher Apr 21, 2021
6edaba1
Apply suggestions from code review
rjmholt Apr 21, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
199 changes: 199 additions & 0 deletions .ci/releaseBuild.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,199 @@
# The name of the build that will be seen in mscodehub
name: PSSA-Release-$(Build.BuildId)
# how is the build triggered
# since this is a release build, no trigger as it's a manual release
trigger: none

pr:
branches:
include:
- master
- release*

# variables to set in the build environment
variables:
DOTNET_CLI_TELEMETRY_OPTOUT: 1
POWERSHELL_TELEMETRY_OPTOUT: 1

# since this build relies on templates, we need access to those
# This needs a service connection in the build to work
# the *name* of the service connection must be the same as the endpoint
resources:
repositories:
- repository: ComplianceRepo
type: github
endpoint: ComplianceGHRepo
name: PowerShell/compliance
# this can be any branch of your choosing
ref: master

# the stages in this build. There are 2
# the assumption for script analyzer is that test is done as part of
# CI so we needn't do it here
stages:
- stage: Build
displayName: Build
pool:
name: Package ES CodeHub Lab E
jobs:
- job: Build_Job
displayName: Build Microsoft.PowerShell.ScriptAnalyzer
# note the variable reference to ESRP.
# this must be created in Project -> Pipelines -> Library -> VariableGroups
# where it describes the link to the SigningServer
variables:
- group: ESRP
steps:
- checkout: self

# the steps for building the module go here
- pwsh: |
Set-Location "$(Build.SourcesDirectory)/OSS_Microsoft_PSSA"
try { ./build.ps1 -Configuration Release -All } catch { throw $_ }
displayName: Execute build

# these are setting vso variables which will be persisted between stages
- pwsh: |
$signSrcPath = "$(Build.SourcesDirectory)/OSS_Microsoft_PSSA/out"
# Set signing src path variable
$vstsCommandString = "vso[task.setvariable variable=signSrcPath]${signSrcPath}"
Write-Host "sending $vstsCommandString"
Write-Host "##$vstsCommandString"

$signOutStep1 = "$(Build.SourcesDirectory)/OSS_Microsoft_PSSA/Step1"
$null = New-Item -ItemType Directory -Path $signOutStep1
# Set signing out path variable
$vstsCommandString = "vso[task.setvariable variable=signOutStep1]${signOutStep1}"
Write-Host "sending $vstsCommandString"
Write-Host "##$vstsCommandString"

$signOutPath = "$(Build.SourcesDirectory)/OSS_Microsoft_PSSA/signed"
$null = New-Item -ItemType Directory -Path $signOutPath
# Set signing out path variable
$vstsCommandString = "vso[task.setvariable variable=signOutPath]${signOutPath}"
Write-Host "sending $vstsCommandString"
Write-Host "##$vstsCommandString"

# Set path variable for guardian codesign validation
$vstsCommandString = "vso[task.setvariable variable=GDN_CODESIGN_TARGETDIRECTORY]${signOutPath}"
Write-Host "sending $vstsCommandString"
Write-Host "##$vstsCommandString"

# Get version and create a variable
$moduleData = Import-PowerShellDataFile "$(Build.SourcesDirectory)/OSS_Microsoft_PSSA/Engine/PSScriptAnalyzer.psd1"
$moduleVersion = $moduleData.ModuleVersion
$vstsCommandString = "vso[task.setvariable variable=moduleVersion]${moduleVersion}"
Write-Host "sending $vstsCommandString"
Write-Host "##$vstsCommandString"


displayName: Setup variables for signing

# checkout the Compliance repository so it can be used to do the actual signing
- checkout: ComplianceRepo

# in script analyzer, we must sign with 2 different certs
# the normal cert for MS created items and the 3rd party cert
# this the MS authored step
# Because this needs 2 certs, we do it in 2 steps.
# the first step signs the binaries and puts them in a staging directory which
# will then be used for the second step.
- template: EsrpSign.yml@ComplianceRepo
parameters:
# the folder which contains the binaries to sign
buildOutputPath: $(signSrcPath)
# the location to put the signed output
signOutputPath: $(signOutStep1)
# the certificate ID to use
certificateId: "CP-230012"
# use minimatch because we need to exclude the NewtonSoft assembly
useMinimatch: true
# the file pattern to use - newtonSoft is excluded
pattern: |
**\*.psd1
**\*.psm1
**\*.ps1xml
**\Microsoft*.dll

# this is the second step of the signing.
# note that the buildOutputPath (where we get the files to sign)
# is the same as the signOutputPath in the previous step
# at the end of this step we will have all the files signed that should be
# signOutPath is the location which contains the files we will use to make the module
- template: EsrpSign.yml@ComplianceRepo
parameters:
# the folder which contains the binaries to sign
buildOutputPath: $(signOutStep1)
# the location to put the signed output
signOutputPath: $(signOutPath)
# the certificate ID to use
# we'll need to change this to the 3rd party cert id
certificateId: "CP-231522"
# the file pattern to use - only sign newtonsoft
pattern: 'Newtonsoft*.dll'

# now create the nupkg which we will use to publish the module
# to the powershell gallery (not part of this yaml)
- pwsh: |
Set-Location "$(Build.SourcesDirectory)/OSS_Microsoft_PSSA"
./build -BuildNupkg -signed
displayName: Create nupkg for publishing

# finally publish the parts of the build which will be used in the next stages
# if it's not published, the subsequent stages will not be able to access it.
# This is the build directory (it contains all of the dll/pdb files)
- publish: "$(Build.SourcesDirectory)/OSS_Microsoft_PSSA"
artifact: build
displayName: publish build directory

# export the nupkg only which will be used in the release pipeline
- publish: "$(signOutPath)/PSScriptAnalyzer.$(moduleVersion).nupkg"
artifact: nupkg
displayName: Publish module nupkg

# Now on to the compliance stage
- stage: compliance
displayName: Compliance
dependsOn: Build
jobs:
- job: Compliance_Job
pool:
name: Package ES CodeHub Lab E
steps:
- checkout: self
- checkout: ComplianceRepo
- download: current
artifact: build

# use the templates in the compliance repo
# since script analyzer has modules, we're using the assembly-module-compliance template
# if you don't have assemblies, you should use script-module-compliance template
- template: assembly-module-compliance.yml@ComplianceRepo
parameters:
# component-governance - the path to sources
sourceScanPath: '$(Build.SourcesDirectory)/OSS_Microsoft_PSSA'
# binskim - this isn't recursive, so you need the path to the assemblies
AnalyzeTarget: '$(Pipeline.Workspace)\build\bin\PSV7Release\netcoreapp3.1\*.dll'
# credscan - scan the repo for credentials
# you can suppress some files with this.
suppressionsFile: '$(Build.SourcesDirectory)/OSS_Microsoft_PSSA/tools/ReleaseBuild/CredScan.Suppressions.json'
# TermCheck
optionsRulesDBPath: ''
optionsFTPath: ''
# tsa-upload
# the compliance scanning must be uploaded, which you need to request
codeBaseName: 'PSSA_202004'
# selections
APIScan: false # set to false when not using Windows APIs.

#- template: template/publish.yml
# parameters:
# stageName: AzArtifactsFeed
# environmentName:
# feedCredential:

#- template: template/publish.yml
# parameters:
# stageName: NuGet
# environmentName: PSMarkdownRenderNuGetApproval
# feedCredential: NugetOrgPush
4 changes: 2 additions & 2 deletions Engine/Engine.csproj
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<VersionPrefix>1.19.1</VersionPrefix>
<VersionPrefix>1.20.0</VersionPrefix>
<TargetFrameworks>netcoreapp3.1;net452</TargetFrameworks>
<AssemblyName>Microsoft.Windows.PowerShell.ScriptAnalyzer</AssemblyName>
<AssemblyVersion>1.19.1</AssemblyVersion>
<AssemblyVersion>1.20.0</AssemblyVersion>
<PackageId>Engine</PackageId>
<RootNamespace>Microsoft.Windows.PowerShell.ScriptAnalyzer</RootNamespace> <!-- Namespace needs to match Assembly name for ressource binding -->
</PropertyGroup>
Expand Down
2 changes: 1 addition & 1 deletion Engine/PSScriptAnalyzer.psd1
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Author = 'Microsoft Corporation'
RootModule = 'PSScriptAnalyzer.psm1'

# Version number of this module.
ModuleVersion = '1.19.1'
ModuleVersion = '1.20.0'

# ID used to uniquely identify this module
GUID = 'd6245802-193d-4068-a631-8863a4342a18'
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<VersionPrefix>1.19.1</VersionPrefix>
<VersionPrefix>1.20.0</VersionPrefix>
<TargetFrameworks>netstandard2.0;net452</TargetFrameworks>
<AssemblyVersion>1.19.1</AssemblyVersion>
<AssemblyVersion>1.20.0</AssemblyVersion>
</PropertyGroup>

<PropertyGroup Condition="'$(TargetFramework)' == 'netstandard2.0'">
Expand Down
2 changes: 1 addition & 1 deletion RuleDocumentation/AvoidUsingCmdletAliases.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ To prevent `PSScriptAnalyzer` from flagging your preferred aliases, create an al
@{
'Rules' = @{
'PSAvoidUsingCmdletAliases' = @{
'Whitelist' = @('cd')
'allowlist' = @('cd')
}
}
}
Expand Down
4 changes: 2 additions & 2 deletions Rules/Rules.csproj
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<VersionPrefix>1.19.1</VersionPrefix>
<VersionPrefix>1.20.0</VersionPrefix>
<TargetFrameworks>netcoreapp3.1;net452</TargetFrameworks>
<AssemblyName>Microsoft.Windows.PowerShell.ScriptAnalyzer.BuiltinRules</AssemblyName>
<AssemblyVersion>1.19.1</AssemblyVersion>
<AssemblyVersion>1.20.0</AssemblyVersion>
<PackageId>Rules</PackageId>
<RootNamespace>Microsoft.Windows.PowerShell.ScriptAnalyzer</RootNamespace> <!-- Namespace needs to match Assembly name for ressource binding -->
</PropertyGroup>
Expand Down
14 changes: 7 additions & 7 deletions Tests/Engine/Settings.tests.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -109,23 +109,23 @@ Describe "Settings Class" {
$settingsHashtable = @{
Rules = @{
PSAvoidUsingCmdletAliases = @{
WhiteList = @("cd", "cp")
allowlist = @("cd", "cp")
}
}
}
$settings = New-Object -TypeName $settingsTypeName -ArgumentList $settingsHashtable
}

It "Should return the rule arguments" {
$settings.RuleArguments["PSAvoidUsingCmdletAliases"]["WhiteList"].Count | Should -Be 2
$settings.RuleArguments["PSAvoidUsingCmdletAliases"]["WhiteList"][0] | Should -Be "cd"
$settings.RuleArguments["PSAvoidUsingCmdletAliases"]["WhiteList"][1] | Should -Be "cp"
$settings.RuleArguments["PSAvoidUsingCmdletAliases"]["allowlist"].Count | Should -Be 2
$settings.RuleArguments["PSAvoidUsingCmdletAliases"]["allowlist"][0] | Should -Be "cd"
$settings.RuleArguments["PSAvoidUsingCmdletAliases"]["allowlist"][1] | Should -Be "cp"
}

It "Should Be case insensitive" {
$settings.RuleArguments["psAvoidUsingCmdletAliases"]["whiteList"].Count | Should -Be 2
$settings.RuleArguments["psAvoidUsingCmdletAliases"]["whiteList"][0] | Should -Be "cd"
$settings.RuleArguments["psAvoidUsingCmdletAliases"]["whiteList"][1] | Should -Be "cp"
$settings.RuleArguments["psAvoidUsingCmdletAliases"]["allowlist"].Count | Should -Be 2
$settings.RuleArguments["psAvoidUsingCmdletAliases"]["allowlist"][0] | Should -Be "cd"
$settings.RuleArguments["psAvoidUsingCmdletAliases"]["allowlist"][1] | Should -Be "cp"
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
}
PSAvoidUsingCmdletAliases = @{
# only allowlist verbs from *-Object cmdlets
Whitelist = @(
allowlist = @(
'%',
'?',
'compare',
Expand Down Expand Up @@ -60,4 +60,4 @@
CheckSeparator = $true
}
}
}
}
2 changes: 1 addition & 1 deletion Tests/Engine/SettingsTest/Project1/ExplicitSettings.psd1
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
"ExcludeRules" = @("PSShouldProcess", "PSAvoidUsingWMICmdlet", "PSUseCmdletCorrectly")
"rules" = @{
PSAvoidUsingCmdletAliases = @{
WhiteList = @("cd", "cp")
allowlist = @("cd", "cp")
}

PSUseConsistentIndentation = @{
Expand Down
4 changes: 2 additions & 2 deletions Tests/Rules/AvoidUsingAlias.tests.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ Configuration MyDscConfiguration {
$settings = @{
'Rules' = @{
'PSAvoidUsingCmdletAliases' = @{
'Whitelist' = @('cd')
'allowlist' = @('cd')
}
}
}
Expand All @@ -83,7 +83,7 @@ Configuration MyDscConfiguration {
$settings = @{
'Rules' = @{
'PSAvoidUsingCmdletAliases' = @{
'Whitelist' = @('cd')
'allowlist' = @('cd')
}
}
}
Expand Down
4 changes: 2 additions & 2 deletions Tests/Rules/TestSettings/AvoidAliasSettings.psd1
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
@{
'Rules' = @{
'PSAvoidUsingCmdletAliases' = @{
'Whitelist' = @('cd')
'allowlist' = @('cd')
}
}
}
}
11 changes: 10 additions & 1 deletion build.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,13 @@ param(
[switch] $Bootstrap,

[Parameter(ParameterSetName='BuildAll')]
[switch] $Catalog
[switch] $Catalog,

[Parameter(ParameterSetName='Package')]
[switch] $BuildNupkg,

[Parameter(ParameterSetName='Package')]
[switch] $Signed

)

Expand Down Expand Up @@ -85,6 +91,9 @@ END {
Install-DotNet
return
}
"Package" {
Start-CreatePackage -signed:$Signed
}
"Test" {
Test-ScriptAnalyzer -InProcess:$InProcess
return
Expand Down
Loading