Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove automatic checksumming #40

Open
TheCakeIsNaOH opened this issue Jan 14, 2024 · 4 comments
Open

Remove automatic checksumming #40

TheCakeIsNaOH opened this issue Jan 14, 2024 · 4 comments

Comments

@TheCakeIsNaOH
Copy link
Member

One potential improvement to make before releasing a version of chocolatey-au would be to remove automatic checksumming.

function get_checksum()
{
function invoke_installer() {
if (!(Test-Path tools\chocolateyInstall.ps1)) { " aborted, chocolateyInstall not found for this package" | result; return }
Import-Module "$choco_tmp_path\helpers\chocolateyInstaller.psm1" -Force -Scope Global
if ($ChecksumFor -eq 'none') { "Automatic checksum calculation is disabled"; return }
if ($ChecksumFor -eq 'all') { $arch = '32','64' } else { $arch = $ChecksumFor }
$Env:ChocolateyPackageFolder = [System.IO.Path]::GetFullPath("$Env:TEMP\chocolatey\$($package.Name)") #https://github.com/majkinetor/au/issues/32
$pkg_path = Join-Path $Env:ChocolateyPackageFolder $global:Latest.Version
New-Item -Type Directory -Force $pkg_path | Out-Null
$Env:ChocolateyPackageName = "chocolatey\$($package.Name)"
$Env:ChocolateyPackageVersion = $global:Latest.Version.ToString()
$Env:ChocolateyAllowEmptyChecksums = 'true'
foreach ($a in $arch) {
$Env:chocolateyForceX86 = if ($a -eq '32') { 'true' } else { '' }
try {
#rm -force -recurse -ea ignore $pkg_path
.\tools\chocolateyInstall.ps1 | result
} catch {
if ( "$_" -notlike 'au_break: *') { throw $_ } else {
$filePath = "$_" -replace 'au_break: '
if (!(Test-Path $filePath)) { throw "Can't find file path to checksum" }
$item = Get-Item $filePath
$type = if ($global:Latest.ContainsKey('ChecksumType' + $a)) { $global:Latest.Item('ChecksumType' + $a) } else { 'sha256' }
$hash = (Get-FileHash $item -Algorithm $type | ForEach-Object Hash).ToLowerInvariant()
if (!$global:Latest.ContainsKey('ChecksumType' + $a)) { $global:Latest.Add('ChecksumType' + $a, $type) }
if (!$global:Latest.ContainsKey('Checksum' + $a)) {
$global:Latest.Add('Checksum' + $a, $hash)
"Package downloaded and hash calculated for $a bit version" | result
} else {
$expected = $global:Latest.Item('Checksum' + $a)
if ($hash -ne $expected) { throw "Hash for $a bit version mismatch: actual = '$hash', expected = '$expected'" }
"Package downloaded and hash checked for $a bit version" | result
}
}
}
}
}
function fix_choco {
Start-Sleep -Milliseconds (Get-Random 500) #reduce probability multiple updateall threads entering here at the same time (#29)
# Copy choco modules once a day
if (Test-Path $choco_tmp_path) {
$ct = Get-Item $choco_tmp_path | ForEach-Object creationtime
if (((get-date) - $ct).Days -gt 1) { Remove-Item -recurse -force $choco_tmp_path } else { Write-Verbose 'Chocolatey copy is recent, aborting monkey patching'; return }
}
Write-Verbose "Monkey patching chocolatey in: '$choco_tmp_path'"
Copy-Item -recurse -force $Env:ChocolateyInstall\helpers $choco_tmp_path\helpers
if (Test-Path $Env:ChocolateyInstall\extensions) { Copy-Item -recurse -force $Env:ChocolateyInstall\extensions $choco_tmp_path\extensions }
$fun_path = "$choco_tmp_path\helpers\functions\Get-ChocolateyWebFile.ps1"
(Get-Content $fun_path) -replace '^\s+return \$fileFullPath\s*$', ' throw "au_break: $fileFullPath"' | Set-Content $fun_path -ea ignore
}
"Automatic checksum started" | result
# Copy choco powershell functions to TEMP dir and monkey patch the Get-ChocolateyWebFile function
$choco_tmp_path = "$Env:TEMP\chocolatey\au\chocolatey"
fix_choco
# This will set the new URLs before the files are downloaded but will replace checksums to empty ones so download will not fail
# because checksums are at that moment set for the previous version.
# SkipNuspecFile is passed so that if things fail here, nuspec file isn't updated; otherwise, on next run
# AU will think that package is the most recent.
#
# TODO: This will also leaves other then nuspec files updated which is undesired side effect (should be very rare)
#
$global:Silent = $true
$c32 = $global:Latest.Checksum32; $c64 = $global:Latest.Checksum64 #https://github.com/majkinetor/au/issues/36
$global:Latest.Remove('Checksum32'); $global:Latest.Remove('Checksum64') # -||-
update_files -SkipNuspecFile | out-null
if ($c32) {$global:Latest.Checksum32 = $c32}
if ($c64) {$global:Latest.Checksum64 = $c64} #https://github.com/majkinetor/au/issues/36
$global:Silent = $false
# Invoke installer for each architecture to download files
invoke_installer
}

The current automatic checksumming implementation uses monkey patching to modify the chocolateyInstall.psm1 module, and then executes the package's chocolateyInstall.ps1 to download the installer. This is a bad idea for a number of reasons:

  1. It relies on Chocolatey CLI not changing the format or method of operation of the chocolateyInstall.psm1 module.
  2. It relies on the chocolateyInstall.ps1 package script not having any other operations beside the call to the powershell helper, or in cases where there is another call (adding desktop shortcuts, etc), then those will happen.
  3. It is non-obvious that this is how the automatic checksumming works.
  4. It does not work on non-Windows systems

The automatic checksumming can be replaced by adding a au_BeforeUpdate function to each update.ps1 (that needs it, embedded packages do not) to explicitly get the checksums. There is an example function below:

function global:au_BeforeUpdate {
    $Latest.Checksum32 = Get-RemoteChecksum $Latest.URL32
    $Latest.Checksum64 = Get-RemoteChecksum $Latest.URL64
}

The downside of this is that it would be a significant breaking change from the way AU currently does things.
Additionally, if the download is more complex, it may require a complex au_BeforeUpdate which duplicates code from the chocolateyInstall.ps1, for example the ddu package requires a referrer page to be downloaded first:
https://github.com/tunisiano187/Chocolatey-packages/blob/58fe4dcc9d3ededdec31593c6a04a37dde22984a/automatic/ddu/tools/chocolateyInstall.ps1#L8

@tunisiano187
Copy link

Hello, just my opinion, but,
that's currently possible to say checksumfor none for local, but the checksum are still relevant for the VERIFICATION.txt file.

I think it's interesting to keep it

@TheCakeIsNaOH
Copy link
Member Author

that's currently possible to say checksumfor none for local, but the checksum are still relevant for the VERIFICATION.txt file.

The checksums are automatically found when using Get-RemoteFiles if I remember correctly, so the automatic checksumming is non-applicable for package that have embedded binaries with a VERIFICATION.txt`

@AdmiringWorm
Copy link
Member

I'm coming across this issue now, this wasn't something we added in the initial release of chocolatey-au as we wanted it to be (for the most part) a drop-in replacement for the original au module.

With that said, it makes sense to remove this functionality. There will be a point in the future where this functionality will no longer work due to the monkey patching of the PowerShell scripts and planned changes to Chocolatey CLI.

I recommend anyone currently using the automatic checksum feature, to switch over to use Get-RemoteChecksum for non-embedded packages, and Get-RemoteFiles for embedded packages when possible.

@TheCakeIsNaOH
Copy link
Member Author

Would it make sense to deprecate it with a warning message?
Possibly added to the same place as the the message from #57

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants