diff --git a/.gitignore b/.gitignore index fdc96d2a52..67ecc85d82 100644 --- a/.gitignore +++ b/.gitignore @@ -6,3 +6,4 @@ test/installer/tmp/* test/tmp/* *~ TestResults.xml +supporting/sqlite/* diff --git a/CHANGELOG.md b/CHANGELOG.md index 4a46492266..f1af1cac61 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,32 @@ +## [v0.5.0](https://github.com/ScoopInstaller/Scoop/compare/v0.4.2...v0.5.0) - 2024-07-01 + +### Features + +- **scoop-search:** Use SQLite for caching apps to speed up local search ([#5851](https://github.com/ScoopInstaller/Scoop/issues/5851), [#5918](https://github.com/ScoopInstaller/Scoop/issues/5918), [#5946](https://github.com/ScoopInstaller/Scoop/issues/5946), [#5949](https://github.com/ScoopInstaller/Scoop/issues/5949), [#5955](https://github.com/ScoopInstaller/Scoop/issues/5955), [#5966](https://github.com/ScoopInstaller/Scoop/issues/5966), [#5967](https://github.com/ScoopInstaller/Scoop/issues/5967), [#5981](https://github.com/ScoopInstaller/Scoop/issues/5981)) +- **core:** New cache filename format ([#5929](https://github.com/ScoopInstaller/Scoop/issues/5929), [#5944](https://github.com/ScoopInstaller/Scoop/issues/5944)) +- **decompress:** Use innounp-unicode as default Inno Setup Unpacker ([#6028](https://github.com/ScoopInstaller/Scoop/issues/6028)) +- **install:** Added the ability to install specific version of app from URL/file link ([#5988](https://github.com/ScoopInstaller/Scoop/issues/5988)) + +### Bug Fixes + +- **scoop-download|install|update:** Use consistent options ([#5956](https://github.com/ScoopInstaller/Scoop/issues/5956)) +- **scoop-info:** Fix download size estimating ([#5958](https://github.com/ScoopInstaller/Scoop/issues/5958)) +- **scoop-search:** Catch error of parsing invalid manifest ([#5930](https://github.com/ScoopInstaller/Scoop/issues/5930)) +- **checkver:** Correct variable 'regex' to 'regexp' ([#5993](https://github.com/ScoopInstaller/Scoop/issues/5993)) +- **checkver:** Correct error messages ([#6024](https://github.com/ScoopInstaller/Scoop/issues/6024)) +- **core:** Search for Git executable instead of any cmdlet ([#5998](https://github.com/ScoopInstaller/Scoop/issues/5998)) +- **core:** Use correct path in 'bash' ([#6006](https://github.com/ScoopInstaller/Scoop/issues/6006)) +- **core:** Limit the number of commands to get when search for git executable ([#6013](https://github.com/ScoopInstaller/Scoop/pull/6013)) +- **decompress:** Match `extract_dir`/`extract_to` and archives ([#5983](https://github.com/ScoopInstaller/Scoop/issues/5983)) +- **json:** Serialize jsonpath return ([#5921](https://github.com/ScoopInstaller/Scoop/issues/5921)) +- **shim:** Restore original path for JAR cmd ([#6030](https://github.com/ScoopInstaller/Scoop/issues/6030)) + +### Code Refactoring + +- **decompress:** Use 7zip to extract Zstd archive ([#5973](https://github.com/ScoopInstaller/Scoop/issues/5973)) +- **install:** Separate archive extraction from downloader ([#5951](https://github.com/ScoopInstaller/Scoop/issues/5951)) +- **install:** Replace 'run_(un)installer()' with 'Invoke-Installer()' ([#5968](https://github.com/ScoopInstaller/Scoop/issues/5968), [#5971](https://github.com/ScoopInstaller/Scoop/issues/5971)) + ## [v0.4.2](https://github.com/ScoopInstaller/Scoop/compare/v0.4.1...v0.4.2) - 2024-05-14 ### Bug Fixes diff --git a/bin/checkver.ps1 b/bin/checkver.ps1 index 07d954c671..57010c9992 100644 --- a/bin/checkver.ps1 +++ b/bin/checkver.ps1 @@ -275,7 +275,7 @@ while ($in_progress -gt 0) { $ver = $Version if (!$ver) { - if (!$regex -and $replace) { + if (!$regexp -and $replace) { next "'replace' requires 're' or 'regex'" continue } @@ -294,13 +294,15 @@ while ($in_progress -gt 0) { } $page = (New-Object System.IO.StreamReader($ms, (Get-Encoding $wc))).ReadToEnd() } + $source = $url if ($script) { $page = Invoke-Command ([scriptblock]::Create($script -join "`r`n")) + $source = 'the output of script' } if ($jsonpath) { # Return only a single value if regex is absent - $noregex = [String]::IsNullOrEmpty($regex) + $noregex = [String]::IsNullOrEmpty($regexp) # If reverse is ON and regex is ON, # Then reverse would have no effect because regex handles reverse # on its own @@ -310,7 +312,7 @@ while ($in_progress -gt 0) { $ver = json_path_legacy $page $jsonpath } if (!$ver) { - next "couldn't find '$jsonpath' in $url" + next "couldn't find '$jsonpath' in $source" continue } } @@ -332,7 +334,7 @@ while ($in_progress -gt 0) { # Getting version from XML, using XPath $ver = $xml.SelectSingleNode($xpath, $nsmgr).'#text' if (!$ver) { - next "couldn't find '$($xpath -replace 'ns:', '')' in $url" + next "couldn't find '$($xpath -replace 'ns:', '')' in $source" continue } } @@ -348,31 +350,31 @@ while ($in_progress -gt 0) { } if ($regexp) { - $regex = New-Object System.Text.RegularExpressions.Regex($regexp) + $re = New-Object System.Text.RegularExpressions.Regex($regexp) if ($reverse) { - $match = $regex.Matches($page) | Select-Object -Last 1 + $match = $re.Matches($page) | Select-Object -Last 1 } else { - $match = $regex.Matches($page) | Select-Object -First 1 + $match = $re.Matches($page) | Select-Object -First 1 } if ($match -and $match.Success) { $matchesHashtable = @{} - $regex.GetGroupNames() | ForEach-Object { $matchesHashtable.Add($_, $match.Groups[$_].Value) } + $re.GetGroupNames() | ForEach-Object { $matchesHashtable.Add($_, $match.Groups[$_].Value) } $ver = $matchesHashtable['1'] if ($replace) { - $ver = $regex.Replace($match.Value, $replace) + $ver = $re.Replace($match.Value, $replace) } if (!$ver) { $ver = $matchesHashtable['version'] } } else { - next "couldn't match '$regexp' in $url" + next "couldn't match '$regexp' in $source" continue } } if (!$ver) { - next "couldn't find new version in $url" + next "couldn't find new version in $source" continue } } diff --git a/bin/uninstall.ps1 b/bin/uninstall.ps1 index 98b5c8d513..2e2dc36ea8 100644 --- a/bin/uninstall.ps1 +++ b/bin/uninstall.ps1 @@ -42,7 +42,7 @@ function do_uninstall($app, $global) { $architecture = $install.architecture Write-Output "Uninstalling '$app'" - run_uninstaller $manifest $architecture $dir + Invoke-Installer -Path $dir -Manifest $manifest -ProcessorArchitecture $architecture -Uninstall rm_shims $app $manifest $global $architecture # If a junction was used during install, that will have been used diff --git a/lib/buckets.ps1 b/lib/buckets.ps1 index 63f6afc0a7..566cbd7119 100644 --- a/lib/buckets.ps1 +++ b/lib/buckets.ps1 @@ -156,6 +156,10 @@ function add_bucket($name, $repo) { $dir = ensure $dir Invoke-Git -ArgumentList @('clone', $repo, $dir, '-q') Write-Host 'OK' + if (get_config USE_SQLITE_CACHE) { + info 'Updating cache...' + Set-ScoopDB -Path (Get-ChildItem (Find-BucketDirectory $name) -Filter '*.json' -Recurse).FullName + } success "The $name bucket was added successfully." return 0 } @@ -168,6 +172,11 @@ function rm_bucket($name) { } Remove-Item $dir -Recurse -Force -ErrorAction Stop + if (get_config USE_SQLITE_CACHE) { + info 'Updating cache...' + Remove-ScoopDBItem -Bucket $name + } + success "The $name bucket was removed successfully." return 0 } diff --git a/lib/core.ps1 b/lib/core.ps1 index ddd274c255..193a8c0020 100644 --- a/lib/core.ps1 +++ b/lib/core.ps1 @@ -216,6 +216,16 @@ function Complete-ConfigChange { } } } + + if ($Name -eq 'use_sqlite_cache' -and $Value -eq $true) { + if ((Get-DefaultArchitecture) -eq 'arm64') { + abort 'SQLite cache is not supported on ARM64 platform.' + } + . "$PSScriptRoot\..\lib\database.ps1" + . "$PSScriptRoot\..\lib\manifest.ps1" + info 'Initializing SQLite cache in progress... This may take a while, please wait.' + Set-ScoopDB + } } function setup_proxy() { @@ -314,10 +324,6 @@ function Invoke-GitLog { # helper functions function coalesce($a, $b) { if($a) { return $a } $b } -function format($str, $hash) { - $hash.keys | ForEach-Object { set-variable $_ $hash[$_] } - $executionContext.invokeCommand.expandString($str) -} function is_admin { $admin = [security.principal.windowsbuiltinrole]::administrator $id = [security.principal.windowsidentity]::getcurrent() @@ -399,7 +405,22 @@ function currentdir($app, $global) { function persistdir($app, $global) { "$(basedir $global)\persist\$app" } function usermanifestsdir { "$(basedir)\workspace" } function usermanifest($app) { "$(usermanifestsdir)\$app.json" } -function cache_path($app, $version, $url) { "$cachedir\$app#$version#$($url -replace '[^\w\.\-]+', '_')" } +function cache_path($app, $version, $url) { + $underscoredUrl = $url -replace '[^\w\.\-]+', '_' + $filePath = "$cachedir\$app#$version#$underscoredUrl" + + # NOTE: Scoop cache files migration. Remove this 6 months after the feature ships. + if (Test-Path $filePath) { + return $filePath + } + + $urlStream = [System.IO.MemoryStream]::new([System.Text.Encoding]::UTF8.GetBytes($url)) + $sha = (Get-FileHash -Algorithm SHA256 -InputStream $urlStream).Hash.ToLower().Substring(0, 7) + $extension = [System.IO.Path]::GetExtension($url) + $filePath = $filePath -replace "$underscoredUrl", "$sha$extension" + + return $filePath +} # apps function sanitary_path($path) { return [regex]::replace($path, "[/\\?:*<>|]", "") } @@ -477,7 +498,7 @@ function Get-HelperPath { [OutputType([String])] param( [Parameter(Mandatory = $true, Position = 0, ValueFromPipeline = $true)] - [ValidateSet('Git', '7zip', 'Lessmsi', 'Innounp', 'Dark', 'Aria2', 'Zstd')] + [ValidateSet('Git', '7zip', 'Lessmsi', 'Innounp', 'Dark', 'Aria2')] [String] $Helper ) @@ -491,12 +512,17 @@ function Get-HelperPath { if ($internalgit) { $HelperPath = $internalgit } else { - $HelperPath = (Get-Command git -ErrorAction Ignore).Source + $HelperPath = (Get-Command git -CommandType Application -TotalCount 1 -ErrorAction Ignore).Source } } '7zip' { $HelperPath = Get-AppFilePath '7zip' '7z.exe' } 'Lessmsi' { $HelperPath = Get-AppFilePath 'lessmsi' 'lessmsi.exe' } - 'Innounp' { $HelperPath = Get-AppFilePath 'innounp' 'innounp.exe' } + 'Innounp' { + $HelperPath = Get-AppFilePath 'innounp-unicode' 'innounp.exe' + if ([String]::IsNullOrEmpty($HelperPath)) { + $HelperPath = Get-AppFilePath 'innounp' 'innounp.exe' + } + } 'Dark' { $HelperPath = Get-AppFilePath 'wixtoolset' 'wix.exe' if ([String]::IsNullOrEmpty($HelperPath)) { @@ -504,7 +530,6 @@ function Get-HelperPath { } } 'Aria2' { $HelperPath = Get-AppFilePath 'aria2' 'aria2c.exe' } - 'Zstd' { $HelperPath = Get-AppFilePath 'zstd' 'zstd.exe' } } return $HelperPath @@ -551,7 +576,7 @@ function Test-HelperInstalled { [CmdletBinding()] param( [Parameter(Mandatory = $true, Position = 0, ValueFromPipeline = $true)] - [ValidateSet('7zip', 'Lessmsi', 'Innounp', 'Dark', 'Aria2', 'Zstd')] + [ValidateSet('7zip', 'Lessmsi', 'Innounp', 'Dark', 'Aria2')] [String] $Helper ) @@ -1013,19 +1038,20 @@ function shim($path, $global, $name, $arg) { warn_on_overwrite "$shim.cmd" $path @( "@rem $resolved_path", - "@cd /d $(Split-Path $resolved_path -Parent)" - "@java -jar `"$resolved_path`" $arg %*" + "@pushd $(Split-Path $resolved_path -Parent)", + "@java -jar `"$resolved_path`" $arg %*", + "@popd" ) -join "`r`n" | Out-UTF8File "$shim.cmd" warn_on_overwrite $shim $path @( "#!/bin/sh", "# $resolved_path", - "if [ `$(echo `$WSL_DISTRO_NAME) ]", + "if [ `$WSL_INTEROP ]", 'then', " cd `$(wslpath -u '$(Split-Path $resolved_path -Parent)')", 'else', - " cd `"$((Split-Path $resolved_path -Parent).Replace('\', '/'))`"", + " cd `$(cygpath -u '$(Split-Path $resolved_path -Parent)')", 'fi', "java.exe -jar `"$resolved_path`" $arg `"$@`"" ) -join "`n" | Out-UTF8File $shim -NoNewLine @@ -1038,7 +1064,7 @@ function shim($path, $global, $name, $arg) { warn_on_overwrite $shim $path @( - "#!/bin/sh", + '#!/bin/sh', "# $resolved_path", "python.exe `"$resolved_path`" $arg `"$@`"" ) -join "`n" | Out-UTF8File $shim -NoNewLine @@ -1046,14 +1072,22 @@ function shim($path, $global, $name, $arg) { warn_on_overwrite "$shim.cmd" $path @( "@rem $resolved_path", - "@bash `"$resolved_path`" $arg %*" + "@bash `"`$(wslpath -u '$resolved_path')`" $arg %* 2>nul", + '@if %errorlevel% neq 0 (', + " @bash `"`$(cygpath -u '$resolved_path')`" $arg %* 2>nul", + ')' ) -join "`r`n" | Out-UTF8File "$shim.cmd" warn_on_overwrite $shim $path @( - "#!/bin/sh", + '#!/bin/sh', "# $resolved_path", - "`"$resolved_path`" $arg `"$@`"" + "if [ `$WSL_INTEROP ]", + 'then', + " `"`$(wslpath -u '$resolved_path')`" $arg `"$@`"", + 'else', + " `"`$(cygpath -u '$resolved_path')`" $arg `"$@`"", + 'fi' ) -join "`n" | Out-UTF8File $shim -NoNewLine } } @@ -1172,7 +1206,7 @@ function applist($apps, $global) { } function parse_app([string]$app) { - if ($app -match '^(?:(?[a-zA-Z0-9-_.]+)/)?(?.*\.json$|[a-zA-Z0-9-_.]+)(?:@(?.*))?$') { + if ($app -match '^(?:(?[a-zA-Z0-9-_.]+)/)?(?.*\.json|[a-zA-Z0-9-_.]+)(?:@(?.*))?$') { return $Matches['app'], $Matches['bucket'], $Matches['version'] } else { return $app, $null, $null diff --git a/lib/database.ps1 b/lib/database.ps1 new file mode 100644 index 0000000000..ae45f46dc1 --- /dev/null +++ b/lib/database.ps1 @@ -0,0 +1,391 @@ +# Description: Functions for interacting with the Scoop database cache + +<# +.SYNOPSIS + Get SQLite .NET driver +.DESCRIPTION + Download and extract the SQLite .NET driver from NuGet. +.PARAMETER Version + System.String + The version of the SQLite .NET driver to download. +.INPUTS + None +.OUTPUTS + System.Boolean + True if the SQLite .NET driver was successfully downloaded and extracted, otherwise false. +#> +function Get-SQLite { + param ( + [string]$Version = '1.0.118' + ) + # Install SQLite + try { + Write-Host "Downloading SQLite $Version..." -ForegroundColor DarkYellow + $sqlitePkgPath = "$env:TEMP\sqlite.nupkg" + $sqliteTempPath = "$env:TEMP\sqlite" + $sqlitePath = "$PSScriptRoot\..\supporting\sqlite" + Invoke-WebRequest -Uri "https://api.nuget.org/v3-flatcontainer/stub.system.data.sqlite.core.netframework/$version/stub.system.data.sqlite.core.netframework.$version.nupkg" -OutFile $sqlitePkgPath + Write-Host "Extracting SQLite $Version..." -ForegroundColor DarkYellow -NoNewline + Expand-Archive -Path $sqlitePkgPath -DestinationPath $sqliteTempPath -Force + New-Item -Path $sqlitePath -ItemType Directory -Force | Out-Null + Move-Item -Path "$sqliteTempPath\build\net45\*" -Destination $sqlitePath -Exclude '*.targets' -Force + Move-Item -Path "$sqliteTempPath\lib\net45\System.Data.SQLite.dll" -Destination $sqlitePath -Force + Remove-Item -Path $sqlitePkgPath, $sqliteTempPath -Recurse -Force + Write-Host ' Done' -ForegroundColor DarkYellow + return $true + } catch { + return $false + } +} + +<# +.SYNOPSIS + Open Scoop SQLite database. +.DESCRIPTION + Open Scoop SQLite database connection and create the necessary tables if not exists. +.INPUTS + None +.OUTPUTS + System.Data.SQLite.SQLiteConnection + The SQLite database connection if **PassThru** is used. +#> +function Open-ScoopDB { + # Load System.Data.SQLite + if (!('System.Data.SQLite.SQLiteConnection' -as [Type])) { + try { + if (!(Test-Path -Path "$PSScriptRoot\..\supporting\sqlite\System.Data.SQLite.dll")) { + Get-SQLite | Out-Null + } + Add-Type -Path "$PSScriptRoot\..\supporting\sqlite\System.Data.SQLite.dll" + } catch { + throw "Scoop's Database cache requires the ADO.NET driver:`n`thttp://system.data.sqlite.org/index.html/doc/trunk/www/downloads.wiki" + } + } + $dbPath = Join-Path $scoopdir 'scoop.db' + $db = New-Object -TypeName System.Data.SQLite.SQLiteConnection + $db.ConnectionString = "Data Source=$dbPath" + $db.ParseViaFramework = $true # Allow UNC path + $db.Open() + $tableCommand = $db.CreateCommand() + $tableCommand.CommandText = "CREATE TABLE IF NOT EXISTS 'app' ( + name TEXT NOT NULL COLLATE NOCASE, + description TEXT NOT NULL, + version TEXT NOT NULL, + bucket VARCHAR NOT NULL, + manifest JSON NOT NULL, + binary TEXT, + shortcut TEXT, + dependency TEXT, + suggest TEXT, + PRIMARY KEY (name, version, bucket) + )" + $tableCommand.CommandType = [System.Data.CommandType]::Text + $tableCommand.ExecuteNonQuery() | Out-Null + $tableCommand.Dispose() + return $db +} + +<# +.SYNOPSIS + Set Scoop database item(s). +.DESCRIPTION + Insert or replace item(s) into the Scoop SQLite database. +.PARAMETER InputObject + System.Object[] + The database item(s) to insert or replace. +.INPUTS + System.Object[] +.OUTPUTS + None +#> +function Set-ScoopDBItem { + [CmdletBinding()] + param ( + [Parameter(Mandatory, Position = 0, ValueFromPipeline)] + [psobject[]] + $InputObject + ) + + begin { + $db = Open-ScoopDB + $dbTrans = $db.BeginTransaction() + # TODO Support [hashtable]$InputObject + $colName = @($InputObject | Get-Member -MemberType NoteProperty | Select-Object -ExpandProperty Name) + $dbQuery = "INSERT OR REPLACE INTO app ($($colName -join ', ')) VALUES ($('@' + ($colName -join ', @')))" + $dbCommand = $db.CreateCommand() + $dbCommand.CommandText = $dbQuery + $dbCommand.CommandType = [System.Data.CommandType]::Text + } + process { + foreach ($item in $InputObject) { + $item.PSObject.Properties | ForEach-Object { + $dbCommand.Parameters.AddWithValue("@$($_.Name)", $_.Value) | Out-Null + } + $dbCommand.ExecuteNonQuery() | Out-Null + } + } + end { + try { + $dbTrans.Commit() + } catch { + $dbTrans.Rollback() + throw $_ + } finally { + $dbCommand.Dispose() + $dbTrans.Dispose() + $db.Dispose() + } + } +} + +<# +.SYNOPSIS + Set Scoop app database item(s). +.DESCRIPTION + Insert or replace Scoop app(s) into the database. +.PARAMETER Path + System.String + The path to the bucket. +.PARAMETER CommitHash + System.String + The commit hash to compare with the HEAD. +.INPUTS + None +.OUTPUTS + None +#> +function Set-ScoopDB { + [CmdletBinding()] + param ( + [Parameter(Position = 0, ValueFromPipeline)] + [string[]] + $Path + ) + + begin { + $list = [System.Collections.Generic.List[psobject]]::new() + $arch = Get-DefaultArchitecture + } + process { + if ($Path.Count -eq 0) { + $bucketPath = Get-LocalBucket | ForEach-Object { Find-BucketDirectory $_ } + $Path = (Get-ChildItem $bucketPath -Filter '*.json' -Recurse).FullName + } + $Path | ForEach-Object { + $manifestRaw = [System.IO.File]::ReadAllText($_) + $manifest = ConvertFrom-Json $manifestRaw -ErrorAction SilentlyContinue + if ($null -ne $manifest.version) { + $list.Add([pscustomobject]@{ + name = $($_ -replace '.*[\\/]([^\\/]+)\.json$', '$1') + description = if ($manifest.description) { $manifest.description } else { '' } + version = $manifest.version + bucket = $($_ -replace '.*buckets[\\/]([^\\/]+)(?:[\\/].*)', '$1') + manifest = $manifestRaw + binary = $( + $result = @() + @(arch_specific 'bin' $manifest $arch) | ForEach-Object { + if ($_ -is [System.Array]) { + $result += "$($_[1]).$($_[0].Split('.')[-1])" + } else { + $result += $_ + } + } + $result -replace '.*?([^\\/]+)?(\.(exe|bat|cmd|ps1|jar|py))$', '$1' -join ' | ' + ) + shortcut = $( + $result = @() + @(arch_specific 'shortcuts' $manifest $arch) | ForEach-Object { + $result += $_[1] + } + $result -replace '.*?([^\\/]+$)', '$1' -join ' | ' + ) + dependency = $manifest.depends -join ' | ' + suggest = $( + $suggest_output = @() + $manifest.suggest.PSObject.Properties | ForEach-Object { + $suggest_output += $_.Value -join ' | ' + } + $suggest_output -join ' | ' + ) + }) + } + } + } + end { + if ($list.Count -ne 0) { + Set-ScoopDBItem $list + } + } +} + +<# +.SYNOPSIS + Select Scoop database item(s). +.DESCRIPTION + Select item(s) from the Scoop SQLite database. + The pattern is matched against the name, binaries, and shortcuts columns for apps. +.PARAMETER Pattern + System.String + The pattern to search for. If is an empty string, all items will be returned. +.PARAMETER From + System.String[] + The fields to search from. +.INPUTS + System.String +.OUTPUTS + System.Data.DataTable + The selected database item(s). +#> +function Select-ScoopDBItem { + [CmdletBinding()] + param ( + [Parameter(Mandatory, Position = 0, ValueFromPipeline)] + [AllowEmptyString()] + [string] + $Pattern, + [Parameter(Mandatory, Position = 1)] + [ValidateNotNullOrEmpty()] + [string[]] + $From + ) + + begin { + $db = Open-ScoopDB + $dbAdapter = New-Object -TypeName System.Data.SQLite.SQLiteDataAdapter + $result = New-Object System.Data.DataTable + $dbQuery = "SELECT * FROM app WHERE $(($From -join ' LIKE @Pattern OR ') + ' LIKE @Pattern')" + $dbQuery = "SELECT * FROM ($($dbQuery + ' ORDER BY version DESC')) GROUP BY name, bucket" + $dbCommand = $db.CreateCommand() + $dbCommand.CommandText = $dbQuery + $dbCommand.CommandType = [System.Data.CommandType]::Text + $dbAdapter.SelectCommand = $dbCommand + } + process { + $dbCommand.Parameters.AddWithValue('@Pattern', $(if ($Pattern -eq '') { '%' } else { '%' + $Pattern + '%' })) | Out-Null + [void]$dbAdapter.Fill($result) + } + end { + $dbAdapter.Dispose() + $db.Dispose() + return $result + } +} + +<# +.SYNOPSIS + Get Scoop database item. +.DESCRIPTION + Get item from the Scoop SQLite database. +.PARAMETER Name + System.String + The name of the item to get. +.PARAMETER Bucket + System.String + The bucket of the item to get. +.PARAMETER Version + System.String + The version of the item to get. If not provided, the latest version will be returned. +.INPUTS + System.String +.OUTPUTS + System.Data.DataTable + The selected database item. +#> +function Get-ScoopDBItem { + [CmdletBinding()] + param ( + [Parameter(Mandatory, Position = 0, ValueFromPipeline)] + [string] + $Name, + [Parameter(Mandatory, Position = 1)] + [string] + $Bucket, + [Parameter(Position = 2)] + [string] + $Version + ) + + begin { + $db = Open-ScoopDB + $dbAdapter = New-Object -TypeName System.Data.SQLite.SQLiteDataAdapter + $result = New-Object System.Data.DataTable + $dbQuery = 'SELECT * FROM app WHERE name = @Name AND bucket = @Bucket' + if ($Version) { + $dbQuery += ' AND version = @Version' + } else { + $dbQuery += ' ORDER BY version DESC LIMIT 1' + } + $dbCommand = $db.CreateCommand() + $dbCommand.CommandText = $dbQuery + $dbCommand.CommandType = [System.Data.CommandType]::Text + $dbAdapter.SelectCommand = $dbCommand + } + process { + $dbCommand.Parameters.AddWithValue('@Name', $Name) | Out-Null + $dbCommand.Parameters.AddWithValue('@Bucket', $Bucket) | Out-Null + $dbCommand.Parameters.AddWithValue('@Version', $Version) | Out-Null + [void]$dbAdapter.Fill($result) + } + end { + $dbAdapter.Dispose() + $db.Dispose() + return $result + } +} + +<# +.SYNOPSIS + Remove Scoop database item(s). +.DESCRIPTION + Remove item(s) from the Scoop SQLite database. +.PARAMETER Name + System.String + The name of the item to remove. +.PARAMETER Bucket + System.String + The bucket of the item to remove. +.INPUTS + System.String +.OUTPUTS + None +#> +function Remove-ScoopDBItem { + [CmdletBinding()] + param ( + [Parameter(Position = 0, ValueFromPipeline, ValueFromPipelineByPropertyName)] + [string] + $Name, + [Parameter(Mandatory, Position = 1, ValueFromPipelineByPropertyName)] + [string] + $Bucket + ) + + begin { + $db = Open-ScoopDB + $dbTrans = $db.BeginTransaction() + $dbQuery = 'DELETE FROM app WHERE bucket = @Bucket' + $dbCommand = $db.CreateCommand() + $dbCommand.CommandText = $dbQuery + $dbCommand.CommandType = [System.Data.CommandType]::Text + } + process { + $dbCommand.Parameters.AddWithValue('@Bucket', $Bucket) | Out-Null + if ($Name) { + $dbCommand.CommandText = $dbQuery + ' AND name = @Name' + $dbCommand.Parameters.AddWithValue('@Name', $Name) | Out-Null + } + $dbCommand.ExecuteNonQuery() | Out-Null + } + end { + try { + $dbTrans.Commit() + } catch { + $dbTrans.Rollback() + throw $_ + } finally { + $dbCommand.Dispose() + $dbTrans.Dispose() + $db.Dispose() + } + } +} diff --git a/lib/decompress.ps1 b/lib/decompress.ps1 index c2b04800d5..9c3f5c6813 100644 --- a/lib/decompress.ps1 +++ b/lib/decompress.ps1 @@ -1,3 +1,67 @@ +# Description: Functions for decompressing archives or installers + +function Invoke-Extraction { + param ( + [string] + $Path, + [string[]] + $Name, + [psobject] + $Manifest, + [Alias('Arch', 'Architecture')] + [string] + $ProcessorArchitecture + ) + + $uri = @(url $Manifest $ProcessorArchitecture) + # 'extract_dir' and 'extract_to' are paired + $extractDir = @(extract_dir $Manifest $ProcessorArchitecture) + $extractTo = @(extract_to $Manifest $ProcessorArchitecture) + $extracted = 0 + + for ($i = 0; $i -lt $Name.Length; $i++) { + # work out extraction method, if applicable + $extractFn = $null + switch -regex ($Name[$i]) { + '\.zip$' { + if ((Test-HelperInstalled -Helper 7zip) -or ((get_config 7ZIPEXTRACT_USE_EXTERNAL) -and (Test-CommandAvailable 7z))) { + $extractFn = 'Expand-7zipArchive' + } else { + $extractFn = 'Expand-ZipArchive' + } + continue + } + '\.msi$' { + $extractFn = 'Expand-MsiArchive' + continue + } + '\.exe$' { + if ($Manifest.innosetup) { + $extractFn = 'Expand-InnoArchive' + } + continue + } + { Test-7zipRequirement -Uri $_ } { + $extractFn = 'Expand-7zipArchive' + continue + } + } + if ($extractFn) { + $fnArgs = @{ + Path = Join-Path $Path $Name[$i] + DestinationPath = Join-Path $Path $extractTo[$extracted] + ExtractDir = $extractDir[$extracted] + } + Write-Host 'Extracting ' -NoNewline + Write-Host $(url_remote_filename $uri[$i]) -ForegroundColor Cyan -NoNewline + Write-Host ' ... ' -NoNewline + & $extractFn @fnArgs -Removal + Write-Host 'done.' -ForegroundColor Green + $extracted++ + } + } +} + function Expand-7zipArchive { [CmdletBinding()] param ( @@ -96,37 +160,9 @@ function Expand-ZstdArchive { [Switch] $Removal ) - $ZstdPath = Get-HelperPath -Helper Zstd - $LogPath = Join-Path (Split-Path $Path) 'zstd.log' - $DestinationPath = $DestinationPath.TrimEnd('\') - ensure $DestinationPath | Out-Null - $ArgList = @('-d', $Path, '--output-dir-flat', $DestinationPath, '-f', '-v') - - if ($Switches) { - $ArgList += (-split $Switches) - } - if ($Removal) { - # Remove original archive file - $ArgList += '--rm' - } - $Status = Invoke-ExternalCommand $ZstdPath $ArgList -LogPath $LogPath - if (!$Status) { - abort "Failed to extract files from $Path.`nLog file:`n $(friendly_path $LogPath)`n$(new_issue_msg $app $bucket 'decompress error')" - } - $IsTar = (strip_ext $Path) -match '\.tar$' - if ($IsTar) { - # Check for tar - $TarFile = Join-Path $DestinationPath (strip_ext (fname $Path)) - Expand-7zipArchive -Path $TarFile -DestinationPath $DestinationPath -ExtractDir $ExtractDir -Removal - } - if (!$IsTar -and $ExtractDir) { - movedir (Join-Path $DestinationPath $ExtractDir) $DestinationPath | Out-Null - # Remove temporary directory - Remove-Item "$DestinationPath\$($ExtractDir -replace '[\\/].*')" -Recurse -Force -ErrorAction Ignore - } - if (Test-Path $LogPath) { - Remove-Item $LogPath -Force - } + # TODO: Remove this function after 2024/12/31 + Show-DeprecatedWarning $MyInvocation 'Expand-7zipArchive' + Expand-7zipArchive -Path $Path -DestinationPath $DestinationPath -ExtractDir $ExtractDir -Switches $Switches -Removal:$Removal } function Expand-MsiArchive { diff --git a/lib/depends.ps1 b/lib/depends.ps1 index bd4ed19cf2..3a38ca2b23 100644 --- a/lib/depends.ps1 +++ b/lib/depends.ps1 @@ -118,11 +118,8 @@ function Get-InstallationHelper { if ($script -like '*Expand-DarkArchive *') { $helper += 'dark' } - if ((Test-ZstdRequirement -Uri $url) -or ($script -like '*Expand-ZstdArchive *')) { - $helper += 'zstd' - } if (!$All) { - '7zip', 'lessmsi', 'innounp', 'dark', 'zstd' | ForEach-Object { + '7zip', 'lessmsi', 'innounp', 'dark' | ForEach-Object { if (Test-HelperInstalled -Helper $_) { $helper = $helper -ne $_ } @@ -144,22 +141,10 @@ function Test-7zipRequirement { $Uri ) return ($Uri | Where-Object { - $_ -match '\.((gz)|(tar)|(t[abgpx]z2?)|(lzma)|(bz2?)|(7z)|(001)|(rar)|(iso)|(xz)|(lzh)|(nupkg))(\.[^\d.]+)?$' + $_ -match '\.(001|7z|bz(ip)?2?|gz|img|iso|lzma|lzh|nupkg|rar|tar|t[abgpx]z2?|t?zst|xz)(\.[^\d.]+)?$' }).Count -gt 0 } -function Test-ZstdRequirement { - [CmdletBinding()] - [OutputType([Boolean])] - param ( - [Parameter(Mandatory = $true)] - [AllowNull()] - [String[]] - $Uri - ) - return ($Uri | Where-Object { $_ -match '\.zst$' }).Count -gt 0 -} - function Test-LessmsiRequirement { [CmdletBinding()] [OutputType([Boolean])] diff --git a/lib/install.ps1 b/lib/install.ps1 index 12d4220015..cbe2b2f8b2 100644 --- a/lib/install.ps1 +++ b/lib/install.ps1 @@ -50,9 +50,10 @@ function install_app($app, $architecture, $global, $suggested, $use_cache = $tru $persist_dir = persistdir $app $global $fname = Invoke-ScoopDownload $app $version $manifest $bucket $architecture $dir $use_cache $check_hash - Invoke-HookScript -HookType 'pre_install' -Manifest $manifest -Arch $architecture + Invoke-Extraction -Path $dir -Name $fname -Manifest $manifest -ProcessorArchitecture $architecture + Invoke-HookScript -HookType 'pre_install' -Manifest $manifest -ProcessorArchitecture $architecture - run_installer $fname $manifest $architecture $dir $global + Invoke-Installer -Path $dir -Name $fname -Manifest $manifest -ProcessorArchitecture $architecture -AppName $app -Global:$global ensure_install_dir_not_in_path $dir $global $dir = link_current $dir create_shims $manifest $dir $global $architecture @@ -65,7 +66,7 @@ function install_app($app, $architecture, $global, $suggested, $use_cache = $tru persist_data $manifest $original_dir $persist_dir persist_permission $manifest $global - Invoke-HookScript -HookType 'post_install' -Manifest $manifest -Arch $architecture + Invoke-HookScript -HookType 'post_install' -Manifest $manifest -ProcessorArchitecture $architecture # save info for uninstall save_installed_manifest $app $bucket $dir $url @@ -539,21 +540,12 @@ function Invoke-ScoopDownload ($app, $version, $manifest, $bucket, $architecture # we only want to show this warning once if (!$use_cache) { warn 'Cache is being ignored.' } - # can be multiple urls: if there are, then installer should go last, - # so that $fname is set properly + # can be multiple urls: if there are, then installer should go first to make 'installer.args' section work $urls = @(script:url $manifest $architecture) # can be multiple cookies: they will be used for all HTTP requests. $cookies = $manifest.cookie - $fname = $null - - # extract_dir and extract_to in manifest are like queues: for each url that - # needs to be extracted, will get the next dir from the queue - $extract_dirs = @(extract_dir $manifest $architecture) - $extract_tos = @(extract_to $manifest $architecture) - $extracted = 0 - # download first if (Test-Aria2Enabled) { Invoke-CachedAria2Download $app $version $manifest $architecture $dir $cookies $use_cache $check_hash @@ -587,44 +579,7 @@ function Invoke-ScoopDownload ($app, $version, $manifest, $bucket, $architecture } } - foreach ($url in $urls) { - $fname = url_filename $url - - $extract_dir = $extract_dirs[$extracted] - $extract_to = $extract_tos[$extracted] - - # work out extraction method, if applicable - $extract_fn = $null - if ($manifest.innosetup) { - $extract_fn = 'Expand-InnoArchive' - } elseif ($fname -match '\.zip$') { - # Use 7zip when available (more fast) - if (((get_config USE_EXTERNAL_7ZIP) -and (Test-CommandAvailable 7z)) -or (Test-HelperInstalled -Helper 7zip)) { - $extract_fn = 'Expand-7zipArchive' - } else { - $extract_fn = 'Expand-ZipArchive' - } - } elseif ($fname -match '\.msi$') { - $extract_fn = 'Expand-MsiArchive' - } elseif (Test-ZstdRequirement -Uri $fname) { - # Zstd first - $extract_fn = 'Expand-ZstdArchive' - } elseif (Test-7zipRequirement -Uri $fname) { - # 7zip - $extract_fn = 'Expand-7zipArchive' - } - - if ($extract_fn) { - Write-Host 'Extracting ' -NoNewline - Write-Host $fname -f Cyan -NoNewline - Write-Host ' ... ' -NoNewline - & $extract_fn -Path "$dir\$fname" -DestinationPath "$dir\$extract_to" -ExtractDir $extract_dir -Removal - Write-Host 'done.' -f Green - $extracted++ - } - } - - $fname # returns the last downloaded file + return $urls.ForEach({ url_filename $_ }) } function cookie_header($cookies) { @@ -696,70 +651,89 @@ function check_hash($file, $hash, $app_name) { return $true, $null } -# for dealing with installers -function args($config, $dir, $global) { - if ($config) { return $config | ForEach-Object { (format $_ @{'dir' = $dir; 'global' = $global }) } } - @() -} - -function run_installer($fname, $manifest, $architecture, $dir, $global) { - $installer = installer $manifest $architecture - if ($installer.script) { - Write-Output 'Running installer script...' - Invoke-Command ([scriptblock]::Create($installer.script -join "`r`n")) - return - } - if ($installer) { - $prog = "$dir\$(coalesce $installer.file "$fname")" - if (!(is_in_dir $dir $prog)) { - abort "Error in manifest: Installer $prog is outside the app directory." +function Invoke-Installer { + [CmdletBinding()] + param ( + [string] + $Path, + [string[]] + $Name, + [psobject] + $Manifest, + [Alias('Arch', 'Architecture')] + [ValidateSet('32bit', '64bit', 'arm64')] + [string] + $ProcessorArchitecture, + [string] + $AppName, + [switch] + $Global, + [switch] + $Uninstall + ) + $type = if ($Uninstall) { 'uninstaller' } else { 'installer' } + $installer = arch_specific $type $Manifest $ProcessorArchitecture + if ($installer.file -or $installer.args) { + # Installer filename is either explicit defined ('installer.file') or file name in the first URL + if (!$Name) { + $Name = url_filename @(url $manifest $architecture) + } + $progName = "$Path\$(coalesce $installer.file $Name[0])" + if (!(is_in_dir $Path $progName)) { + abort "Error in manifest: $((Get-Culture).TextInfo.ToTitleCase($type)) $progName is outside the app directory." + } elseif (!(Test-Path $progName)) { + abort "$((Get-Culture).TextInfo.ToTitleCase($type)) $progName is missing." } - $arg = @(args $installer.args $dir $global) - if ($prog.endswith('.ps1')) { - & $prog @arg + $substitutions = @{ + '$dir' = $Path + '$global' = $Global + '$version' = $Manifest.version + } + $fnArgs = substitute $installer.args $substitutions + if ($progName.EndsWith('.ps1')) { + & $progName @fnArgs } else { - $installed = Invoke-ExternalCommand $prog $arg -Activity 'Running installer...' - if (!$installed) { - abort "Installation aborted. You might need to run 'scoop uninstall $app' before trying again." + $status = Invoke-ExternalCommand $progName -ArgumentList $fnArgs -Activity "Running $type ..." + if (!$status) { + if ($Uninstall) { + abort 'Uninstallation aborted.' + } else { + abort "Installation aborted. You might need to run 'scoop uninstall $AppName' before trying again." + } } # Don't remove installer if "keep" flag is set to true - if (!($installer.keep -eq 'true')) { - Remove-Item $prog + if (!$installer.keep) { + Remove-Item $progName } } } + Invoke-HookScript -HookType $type -Manifest $Manifest -ProcessorArchitecture $ProcessorArchitecture } -function run_uninstaller($manifest, $architecture, $dir) { - $uninstaller = uninstaller $manifest $architecture - $version = $manifest.version - if ($uninstaller.script) { - Write-Output 'Running uninstaller script...' - Invoke-Command ([scriptblock]::Create($uninstaller.script -join "`r`n")) - return - } - - if ($uninstaller.file) { - $prog = "$dir\$($uninstaller.file)" - $arg = args $uninstaller.args - if (!(is_in_dir $dir $prog)) { - warn "Error in manifest: Installer $prog is outside the app directory, skipping." - $prog = $null - } elseif (!(Test-Path $prog)) { - warn "Uninstaller $prog is missing, skipping." - $prog = $null - } +function Invoke-HookScript { + [CmdletBinding()] + param( + [Parameter(Mandatory = $true)] + [ValidateSet('installer', 'pre_install', 'post_install', 'uninstaller', 'pre_uninstall', 'post_uninstall')] + [String] $HookType, + [Parameter(Mandatory = $true)] + [ValidateNotNullOrEmpty()] + [PSCustomObject] $Manifest, + [Parameter(Mandatory = $true)] + [Alias('Arch', 'Architecture')] + [ValidateSet('32bit', '64bit', 'arm64')] + [string] + $ProcessorArchitecture + ) - if ($prog) { - if ($prog.endswith('.ps1')) { - & $prog @arg - } else { - $uninstalled = Invoke-ExternalCommand $prog $arg -Activity 'Running uninstaller...' - if (!$uninstalled) { - abort 'Uninstallation aborted.' - } - } - } + $script = arch_specific $HookType $Manifest $ProcessorArchitecture + if ($HookType -in @('installer', 'uninstaller')) { + $script = $script.script + } + if ($script) { + Write-Host "Running $HookType script..." -NoNewline + Invoke-Command ([scriptblock]::Create($script -join "`r`n")) + Write-Host 'done.' -ForegroundColor Green } } @@ -929,7 +903,7 @@ function env_set($manifest, $dir, $global, $arch) { if ($env_set) { $env_set | Get-Member -Member NoteProperty | ForEach-Object { $name = $_.name - $val = format $env_set.$($_.name) @{ 'dir' = $dir } + $val = substitute $env_set.$($_.name) @{ '$dir' = $dir } Set-EnvVar -Name $name -Value $val -Global:$global Set-Content env:\$name $val } @@ -946,28 +920,6 @@ function env_rm($manifest, $global, $arch) { } } -function Invoke-HookScript { - [CmdletBinding()] - param( - [Parameter(Mandatory = $true)] - [ValidateSet('pre_install', 'post_install', - 'pre_uninstall', 'post_uninstall')] - [String] $HookType, - [Parameter(Mandatory = $true)] - [ValidateNotNullOrEmpty()] - [PSCustomObject] $Manifest, - [Parameter(Mandatory = $true)] - [ValidateSet('32bit', '64bit', 'arm64')] - [String] $Arch - ) - - $script = arch_specific $HookType $Manifest $Arch - if ($script) { - Write-Output "Running $HookType script..." - Invoke-Command ([scriptblock]::Create($script -join "`r`n")) - } -} - function show_notes($manifest, $dir, $original_dir, $persist_dir) { if ($manifest.notes) { Write-Output 'Notes' diff --git a/lib/json.ps1 b/lib/json.ps1 index 2469204dc5..d8d07f50a6 100644 --- a/lib/json.ps1 +++ b/lib/json.ps1 @@ -116,7 +116,7 @@ function json_path([String] $json, [String] $jsonpath, [Hashtable] $substitution # Convert first value to string $result = $result.ToString() } else { - $result = "$([String]::Join('\n', $result))" + $result = [Newtonsoft.Json.JsonConvert]::SerializeObject($result) } return $result } catch [Exception] { diff --git a/lib/manifest.ps1 b/lib/manifest.ps1 index f66755c6f4..9ca618158b 100644 --- a/lib/manifest.ps1 +++ b/lib/manifest.ps1 @@ -23,7 +23,7 @@ function url_manifest($url) { } catch { throw } - if(!$str) { return $null } + if (!$str) { return $null } try { $str | ConvertFrom-Json -ErrorAction Stop } catch { @@ -137,16 +137,26 @@ function generate_user_manifest($app, $bucket, $version) { warn "Given version ($version) does not match manifest ($($manifest.version))" warn "Attempting to generate manifest for '$app' ($version)" + ensure (usermanifestsdir) | Out-Null + $manifest_path = "$(usermanifestsdir)\$app.json" + + if (get_config USE_SQLITE_CACHE) { + $cached_manifest = (Get-ScoopDBItem -Name $app -Bucket $bucket -Version $version).manifest + if ($cached_manifest) { + $cached_manifest | Out-UTF8File $manifest_path + return $manifest_path + } + } + if (!($manifest.autoupdate)) { abort "'$app' does not have autoupdate capability`r`ncouldn't find manifest for '$app@$version'" } - ensure (usermanifestsdir) | out-null try { - Invoke-AutoUpdate $app "$(Convert-Path (usermanifestsdir))\$app.json" $manifest $version $(@{ }) - return Convert-Path (usermanifest $app) + Invoke-AutoUpdate $app $manifest_path $manifest $version $(@{ }) + return $manifest_path } catch { - write-host -f darkred "Could not install $app@$version" + Write-Host -ForegroundColor DarkRed "Could not install $app@$version" } return $null @@ -156,5 +166,5 @@ function url($manifest, $arch) { arch_specific 'url' $manifest $arch } function installer($manifest, $arch) { arch_specific 'installer' $manifest $arch } function uninstaller($manifest, $arch) { arch_specific 'uninstaller' $manifest $arch } function hash($manifest, $arch) { arch_specific 'hash' $manifest $arch } -function extract_dir($manifest, $arch) { arch_specific 'extract_dir' $manifest $arch} -function extract_to($manifest, $arch) { arch_specific 'extract_to' $manifest $arch} +function extract_dir($manifest, $arch) { arch_specific 'extract_dir' $manifest $arch } +function extract_to($manifest, $arch) { arch_specific 'extract_to' $manifest $arch } diff --git a/libexec/scoop-bucket.ps1 b/libexec/scoop-bucket.ps1 index 6a2e90e6cb..ceb28865a5 100644 --- a/libexec/scoop-bucket.ps1 +++ b/libexec/scoop-bucket.ps1 @@ -19,6 +19,11 @@ # scoop bucket known param($cmd, $name, $repo) +if (get_config USE_SQLITE_CACHE) { + . "$PSScriptRoot\..\lib\manifest.ps1" + . "$PSScriptRoot\..\lib\database.ps1" +} + $usage_add = 'usage: scoop bucket add []' $usage_rm = 'usage: scoop bucket rm ' diff --git a/libexec/scoop-cache.ps1 b/libexec/scoop-cache.ps1 index 959c73fcde..30e8354fca 100644 --- a/libexec/scoop-cache.ps1 +++ b/libexec/scoop-cache.ps1 @@ -16,7 +16,7 @@ param($cmd) function cacheinfo($file) { $app, $version, $url = $file.Name -split '#' - New-Object PSObject -Property @{ Name = $app; Version = $version; Length = $file.Length; URL = $url } + New-Object PSObject -Property @{ Name = $app; Version = $version; Length = $file.Length } } function cacheshow($app) { @@ -28,7 +28,7 @@ function cacheshow($app) { $files = @(Get-ChildItem $cachedir | Where-Object -Property Name -Value "^$app#" -Match) $totalLength = ($files | Measure-Object -Property Length -Sum).Sum - $files | ForEach-Object { cacheinfo $_ } | Select-Object Name, Version, Length, URL + $files | ForEach-Object { cacheinfo $_ } | Select-Object Name, Version, Length Write-Host "Total: $($files.Length) $(pluralize $files.Length 'file' 'files'), $(filesize $totalLength)" -ForegroundColor Yellow } @@ -48,7 +48,7 @@ function cacheremove($app) { $files | ForEach-Object { $curr = cacheinfo $_ - Write-Host "Removing $($curr.URL)..." + Write-Host "Removing $($_.Name)..." Remove-Item $_.FullName if(Test-Path "$cachedir\$($curr.Name).txt") { Remove-Item "$cachedir\$($curr.Name).txt" diff --git a/libexec/scoop-config.ps1 b/libexec/scoop-config.ps1 index ff0bff3eee..6007bd6434 100644 --- a/libexec/scoop-config.ps1 +++ b/libexec/scoop-config.ps1 @@ -27,6 +27,9 @@ # use_lessmsi: $true|$false # Prefer lessmsi utility over native msiexec. # +# use_sqlite_cache: $true|$false +# Use SQLite database for caching. This is useful for speeding up 'scoop search' and 'scoop shim' commands. +# # no_junction: $true|$false # The 'current' version alias will not be used. Shims and shortcuts will point to specific version instead. # diff --git a/libexec/scoop-download.ps1 b/libexec/scoop-download.ps1 index ef43f8f41d..1901527b09 100644 --- a/libexec/scoop-download.ps1 +++ b/libexec/scoop-download.ps1 @@ -15,7 +15,7 @@ # # Options: # -f, --force Force download (overwrite cache) -# -h, --no-hash-check Skip hash verification (use with caution!) +# -s, --skip-hash-check Skip hash verification (use with caution!) # -u, --no-update-scoop Don't update Scoop before downloading if it's outdated # -a, --arch <32bit|64bit|arm64> Use the specified architecture, if the app supports it @@ -24,11 +24,14 @@ . "$PSScriptRoot\..\lib\autoupdate.ps1" # 'generate_user_manifest' (indirectly) . "$PSScriptRoot\..\lib\manifest.ps1" # 'generate_user_manifest' 'Get-Manifest' . "$PSScriptRoot\..\lib\install.ps1" +if (get_config USE_SQLITE_CACHE) { + . "$PSScriptRoot\..\lib\database.ps1" +} -$opt, $apps, $err = getopt $args 'fhua:' 'force', 'no-hash-check', 'no-update-scoop', 'arch=' +$opt, $apps, $err = getopt $args 'fsua:' 'force', 'skip-hash-check', 'no-update-scoop', 'arch=' if ($err) { error "scoop download: $err"; exit 1 } -$check_hash = !($opt.h -or $opt.'no-hash-check') +$check_hash = !($opt.s -or $opt.'skip-hash-check') $use_cache = !($opt.f -or $opt.force) $architecture = Get-DefaultArchitecture try { diff --git a/libexec/scoop-info.ps1 b/libexec/scoop-info.ps1 index 853de647f9..3907b9f71a 100644 --- a/libexec/scoop-info.ps1 +++ b/libexec/scoop-info.ps1 @@ -166,7 +166,7 @@ if ($status.installed) { $cached = $null } - [int]$urlLength = (Invoke-WebRequest $url -Method Head).Headers.'Content-Length'[0] + $urlLength = (Invoke-WebRequest $url -Method Head).Headers.'Content-Length' | ForEach-Object { [int]$_ } $totalPackage += $urlLength } catch [System.Management.Automation.RuntimeException] { $totalPackage = 0 @@ -210,7 +210,7 @@ $env_set = arch_specific 'env_set' $manifest $install.architecture if ($env_set) { $env_vars = @() $env_set | Get-Member -member noteproperty | ForEach-Object { - $env_vars += "$($_.name) = $(format $env_set.$($_.name) @{ "dir" = $dir })" + $env_vars += "$($_.name) = $(substitute $env_set.$($_.name) @{ '$dir' = $dir })" } $item.Environment = $env_vars -join "`n" } diff --git a/libexec/scoop-install.ps1 b/libexec/scoop-install.ps1 index fac03d71f4..173bcd868f 100644 --- a/libexec/scoop-install.ps1 +++ b/libexec/scoop-install.ps1 @@ -10,15 +10,21 @@ # To install an app from a manifest at a URL: # scoop install https://raw.githubusercontent.com/ScoopInstaller/Main/master/bucket/runat.json # +# To install a different version of the app from a URL: +# scoop install https://raw.githubusercontent.com/ScoopInstaller/Main/master/bucket/neovim.json@0.9.0 +# # To install an app from a manifest on your computer # scoop install \path\to\app.json # +# To install an app from a manifest on your computer +# scoop install \path\to\app.json@version +# # Options: # -g, --global Install the app globally # -i, --independent Don't install dependencies automatically # -k, --no-cache Don't use the download cache +# -s, --skip-hash-check Skip hash validation (use with caution!) # -u, --no-update-scoop Don't update Scoop before installing if it's outdated -# -s, --skip Skip hash validation (use with caution!) # -a, --arch <32bit|64bit|arm64> Use the specified architecture, if the app supports it . "$PSScriptRoot\..\lib\getopt.ps1" @@ -32,12 +38,15 @@ . "$PSScriptRoot\..\lib\psmodules.ps1" . "$PSScriptRoot\..\lib\versions.ps1" . "$PSScriptRoot\..\lib\depends.ps1" +if (get_config USE_SQLITE_CACHE) { + . "$PSScriptRoot\..\lib\database.ps1" +} -$opt, $apps, $err = getopt $args 'gikusa:' 'global', 'independent', 'no-cache', 'no-update-scoop', 'skip', 'arch=' +$opt, $apps, $err = getopt $args 'giksua:' 'global', 'independent', 'no-cache', 'skip-hash-check', 'no-update-scoop', 'arch=' if ($err) { "scoop install: $err"; exit 1 } $global = $opt.g -or $opt.global -$check_hash = !($opt.s -or $opt.skip) +$check_hash = !($opt.s -or $opt.'skip-hash-check') $independent = $opt.i -or $opt.independent $use_cache = !($opt.k -or $opt.'no-cache') $architecture = Get-DefaultArchitecture diff --git a/libexec/scoop-search.ps1 b/libexec/scoop-search.ps1 index a9013afd02..4254099989 100644 --- a/libexec/scoop-search.ps1 +++ b/libexec/scoop-search.ps1 @@ -3,6 +3,8 @@ # Help: Searches for apps that are available to install. # # If used with [query], shows app names that match the query. +# - With 'use_sqlite_cache' enabled, [query] is partially matched against app names, binaries, and shortcuts. +# - Without 'use_sqlite_cache', [query] can be a regular expression to match against app names and binaries. # Without [query], shows all the available apps. param($query) @@ -11,16 +13,10 @@ param($query) $list = [System.Collections.Generic.List[PSCustomObject]]::new() -try { - $query = New-Object Regex $query, 'IgnoreCase' -} catch { - abort "Invalid regular expression: $($_.Exception.InnerException.Message)" -} - $githubtoken = Get-GitHubToken $authheader = @{} if ($githubtoken) { - $authheader = @{'Authorization' = "token $githubtoken"} + $authheader = @{'Authorization' = "token $githubtoken" } } function bin_match($manifest, $query) { @@ -39,16 +35,16 @@ function bin_match($manifest, $query) { function bin_match_json($json, $query) { [System.Text.Json.JsonElement]$bin = [System.Text.Json.JsonElement]::new() - if (!$json.RootElement.TryGetProperty("bin", [ref] $bin)) { return $false } + if (!$json.RootElement.TryGetProperty('bin', [ref] $bin)) { return $false } $bins = @() - if($bin.ValueKind -eq [System.Text.Json.JsonValueKind]::String -and [System.IO.Path]::GetFileNameWithoutExtension($bin) -match $query) { + if ($bin.ValueKind -eq [System.Text.Json.JsonValueKind]::String -and [System.IO.Path]::GetFileNameWithoutExtension($bin) -match $query) { $bins += [System.IO.Path]::GetFileName($bin) } elseif ($bin.ValueKind -eq [System.Text.Json.JsonValueKind]::Array) { - foreach($subbin in $bin.EnumerateArray()) { - if($subbin.ValueKind -eq [System.Text.Json.JsonValueKind]::String -and [System.IO.Path]::GetFileNameWithoutExtension($subbin) -match $query) { + foreach ($subbin in $bin.EnumerateArray()) { + if ($subbin.ValueKind -eq [System.Text.Json.JsonValueKind]::String -and [System.IO.Path]::GetFileNameWithoutExtension($subbin) -match $query) { $bins += [System.IO.Path]::GetFileName($subbin) } elseif ($subbin.ValueKind -eq [System.Text.Json.JsonValueKind]::Array) { - if([System.IO.Path]::GetFileNameWithoutExtension($subbin[0]) -match $query) { + if ([System.IO.Path]::GetFileNameWithoutExtension($subbin[0]) -match $query) { $bins += [System.IO.Path]::GetFileName($subbin[0]) } elseif ($subbin.GetArrayLength() -ge 2 -and $subbin[1] -match $query) { $bins += $subbin[1] @@ -65,25 +61,33 @@ function search_bucket($bucket, $query) { $apps = Get-ChildItem (Find-BucketDirectory $bucket) -Filter '*.json' -Recurse $apps | ForEach-Object { - $json = [System.Text.Json.JsonDocument]::Parse([System.IO.File]::ReadAllText($_.FullName)) + $filepath = $_.FullName + + $json = try { + [System.Text.Json.JsonDocument]::Parse([System.IO.File]::ReadAllText($filepath)) + } catch { + debug "Failed to parse manifest file: $filepath (error: $_)" + return + } + $name = $_.BaseName if ($name -match $query) { $list.Add([PSCustomObject]@{ - Name = $name - Version = $json.RootElement.GetProperty("version") - Source = $bucket - Binaries = "" - }) + Name = $name + Version = $json.RootElement.GetProperty('version') + Source = $bucket + Binaries = '' + }) } else { $bin = bin_match_json $json $query if ($bin) { $list.Add([PSCustomObject]@{ - Name = $name - Version = $json.RootElement.GetProperty("version") - Source = $bucket - Binaries = $bin -join ' | ' - }) + Name = $name + Version = $json.RootElement.GetProperty('version') + Source = $bucket + Binaries = $bin -join ' | ' + }) } } } @@ -99,20 +103,20 @@ function search_bucket_legacy($bucket, $query) { if ($name -match $query) { $list.Add([PSCustomObject]@{ - Name = $name - Version = $manifest.Version - Source = $bucket - Binaries = "" - }) + Name = $name + Version = $manifest.Version + Source = $bucket + Binaries = '' + }) } else { $bin = bin_match $manifest $query if ($bin) { $list.Add([PSCustomObject]@{ - Name = $name - Version = $manifest.Version - Source = $bucket - Binaries = $bin -join ' | ' - }) + Name = $name + Version = $manifest.Version + Source = $bucket + Binaries = $bin -join ' | ' + }) } } } @@ -154,7 +158,7 @@ function search_remotes($query) { $names = $buckets | Get-Member -MemberType NoteProperty | Select-Object -ExpandProperty name $results = $names | Where-Object { !(Test-Path $(Find-BucketDirectory $_)) } | ForEach-Object { - @{ "bucket" = $_; "results" = (search_remote $_ $query) } + @{ 'bucket' = $_; 'results' = (search_remote $_ $query) } } | Where-Object { $_.results } if ($results.count -gt 0) { @@ -175,25 +179,45 @@ function search_remotes($query) { $remote_list } -$jsonTextAvailable = [System.AppDomain]::CurrentDomain.GetAssemblies() | Where-object { [System.IO.Path]::GetFileNameWithoutExtension($_.Location) -eq "System.Text.Json" } +if (get_config USE_SQLITE_CACHE) { + . "$PSScriptRoot\..\lib\database.ps1" + Select-ScoopDBItem $query -From @('name', 'binary', 'shortcut') | + Select-Object -Property name, version, bucket, binary | + ForEach-Object { + $list.Add([PSCustomObject]@{ + Name = $_.name + Version = $_.version + Source = $_.bucket + Binaries = $_.binary + }) + } +} else { + try { + $query = New-Object Regex $query, 'IgnoreCase' + } catch { + abort "Invalid regular expression: $($_.Exception.InnerException.Message)" + } + + $jsonTextAvailable = [System.AppDomain]::CurrentDomain.GetAssemblies() | Where-Object { [System.IO.Path]::GetFileNameWithoutExtension($_.Location) -eq 'System.Text.Json' } -Get-LocalBucket | ForEach-Object { - if ($jsonTextAvailable) { - search_bucket $_ $query - } else { - search_bucket_legacy $_ $query + Get-LocalBucket | ForEach-Object { + if ($jsonTextAvailable) { + search_bucket $_ $query + } else { + search_bucket_legacy $_ $query + } } } if ($list.Count -gt 0) { - Write-Host "Results from local buckets..." + Write-Host 'Results from local buckets...' $list } if ($list.Count -eq 0 -and !(github_ratelimit_reached)) { $remote_results = search_remotes $query if (!$remote_results) { - warn "No matches found." + warn 'No matches found.' exit 1 } $remote_results diff --git a/libexec/scoop-uninstall.ps1 b/libexec/scoop-uninstall.ps1 index 7931158eec..5ad606a461 100644 --- a/libexec/scoop-uninstall.ps1 +++ b/libexec/scoop-uninstall.ps1 @@ -74,7 +74,7 @@ if (!$apps) { exit 0 } continue } - run_uninstaller $manifest $architecture $dir + Invoke-Installer -Path $dir -Manifest $manifest -ProcessorArchitecture $architecture -Uninstall rm_shims $app $manifest $global $architecture rm_startmenu_shortcuts $manifest $global $architecture diff --git a/libexec/scoop-update.ps1 b/libexec/scoop-update.ps1 index 354fb2e839..bd825731c3 100644 --- a/libexec/scoop-update.ps1 +++ b/libexec/scoop-update.ps1 @@ -6,13 +6,13 @@ # You can use '*' in place of to update all apps. # # Options: -# -f, --force Force update even when there isn't a newer version -# -g, --global Update a globally installed app -# -i, --independent Don't install dependencies automatically -# -k, --no-cache Don't use the download cache -# -s, --skip Skip hash validation (use with caution!) -# -q, --quiet Hide extraneous messages -# -a, --all Update all apps (alternative to '*') +# -f, --force Force update even when there isn't a newer version +# -g, --global Update a globally installed app +# -i, --independent Don't install dependencies automatically +# -k, --no-cache Don't use the download cache +# -s, --skip-hash-check Skip hash validation (use with caution!) +# -q, --quiet Hide extraneous messages +# -a, --all Update all apps (alternative to '*') . "$PSScriptRoot\..\lib\getopt.ps1" . "$PSScriptRoot\..\lib\json.ps1" # 'save_install_info' in 'manifest.ps1' (indirectly) @@ -24,12 +24,15 @@ . "$PSScriptRoot\..\lib\versions.ps1" . "$PSScriptRoot\..\lib\depends.ps1" . "$PSScriptRoot\..\lib\install.ps1" +if (get_config USE_SQLITE_CACHE) { + . "$PSScriptRoot\..\lib\database.ps1" +} -$opt, $apps, $err = getopt $args 'gfiksqa' 'global', 'force', 'independent', 'no-cache', 'skip', 'quiet', 'all' +$opt, $apps, $err = getopt $args 'gfiksqa' 'global', 'force', 'independent', 'no-cache', 'skip-hash-check', 'quiet', 'all' if ($err) { "scoop update: $err"; exit 1 } $global = $opt.g -or $opt.global $force = $opt.f -or $opt.force -$check_hash = !($opt.s -or $opt.skip) +$check_hash = !($opt.s -or $opt.'skip-hash-check') $use_cache = !($opt.k -or $opt.'no-cache') $quiet = $opt.q -or $opt.quiet $independent = $opt.i -or $opt.independent @@ -38,21 +41,21 @@ $all = $opt.a -or $opt.all # load config $configRepo = get_config SCOOP_REPO if (!$configRepo) { - $configRepo = "https://github.com/ScoopInstaller/Scoop" + $configRepo = 'https://github.com/ScoopInstaller/Scoop' set_config SCOOP_REPO $configRepo | Out-Null } # Find current update channel from config $configBranch = get_config SCOOP_BRANCH if (!$configBranch) { - $configBranch = "master" + $configBranch = 'master' set_config SCOOP_BRANCH $configBranch | Out-Null } -if(($PSVersionTable.PSVersion.Major) -lt 5) { +if (($PSVersionTable.PSVersion.Major) -lt 5) { # check powershell version - Write-Output "PowerShell 5 or later is required to run Scoop." - Write-Output "Upgrade PowerShell: https://docs.microsoft.com/en-us/powershell/scripting/install/installing-powershell-core-on-windows" + Write-Output 'PowerShell 5 or later is required to run Scoop.' + Write-Output 'Upgrade PowerShell: https://docs.microsoft.com/en-us/powershell/scripting/install/installing-powershell-core-on-windows' break } $show_update_log = get_config SHOW_UPDATE_LOG $true @@ -63,14 +66,14 @@ function Sync-Scoop { [Switch]$Log ) # Test if Scoop Core is hold - if(Test-ScoopCoreOnHold) { + if (Test-ScoopCoreOnHold) { return } # check for git if (!(Test-GitAvailable)) { abort "Scoop uses Git to update itself. Run 'scoop install git' and try again." } - Write-Host "Updating Scoop..." + Write-Host 'Updating Scoop...' $currentdir = versiondir 'scoop' 'current' if (!(Test-Path "$currentdir\.git")) { $newdir = "$currentdir\..\new" @@ -108,10 +111,10 @@ function Sync-Scoop { # Stash uncommitted changes if (Invoke-Git -Path $currentdir -ArgumentList @('diff', 'HEAD', '--name-only')) { if (get_config AUTOSTASH_ON_CONFLICT) { - warn "Uncommitted changes detected. Stashing..." + warn 'Uncommitted changes detected. Stashing...' Invoke-Git -Path $currentdir -ArgumentList @('stash', 'push', '-m', "WIP at $([System.DateTime]::Now.ToString('o'))", '-u', '-q') } else { - warn "Uncommitted changes detected. Update aborted." + warn 'Uncommitted changes detected. Update aborted.' return } } @@ -152,7 +155,7 @@ function Sync-Bucket { Param ( [Switch]$Log ) - Write-Host "Updating Buckets..." + Write-Host 'Updating Buckets...' if (!(Test-Path (Join-Path (Find-BucketDirectory 'main' -Root) '.git'))) { info "Converting 'main' bucket to git repo..." @@ -170,40 +173,88 @@ function Sync-Bucket { $buckets = Get-LocalBucket | ForEach-Object { $path = Find-BucketDirectory $_ -Root return @{ - name = $_ + name = $_ valid = Test-Path (Join-Path $path '.git') - path = $path + path = $path } } $buckets | Where-Object { !$_.valid } | ForEach-Object { Write-Host "'$($_.name)' is not a git repository. Skipped." } + $updatedFiles = [System.Collections.ArrayList]::Synchronized([System.Collections.ArrayList]::new()) + $removedFiles = [System.Collections.ArrayList]::Synchronized([System.Collections.ArrayList]::new()) if ($PSVersionTable.PSVersion.Major -ge 7) { # Parallel parameter is available since PowerShell 7 $buckets | Where-Object { $_.valid } | ForEach-Object -ThrottleLimit 5 -Parallel { . "$using:PSScriptRoot\..\lib\core.ps1" + . "$using:PSScriptRoot\..\lib\buckets.ps1" - $bucketLoc = $_.path $name = $_.name + $bucketLoc = $_.path + $innerBucketLoc = Find-BucketDirectory $name $previousCommit = Invoke-Git -Path $bucketLoc -ArgumentList @('rev-parse', 'HEAD') Invoke-Git -Path $bucketLoc -ArgumentList @('pull', '-q') if ($using:Log) { Invoke-GitLog -Path $bucketLoc -Name $name -CommitHash $previousCommit } + if (get_config USE_SQLITE_CACHE) { + Invoke-Git -Path $bucketLoc -ArgumentList @('diff', '--name-status', $previousCommit) | ForEach-Object { + $status, $file = $_ -split '\s+', 2 + $filePath = Join-Path $bucketLoc $file + if ($filePath -match "^$([regex]::Escape($innerBucketLoc)).*\.json$") { + switch ($status) { + { $_ -in 'A', 'M', 'R' } { + [void]($using:updatedFiles).Add($filePath) + } + 'D' { + [void]($using:removedFiles).Add([pscustomobject]@{ + Name = ([System.IO.FileInfo]$file).BaseName + Bucket = $name + }) + } + } + } + } + } } } else { $buckets | Where-Object { $_.valid } | ForEach-Object { - $bucketLoc = $_.path $name = $_.name + $bucketLoc = $_.path + $innerBucketLoc = Find-BucketDirectory $name $previousCommit = Invoke-Git -Path $bucketLoc -ArgumentList @('rev-parse', 'HEAD') Invoke-Git -Path $bucketLoc -ArgumentList @('pull', '-q') if ($Log) { Invoke-GitLog -Path $bucketLoc -Name $name -CommitHash $previousCommit } + if (get_config USE_SQLITE_CACHE) { + Invoke-Git -Path $bucketLoc -ArgumentList @('diff', '--name-status', $previousCommit) | ForEach-Object { + $status, $file = $_ -split '\s+', 2 + $filePath = Join-Path $bucketLoc $file + if ($filePath -match "^$([regex]::Escape($innerBucketLoc)).*\.json$") { + switch ($status) { + { $_ -in 'A', 'M', 'R' } { + [void]($updatedFiles).Add($filePath) + } + 'D' { + [void]($removedFiles).Add([pscustomobject]@{ + Name = ([System.IO.FileInfo]$file).BaseName + Bucket = $name + }) + } + } + } + } + } } } + if ((get_config USE_SQLITE_CACHE) -and ($updatedFiles.Count -gt 0 -or $removedFiles.Count -gt 0)) { + info 'Updating cache...' + Set-ScoopDB -Path $updatedFiles + $removedFiles | Remove-ScoopDBItem + } } function update($app, $global, $quiet = $false, $independent, $suggested, $use_cache = $true, $check_hash = $true) { @@ -251,7 +302,7 @@ function update($app, $global, $quiet = $false, $independent, $suggested, $use_c # region Workaround # Workaround for https://github.com/ScoopInstaller/Scoop/issues/2220 until install is refactored # Remove and replace whole region after proper fix - Write-Host "Downloading new version" + Write-Host 'Downloading new version' if (Test-Aria2Enabled) { Invoke-CachedAria2Download $app $version $manifest $architecture $cachedir $manifest.cookie $true $check_hash } else { @@ -269,12 +320,12 @@ function update($app, $global, $quiet = $false, $independent, $suggested, $use_c error $err if (Test-Path $source) { # rm cached file - Remove-Item -force $source + Remove-Item -Force $source } if ($url.Contains('sourceforge.net')) { Write-Host -f yellow 'SourceForge.net is known for causing hash validation fails. Please try again before opening a ticket.' } - abort $(new_issue_msg $app $bucket "hash check failed") + abort $(new_issue_msg $app $bucket 'hash check failed') } } } @@ -289,7 +340,7 @@ function update($app, $global, $quiet = $false, $independent, $suggested, $use_c Invoke-HookScript -HookType 'pre_uninstall' -Manifest $old_manifest -Arch $architecture Write-Host "Uninstalling '$app' ($old_version)" - run_uninstaller $old_manifest $architecture $dir + Invoke-Installer -Path $dir -Manifest $old_manifest -ProcessorArchitecture $architecture -Uninstall rm_shims $app $old_manifest $global $architecture # If a junction was used during install, that will have been used @@ -404,11 +455,11 @@ if (-not ($apps -or $all)) { } elseif ($outdated.Length -eq 0) { Write-Host -f Green "Latest versions for all apps are installed! For more information try 'scoop status'" } else { - Write-Host -f DarkCyan "Updating one outdated app:" + Write-Host -f DarkCyan 'Updating one outdated app:' } } - $suggested = @{}; + $suggested = @{} # $outdated is a list of ($app, $global) tuples $outdated | ForEach-Object { update @_ $quiet $independent $suggested $use_cache $check_hash } } diff --git a/test/Scoop-Core.Tests.ps1 b/test/Scoop-Core.Tests.ps1 index 3fb7fbd663..f6846e5456 100644 --- a/test/Scoop-Core.Tests.ps1 +++ b/test/Scoop-Core.Tests.ps1 @@ -259,6 +259,23 @@ Describe 'get_app_name_from_shim' -Tag 'Scoop', 'Windows' { } } +Describe 'cache_path' -Tag 'Scoop' { + It 'returns the correct cache path for a given input' { + $url = 'https://example.com/git.zip' + $ret = cache_path 'git' '2.44.0' $url + $inputStream = [System.IO.MemoryStream]::new([System.Text.Encoding]::UTF8.GetBytes($url)) + $sha = (Get-FileHash -Algorithm SHA256 -InputStream $inputStream).Hash.ToLower().Substring(0, 7) + $ret | Should -Be "$cachedir\git#2.44.0#$sha.zip" + } + + # # NOTE: Remove this 6 months after the feature ships. + It 'returns the old format cache path for a given input' { + Mock Test-Path { $true } + $ret = cache_path 'git' '2.44.0' 'https://example.com/git.zip' + $ret | Should -Be "$cachedir\git#2.44.0#https_example.com_git.zip" + } +} + Describe 'sanitary_path' -Tag 'Scoop' { It 'removes invalid path characters from a string' { $path = 'test?.json' diff --git a/test/Scoop-Decompress.Tests.ps1 b/test/Scoop-Decompress.Tests.ps1 index 86220af5cb..4ec31a4ece 100644 --- a/test/Scoop-Decompress.Tests.ps1 +++ b/test/Scoop-Decompress.Tests.ps1 @@ -25,7 +25,7 @@ Describe 'Decompression function' -Tag 'Scoop', 'Windows', 'Decompress' { } It 'Test cases should exist and hash should match' { $testcases | Should -Exist - (Get-FileHash -Path $testcases -Algorithm SHA256).Hash.ToLower() | Should -Be '23a23a63e89ff95f5ef27f0cacf08055c2779cf41932266d8f509c2e200b8b63' + (Get-FileHash -Path $testcases -Algorithm SHA256).Hash.ToLower() | Should -Be 'afb86b0552187b8d630ce25d02835fb809af81c584f07e54cb049fb74ca134b6' } It 'Test cases should be extracted correctly' { { Microsoft.PowerShell.Archive\Expand-Archive -Path $testcases -DestinationPath $working_dir } | Should -Not -Throw @@ -152,41 +152,6 @@ Describe 'Decompression function' -Tag 'Scoop', 'Windows', 'Decompress' { } } - Context 'zstd extraction' { - - BeforeAll { - if ($env:CI) { - Mock Get-AppFilePath { $env:SCOOP_ZSTD_PATH } -ParameterFilter { $Helper -eq 'zstd' } - Mock Get-AppFilePath { '7z.exe' } -ParameterFilter { $Helper -eq '7zip' } - } elseif (!(installed zstd)) { - scoop install zstd - } - - $test1 = "$working_dir\ZstdTest.zst" - $test2 = "$working_dir\ZstdTest.tar.zst" - } - - It 'extract normal compressed file' { - $to = test_extract 'Expand-ZstdArchive' $test1 - $to | Should -Exist - "$to\ZstdTest" | Should -Exist - (Get-ChildItem $to).Count | Should -Be 1 - } - - It 'extract nested compressed file' { - $to = test_extract 'Expand-ZstdArchive' $test2 - $to | Should -Exist - "$to\ZstdTest" | Should -Exist - (Get-ChildItem $to).Count | Should -Be 1 - } - - It 'works with "-Removal" switch ($removal param)' { - $test1 | Should -Exist - test_extract 'Expand-ZstdArchive' $test1 $true - $test1 | Should -Not -Exist - } - } - Context 'msi extraction' { BeforeAll { diff --git a/test/Scoop-Depends.Tests.ps1 b/test/Scoop-Depends.Tests.ps1 index d80d7d2652..79b868ef90 100644 --- a/test/Scoop-Depends.Tests.ps1 +++ b/test/Scoop-Depends.Tests.ps1 @@ -14,11 +14,6 @@ Describe 'Package Dependencies' -Tag 'Scoop' { Test-7zipRequirement -Uri 'test.bin' | Should -BeFalse Test-7zipRequirement -Uri @('test.xz', 'test.bin') | Should -BeTrue } - It 'Test Zstd requirement' { - Test-ZstdRequirement -Uri 'test.zst' | Should -BeTrue - Test-ZstdRequirement -Uri 'test.bin' | Should -BeFalse - Test-ZstdRequirement -Uri @('test.zst', 'test.bin') | Should -BeTrue - } It 'Test lessmsi requirement' { Mock get_config { $true } Test-LessmsiRequirement -Uri 'test.msi' | Should -BeTrue @@ -27,7 +22,6 @@ Describe 'Package Dependencies' -Tag 'Scoop' { } It 'Allow $Uri be $null' { Test-7zipRequirement -Uri $null | Should -BeFalse - Test-ZstdRequirement -Uri $null | Should -BeFalse Test-LessmsiRequirement -Uri $null | Should -BeFalse } } diff --git a/test/bin/test.ps1 b/test/bin/test.ps1 index ffb35351a7..a25940ccb9 100644 --- a/test/bin/test.ps1 +++ b/test/bin/test.ps1 @@ -70,19 +70,6 @@ if ($env:CI -eq $true) { Invoke-WebRequest -Uri $source -OutFile $destination & 7z.exe x "$env:SCOOP_HELPERS_PATH\innounp.rar" -o"$env:SCOOP_HELPERS_PATH\innounp" -y | Out-Null } - - # Only download zstd for AppVeyor, GitHub Actions has zstd installed by default - if ($env:BHBuildSystem -eq 'AppVeyor') { - $env:SCOOP_ZSTD_PATH = "$env:SCOOP_HELPERS_PATH\zstd\zstd.exe" - if (!(Test-Path $env:SCOOP_ZSTD_PATH)) { - $source = 'https://github.com/facebook/zstd/releases/download/v1.5.1/zstd-v1.5.1-win32.zip' - $destination = "$env:SCOOP_HELPERS_PATH\zstd.zip" - Invoke-WebRequest -Uri $source -OutFile $destination - & 7z.exe x "$env:SCOOP_HELPERS_PATH\zstd.zip" -o"$env:SCOOP_HELPERS_PATH\zstd" -y | Out-Null - } - } else { - $env:SCOOP_ZSTD_PATH = (Get-Command zstd.exe).Path - } } } diff --git a/test/fixtures/decompress/TestCases.zip b/test/fixtures/decompress/TestCases.zip index d558faac39..d2cd98c0f2 100644 Binary files a/test/fixtures/decompress/TestCases.zip and b/test/fixtures/decompress/TestCases.zip differ