[Bug] Shim path with umlauts is not encoded in utf-8

[kunago]

Bug Report

Current Behavior

Often when I try to run commands I receive an error because of the path. My username contains a space and umlauts. Unfortunately scoop does not seem to use utf-8 or so it seems. So instead of "á" in the username in the path I see a question mark instead which obviously results in an error. This is a current issue, I guess a month old.

Expected Behavior

I expect the path to be encoded correctly with umlauts as well.

Additional context/output

PS C:\Users\Peter Novám> scoop bucket list
Could not create process with command 'C:\Users\Peter Nov?k\scoop\apps\git\current\cmd\git.exe  -C "C:\Users\Peter Novßk\scoop\buckets\extras" log --format=%aD -n 1'.
Could not create process with command 'C:\Users\Peter Nov?k\scoop\apps\git\current\cmd\git.exe  -C "C:\Users\Peter Novßk\scoop\buckets\extras" config remote.origin.url'.
Could not create process with command 'C:\Users\Peter Nov?k\scoop\apps\git\current\cmd\git.exe  -C "C:\Users\Peter Novßk\scoop\buckets\games" log --format=%aD -n 1'.
Could not create process with command 'C:\Users\Peter Nov?k\scoop\apps\git\current\cmd\git.exe  -C "C:\Users\Peter Novßk\scoop\buckets\games" config remote.origin.url'.
Could not create process with command 'C:\Users\Peter Nov?k\scoop\apps\git\current\cmd\git.exe  -C "C:\Users\Peter Novßk\scoop\buckets\java" log --format=%aD -n 1'.
Could not create process with command 'C:\Users\Peter Nov?k\scoop\apps\git\current\cmd\git.exe  -C "C:\Users\Peter Novßk\scoop\buckets\java" config remote.origin.url'.
Could not create process with command 'C:\Users\Peter Nov?k\scoop\apps\git\current\cmd\git.exe  -C "C:\Users\Peter Novßk\scoop\buckets\nirsoft" log --format=%aD -n 1'.
Could not create process with command 'C:\Users\Peter Nov?k\scoop\apps\git\current\cmd\git.exe  -C "C:\Users\Peter Novßk\scoop\buckets\nirsoft" config remote.origin.url'.
Could not create process with command 'C:\Users\Peter Nov?k\scoop\apps\git\current\cmd\git.exe  -C "C:\Users\Peter Novßk\scoop\buckets\versions" log --format=%aD -n 1'.
Could not create process with command 'C:\Users\Peter Nov?k\scoop\apps\git\current\cmd\git.exe  -C "C:\Users\Peter Novßk\scoop\buckets\versions" config remote.origin.url'.

Name     Source Updated Manifests
----     ------ ------- ---------
extras                       1430
games                         215
java                          220
nirsoft                       266
versions                      278

Possible Solution

System details

Windows version: [e.g. 7, 8, 10]
Windows 11

OS architecture: [e.g. 32bit, 64bit]
64bit

PowerShell version: [output of "$($PSVersionTable.PSVersion)"]

Major  Minor  Build  Revision
-----  -----  -----  --------
5      1      22000  282

Additional software: [(optional) e.g. ConEmu, Git]
I also use conemu but in this case it happens also with pure PowerShell

Scoop Configuration

{
    "lastupdate":  "2022-03-08T09:11:13.4293955+01:00",
    "SCOOP_REPO":  "https://github.com/lukesampson/scoop",
    "SCOOP_BRANCH":  "master"
}

[HUMORCE]

PS C:\Users\Peter Novßk> cat ~\scoop\shims\git.shim
path = "C:\Users\Peter Nov?k\scoop\apps\git\current\cmd\git.exe"

/cc @niheaven

EDIT:

a similar issue exists in installing app(creating shortcut) also.

PS C:\Users\Peter Novßk> scoop install git
Installing 'git' (2.35.1.windows.2) [64bit]
Loading PortableGit-2.35.1.2-64-bit.7z.exe from cache
Checking hash of PortableGit-2.35.1.2-64-bit.7z.exe ... ok.
Extracting dl.7z ... done.
Linking ~\scoop\apps\git\current => ~\scoop\apps\git\2.35.1.windows.2
Creating shim for 'git'.
Creating shim for 'gitk'.
Creating shim for 'git-gui'.
Creating shim for 'tig'.
Creating shim for 'git-bash'.
Value does not fall within the expected range.
At C:\Users\Peter Novßk\scoop\apps\scoop\current\lib\shortcuts.ps1:48 char:5
+     $wsShell.TargetPath = $target.FullName
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (:) [], ArgumentException
    + FullyQualifiedErrorId : System.ArgumentException

Unable to save shortcut "C:\Users\Peter Nov?k\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Scoop
Apps\Git Bash.lnk".
At C:\Users\Peter Novßk\scoop\apps\scoop\current\lib\shortcuts.ps1:56 char:5
+     $wsShell.Save()
+     ~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (:) [], FileNotFoundException
    + FullyQualifiedErrorId : System.IO.FileNotFoundException

Creating shortcut for Git Bash (git-bash.exe)
Value does not fall within the expected range.
At C:\Users\Peter Novßk\scoop\apps\scoop\current\lib\shortcuts.ps1:48 char:5
+     $wsShell.TargetPath = $target.FullName
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (:) [], ArgumentException
    + FullyQualifiedErrorId : System.ArgumentException

Unable to save shortcut "C:\Users\Peter Nov?k\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Scoop
Apps\Git GUI.lnk".
At C:\Users\Peter Novßk\scoop\apps\scoop\current\lib\shortcuts.ps1:56 char:5
+     $wsShell.Save()
+     ~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (:) [], FileNotFoundException
    + FullyQualifiedErrorId : System.IO.FileNotFoundException

Creating shortcut for Git GUI (git-gui.exe)
'git' (2.35.1.windows.2) was installed successfully!
Notes
-----
Set Git Credential Manager Core by running: "git config --global credential.helper manager-core"

[kunago]

Yes, apparently, that is what is in the shim. It is in all shims so the questions is whether there needs to be some encoding changed or turned on.

[rashil2000]

Did you face this issue just today or before today also?

[kunago]

Other days as well. I noticed for the first time a few weeks ago when using adb which needed to be launched directly from the dir to bypass the wrong shim path. As of today, it started happening with other commands too.

[niheaven]

We have Out-UTF8File now, but it doesn't support -Append so I haven't replaced Out-File in shim creation. Will be fixed in 'develop' and may in next main release.

[rashil2000]

Similar issues:

• Autoupdate error when used directory name with multibyte characters #2573
• Special characters in Windows user names will cause command scripts in "shims" to fail. #2230

[chawyehsu]

@kunago @HUMORCE Could you please type $OutputEncoding and paste the result?

[kunago]

PS C:\Users\Petr Novák> $OutputEncoding

IsSingleByte      : True
BodyName          : us-ascii
EncodingName      : US-ASCII
HeaderName        : us-ascii
WebName           : us-ascii
WindowsCodePage   : 1252
IsBrowserDisplay  : False
IsBrowserSave     : False
IsMailNewsDisplay : True
IsMailNewsSave    : True
EncoderFallback   : System.Text.EncoderReplacementFallback
DecoderFallback   : System.Text.DecoderReplacementFallback
IsReadOnly        : True
CodePage          : 20127

[HUMORCE]

PS C:\Users\Peter Novßk> $OutputEncoding

IsSingleByte      : True
BodyName          : us-ascii
EncodingName      : US-ASCII
HeaderName        : us-ascii
WebName           : us-ascii
WindowsCodePage   : 1252
IsBrowserDisplay  : False
IsBrowserSave     : False
IsMailNewsDisplay : True
IsMailNewsSave    : True
EncoderFallback   : System.Text.EncoderReplacementFallback
DecoderFallback   : System.Text.DecoderReplacementFallback
IsReadOnly        : True
CodePage          : 20127

i simply created a user with samename as @kunago in the VM.

[kunago]

As it seems, I am uble to run "scoop" from cmd with powershell at the beginning. While simply running scoop returns the invalid path error, running powershell scoop passes fine. I am update, see the status of apps and use all other scoop commands.

The problem can be partially solved this way which is the way I am using now to run scoop.

However, the umlauts in *.shim files returns an error anytime running any other app from the command line. For instance running php from the command line, which was previously installed with scoop, once again returns the error, and regardless whether I run it with php or powershell php.

Do you know of a workaround for this use case?

[niheaven]

Wait for scoop version bump, and reset all your shims may help. And if bugs are still there, please report again.