Git doesn't consider actual encoding in diff view

[izacsc]

• VSCode Version: 1.9.1
• OS Version: Windows 10

Steps to Reproduce:

1. Open a file with especific file encoding and change it.
2. Observe encoding diferences in diff view

In the example above, I just added a blank line in a Windows 1252 file.

[bizoo]

Note that this issue is not related to SCM Preview, I've the same behavior in 1.10.2 without it enabled.

[eamodio]

@joaomoreno I have an open request to bring this same functionality to GitLens -- so it would be wonderful if what is done here would be accessible to extensions.

[zabel-xyz]

For now, I have to go back to the previous version with the command "Disable Preview SCM".
Because, it's too difficult to check the git differences with this encoding issue.
(VSCode 1.11.1)

[eamodio]

I was hoping the new auto-detect of file encoding would help here, but it looks like that feature only works for actual files on disk and not via content providers. :(

[joaomoreno]

Merged #24911 which enables decoding with the encoding configured in settings. This helps most users.

Can't really fix it all the way without #824.

[PorkyProgs]

I guess that I am suffering from the same problem. I am not wise with GIT (I just hope it works) so I cannot say what encoding my files are being saved within the Git Repository.
I am having the following special effect when I compare my current file wit the one in the repository:

I am using:

And when I did a Git Diff from the cmdline I got:
Dev2-GIT[Develop]:09.13 14:09>git diff HEAD .\Runbooks\Manage-RBADObjects.ps1 diff --git a/Runbooks/Manage-RBADObjects.ps1 b/Runbooks/Manage-RBADObjects.ps1 index bf755e8..9869c25 100644 Binary files a/Runbooks/Manage-RBADObjects.ps1 and b/Runbooks/Manage-RBADObjects.ps1 differ
However, I am left no wiser after the event.
I wonder if the problem occurs during the uploading of the files into the Git Repository or if the data is per default saved in Git using different encoding.
When I view my Git Settings I have .....
Dev2-GIT[Develop]:09.13 14:09>git config --list core.symlinks=false core.autocrlf=true core.fscache=true color.diff=auto color.status=auto color.branch=auto color.interactive=true help.format=html rebase.autosquash=true http.sslcainfo=C:/Program Files/Git/mingw64/ssl/certs/ca-bundle.crt diff.astextplain.textconv=astextplain filter.lfs.clean=git-lfs clean -- %f filter.lfs.smudge=git-lfs smudge -- %f filter.lfs.required=true filter.lfs.process=git-lfs filter-process credential.helper=manager gui.recentrepo=C:/Depo/RB/Dev2-GIT user.name=Porky user.email=ilovegit@gmx.de difftool.sourcetree.cmd='C:/Users/Porky/Documents/PortableApps/WinMergePortable/WinMergePortable.exe' "$LOCAL" "$REMOTE" core.excludesfile=C:\Depo\RB\Dev2-GIT\.gitignore core.editor=code --wait http.proxy=http://149.249.158.30:8080 credential.helper=wincred core.repositoryformatversion=0 core.filemode=false core.bare=false core.logallrefupdates=true core.symlinks=false core.ignorecase=true core.hidedotfiles=dotGitOnly gui.wmstate=normal remote.origin.url=ssh://git@Our.Git.Repository.net:7999/Dev/sma.git remote.origin.fetch=+refs/heads/*:refs/remotes/origin/* remote.origin.pushurl=ssh://git@Our.Git.Repository.net:7999/Dev/sma.git gitflow.branch.master=master gitflow.branch.develop=develop gitflow.prefix.feature=feature/ gitflow.prefix.release=release/ gitflow.prefix.hotfix=hotfix/ gitflow.prefix.support=support/ gitflow.prefix.versiontag= branch.master.remote=origin branch.master.merge=refs/heads/master branch.develop.remote=origin branch.develop.merge=refs/heads/develop user.name=Porky user.email=ilovegit@gmx.de

So far I have not been able to isolate exactly where the problem is, is it a Git problem or a Visual Team Studio problem? Incidentally, if it helps I am on a Win7SP1 box. I often edit files in the Powershell ISE and presume files are being save using UTF8.
Thanks for your suggestions in advance,
Porky

[larsxschneider]

I am working on a patch series for core Git to help Git understand different encodings. If this sounds interesting to you, then I would love to hear your feedback on my proposal: https://github.com/git/git/pull/440/files#diff-6fb066a3d66afe00c0995f97f5c680fd

The current mailing list discussion is here if you want to chime in:
https://public-inbox.org/git/DC552BF4-3E87-41E0-BF92-4BA9633D374E@gmail.com/

[vkavalchuk]

I have the same issue with the Mercurial plugin (https://marketplace.visualstudio.com/items?itemName=mrcrowl.hg). Problem is the same on both views.

Inline diff view:

Diff view. Whole document marked as changed, but 2 lines deleted only:

Preferenced file encoding is set to "files.encoding": "windows-1251". VSCode version:

[joaomoreno]

Fixed by having git attempt to detect encoding when files.autoGuessEncoding is enabled.

Some encodings are still improperly detected: https://github.com/Microsoft/vscode/issues?q=is%3Aopen+is%3Aissue+label%3Afile-encoding+label%3Aupstream

[bizoo]

IMHO, the only good way to solve this issue is to use the encoding of the view, so to first resolve #824.

Why having a proper API to set/get encoding is so long to be done ? Seriously, it's not a rhetorical question: Is it because it's difficult or because it's a very low priority thing ? Or something else ?