Git staging individual changes converts UTF-8 with BOM to UTF-8

[ridago]

Type: Bug

1. Start with a UTF-8 with BOM file
2. Make a change to some line
3. Click the change indicator to the left and choose "Stage change". Do NOT stage the whole file
4. Commit the staged change

The file in HEAD is now UTF-8, not UTF-8 with BOM. I.e. VSCode removed the EF BB BF bytes at the start of the file. The file in the current workspace is still UTF-8 with BOM, meaning if you now stage the whole file it will revert back to a UTF-8 with BOM file, but the history will show it first being converted to UTF-8, then back to UTF-8 with BOM.

This is not the same behaviour as staging individual changes with git add -p

VS Code version: Code 1.81.1 (6c3e3db, 2023-08-09T22:22:42.175Z)
OS version: Windows_NT x64 10.0.22621
Modes:

System Info

Item	Value
CPUs	Intel(R) Core(TM) i7-8700K CPU @ 3.70GHz (12 x 3696)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: disabled_off direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok video_decode: enabled video_encode: enabled vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: enabled
Load (avg)	undefined
Memory (System)	31.93GB (22.85GB free)
Process Argv	--crash-reporter-id 11feb52b-844e-4d4e-a78a-c5f927f3e2df
Screen Reader	no
VM	0%

Extensions (17)

Extension	Author (truncated)	Version
vscode-docker	ms-	1.26.0
python	ms-	2023.14.0
vscode-pylance	ms-	2023.8.40
jupyter	ms-	2023.7.1002162226
jupyter-keymap	ms-	1.1.2
jupyter-renderers	ms-	1.0.17
vscode-jupyter-cell-tags	ms-	0.1.8
vscode-jupyter-slideshow	ms-	0.1.5
remote-containers	ms-	0.304.0
remote-ssh	ms-	0.102.0
remote-ssh-edit	ms-	0.86.0
remote-wsl	ms-	0.81.0
vscode-remote-extensionpack	ms-	0.24.0
cpptools	ms-	1.17.4
hexeditor	ms-	1.9.12
remote-explorer	ms-	0.4.1
remote-server	ms-	1.4.3

A/B Experiments

vsliv368cf:30146710
vsreu685:30147344
python383:30185418
vspor879:30202332
vspor708:30202333
vspor363:30204092
vslsvsres303:30308271
vserr242cf:30382550
pythontb:30283811
vsjup518:30340749
pythonptprofiler:30281270
vshan820:30294714
vstes263:30335439
vscod805:30301674
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
vsaa593cf:30376535
pythonvs932:30410667
py29gd2263:30792226
vsclangdf:30486550
c4g48928:30535728
dsvsc012:30540252
pynewext54:30695312
azure-dev_surveyonecf:30548226
vsccc:30803844
282f8724:30602487
f6dab269:30613381
2i9eh265:30646982
showlangstatbar:30737416
a2ce3375:30757347
pythonfmttext:30731395
fixshowwlkth:30771522
showindicator:30805244
pythongtdpath:30769146
i26e3531:30792625
gsofb:30804716
pythonnosmt12:30797651
pythonidxptcf:30805731
pythonnoceb:30805159
asynctok:30821568
dsvsc013:30795093
dsvsc014:30804076
diffeditorv2:30821572

[Gouvernathor]

Happens to me as well, and causes issues on the repos I'm a part of. I would very much like a solution to this.

[hhmmjjnn]

Me too.

Not only with the Stage Selected Ranges command, but with Unstage Selected Ranges and Revert Selected Ranges, as well.

[Scorg]

Can confirm this unfixed as of 1.90.1

[michaelmesser]

This bug is annoying when working on Visual Studio projects because Visual Studio creates BOMs by default, but VSCode silently removes them leading to a mix of encodings.

[sirtheta]

Our old codebase uses mostly windows-1252, and we need to use this encoding. When we stage only selected ranges, it will commit the file as utf-8. This is very annoying.

[hhmmjjnn]

Right. It doesn't have anything to do with the BOM. Any encoding will be lost in favor of (BOM-less) UTF-8 regardless.

[ridago]

Had another look into this today and realised it's a duplicate of #111915 so it should probably me marked as such