Error in MSBuild package when code contains an acute accent

[Meligy]

Let's say I have this enum:

public enum MeetingLocation
{
  Café,
  Restaurant
}

I can build the project just fine even though it has the acute accent (é).

But if I add the CSharpier MSBuild package, I get an error on build:

  Error <FULL_FILE_PATH> - Failed to compile so was not formatted.

Of course the file does not get formatted, but it does not block formatting other files or fail the build itself.

Adding the comment // csharpier-ignore to the beginning of the file or the previous line does not seem to help.

This is tested agains CSharpier.MsBuild@0.20.0, on .NET SDK 6.0.403.

The VS Code extension does not seem to have a problem (yes, I checked the extension logs).

[belav]

I wasn't able to reproduce this until I installed 6.0.403, as a workaround going back to 6.0.401 should get you past the problem.

This does not appear specific to csharpier.msbuild, when I install csharpier 0.20.0 in my test project the dotnet csharpier . command fails with the same error.

[Meligy]

Wow. I totally did not expect it to be related to .NET SDK version. I mentioned it just to provide as much info as I could.

Thanks for the hint.

[belav]

I'm no longer sure downgrading will work. After downgrading I still have the problem. I did track it down to an issue with incorrectly detecting file encoding.

When the file is UTF8, csharpier is detecting it as SBCSCodePageEncoding, which leads to

    Café,
// instead of
    Café,

When I switch the file encoding to UTF-8-BOM, then csharpier can format the file.

I'm using https://github.com/CharsetDetector/UTF-unknown to do the file encoding detection, so this may be a bug with that library. The latest version of the library still has the issue.

Another option that will probably work is adding the file to a csharpierignore file, because then csharpier will not try to read the file at all. Using // csharpier-ignore requires that csharpier is able to compile the file in order to read the comments.

[belav]

So it appears that UTF.Unknown is detecting 3 possible encodings for a file with the given enum. UFT8 has a confidence of only .5, and windows-1250 has a confidence of .75. I opened this issue - CharsetDetector/UTF-unknown#159

If I remember correctly the IDE extensions stream all of the files to csharpier using UTF8. Which explains why you don't run into this issue with them. I an inclined to change csharpier to use UTF8 if it is in the possible detected encodings. But I don't know enough about file encodings to know if that could cause a problem for people.