dotnet new creates files with CRLF line endings on linux

[livarcocc]

From @kromatic on September 21, 2017 20:58

Steps to reproduce

Run dotnet new classlib in new project folder.

Expected behavior

On Linux, this should create Class1.cs and .csproj files with LF line endings.

Actual behavior

Class1.cs and the .csproj file are both created with CRLF line endings. (Files in bin and obj folders appear to all have the correct LF line endings.)

Environment data

dotnet --info output:

.NET Command Line Tools (2.0.0)

Product Information:
Version: 2.0.0
Commit SHA-1 hash: cdcd1928c9

Runtime Environment:
OS Name: fedora
OS Version: 26
OS Platform: Linux
RID: fedora.26-x64
Base Path: /usr/share/dotnet/sdk/2.0.0/

Microsoft .NET Core Shared Framework Host

Version : 2.0.0
Build : e8b8861ac7faf042c87a5c2f9f2d04c98b69f28d

Copied from original issue: dotnet/cli#7695

[mlorbetske]

This is currently "by design" (see #350 (comment)). Originally, we had standardized all content on LF line endings, but this caused issues with certain editors on Windows and produced a ton of messages when adding the files to Git.

Since the content is not produced line by line, we don't explicitly set what the line endings should be, we just emit what was included in the original content. We are looking in to a feature in the future (if there's enough interest) where templates could indicate that files matched by certain patterns would have their line endings replaced with ones matching the platform.

[SteffenL]

I just ran dotnet new [...] for the first time on Linux, and I was rather shocked to see CRLF line endings as I staged the files in the Git repository.

I would personally want native/expected behavior from a cross-platform tool, regardless of what elementary things certain editors have problems with. For example, if an editor (let's say Notepad on Windows) can't deal with LF, then I would rather fix the editor or use a different one.

At least for now, it seems like git config --global core.autocrlf input will work okay for me personally on Linux and macOS, and git config --global core.autocrlf true on Windows.

[sandyarmstrong]

This is incredibly frustrating on non-Windows platforms.

If you forget about this bug before you start making changes to your code, you can easily end up with mixed line endings, which confuses all sorts of tooling.

Fixing it for the whole repo requires dos2unix and, realistically, more shell scripting know-how than most devs possess to avoid accidentally modifying the contents of .git and node_modules, etc.

[laktak]

Could we get an update on the current status of this bug?

The last comment in the referenced issue says:

This is now fixed

but dotnet new still produces CRLF?

[michelesolytic]

I also encountered this problem today, and I disapprove the decision of keeping this behaviour by design. If there are some editors on Windows which have troubles handling LF endings, then it should be their responsibility to fix their behaviour, not yours.
I believe you should rather provide instructions to those users (or a command flag, or a tool) to counter the issue in those scenarios: once the users are aware of the problem, they'd eventually push the editor's developers to fix it.

[laymanmu]

Carriage Returns were designed for mechanical typewriters. What year is it again? Is there seriously no option to tell the dotnet cli to not add carriage returns? There's no config file or environment variable? What bubble do you live in to think that this is Ok? I think the emperor needs to put on some clothes - I'm here to tell you that the world does not use CRLF anymore and if you depend on it then you're being left behind.

[volkoff-pro]

This behavior of dotnet cli is very frustrating for me. I'm waiting maybe some flag for line ending, or something else like this.
If I generating big project using built-in templates it is very hard to change crlf to lf(

[genalt]

Looks like this issue is resolved. I have just checked on Linux, all files contain LF endings.
The issue will be closed next triage session.

[n-romaniv]

@grinrag, hi! It still repduces for me. It is very frustrating because OmniSharp in VSCode seems to work only when the files have LF as line endings. Otherwise, there is no highlighting or hints.

.NET Core SDK (reflecting any global.json):
Version: 3.1.301
Commit: 7feb845744

Runtime Environment:
OS Name: ubuntu
OS Version: 20.04
OS Platform: Linux
RID: linux-x64
Base Path: /usr/share/dotnet/sdk/3.1.301/

Host (useful for support):
Version: 3.1.5
Commit: 65cd789777

.NET Core SDKs installed:
3.1.301 [/usr/share/dotnet/sdk]

.NET Core runtimes installed:
Microsoft.AspNetCore.App 3.1.5 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 3.1.5 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

[arcaartem]

I can replicate the issue on MacOS Catalina too.

.net version installed: 3.1.403

[1ma]

Still reproducible on .NET 5.0

marcel@home:~/test$ dotnet --info
.NET SDK (reflecting any global.json):
 Version:   5.0.100
 Commit:    5044b93829

Runtime Environment:
 OS Name:     ubuntu
 OS Version:  18.04
 OS Platform: Linux
 RID:         ubuntu.18.04-x64
 Base Path:   /usr/share/dotnet/sdk/5.0.100/

Host (useful for support):
  Version: 5.0.0
  Commit:  cf258a14b7

.NET SDKs installed:
  5.0.100 [/usr/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 5.0.0 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 5.0.0 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET runtimes or SDKs:
  https://aka.ms/dotnet-download


marcel@home:~/test$ dotnet new classlib
The template "Class library" was created successfully.

Processing post-creation actions...
Running 'dotnet restore' on /home/marcel/test/test.csproj...
  Determining projects to restore...
  Restored /home/marcel/test/test.csproj (in 64 ms).
Restore succeeded.


marcel@home:~/test$ file Class1.cs 
Class1.cs: C++ source, UTF-8 Unicode (with BOM) text, with CRLF line terminators

[keegandent]

Still reproducible in .NET 6 Preview 3. Worse yet, it even uses \ path separators. Check how ExProj is included in ExSln in my below example. This really makes it seem like Unix/Linux is a second-class citizen for .NET and that changing it is low priority.

[keegan@t450s Projects]$ dotnet --info
.NET SDK (reflecting any global.json):
 Version:   6.0.100-preview.3.21202.5
 Commit:    aee38a6dd4

Runtime Environment:
 OS Name:     fedora
 OS Version:  33
 OS Platform: Linux
 RID:         fedora.33-x64
 Base Path:   /home/keegan/dotnet/sdk/6.0.100-preview.3.21202.5/

Host (useful for support):
  Version: 6.0.0-preview.3.21201.4
  Commit:  236cb21e3c

.NET SDKs installed:
  6.0.100-preview.3.21202.5 [/home/keegan/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.0-preview.3.21201.13 [/home/keegan/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 6.0.0-preview.3.21201.4 [/home/keegan/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET runtimes or SDKs:
  https://aka.ms/dotnet-download

[keegan@t450s Projects]$ mkdir ExSln
[keegan@t450s Projects]$ cd !$
cd ExSln
[keegan@t450s ExSln]$ dotnet new sln
The template "Solution File" was created successfully.
[keegan@t450s ExSln]$ mkdir -p src/ExProj
[keegan@t450s ExSln]$ cd !$
cd src/ExProj
[keegan@t450s ExProj]$ dotnet new classlib
The template "Class Library" was created successfully.

Processing post-creation actions...
Running 'dotnet restore' on /home/keegan/Projects/ExSln/src/ExProj/ExProj.csproj...
  Determining projects to restore...
  Restored /home/keegan/Projects/ExSln/src/ExProj/ExProj.csproj (in 64 ms).
Restore succeeded.

[keegan@t450s ExProj]$ cd -
/home/keegan/Projects/ExSln
[keegan@t450s ExSln]$ dotnet sln add src/ExProj/
Project `src/ExProj/ExProj.csproj` added to the solution.
[keegan@t450s ExSln]$ file ExSln.sln 
ExSln.sln: UTF-8 Unicode (with BOM) text, with CRLF line terminators
[keegan@t450s ExSln]$ cat !$
cat ExSln.sln

Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio Version 16
VisualStudioVersion = 16.6.30114.105
MinimumVisualStudioVersion = 10.0.40219.1
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "src", "src", "{979BB498-A893-42EC-B4D7-CBA461FAFAE9}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "ExProj", "src\ExProj\ExProj.csproj", "{D6CA4D2E-E7FF-48B9-B433-921E9B4BB4E6}"
EndProject
Global
	GlobalSection(SolutionConfigurationPlatforms) = preSolution
		Debug|Any CPU = Debug|Any CPU
		Debug|x64 = Debug|x64
		Debug|x86 = Debug|x86
		Release|Any CPU = Release|Any CPU
		Release|x64 = Release|x64
		Release|x86 = Release|x86
	EndGlobalSection
	GlobalSection(SolutionProperties) = preSolution
		HideSolutionNode = FALSE
	EndGlobalSection
	GlobalSection(ProjectConfigurationPlatforms) = postSolution
		{D6CA4D2E-E7FF-48B9-B433-921E9B4BB4E6}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
		{D6CA4D2E-E7FF-48B9-B433-921E9B4BB4E6}.Debug|Any CPU.Build.0 = Debug|Any CPU
		{D6CA4D2E-E7FF-48B9-B433-921E9B4BB4E6}.Debug|x64.ActiveCfg = Debug|Any CPU
		{D6CA4D2E-E7FF-48B9-B433-921E9B4BB4E6}.Debug|x64.Build.0 = Debug|Any CPU
		{D6CA4D2E-E7FF-48B9-B433-921E9B4BB4E6}.Debug|x86.ActiveCfg = Debug|Any CPU
		{D6CA4D2E-E7FF-48B9-B433-921E9B4BB4E6}.Debug|x86.Build.0 = Debug|Any CPU
		{D6CA4D2E-E7FF-48B9-B433-921E9B4BB4E6}.Release|Any CPU.ActiveCfg = Release|Any CPU
		{D6CA4D2E-E7FF-48B9-B433-921E9B4BB4E6}.Release|Any CPU.Build.0 = Release|Any CPU
		{D6CA4D2E-E7FF-48B9-B433-921E9B4BB4E6}.Release|x64.ActiveCfg = Release|Any CPU
		{D6CA4D2E-E7FF-48B9-B433-921E9B4BB4E6}.Release|x64.Build.0 = Release|Any CPU
		{D6CA4D2E-E7FF-48B9-B433-921E9B4BB4E6}.Release|x86.ActiveCfg = Release|Any CPU
		{D6CA4D2E-E7FF-48B9-B433-921E9B4BB4E6}.Release|x86.Build.0 = Release|Any CPU
	EndGlobalSection
	GlobalSection(NestedProjects) = preSolution
		{D6CA4D2E-E7FF-48B9-B433-921E9B4BB4E6} = {979BB498-A893-42EC-B4D7-CBA461FAFAE9}
	EndGlobalSection
EndGlobal
[keegan@t450s ExSln]$ 

[DavidKarlas]

@keegandent notice .sln file format is Visual Studio file format which was in past Windows only and as far as I know doesn't work with parsing / as path separator, but all tools in ecosystem that read .sln file format on unix systems understand that and handle that correctly.

When it comes to line endings its up to template author to decide which line ending which file will be, we should probably reconsider default for console, library and other SDK provided templates to switch to LF...

Engine itself could provide is ability for template author to specify for which files in template line ending can change and then engine changes line endings based on operating system + add flag for user to override that for example dotnet new console --line-eding=crlf...

But notice that we can't just enable this for all files, because for example some file formats/extension must stay in CRLF format I think .sln is one example of that.

[keegandent]

@DavidKarlas For the record the *.cs files are also CRLF.

Changing the *.sln and *.csproj files to Unix line endings and path separators does not seem to stop them from functioning, at least on Linux and using the CLI, VSCode, and/or MonoDevelop.

[laktak]

But notice that we can't just enable this for all files, because for example some file formats/extension must stay in CRLF format I think .sln is one example of that.

Wouldn't it make more sense to fix the tools instead of keeping legacy baggage around?

[arcaartem]

When it comes to line endings its up to template author to decide which line ending which file will be, we should probably reconsider default for console, library and other SDK provided templates to switch to LF...

Wouldn’t that create issues for Windows users in that case? (i.e. changing default to LF)

Engine itself could provide is ability for template author to specify for which files in template line ending can change and then engine changes line endings based on operating system + add flag for user to override that for example dotnet new console --line-eding=crlf...

Sounds sensible

But notice that we can't just enable this for all files, because for example some file formats/extension must stay in CRLF format I think .sln is one example of that.

Why should they stay in that format?

[DavidKarlas]

Notice that bug is open and we plan to fix this, question is just how

[keegandent]

Well the "poor man's" fix would be to generate the solution or project template as normal, but run the human-readable files through a dos2unix-esque step. This still would not correct the path separators. What I find interesting is that in the standard output for most dotnet CLI commands, the paths use *nix separators.

[laktak]

I don't think that solution would be too bad if you also run it through dotnet-format to get a consistent coding style (though dotnet-format still has some gaps).

[DavidKarlas]

Just wanted to mention we didn't forget about this issue and I found nice example why we can't just blindly convert \n to \r\n and vice-versa.

Lets assume that template has 2 files:

• build.bat(for Windows)
• build.sh(for Unix)

Lets say that user creates this template on Windows, what will end up happening is that build.sh will have \r\n.
Later when someone on Linux downloads this project and tries to use build.sh it will fail with bash: ./build.sh: /bin/bash^M: bad interpreter: No such file or directory. Hence for .sh files we need to disable this functionality, hence we need to introduce new syntax for template authors to have ability to disable this automatic converting for some files in template.

[laktak]

Is it really necessary to keep \r\n at all?

I know that most (all?) tools on Windows will work with \n (this includes the shell and common utilities like Notepad).

[vlada-shubina]

discussed with @baronfel, there are 2 approaches to implement this feature:

• extend SpecialOperationConfig as explained in dotnet new creates files with CRLF line endings on linux #1248 (comment) (3-4 weeks of work)
• Developers enjoy dotnet format UXs, in line with VS UXs #5505 might fix the issue too, but performance heavy.

1st approach is preferred, but need to be planned accordingly as it's a big chunk of work.

[ylluminate]

Are we any closer here? I have to say this is very frustrating on macOS and requires the creation and use of a script that detects and converts files via dos2unix. While I know this is not the case, this "feels" lazy and inconsiderate on Microsoft's part towards non-Windows systems...

[rjhind]

Yes this is something that really does need sorting. Creating a blazer or Maui app on Mac creates a lot of files that you have to cover before committing to git. Really frustrating.

[ZeroAurora]

Additionally, I'd wish to see the templating produces files without BOM.