[Obsolete] How to enable type providers with new-style .NET SDK project files, dotnet build, .NET Standard and .NET Core programming

[dsyme]

This issue documents how to enable the use of type providers when using a .NET SDK 2.0 "new style" project file.

This also allows the use of cross-targeting type providers as part of .NET Core 2.0 app programming or .NET Standard 2.0 library programming.

If you don't follow one of the workarounds below you may get a silent failure or RunFsc.cmd failed from the F# compiler when executing dotnet build or "StackOverflowException", if your project references a type provider.

What you need to do

1. Install an F# Compiler that runs on .NET Framework or Mono, see Linux, Windows, OSX. It is likely you already have one installed.

2. Add the file fsc.props to the root of your project and import it into each of your project files that reference a type provider using something along the lines of: <Import Project="fsc.props" />,

The F# compiler will then run using .NET Framework and/or Mono.

Alternative Workaround

On Windows:

• Install Visual Studio 2017 15.3 or later
• Use "msbuild.exe" from a Visual Studio 2017 command line prompt to build the project instead of "dotnet build"

On Linux/OSX/Xamarin/Mono:

• Install Mono 5.0 or later
• Use "msbuild" to build the project instead of "dotnet build"

Explanation: when you compile your project using msbuild.exe the F# support in the .NET SDK will assume .NET Framework execution of the F# compiler is required. This workaround is also compatible with using an fsc.props file as mentioned above.

Examples where the problem occurs

Example 1 (building a .NET 4.6.1 application that uses FSharp.Data)

dotnet new console -o -lang F# --target-framework-override net461 app3
cd app3
dotnet add package FSharp.Data
dotnet restore
dotnet build

Example 2 (building a .NET Core 2.0 application that uses FSharp.Data)

dotnet new console -o -lang F# app4
cd app4
dotnet add package FSharp.Data
dotnet restore
dotnet build

Expected behaviour is compilation success, or at least nice error point to workaround. Actual behavior is a silent compilation failure, and dotnet build /v:d shows l Process is terminating due to StackOverflowException. Done executing task "Fsc" -- FAILED.

Explanation

As explained in this issue, type providers are not yet supported when the F# compiler runs using .NET Core. By default, the .NET SDK tooling runs the F# compiler using .NET Core.

The workaround is to have the .NET SDK tooling use an F# Compiler running on the .NET Framework or Mono, regardless of which version of .NET you're targeting. By doing this, the type provider is hosted and executed using the .NET Framework. This technique applies even if you are doing .NET Core programming (for cross-targeting type providers), or building a .NET Standard 2.0 library. The F# Compiler cross-generates code for the correct target regardless of how it executes.

Although this workaround may seem "wrong" if targeting .NET Core, it has advantages:

• the use of .NET Framework/Mono is only at compile-time
• the technique stress-tests the cross-targeting capabilities of the type provider
• the technique unblocks TP authors to test and adjust TPs to be cross-targeting

In the future, the plan of record is to allow .NET Standard 2.0 type providers to be used when the F# compiler is running on .NET Core. This is an interim workaround for those who wish to unblock the testing and use of type providers for .NET SDK project files and .NET Core/.NET Standard 2.0 programming.

[dsyme]

Note, I think we can have a better setting for FscToolPath which works on both Windows and Mono, though it needs a conditional to determine the right location

[larjo]

Thanks @dsyme!

The FscToolPath/FscToolExe workaround seems to work just fine even with SDK 1.0.4.

[dsyme]

@cartermp @KevinRansom As I've now documented in the issue description, it is possible to use the above technique to enable type providers for .NET Core programming, as long as the type providers have been written as a cross targeting type provider. The main cross-targeting type provider is FSharp.Data.

Here are the steps I used to verify that this can be used with .NET Core 2.0 programming:

• dotnet new console -o -lang F# tpapp2
• cd tpapp2
• dotnet add package FSharp.Data
• dotnet restore
• edit the .fs to contain

open System
open FSharp.Data

type NugetStats = HtmlProvider<"https://www.nuget.org/packages/FSharp.Data">

let rawStats = NugetStats().Tables.``Version History``

rawStats.Rows |> Seq.iter (printfn "row = %A")

• edit the tpapp2.fsproj to contain

    <FscToolPath>C:\Program Files (x86)\Microsoft SDKs\F#\4.1\Framework\v4.0</FscToolPath>
    <FscToolExe>fsc.exe</FscToolExe>

• dotnet build
• dotnet run

output:

row = ("F# Data 2.3.3 (this version)", 22954M, 10/04/2017 00:00:00)
...
row = ("FSharp.Data 1.0.2", 234M, 14/12/2012 00:00:00)
row = ("FSharp.Data 1.0.1", 187M, 14/12/2012 00:00:00)
row = ("FSharp.Data 1.0.0", 469M, 13/12/2012 00:00:00)

🎉🎉🎉🎉 we have used type providers for .NET Core programming for the first time :)

Notes:

• When using VSCode/Ionide you also get full intellisense:

• The above technique also works for .NET Standard 2.0 programming (dotnet new lib -lang F# tplib1). However at the time of writing you need an F# compiler that includes this fix, e.g. use Visual Studio 2017 preview 3, or use an F# compiler built from source. I tested with <FscToolPath>C:\GitHub\dsyme\visualfsharp\release\net40\bin</FscToolPath>

• You get some warnings because FSharp.Data for .NET 4.x is being used. However it should be possible to produce a fully compliant .NET Standard 2.0 version of FSharp.Data, that is TBD, and until then the .NET Framework version is not that likely to cause major problems since the surface area of FSharp.Dataa is relatively low

[buvinghausen]

@dsyme you are the freaking man! If I'm ever at a conference you're speaking at dinner one night is on me...

So I journeyed into the matrix to get my application running inside a Docker container on Linux despite it's heavy JSON Type Provider usage due to your workaround and I'm happy to report it works like a champion!

Pros:
This workaround is elegant, nice, and awesome
Runs in Docker on Linux!!!!!
Only developer IDEs and build servers need Windows which is perfectly livable until type providers compile on Core
I can still write all my build scripts for the build servers to be dotnet cli compliant

Cons:
There were 3 things that came up and I've attached a screenshot for all of them to see if I'm doing something wrong..

1. FSC was complaining about netstandard2.0 but it works when I put in netcoreapp2.0
   

2. NuGet shows no packages installed for F#
   

3. My Intellisense is broken but I'm pretty certain this probably has something to do with item 2
   

The only other con was dotnet build, run, restore, publish all throw errors when I include the docker-compose.dcproj file in the .sln file. I have read that is because the SDK hasn't been open sourced which is a requirement to be included in the tool chain. This is not a concern as I can simply script the commands to all run against the top level project file rather than the solution.

I have a feeling my cons are entirely a PEBCAK and I've thought up workarounds in the event they are not. I'm still way happier about actually being able to build out my services against dockerswarm on Linux and still being able to use the Type Providers.

Regards,
Brian

[buvinghausen]

The PEBCAK is that I couldn't read.... vscode/ionide I will leave the items up here just in case we want the VS team to see they are very close.

[cartermp]

Not sure about (2), but (3) is due to an issue in the language service. #3260 is tracking that issue, and @KevinRansom is currently working on it. This is wonderful to hear, though, @buvinghausen! Happy to hear that it's working for you in Docker/Linux.

[enricosada]

Only developer IDEs and build servers need Windows which is perfectly livable until type providers compile on Core

Atm will work also on non windows, using mono. two big ways:

1. use mono on dev on non win. same workaround

2. use docker multi stage setup

   • build a mono docker image for build, build there => get output
   • run run the build output in another container with just .net core

pro of 2 is that with multi stage, the image size contains only netcore (use official dotnet core docker image) and is slimmer

maybe i should gist the dockerfile i use for that. is a good setup, also for ci who are docker based (we use jenkins).

[enricosada]

@dsyme as a note, dotnet new lib -lang F# works the same of classlib, and is less wierd 😄

[buvinghausen]

@cartermp thanks for the clarification I'm actually working on getting VS Code launch & debug into the Docker container like I can from VS 2017 then I may just cut the cord on VS full I've been looking for an excuse to do that as I have been using VS Code exclusively for TypeScript development.

@enricosada Those are some excellent points but for now I'm totally happy with simply making my CI server run on Windows and letting the FE developers run everything in Node but talking to our integration services on the back end. That should give me enough runway to let all the frameworks and tooling catch up such that I can then tell them they have to install .NET Core & OmniSharp and learn the dotnet cli. As an aside I think today is the day that I'm going to see how feasible it is to run the React Redux template in Docker. I know when you create the project they have the docker option disabled. Should be fun.

I really appreciate all the work you guys are putting into this platform I cannot stress that enough it really is amazing to think what the future holds.

[orient-man]

Thx for the workaround but I encountered another issue:

error FS3033: The type provider 'FSharp.Data.TypeProviders.DesignTime.DataProviders' reported an error: The .NET SDK 4.0 or 4.5 tools could not be found

Source code tells the truth: SDK paths are fixed. On my relatively new machine (or on VSTS agent) there're no such registry keys. I added new ones (which existed on my machine):

@"Software\Microsoft\Microsoft SDKs\NETFXSDK\4.6.1\WinSDK-NetFx40Tools"
@"Software\Microsoft\Microsoft SDKs\NETFXSDK\4.6.2\WinSDK-NetFx40Tools"

and now it works. Are you interested in pull request for that change?

[ctaggart]

I so confused on what I need to do to get the OpenAPITypeProvider to work. It is build for netstandard2.0. fsprojects/OpenAPITypeProvider#6

[dsyme]

Recording a link to my reply here: fsprojects/OpenAPITypeProvider#6 (comment)

[SpiegelSoft]

I'm trying to use FSharp.Configuration in my project. On my TeamCity build environment (Linux), I have installed the fsharp compiler and imported fsc.props, but the build breaks with the error message

"fsharpc" exited with code 1.

For now, I'm not sure where to go, as it's not clear how to get a detailed error description from fsharpc.

[larjo]

The fsc.props workaround stopped working in version 2.2.203 of the SDK.

I posted a workaround in fsprojects/FSharp.TypeProviders.SDK/issues/244

[buvinghausen]

Why would you still be using the fsc.props workaround? The 3.0 beta (and all subsequent 3.x releases) has been working without the workaround for over a year now...

[Thorium]

After the latest caching improvements I started to get this error FS3033(0,0): The design-time type 'System.String' utilized by a type provider was not found in the target reference assembly set '[ ...(list of assemblies)... ]'. You may be referencing a profile which contains fewer types than those needed by the type provider you are using. also in the .NET Framework full.

ProvidedTypes.fs commit 18459e8b23b44b389dc8f2b8fe444fedf2b47a8c did work, but after commit b08fe13625b2ffc08529cb9350db8f86c6b3c537 there are problems.

[patham9]

It's fixed in 3.0? With dotnet-sdk-3.0.100-preview5-011568-linux-x64.tar.gz I still get the same error, after invoking "dotnet build" on a F# project:
error FS3053 : The type provider 'FSharp.Configuration.ConfigTypeProvider+FSharpConfigurationProvider' reported an error : The type provider constructor has thrown an exception: Exception has been thrown by the target of an invocation.
Will it be fixed properly, if so when is a fix planned to appear approximately? Can a out-of-the-box solution that does not demand manual workarounds with path hacks be expected in the future?

[cartermp]

@patham9 it’s highly unlikely that this issue is the right place to report that error, as the mechanism to allow type providers to work on .NET Core has been shipped and stable for quite some time. I suspect your issue lies in the library, not this type provider mechanism.

[baronfel]

I agree with @cartermp. My initial porting of the FSharp.Configuration TP was incomplete and it hasn't been fully updated with a .netstandard version, which would be required for seamless usage. There's an issue on the repo for this I believe, but someone needs to do/finish the work. It's on my list to do, but like many others I have many competing priorities.

[jfrank-summit]

@Thorium Did you ever figure out a workaround or fix? Having this same problem...

After the latest caching improvements I started to get this error FS3033(0,0): The design-time type 'System.String' utilized by a type provider was not found in the target reference assembly set '[ ...(list of assemblies)... ]'. You may be referencing a profile which contains fewer types than those needed by the type provider you are using. also in the .NET Framework full.

[Thorium]

Yes, I figured out what happened.
FSharp.TypeProviders.SDK has the file ProvidedTypes.fs which is loading types from netstandard.dll which contains the base types of .NET Standard.
If you follow the tutorial, LemonadeProvider, and open the file
src\LemonadeProvider.DesignTime\LemonadeProvider.DesignTime.fsproj you will notice the dependency:

    <None Include="..\..\packages\NETStandard.Library.NETFramework\build\net461\lib\netstandard.dll" Condition="'$(TargetFramework)' == 'net45'">
        <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
    </None>

So the typeprovider loads the netstandard.dll from this NuGet package:
https://www.nuget.org/packages/NETStandard.Library.NETFramework/2.0.0-preview2-25405-01

But the netstandard.dll there is an old version, and if you a later use netstandard.dll in your machine (you have the latest .NET Core installed) there will be version conflict between the loaded netstandard.dll and the used netstandard.dll. That might also lead the type-provider to reflection-loading-loop between the two dlls, causing stackoverflow, or just not to find the core types. Using only the old-one works.

I went the simplest fix which was uninstall VS2019 and take VS2017 back, and everything worked correctly.
:-P

Edit: Also, the ProvidedTypes.fs I'm using is the commit 18459e8b23b44b389dc8f2b8fe444fedf2b47a8c