.NET Standard 2.0 TPDTC with external dependencies example

[dmitry-a-morozov]

I wanted to make https://github.com/demetrixbio/FSharp.Data.Npgsql true .NET Standard 2.0 TPDTC such that consuming projects can be compiled only with .NET Core SDK tools.

I followed
https://github.com/fsprojects/FSharp.TypeProviders.SDK#making-a-net-standard-20-tpdtc
and
https://github.com/fsharp/fslang-design/blob/master/tooling/FST-1003-loading-type-provider-design-time-components.md
but I get an error

error FS3033: The type provider 'FSharp.Data.Npgsql.NpgsqlProviders' reported an error: Could not load file or assembly 'Npgsql, Version=4.0.0.0, Culture=neutral, PublicKeyToken=5d8b90d52f46fda7'. The system cannot find the file specified.

So unless workaround applied consuming projects fail to compile.

After followed @dsyme suggestion
https://twitter.com/mitekm/status/1024451317526233089

It helped with to solve TPDTC dependency loading issue but I got stuck with following error

error FS1108: The type 'Void' is required here and is unavailable. You must add a reference to assembly 'System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e'.

It's not a first time I hit TPDTC dependency management issue while working on type provider that targets .NET Core runtime.
It would be helpful either to extend BasicProvider.DesignTime project to demo how to handle external dependencies or create separate sample type provider to cover that case.

[dmitry-a-morozov]

@dsyme Any updates on how to solve

error FS1108: The type 'Void' is required here and is unavailable. You must add a reference to assembly 'System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e'.

?

[dsyme]

@dmitry-a-morozov Apologies for the slow reply, going through TPSDK issues now.

Do you have a work-in-progress branch that I could use to trial different solutions?

thanks
don

[dmitry-a-morozov]

Hi @dsyme
In demetrixbio/FSharp.Data.Npgsql@e0f32ac
commit I created the simplest repro I can think of.

There is a sample console app
https://github.com/demetrixbio/FSharp.Data.Npgsql/tree/master/tests/TpSdkIssue244
It still requires PostgresSQL instance.
I suggest you one of the following options:

• Install PostgresSQL locally
• Use Azure to create PostgresSQL instance
• Use AWS Free-tier to create PostgresSQL instance

Don't forget to adjust connection string
https://github.com/demetrixbio/FSharp.Data.Npgsql/blob/e0f32ac176f0afac1b22d38f2d07babf3cb46ab2/tests/TpSdkIssue244/Program.fs#L6

If you'll choose default local installation no changes to connection string are necessary.

[dmitry-a-morozov]

@dsyme The sample app uses system schema so no need to create additional database on top of default installation.

[daz10000]

I will lay my reputation on the line and claim postgres can be installed in ~ 5 mins post download. Just pick default options, ignore stack builder and make sure you remember the password you create for the postgres user, and then connection string is something like

User ID=postgres;Password=myPassword;Host=localhost;Port=5432;Database=postgres; Pooling=true;Min Pool Size=0;Max Pool Size=100;Connection Lifetime=0;

[dsyme]

I've find the cause of this:

error FS1108: The type 'Void' is required here and is unavailable. You must add a reference to assembly 'System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e'.

It's a painful problem and will require a painful workaround. More later tonight

[dmitry-a-morozov]

Great effort @dsyme !
I still strongly suggest to expand sample providers to have external dependencies so these test cases can be covered.

[dsyme]

I still strongly suggest to expand sample providers to have external dependencies so these test cases can be covered.

Yes, I'll do this. Could you point me to the code you added for Assembly resolve events? I'd like to make the TPSDK have some version of that built-in

[dmitry-a-morozov]

All I did is
https://github.com/demetrixbio/FSharp.Data.Npgsql/blob/e0f32ac176f0afac1b22d38f2d07babf3cb46ab2/src/DesignTime/TypeProviderAssembly.fs#L32

[dsyme]

Long term fix for painful problem is here: dotnet/fsharp#5621

[dsyme]

Short term painful hack fix for painful problem is here: #259

[dsyme]

@dmitry-a-morozov Ah yes! I forgot we had the RegisterProbingFOlder stuff still in the TPSDK

[dsyme]

@dmitry-a-morozov Assuming #259 goes through ok this should deal with all outstanding issues blocking you I hope.

[dsyme]

This is now addressed, and the template recently added is setup for TPDTCs with dependencies

[piaste]

I'm working on converting the SQLProvider to .NET Standard 2.0, so I'd like to get this right. Sorry for the wall of text.

My current fork(s) are a set of ComboProvider-style projects, each of which is just one .NET Standard 2.0 library each, except they have NuGet dependencies (a common package for shared code, plus each database's driver packages). These dependencies are used both at design-time and at runtime.

Right now, to use e.g. Npgsql as a dependency, I need to copy Npgsql.dll as well as its transitive dependencies into the /lib/netstandard20/ folder of the FSharp.Data.SqlProvider .nupkg file. If I don't do this, it doesn't work at design-time.

But this means that any type provider dependencies are hidden from NuGet. When a user builds a project that references SQLProvider, the Npgsql.dll file included in the SQLProvider NuGet package will be copied to the output folder. If Npgsql also appears as a regular NuGet dependency of the user's project, the .dll from its own NuGet and the one from SQLProvider's will conflict.

So the only way to make it play alongside with NuGet (what if the user adds a more recent patch version of Npgsql?) is to declare Npgsql as a dependency and pin it to the exact version included in the package. And I need to do the same for other transitive dependencies, which can inconvenience the user if one of them is, let's say, JSON.NET.

The alternative is to split each provider into runtime and design-time components only for the purpose of keeping design-time dependencies managed separatedly, even though they're the same as runtime dependencies.

Is the above correct? If it is, is it on the roadmap to make "combo" type providers work with regular NuGet dependencies?

[dsyme]

I believe if there are dependencies then you should split.

[piaste]

With the preview dotnet SDK for 3.0 the FscToolPath / FscToolExe path don't seem to be recognized correctly:

  <PropertyGroup Condition="'$(IsWindows)' == 'true' AND Exists('C:\Program Files (x86)\Microsoft SDKs\F#\10.1\Framework\v4.0\fsc.exe')">
    <FscToolPath>C:\Program Files (x86)\Microsoft SDKs\F#\10.1\Framework\v4.0</FscToolPath>
    <FscToolExe>fsc.exe</FscToolExe>
  </PropertyGroup>

 error FS0226: The file extension of 'C:\Program Files (x86)\dotnet\sdk\3.0.100-preview-010154\FSharp\fsc.exe' is not recognized. Source files must have extension .fs, .fsi, .fsx, .fsscript, .ml or .mli

Note that the path in the error message is that of the dotnet SDK, rather than the one specified in FscToolPath.

As a test, I renamed the fsc.exe file in the framework folder to fsc2.exe , and changed the .fsproj accordingly, but the error remained the same.

[larjo]

There is the same problem with the latest released version as well, v2.2.203.

My workaround is to replace

    <FscToolPath>C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\Common7\IDE\CommonExtensions\Microsoft\FSharp</FscToolPath>
    <FscToolExe>fsc.exe</FscToolExe>

with

    <FscToolPath>$(MSBuildThisFileDirectory)</FscToolPath>
    <FscToolExe>fscfix.bat</FscToolExe>

where fscfix.bat is

fsc %2

That is it ignores the first argument and calls fsc.exe (which I have in my path) with the file that is going to be compiled.

However this fix isn't needed when building in visual studio 2019.

Edit:

This batch file works both with dotnet build and from visual studio:

for %%a in (%*) do set last=%%a
fsc %last%

[isaacabraham]

@larjo I'm really confused - the Azure Storage TP relied on the compiler tools nuget package and promptly broke when I installed VS2019, but I have since tried removing the Tools nuget package completely and the package appears to suddenly now work out of the box. I have no idea why the dependency is no longer required.

[larjo]

@isaacabraham Interesting. So now, your type provider compiles with dotnet build and no "fsc.props-workaround" is needed?

SqlProvider e.g. still needs the external fsc to compile though.

[drk-mtr]

@larjo Thanks very much for posting your workaround, this was what eventually got it working for me (SqlProvider with MsSql).

I naively thought Type Providers might save me time, instead I spend my life just trying to get them working hehe.

[theimowski]

thanks @larjo for the workaround

Just to add for reference so someone might find it useful - my case is compiling project with FSharp.Data to net40 using .NET SDK 3.0 preview and I'm using the FSharp.Compiler.Tools NuGet instead of preinstalled F# SDK

I had to modify the fscfix.bat to point to the fsc.exe path relative to my fsproj:

..\..\packages\tools\FSharp.Compiler.Tools\tools\fsc.exe %2

(note I'm using Paket tools group name for the package, hence the additional directory there)