F# Status on CoreRT and UWP

[charlesroddie]

[2018-12-06 update] F# UWP apps can be released on the Windows Store.
#6055 (comment)

---

Recent testing has assessed the compatibility of F# with CoreRT and .NET Native. Only minor issues have been found, which are easily worked around (see specific issues below).

CoreRT issues can always be tracked and worked on here, and .NET Native issues can now also be tracked now that users can compile using it (See gatekeeper.xml section).

This post is a documentation of the status of F# and CoreRT as requested by @dsyme. This post will be kept up to date. One purpose is to inform F# developers using CoreRT and .NET Native. The second purpose is to inform the request for allowing F# usage on the Windows Store, a 1 line change that may require approval.

Testing

CoreRT and .Net Native went through basic tests (the F# cheatsheet), and specific tests (tail calls and highly nested generics as recommended by @MichalStrehovsky, and some of the F# test suite). .Net Native was further tested in a production app using typical F# structures (a symbolic algebra engine using large discriminated union trees, standard usage of record types, reactive UI using Xamarin.Forms, SQLite). CoreRT is used by a number of F# users already. (@ChrisPritchard has written a number of games without encountering any issues.)

Specific Issues

Issue	UWP (.Net Native)	CoreRT	Workaround
sprintf	Fails	Can work for some types using rd.xml	Use concrete string methods like String.Format.
.ToString() on discriminated unions and record types	Fails	Fails	Override .ToString() on these types.
LINQ expressions	Fails	Fails	Use an alternative to LINQ (e.g. SQL) or use LINQ from C#.
.tail instructions	Ignored	OK	Avoid a style that is dependent on optimization of complex tail calls.
F# Events	OK	Fails	Use standard .Net Events on CoreRT.
Highly nested generics	OK	Fails	(Not yet encountered in realistic code.)
char enums	Fails	?	Use int enums

Of the issues affecting UWP, lack of support for .tail does not break typical F# code (as Fable and historically mono AOT prove), and the other two issues have simple workarounds.

See https://github.com/FoggyFinder/FSharpCoreRtTest/blob/master/README.md#description-of-current-issues for more detail, and the MainIssues branch for relevant code.

.Net Native: gatekeeper.xml blocks F#

The remaining problem is a non-technical one. F# usage of .Net Native is blocked by gatekeeper.xml.

<FSharpRule on="true">
<ErrorLevel>error</ErrorLevel>
...
</FSharpRule>

This can be disabled on user systems to enable compilation with .Net Native, allowing creation of working UWP apps. However these apps cannot be successfully uploaded to the store since they will fail the F# test when compiling on Microsoft servers. We therefore need official editing of GatekeeperConfig.xml, either by turning it off or by replacing error with warning.

Reasons to allow F# UWP applications on the Windows Store:

1. Already promised. 2015-10 @crutkas: "We are committed to bringing F# support to .NET Native."

2. No technical work required. There are no technical issues currently blocking F# usage.

3. Large user demand. 1500+ votes for Add F# support for .NET Native and 2000+ votes for F# support in .Net native for UWP when those were active. This thread is a continuation of F# Support in .NET Native and UWP and .Net Native F# Support; the latter contains use cases.

[morganbr]

@charlesroddie, thank you for investigating this in depth. Can you please tell us more about what goes wrong when you try to use sprintf, ToString and Quotations on .NET Native (UWP)?

[charlesroddie]

sprintf gives a System.IndexOutOfRangeException on UWP, as does using ToString on DUs and record types. On CoreRT there is an NRE. @zpodlovics identified the sprintf issue as coming from dynamic code generation:

let mi = typeof<ObjectPrinter>.GetMethod("GenericToString", NonPublicStatics)
let mi = mi.MakeGenericMethod(ty)
mi.Invoke(null, [| box spec |])

[charlesroddie]

There is no general issue with F# quotations; the only issue we have found is with LINQ. The function causing the problem is Microsoft.FSharp.Linq.RuntimeHelpers.LeafExpressionConverter.QuotationToExpression. This gives a fail fast error.

E.g. if db is an sqlite connection, db.QueryAsync<T>() .Table<T>() .OrderByDescending(fun x -> x.Timestamp) fails. A workaround is to use SQL: db.QueryAsync<T>("select * from T order by Timestamp desc").

LINQ requires care even from C# and inherently uses a lot of reflection I suppose.

F# query expressions seem to work OK.

[oldjuanito]

Has there been any progress on this (official editing of GatekeeperConfig.xml)?

Thank you all for all your hard work.

[MichalStrehovsky]

We are looking into removing the F# blocking in the Store compilation machines for the next release of .NET Native for UWP apps.

F# will remain officially unsupported, but we'll not prevent people who understand the implications (and who did enough testing to make sure it works okay for their use case) from uploading to the Store. People will still need to hack their local Visual Studio installation to remove the blocking locally because we don't believe the experience of using F# with the level of support the .NET Native compiler has is anywhere near pleasant.

The focus of the .NET Native team has been on completing ARM64 support in the next release. The work to lift the remaining technical limitations to fully support F# is unfortunately non-trivial.

For an example of the technical limitations on top of the issues found by the testing in the top post, you can try this:

1. Create a blank UWP app project in Visual Studio
2. Add a new F# .NET Standard 2.0 library project
3. Put this in the F# project:

namespace FSharpLibrary

module Fib =
    let rec fib = function
        | n when n=0I -> 0I
        | n when n=1I -> 1I
        | n -> fib(n - 1I) + fib(n - 2I)

4. Add a reference to the F# project from the UWP app and add a call to the method in F# from it.
5. Build a Release version of the app

The .NET Native compiler will emit a couple warnings about unsupported LDTOKEN instructions to stdout (there's several places in FSharp.Core.dll where the IL is doing LDTOKEN of an uninstantiated generic method, such as in static System.Void <StartupCode$FSharp-Core>.$Query..cctor() - this is not currently supported by the native code generator backend we use. Then the compiler will crash with an internal error.

[charlesroddie]

Great news! Allowed but not officially supported is the sensible decision.

For an example of the technical limitations on top of the issues found by the testing in the top post...

Thanks. We will check this and keep the issue list updated.

[dsyme]

The .NET Native compiler will emit a couple warnings about unsupported LDTOKEN instructions to stdout (there's several places in FSharp.Core.dll where the IL is doing LDTOKEN of an uninstantiated generic method, such as in static System.Void <StartupCode$FSharp-Core>.$Query..cctor() - this is not currently supported by the native code generator backend we use. Then the compiler will crash with an internal error.

We can likely fix this easily in a future FSharp.Core (e.g. change to emit ldtoken of an instantiated generic method and then call GetGenericMethodDefinition() )

[FoggyFinder]

For an example of the technical limitations on top of the issues found by the testing in the top post, you can try this:

hm, I can't reproduce it:

Did I do something wrong?

[MichalStrehovsky]

We can likely fix this easily in a future FSharp.Core (e.g. change to emit ldtoken of an instantiated generic method and then call GetGenericMethodDefinition() )

That would help! We've had an issue open to make it work for a long time. It never met the bar because we thought only IL obfuscators emit that kind of IL (and we have hack in place that checks if the subsequent instruction is a simple pop, which fixes the obfuscator...).

hm, I can't reproduce it

What version of the .NET Native compiler did you use? I used 2.1.8. Also, do did you modify the RD.XML in the empty UWP app template? The problem goes away after removing the *Application* line because we no longer need to compile all of FSharp.Core, only a subset that's statically needed.

[FoggyFinder]

What version of the .NET Native compiler did you use? I used 2.1.8.

The same.

Also, do did you modify the RD.XML in the empty UWP app template?

No, I didn't. My rd (without comments and whitespaces):

<Directives xmlns="http://schemas.microsoft.com/netfx/2013/01/metadata">
  <Application>
    <Assembly Name="*Application*" Dynamic="Required All" />
  </Application>
</Directives>

Version of FSharp.Core: 4.5.2

[MichalStrehovsky]

Here's my project: FSharpApp.zip

One thing I noticed is that the compiler only crashes when building x86. It compiles fine for x64.

[FoggyFinder]

One thing I noticed is that the compiler only crashes when building x86. It compiles fine for x64.

yep, exactly, I had x64

[JaggerJo]

Are we stuck on "Allowed but not officially supported" for now or are there any plans towards "Officially supported" ?

[charlesroddie]

@JaggerJo: yes we are, as fully described in @MichalStrehovsky's post.

We can confirm that .Net Native compilation works now on Store compilation machines, and we have released our beta F# app on the store (link).

[Happypig375]

Featured on r/fsharp!
https://www.reddit.com/r/fsharp/comments/acj7dn/f_winrtuwp_apps_on_net_native_are_now_releasable/?utm_source=reddit-android

[charlesroddie]

Direct access to UWP UI classes from F# depends on either

1. Creating an F# template for a UWP class library analogous to the current C#/VB templates, or
2. Exposing Windows.Foundation.UniversalApiContract as NuGet package, which would allow creating a plain net core/standard class library and adding this package.

Until then F# can only be used 1. in parts of a solution that don't have to reference controls directly (i.e. models, viewmodels), or 2. indirectly via Xamarin.Forms.

[pauldorehill]

@charlesroddie are there any workarounds to getting access to the UWP UIs (similar to the gatekeeper hack)? I'm using Fabulous and writing some custom UWP renderers, but as it stands I need to use a C# UWP Library for the views.

[charlesroddie]

@pauldorehill Yes using C# for the custom renderers is the right approach and will be for the next year or so.

WinUI 3.0, in whatever form that takes, is likely to have better F# support.

[TheFo2sh]

ok for uwp , now i can compile the x64 and submit it to the store which is good but what about the Arm and arm64 ? i hope that MS can find a solution for it as both F# and UWP are part of the .NET ecosystem.

[charlesroddie]

@TheFo2sh Are you having a specific problem on ARM or are you just conjecturing that there may be a problem?

it

It's possible that you are making a mistake here. Compiled native code is not submitted to the store.

[knocte]

This doesn't work for me. Trying to publish a C# app (which uses a lot of F# libs) fails to compile, with this error:

Severity	Code	Description	Project	File	Line	Suppression State
Error		ILT0005: 'C:\Users\knoct\.nuget\packages\microsoft.net.native.compiler\2.0.2\tools\x64\ilc\ilc.exe --gatekeeper @"C:\Users\knoct\Source\Repos\geewalletFRONTEND\src\GWallet.Frontend.XF.UWP\obj\x64\Release\ilc\intermediate\gkargs.rsp"' returned exit code 1	GWallet.Frontend.XF.UWP			

When googling this error, one can only find this stackoverflow answer (https://stackoverflow.com/a/49142932/544947) which just recommends to disable the Compile with .NET Native toolchain checkbox, but if you do this and try to submit to the appStore, you get the error:

Package acceptance validation error: This package wasn't built for submission to the Microsoft Store. Make sure you're uploading a Release build with the .NET Native tool chain enabled.

[MichalStrehovsky]

@knocte You have to hack your local installation to remove the blocking locally, as described here: #5780 (comment). We don't have a nicer way to do that because this makes it very clear that it's getting into unsupported territories.

See my post above:

F# will remain officially unsupported, but we'll not prevent people who understand the implications (and who did enough testing to make sure it works okay for their use case) from uploading to the Store. People will still need to hack their local Visual Studio installation to remove the blocking locally because we don't believe the experience of using F# with the level of support the .NET Native compiler has is anywhere near pleasant.

[TheFo2sh]

@charlesroddie , sure you can submit a package with f# to the store after turning off the .net native compiler guard , but Unfortunately this works only for the x64 bit. so now after Microsoft released the arm based surface pro x i need a solution to create an armx64 package that can be submitted to the store while keep using f#.

[knocte]

@MichalStrehovsky I appreciate your comment. However I have tested what is advised in that issue/comment, and while some compilation errors go away, very basic F# functionality such as sprintf and printf simply throws NullReferenceExceptions. Where can I submit this bug?

[charlesroddie]

@TheFo2sh If you'd like help I'd suggest posting a precise description of the successful process with just C# and at what point an identical process with F# code fails.

@knocte The current way round is to remove sprintf and printf. If you want to use sprintf in reflection-free/unfriendly toolchains, or anywhere where performance is important, then the best places to raise an issue are https://github.com/dotnet/fsharp or https://github.com/fsharp/fslang-suggestions to discuss reflection-free string formatting. Or add to dotnet/fsharp#4954 .

[knocte]

@charlesroddie thanks, I didn't know it was because of reflection. I have a workaround in mind, I will test it and will report about my findings after that, in case I see more limitations.

[AngelMunoz]

Hey there not sure if this is the right issue (if not please point me to the correct one) but I have an issue trying to use Microsoft.Windows.SDK.Contracts that as far as I know, these are WinRT API's
here's a reproduction repository
https://github.com/AngelMunoz/ContractsRepo