F# support #2057
Comments
|
Related dotnet/fsharp#1096 |
|
First blocker: There's no support for MSBuild based F# projects with the dotnet cli. Is there an alternative to https://github.com/dotnet/corert/blob/master/Documentation/how-to-build-and-run-ilcompiler-in-console-shell-prompt.md#compiling-source-to-native-code-using-the-ilcompiler-you-built for building .NET Core apps with CoreRT? |
|
@tsolarin Thanks for being interested in F# on CoreRT! You can build the F# app using the current non-msbuild based .NET Core tooling, and then follow steps in https://github.com/dotnet/corert/blob/master/Documentation/how-to-build-and-run-ilcompiler-in-visual-studio-2015.md with modified c:\corert\bin\obj\Windows_NT.x64.Debug\ryujit.rsp:
|
|
@jkotas I'm on OSX so unfortunately I couldn't use the method above. However, thanks to https://github.com/dotnet/netcorecli-fsc/tree/master/examples/preview3/console and a whole lot of tweaking on my end I was able to get MSBuild to successfully build F# on CoreCLR on OSX. CoreRTI went ahead to make a trial build on CoreRT. Below are the results Using RyuJITOn first build attempt (possibly due to the lack of existing output files) Subsequent build attempts produced a Using CPP Code GeneratorThe CppCodeGen approach achieves a successful build of an F# console project on CoreRT. Below is the build process output: However, when I try to run the resulting native binary instead of the expected Environment Details
|
This error is caused by FSharp.Core.dll missing on ilc command line. The project dependencies are not picked up automatically yet, Could you please manually specify FSharp.Core.dll for now - adding the following lines to your .fsproj file should do the trick: Also, do not worry about CppCodeGen for now. It is behind RyuJIT codegen, so you would only find more problems with it; not less. |
|
Thanks for that @jkotas I added that and tried another RyuJIT based build. Below is the build output: |
This error is caused by CoreRT version of System.Diagnostics.Tracing not being ported to Unix yet. To workaround it for now, add a dummy throwing stub for EventWriteTransfer to https://github.com/dotnet/corert/blob/master/src/Native/Bootstrap/platform.unix.cpp |
|
Awesome! Followed your directive and it built successfully. However, when I try to run the resulting native binary I get the following error: |
|
/cc @jkotas |
|
As the message says, this means you are hitting unimplemented part of the system. The first step is to find out where it is coming from exactly: Start your program under I suspect that it will be coming from Verify that it works, e.g. by running a simple program that does |
|
@tsolarin Thank you for your help! |
|
PR open. Had to port the C++ impl though, I couldn't achieve a successful build without it. |
* add a dummy throwing stub for EventWriteTransfer (#2057) This enables F# apps to be built against CoreRT until System.Diagnostics.Tracing is ported to Unix * add implementation for Environment.ProcessorCount (#2057) - This fixes "unimplemented runtime method" runtime error for GetNativeSystemInfo method. - Achieves successful run of a simple "Hello Wolrd!" F# console app * Remove GetNativeSystemInfo unix stub (#2057) The GetNativeSystemInfo method is no longer called on unix so can be removed
|
So we got |
|
Try more complex tests - I believe F# repo has a bunch of them. |
|
Hey @jkotas it's me again 😀. I've been trying to build more complex F# programs with CoreRT and I've run into another error. Whenever I try to print formatted text (e.g. My question: Do I help with porting required reflection apis from CoreCLR or would my time be better spent discovering/fixing other F# features that should work with the current state of CoreRT but don't? |
Where is this runtime internal error coming from? It looks like a more fundamental problem than just a missing reflection.
The simple @MichalStrehovsky Anything you would add? |
Yes, the rule is that if the method is statically reachable in the callgraph, and the type that owns the method was allocated (with a You can force the type and all of it's methods to be generated through RD.XML if the above conditions are not naturally met. We only support a limited subset of the RD.XML format. Callstack at the time of failfast would help. |
|
Hi @jkotas thanks for pointing out the rdxml workaround I'll give it a shot. @MichalStrehovsky here's the callstack at the time of failfast: * thread #1: tid = 0x42c89, 0x000000010018edca FsCore`FSharp_Core_Microsoft_FSharp_Core_PrintfImpl__getValueConverter + 1394, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
* frame #0: 0x000000010018edca FsCore`FSharp_Core_Microsoft_FSharp_Core_PrintfImpl__getValueConverter + 1394
frame #1: 0x000000010011bdc8 FsCore`System_Private_CoreLib_System_Diagnostics_Debug__Assert + 12
frame #2: 0x00000001001141f4 FsCore`System_Private_CoreLib_System_Array__GetRawSzArrayData + 28
frame #3: 0x0000000100113082 FsCore`RhpStelemRef + 250
frame #4: 0x000000010018edbd FsCore`FSharp_Core_Microsoft_FSharp_Core_PrintfImpl__getValueConverter + 1381
frame #5: 0x0000000100186c8c FsCore`FSharp_Core_Microsoft_FSharp_Core_PrintfImpl_PrintfBuilder_3<System_IO_System_IO_TextWriter__FSharp_Core_Microsoft_FSharp_Core_Unit__FSharp_Core_Microsoft_FSharp_Core_Unit>__FSharp_Core_Microsoft_FSharp_Core_PrintfImpl_PrintfBuilder_3__parseFromFormatSpecifier<> + 1152
frame #6: 0x0000000100162b9e FsCore`FSharp_Core_Microsoft_FSharp_Core_PrintfImpl_PrintfBuilder_3<System_IO_System_IO_TextWriter__FSharp_Core_Microsoft_FSharp_Core_Unit__FSharp_Core_Microsoft_FSharp_Core_Unit>__FSharp_Core_Microsoft_FSharp_Core_PrintfImpl_PrintfBuilder_3__parseFormatString<> + 118
frame #7: 0x000000010015b265 FsCore`FSharp_Core_Microsoft_FSharp_Core_PrintfImpl_PrintfBuilder_3<System_IO_System_IO_TextWriter__FSharp_Core_Microsoft_FSharp_Core_Unit__FSharp_Core_Microsoft_FSharp_Core_Unit>__FSharp_Core_Microsoft_FSharp_Core_PrintfImpl_PrintfBuilder_3__Build<FSharp_Core_Microsoft_FSharp_Core_FSharpFunc_2<FSharp_Core_Microsoft_FSharp_Core_FSharpOption_1<System_Private_CoreLib_System_Int32>__FSharp_Core_Microsoft_FSharp_Core_Unit>> + 89
frame #8: 0x000000010014f38f FsCore`FSharp_Core_Microsoft_FSharp_Core_PrintfImpl__cctor_1262_175<FSharp_Core_Microsoft_FSharp_Core_FSharpFunc_2<FSharp_Core_Microsoft_FSharp_Core_FSharpOption_1<System_Private_CoreLib_System_Int32>__FSharp_Core_Microsoft_FSharp_Core_Unit>__System_IO_System_IO_TextWriter__FSharp_Core_Microsoft_FSharp_Core_Unit__FSharp_Core_Microsoft_FSharp_Core_Unit>__FSharp_Core_Microsoft_FSharp_Core_PrintfImpl__cctor_1262_175__Invoke<> + 39
frame #9: 0x0000000100141f58 FsCore`System_Collections_Concurrent_System_Collections_Concurrent_ConcurrentDictionary_2<System_Private_CoreLib_System_String__System_Private_CoreLib_System_Tuple_2<FSharp_Core_Microsoft_FSharp_Core_FSharpFunc_2<FSharp_Core_Microsoft_FSharp_Core_FSharpFunc_2<FSharp_Core_Microsoft_FSharp_Core_Unit__FSharp_Core_Microsoft_FSharp_Core_PrintfImpl_PrintfEnv_3<System_IO_System_IO_TextWriter__FSharp_Core_Microsoft_FSharp_Core_Unit__FSharp_Core_Microsoft_FSharp_Core_Unit>>__FSharp_Core_Microsoft_FSharp_Core_FSharpFunc_2<FSharp_Core_Microsoft_FSharp_Core_FSharpOption_1<System_Private_CoreLib_System_Int32>__FSharp_Core_Microsoft_FSharp_Core_Unit>>__System_Private_CoreLib_System_Int32>>__System_Collections_Concurrent_System_Collections_Concurrent_ConcurrentDictionary_2__GetOrAdd<> + 100
frame #10: 0x0000000100126b2c FsCore`FSharp_Core_Microsoft_FSharp_Core_PrintfImpl_Cache_4<FSharp_Core_Microsoft_FSharp_Core_FSharpFunc_2<FSharp_Core_Microsoft_FSharp_Core_FSharpOption_1<System_Private_CoreLib_System_Int32>__FSharp_Core_Microsoft_FSharp_Core_Unit>__System_IO_System_IO_TextWriter__FSharp_Core_Microsoft_FSharp_Core_Unit__FSharp_Core_Microsoft_FSharp_Core_Unit>__FSharp_Core_Microsoft_FSharp_Core_PrintfImpl_Cache_4__Get<> + 104
frame #11: 0x00000001001250f7 FsCore`FSharp_Core_Microsoft_FSharp_Core_PrintfModule__FSharp_Core_Microsoft_FSharp_Core_PrintfModule__PrintFormatToTextWriterThen<FSharp_Core_Microsoft_FSharp_Core_Unit__FSharp_Core_Microsoft_FSharp_Core_FSharpFunc_2<FSharp_Core_Microsoft_FSharp_Core_FSharpOption_1<System_Private_CoreLib_System_Int32>__FSharp_Core_Microsoft_FSharp_Core_Unit>> + 27
frame #12: 0x000000010011877b FsCore`FsCore_Program__main + 67
frame #13: 0x0000000100110f43 FsCore`FsCore__Module___StartupCodeMain + 87
frame #14: 0x0000000100010a82 FsCore`main(argc=1, argv=0x00007fff5fbff3e0) + 242 at main.cpp:314
frame #15: 0x00007fff8e5f85ad libdyld.dylib`start + 1For further investigation here's the simple F# program I compiled: [<EntryPoint>]
let main argv =
let div x y =
match y with
| 0 -> None
| _ -> Some(x/y)
let res : int option = div 20 4
printfn "Result: %A " res
0 // return an integer exit code |
|
Just as a quick heads-up, I noticed that a design for how to handle tail calls in CoreRT has been merged recently. Progress being made! |
|
Hi @tsolarin , may I ask how did you make the Thanks! |
|
@wecing here's a copy of my FsProj I use for building both on CoreCLR and CoreRT <Project Sdk="FSharp.NET.Sdk;Microsoft.NET.Sdk">
<Import Project="\Users\Toni\.nuget\packages\fsharp.net.sdk\1.0.0\build\FSharp.NET.Sdk.props" />
<Import Project="$(MSBuildSDKsPath)\Microsoft.NET.Sdk\Sdk\Sdk.props" />
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp1.0</TargetFramework>
<Version>1.0.0-alpha</Version>
<EnableDefaultCompileItems>false</EnableDefaultCompileItems>
</PropertyGroup>
<ItemGroup>
<Compile Include="Program.fs" />
<EmbeddedResource Include="**\*.resx" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="FSharp.NET.Sdk" Version="1.0.0-beta-*" PrivateAssets="All" />
<PackageReference Include="Microsoft.FSharp.Core.netcore" Version="1.0.0-alpha-161023" />
</ItemGroup>
<ItemGroup>
<DotNetCliToolReference Include="dotnet-compile-fsc">
<Version>1.0.0-preview2-020000</Version>
</DotNetCliToolReference>
</ItemGroup>
<ItemGroup>
<IlcReference Include="/Users/Toni/.nuget/packages/microsoft.fsharp.core.netcore/1.0.0-alpha-161023/lib/netstandard1.6/FSharp.Core.dll" />
</ItemGroup>
<Import Project="\Users\Toni\.nuget\packages\fsharp.net.sdk\1.0.0\build\FSharp.NET.Core.Sdk.targets" />
<Import Project="$(MSBuildSDKsPath)\Microsoft.NET.Sdk\Sdk\Sdk.targets" />
<Import Project="$(IlcPath)\Microsoft.NETCore.Native.targets" />
</Project>You'd have to import |
|
Hi @tsolarin , thanks for your answer. I copied your .fsproj and changed all occurences of your home directory to mine, but It is probably breaked by dotnet/netcorecli-fsc@9392487 , which was checked in in early Jan. So, @tsolarin , are you still able to build F# programs on CoreRT? (also cc @enricosada who submitted the patch) |
|
Hi everyone. As a note, you are using old templates for fsproj /cc @wecing @tsolarin More info in netcorecli-fsc wiki page For previous dotnet/cli releases, see wiki and there are also templates in https://github.com/dotnet/netcorecli-fsc/tree/master/examples (call me if you get issues) and example (generated by <Project Sdk="FSharp.NET.Sdk;Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp1.0</TargetFramework>
</PropertyGroup>
<ItemGroup>
<Compile Include="Program.fs" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="FSharp.NET.Sdk" Version="1.0.*" PrivateAssets="All" />
<PackageReference Include="FSharp.Core" Version="4.1.*" />
</ItemGroup>
</Project>In general, the diff with c# it's just:
IDE tooling is not mature yet, but from fsproj from cli and fsc compiler are ok afaik. @jkotas @wecing @tsolarin @MichalStrehovsky any issue feel free to ping me directly anywhere (skype/twitter/github), or open an issue in https://github.com/dotnet/netcorecli-fsc |
|
@tsolarin @enricosada Okay, sorry for spamming responses, but I fixed the issue by using FSharp.NET.Sdk/Sdk/Sdk.{props,targets} in <Project>
<Import Project="$(MSBuildSDKsPath)\FSharp.NET.Sdk\Sdk\Sdk.props" />
<Import Project="$(MSBuildSDKsPath)\Microsoft.NET.Sdk\Sdk\Sdk.props" />
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp1.0</TargetFramework>
<Version>1.0.0-alpha</Version>
<EnableDefaultCompileItems>false</EnableDefaultCompileItems>
</PropertyGroup>
<ItemGroup>
<Compile Include="Program.fs" />
<EmbeddedResource Include="**\*.resx" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="FSharp.NET.Sdk" Version="1.0.0-beta-*" PrivateAssets="All" />
<PackageReference Include="Microsoft.FSharp.Core.netcore" Version="1.0.0-alpha-161023" />
</ItemGroup>
<ItemGroup>
<DotNetCliToolReference Include="dotnet-compile-fsc">
<Version>1.0.0-preview2-020000</Version>
</DotNetCliToolReference>
</ItemGroup>
<ItemGroup>
<IlcReference Include="/home/w/.nuget/packages/microsoft.fsharp.core.netcore/1.0.0-alpha-161023/lib/netstandard1.6/FSharp.Core.dll" />
</ItemGroup>
<Import Project="$(MSBuildSDKsPath)\FSharp.NET.Sdk\Sdk\Sdk.targets" />
<Import Project="$(MSBuildSDKsPath)\Microsoft.NET.Sdk\Sdk\Sdk.targets" />
<Import Project="$(IlcPath)\Microsoft.NETCore.Native.targets" />
</Project>My What really confused me is that if you stick to @tsolarin 's |
|
@wecing the import of .props/target is automatic from What version of dotnet/cli are you building? maybe this will do: <Project Sdk="FSharp.NET.Sdk;Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp1.0</TargetFramework>
</PropertyGroup>
<ItemGroup>
<Compile Include="Program.fs" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="FSharp.NET.Sdk" Version="1.0.*" PrivateAssets="All" />
<PackageReference Include="FSharp.Core" Version="4.1.*" />
</ItemGroup>
<ItemGroup>
<IlcReference Include="$(NuGetPackageFolders)/fsharp.core/4.1.0/lib/netstandard1.6/FSharp.Core.dll" />
</ItemGroup>
<Import Project="$(IlcPath)\Microsoft.NETCore.Native.targets" />
</Project>as a note, you can do |
|
@enricosada , my But the FSharp.NET.Sdk/Sdk/Sdk.props file from $(MSBuildSDKsPath) is really simple: |
|
I'll try to check this weekend. |
|
@wecing @enricosada I've taken a look and did the upgrade to use dotnet/cli 1.0.1 and here are my findings: Using the template generated fsproj works ok when just building for CoreCLR, however some modifications are needed when trying to build for CoreRT. Here's my final layout as well as the steps I took to achieve a successful build:
Try it with a simple "Hello World" #2165 app at first just to be sure. I'm currently working on making more complex F# programs work with CoreRT. Here's a text version of the image for easy copy-paste <Project Sdk="FSharp.NET.Sdk;Microsoft.NET.Sdk">
<Import Project="\Users\Toni\.nuget\packages\FSharp.Net.Sdk\1.0.2\build\FSharp.NET.Sdk.props" />
<Import Project="$(MSBuildSDKsPath)\Microsoft.NET.Sdk\Sdk\Sdk.props" />
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp1.1</TargetFramework>
</PropertyGroup>
<ItemGroup>
<Compile Include="Program.fs" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="FSharp.Core" Version="4.1.*" />
<PackageReference Include="FSharp.NET.Sdk" Version="1.0.*" PrivateAssets="All" />
</ItemGroup>
<ItemGroup>
<IlcReference Include="\Users\Toni\.nuget\packages\FSharp.Core\4.1.0.2\lib\netstandard1.6\FSharp.Core.dll" />
</ItemGroup>
<Import Project="\Users\Toni\.nuget\packages\FSharp.Net.Sdk\1.0.2\build\FSharp.NET.Core.Sdk.targets" />
<Import Project="$(MSBuildSDKsPath)\Microsoft.NET.Sdk\Sdk\Sdk.targets" />
<Import Project="$(IlcPath)\Microsoft.NETCore.Native.targets" />
</Project>At this point building F# on CoreRT is not as smooth as the C# experience and the above workaround is needed as there's still a lot of work to be done @wecing Let me know if you have any more questions. |
|
@tsolarin @MichalStrehovsky I think the problem of printfn originates from GetMethod returning null on PrintfImpl.Specializations. GetMethod is invoked here. I am not quite sure how to whitelist it in rd.xml (tried but not successful), but it seems that all those |
|
@wecing that I already know. Calls to |
|
@tsolarin If you comment out this line (so that The |
We not only don't generate the metadata, but we also don't bother generating the code. The experience is pretty bare bones right now. As @wecing mentioned, you either need to make sure the code path is statically reachable, or provide it as an additional compilation root (a place from which compilation starts compiling the static callgraph) externally with an RD.XML file. The RD.XML will have something like this in it: <?xml version="1.0" encoding="utf-8"?>
<Directives xmlns="http://schemas.microsoft.com/netfx/2013/01/metadata">
<Application>
<Assembly Name="repro">
<Type Name="IFrobber" Dynamic="Required All" />
</Assembly>
</Application>
</Directives>Assembly name is the name of the assembly that contains the definition of the type. The type name needs to be in the ECMA-335 SerString format (note it's a different format than the "real" RD.XML uses, and a bit more cumbersome). The compiler is pretty unfriendly if you don't get the exact string right, so it might require a bit of trial and error. (Useful piece of information is that generic type arguments need extra brackets, so it's Then you pass the XML file to ilc.exe as an additional parameter ( |
|
Can anyone put a test repo online? Could be just a hello world project. Then we can start testing to see what works and doesn't work. |
YES. Shall we set up a repo to test F#? |
|
@charlesroddie Good question. I think this needs coordination (to keep them in sync) with the F# team especially with @dsyme The F# testsuite (as far as I know) are available here, but it will needs some fsproj / test runner script changes (some of them written in perl) to target CoreRT: |
|
You can just take any of the source code in one of these subdirectories, e.g. start with this one, compile it and run it. In most cases it's just this for .NET Framework: so adjust that in the natural way. The test runner here runs various variations (debug on/off, optimize on/off etc). If you are based on .NET SDK projects then you can also dump the contents of those test files into a .NET SDK project created following these instructions |
|
If you want to somehow add automated CoreRT testing to http://github.com/Microsoft/visualfsharp that would be fine I think. We already run Mono, .NET Framework and .NET Core. It looks like all you have to add is to the project file? |
|
@dsyme Basically yes, and a nuget.config also required because the ILCompiler is not yet available on nuget.org. I guess the nuget.config could be moved to the tests root directory. Using printf in AOT environment is usually fails, and fixing it is usually horrible (learned from earlier mono aot experience). I have attached an example test project from tests/fsharp/core/array. Unfortunately it fails with Usage (Ubuntu 16.04, x86_64): |
|
I did a quick (and probably non-correct) search & replace with System.String.Format and System.Console.WriteLine, but sill fails with the following: Update (without zip update): commented out all multidimension array test modules, but still have the same exception: tests_fsharp_core_array_without_printf.zip Update2: after disabling bug6447 it runs, but most 2d/3d/4d tests are disabled due the non-zero lower bound missing support. |
|
CoreRT/.Net Native has no plans to implement non-zero based arrays across all platforms. That shouldn't be a problem. Are any tests failing with this error that you wouldn't expect to fail? I.e. any tests that don't explicity ask for non-zero-based arrays? |
Issue to track progress towards enabling native F# support.
The first step: try to compile F# "hello world" program using CoreRT and identify where the first blocking issue.
The text was updated successfully, but these errors were encountered: