How to implement System.Net.NameResolution on FreeBSD?

[janhenke]

The assembly System.Net.NameResolution does not compile on FreeBSD. Looking at the sources it seems to have corresponding source files implementing it on Linux and OS X.

Looks like a good candidate for a shim, to unify the implementation. Especially given that the sockets API is pretty consistent across POSIX systems.

@nguerrera do you plan a System.Net.Native shim to cover this?

\cc @josteink

[ellismg]

FYI, The plan for 1.0 is to move all OS specific P/Invokes from managed code onto shims. I can't speak to the sequencing of the conversion, but it is something we want to do.

[janhenke]

@ellismg Thanks for the information. Sounds good. The shims are really good.

I will leave this open for now though. Once the corresponding shim is implemented, we need to test if it compiles cleanly on FreeBSD and can close it on success.

[jonmill]

I'll be shim'ing the System.Net layer that is currently in Master next; I'll take a look at this and make sure I get the right calls for FreeBSD

[janhenke]

Thanks for the update.

[ghost]

Thank you @sokket! With this done, FreeBSD would probably be fully supported platform by both .NET core repos (as well as all aspnet, and Roslyn!).

and make sure I get the right calls for FreeBSD

I assume your approach is to look for lowest common denominator: POSIX sys calls first? If available use POSIX, otherwise fallback to kernel-specific system calls.
That would be my choice too. :)

[ghost]

@janhenke, FYI one important (qasi-related) PR is close to getting merged dotnet/corefx#3257 :)

[janhenke]

@jasonwilliams200OK I already updated the list just before you wrote it. Once dotnet/corefx#3257 is merged, this one should be the last blocker on FreeBSD.

[ghost]

@stephentoub, @ellismg, since this is the last stopper, is it a good time to resurrect dotnet/corefx#2533 and exclude FreeBSD from building System.Net.NameResolution? We can then probably request @mmitche to kick off FreeBSD instance for CoreFX on CI, so future PRs don't break our beloved OS support. :)

[stephentoub]

@jasonwilliams200OK, I'm not following. That PR was about adding a property to OSPlatform, but the implementation is already there to let you use OSPlatform.Create("FREEBSD"). And that let's you dynamically detect in code at runtime whether you're on FreeBSD or not, but that wouldn't help in getting the build system to skip a particular assembly when targeting a particular platform.

[janhenke]

@jasonwilliams200OK I do not think that is a good plan. Let us wait for the last shim, which properly implements this assembly. Once we have archived that, we can think about adding FreeBSD to the CI/front page as "community supported platform".

[stephentoub]

@janhenke, are any of the corefx tests running successfully on FreeBSD? If so, do you have a sense for what the success/failure rate looks like?

[ghost]

Ops, ok agree with you guys. I was considering @stephentoub's comment dotnet/corefx#2533 (comment), since we are very close to completion now. But it makes sense to have all assemblies compiling and tests passing before declaring the platform on public surface area.

[stephentoub]

since we are very close to completion now

Are we? That's why I was asking about corefx test passing rates. Successfully building is just the first step... the code actually needs to work 😉 If the tests are all or mostly already passing, that's exciting news. I've just not heard any reports of that yet.

[janhenke]

@stephentoub there is no way of telling until we can build all assemblies for FreeBSD as far as I can see. It would be of course the logical step once we have the assemblies to make sure the test work on the same level as they do on Linux/OS X.
As all the platform code went to the shims, I am fairly confident that it will "just work". If not, we will fix the fall out.
I do share the opinion that we should be fairly close to reach the same level with Linux and OS X soon.

[nguerrera]

@janhenke You should still be able to run tests and ignore failures from (or skip) unimplemented API. See https://github.com/dotnet/corefx/blob/master/Documentation/building/cross-platform-testing.md

[ghost]

I am building current CoreFX master of FreeBSD 10.1 amd64 as we speak. :)
Running via Hyper-V on Windows 10: dotnet/roslyn#2844 (comment) 😉

[josteink]

I'm just going to be cheerleading from the side here. This sounds like awesome news and is all in all about as exciting as an OSS project can get :D

[jonmill]

Just an update - I almost have the code in System.Net.Primitives and System.Net.NameResolution compiling; will be trying to get that finished today and then ensure the tests pass on Linux and OS X. I don't have a CoreFX and CoreCLR build for my FreeBSD instance so if someone has that and wants to test, great; if not, we can rely on the test coverage in Linux and OS X

[ghost]

Thanks for the update @sokket! :)
As far as I can tell, once all FreeBSD assemblies build (which I will test and report back), we can probably request @mmitche to setup FreeBSD on CI server as he did for CoreCLR. I hope the tests projects don't give an incredibly tough time to build on FreeBSD. :)

[janhenke]

@sokket Once it is merged I will try to build the native component. Do not worry, if it does not work on FreeBSD you will see an issue or PR about that soon. :D
The managed part should be identical for Unixes as far as I understand so it should be fine.

[mmitche]

@jasonwilliams200OK Yeah we should be able to do that.

[ghost]

@mmitche, thanks! :)

@janhenke, curiously, how much time does it usually take on "Restoring all packages" step? In my setup it takes forever to complete it.. Last time I manually copied the packages from somewhere I don't remember..

⌚

[ghost]

BTW I am testing with @sokket's pal branch with rebased on top of master:

git clone https://github.com/sokket/corefx/ -b pal
cd corefx
git remote add dotnet https://github.com/dotnet/corefx
git pull --rebase dotnet master
./build.sh

[jonmill]

@jasonwilliams200OK how's it look? I tried to make sure pal_networking.cpp compiled on my FreeBSD box but, I admit, I haven't used it before so I may have a wonky or incorrect configuration.

Also, running through Jenkins shows only 5 failing unit tests (woot!) so I'll be fixing those today and hopefully getting a PR out today or tomorrow (depending on how tricky the bugs are)

[janhenke]

@jasonwilliams200OK That step indeed takes a long time to run. I did not measure, but I usually let it run in the background and check back. That is apparently normal, since it updates/downloads a lot of packages.

[ghost]

@sokket, @janhenke,
old pedantic habit die hard; I am doing all of it from scratch. But it is hanging at "Restoring all packages". Yesterday I tried it as well, and I assumed it is hanging there because the VM has less memory or storage, I have increased but still "Restoring all packages.." step seems never ending.
On Windows10 host, it took about 7 minutes using same network to finish Restoring all packages step. On FreeBSD, it is hanging there for over 50 minutes now.. (this is my third test today, for first two, I checked back after hours then manually aborted and restarted the VM..)
Either something wrong with my FreeBSD installation, that it doesn't scale well in recursion cases. I think I should create a new FreeBSD v10.2 (latest) VM (currently I am running v10.1)

[jonmill]

@jasonwilliams200OK @janhenke PR dotnet/corefx#3471 is open to address this :)

[akoeplinger]

@jasonwilliams200OK try setting MONO_THREADS_PER_CPU=50 env var, that usually helps working around a Mono+dnx restore bug.

[ghost]

@akoeplinger, thanks that fixed the SendFailure issue. 👍

Now I am getting compilation errors like:

/root/projects/corefx/packages/Microsoft.Build.Mono.Debug.14.1.0.0-prerelease/lib/Microsoft.CSharp.CurrentVersion.target
s(383,9): error MSB6006: "csc2.exe" exited with code 1.

for:

Microsoft.Win32.Primitives
Microsoft.Win32.Registry
System.Collections.Concurrent
System.Collections.Immutable
System.Collections.NonGeneric
System.Collections.Specialized
System.Private.Uri
System.Diagnostics.Debug
System.ComponentModel.Annotations
System.ComponentModel.EventBasedAsync
System.ComponentModel.Primitives
System.ComponentModel.TypeConverter
System.ComponentModel
System.Composition.AttributedModel

...
etc.

Running csc2 command separately like:

mono ./packages/Microsoft.Net.ToolsetCompilers.1.0.0-rc3-20150510-01/tools/csc2.exe
/root/projects/corefx/src/System.ComponentModel/src/System.ComponentModel.csproj

shows loads of missing type and compilation errors.

[janhenke]

@jasonwilliams200OK Please open a new issue for that. Build problems are off topic for this issue.

[ghost]

@janhenke, good idea but, how can we verify dotnet/corefx#3471 if it is not building in first place? I am only building to validate this issue. :/

[janhenke]

Still it is a separate issue, since your build problems are not related to this particular feature, but of general nature. We should keep separate issues separate.

[janhenke]

Since this assembly has been shim'ed, I close this issue. It will be resolved once dotnet/corefx#3471 has been merged.