DnsNameResolver hangs for 5 seconds in InetAddress.getAllByName0 on Mac OSX

[yschimke]

Expected behavior

DnsNameResolver constructor should be fast, anything slow offloaded until actually used. Or a warning given that environment in badly configured.

This is an environmental problem, but raising because its a regression from java DNS which doesn't hang like this.

Feel free to close if you don't want to worry about it in Netty.

Actual behavior

constructor hangs at program startup

Steps to reproduce

    System.out.println("started " + new Date());

    ThreadFactory threadFactory = new DefaultThreadFactory("netty", true);
    NioEventLoopGroup eventLoopGroup = new NioEventLoopGroup(1, threadFactory);

    DnsNameResolverBuilder builder = new DnsNameResolverBuilder(eventLoopGroup.next())
            .channelType(NioDatagramChannel.class)
            .optResourceEnabled(false)
            .maxQueriesPerResolve(3)
            .recursionDesired(true);

    DnsNameResolver re = builder.build();

    System.out.println("finished " + new Date());

started Sat Feb 25 13:17:16 GMT 2017
finished Sat Feb 25 13:17:21 GMT 2017

Fix

Fix is http://justthesam.com/2016/10/fixing-java-net-inet6addressimpl-lookupallhostaddr-slowdown/

Netty version

4.1.8.Final

JVM version (e.g. java -version)

$ java -version
java version "9-ea"
Java(TM) SE Runtime Environment (build 9-ea+157)
Java HotSpot(TM) 64-Bit Server VM (build 9-ea+157, mixed mode)

OS version (e.g. uname -a)

Darwin xxx 16.4.0 Darwin Kernel Version 16.4.0: Thu Dec 22 22:53:21 PST 2016; root:xnu-3789.41.3~3/RELEASE_X86_64 x86_64

"main" #1 prio=5 os_prio=31 tid=0x00007f8c60809800 nid=0x1d03 runnable [0x00007000027a3000]
   java.lang.Thread.State: RUNNABLE
	at java.net.Inet6AddressImpl.lookupAllHostAddr(java.base@9-ea/Native Method)
	at java.net.InetAddress$PlatformNameService.lookupAllHostAddr(java.base@9-ea/InetAddress.java:921)
	at java.net.InetAddress.getAddressesFromNameService(java.base@9-ea/InetAddress.java:1501)
	at java.net.InetAddress$NameServiceAddresses.get(java.base@9-ea/InetAddress.java:840)
	- locked <0x00000006cfce83e8> (a java.net.InetAddress$NameServiceAddresses)
	at java.net.InetAddress.getAllByName0(java.base@9-ea/InetAddress.java:1491)
	at java.net.InetAddress.getLocalHost(java.base@9-ea/InetAddress.java:1623)
	at sun.management.VMManagementImpl.getVmId(java.management@9-ea/VMManagementImpl.java:144)
	at sun.management.RuntimeImpl.getName(java.management@9-ea/RuntimeImpl.java:59)
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-ea/Native Method)
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-ea/NativeMethodAccessorImpl.java:62)
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-ea/DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(java.base@9-ea/Method.java:547)
	at io.netty.channel.DefaultChannelId.defaultProcessId(DefaultChannelId.java:121)
	at io.netty.channel.DefaultChannelId.<clinit>(DefaultChannelId.java:78)
	at io.netty.channel.AbstractChannel.newId(AbstractChannel.java:107)
	at io.netty.channel.AbstractChannel.<init>(AbstractChannel.java:79)
	at io.netty.channel.nio.AbstractNioChannel.<init>(AbstractNioChannel.java:84)
	at io.netty.channel.nio.AbstractNioMessageChannel.<init>(AbstractNioMessageChannel.java:42)
	at io.netty.channel.socket.nio.NioDatagramChannel.<init>(NioDatagramChannel.java:147)
	at io.netty.channel.socket.nio.NioDatagramChannel.<init>(NioDatagramChannel.java:115)
	at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(java.base@9-ea/Native Method)
	at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(java.base@9-ea/NativeConstructorAccessorImpl.java:62)
	at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(java.base@9-ea/DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(java.base@9-ea/Constructor.java:473)
	at java.lang.Class.newInstance(java.base@9-ea/Class.java:558)
	at io.netty.channel.ReflectiveChannelFactory.newChannel(ReflectiveChannelFactory.java:38)
	at io.netty.bootstrap.AbstractBootstrap.initAndRegister(AbstractBootstrap.java:321)
	at io.netty.bootstrap.AbstractBootstrap.register(AbstractBootstrap.java:235)
	at io.netty.resolver.dns.DnsNameResolver.<init>(DnsNameResolver.java:249)
	at io.netty.resolver.dns.DnsNameResolverBuilder.build(DnsNameResolverBuilder.java:347)

[yschimke]

The more I think about it, the more I realise it is

a) actually related to defaultProcessId
b) kind of funny that Netty DNS relies on built in java DNS

Feel free to close as not an issue, just raiding as it was surprising and annoying

[yschimke]

Working around with

-Dio.netty.processId=$$

[normanmaurer]

WIll take a look

[testn]

adding system property

-Djava.net.preferIPv4Stack=true

should help avoid the problem

[johnou]

The current method of pulling the process id is a bit of a hack since it relies on VMManagementImpl.getVmId however this now seems to be the common way of obtaining it for a lot of projects < JDK9.

    public String getVmId() {
        int var1 = this.getProcessId();
        String var2 = "localhost";

        try {
            var2 = InetAddress.getLocalHost().getHostName();
        } catch (UnknownHostException var4) {
            ;
        }

        return var1 + "@" + var2;
    }

    private native int getProcessId();

Java 9 provides an API to pull the process id.

long pid = ProcessHandle.current().getPid();

@normanmaurer not sure there is much that can be done apart from time how long it takes and issue a warn if it exceeds a threshold, or implement pulling the pid via Java 9 API using reflection, that said this seems like a machine misconfiguration not an issue with Netty.. but just so you know @yschimke, Norman loves contributions ;)

[yschimke]

I'm approved at work for my own projects, Finagle, OkHttp but didn't ask for Netty. I don't have that much spare time so it's not worth the CLA effort just for this fix.

[yschimke]

BTW I agree about the misconfiguration, I fixed this on my machine and also commented with a workaround.

Two things set off my spidey sense

1. its something that is affecting Netty performance for some # of Mac users/developers. potentially this could impact some GUI client software run on end users Macs.
2. When you choose to use Netty DnsNameResolver and google public servers, having it block because of your local Java default dns configuration seems weird.

[askiros]

It looks like an Mac OSX issue (resolution of the local host must happens inside java.lang.management.RuntimeMXBean#getName() causing the delay). The StackOverflow solution remove the delay.

[Scottmitch]

Let me close this issue for now as I think with PR #6592 we should no longer require JDK DNS resolution as part of codec-dns's initialization