Skip to content

Commit

Permalink
[LIVY-697] Rsc client cannot resolve the hostname of driver in yarn-c…
Browse files Browse the repository at this point in the history
…luster mode

## What changes were proposed in this pull request?

[LIVY-697] Rsc client cannot resolve the hostname of driver in yarn-cluster mode

1. The content of Driver in /etc/hosts are as follows:
   127.0.0.1 localhost
   10.10.10.10 test_hostname

2. The content of Driver in /etc/hostname are as follows:
   test_hostname

3. The findLocalAddress method in livy cannot return 10.10.10.10, but return  test_hostname.
   Because the  test_hostname point to 10.10.10.10,  which doesn't pass the check
   address.isLoopbackAddress(), so findLocalAddress return test_hostname .

4. The rsc client cannot resolve the test_hostname, which cause rsc client cannot connect to
   driver.

5. The findLocalAddress method in livy can return 10.10.10.10 as expected if the content
   of Driver in  /etc/hosts are as follows, which is not correct in our environment.
   127.0.0.1 localhost
   127.0.0.1 test_hostname

6. Though I can modify the findLocalAddress method to return 10.10.10.10, but it maybe cause
  error if the machine has multiple  network  cards.  So, rsc client gets the driver ip from the
  connection.

## How was this patch tested?

1. The content of Driver in /etc/hosts are as follows:
   127.0.0.1 localhost
   127.0.0.1 test_hostname

2. The content of Driver in /etc/hostname are as follows:
   test_hostname

3. rsc client can get the driver ip in connection, and connect to driver successfully.

Author: runzhiwang <runzhiwang@tencent.com>

Closes #246 from runzhiwang/hostname-2-ip.
  • Loading branch information
runzhiwang authored and jerryshao committed Oct 21, 2019
1 parent 85837e3 commit ba12b51
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion rsc/src/main/java/org/apache/livy/rsc/ContextLauncher.java
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
import java.io.OutputStreamWriter;
import java.io.Reader;
import java.io.Writer;
import java.net.InetSocketAddress;
import java.nio.file.Files;
import java.util.ArrayList;
import java.util.Arrays;
Expand Down Expand Up @@ -340,7 +341,9 @@ void dispose() {
}

private void handle(ChannelHandlerContext ctx, RemoteDriverAddress msg) {
ContextInfo info = new ContextInfo(msg.host, msg.port, clientId, secret);
InetSocketAddress insocket = (InetSocketAddress) ctx.channel().remoteAddress();
String ip = insocket.getAddress().getHostAddress();
ContextInfo info = new ContextInfo(ip, msg.port, clientId, secret);
if (promise.trySuccess(info)) {
timeout.cancel(true);
LOG.debug("Received driver info for client {}: {}/{}.", client.getChannel(),
Expand Down

0 comments on commit ba12b51

Please sign in to comment.