[SPARK-11096] Post-hoc review Netty based RPC implementation - round 2

[rxin]

A few more changes:

1. Renamed IDVerifier -> RpcEndpointVerifier
2. Renamed NettyRpcAddress -> RpcEndpointAddress
3. Simplified NettyRpcHandler a bit by removing the connection count tracking. This is OK because I now force spark.shuffle.io.numConnectionsPerPeer to 1
4. Reduced spark.rpc.connect.threads to 64. It would be great to eventually remove this extra thread pool.
5. Minor cleanup & documentation.

[rxin]

cc @zsxwing

[rxin]

this is not used anywhere

[zsxwing]

LGTM

[SparkQA]

Test build #43706 has finished for PR 9112 at commit 48b352d.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[zsxwing]

Oh, NettyRpcHandlerSuite.receive tests the case that a remote client fires multiple connections. You can remove L45 and L46 here:

spark/core/src/test/scala/org/apache/spark/rpc/netty/NettyRpcHandlerSuite.scala

Line 45 in 390b22f

when(channel.remoteAddress()).thenReturn(new InetSocketAddress("localhost", 40001))

[SparkQA]

Test build #43723 has finished for PR 9112 at commit e105f3f.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[rxin]

Merging this .