Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve service URL DNS resolution #929

Closed
merlimat opened this issue Nov 30, 2017 · 0 comments
Closed

Improve service URL DNS resolution #929

merlimat opened this issue Nov 30, 2017 · 0 comments
Assignees
@merlimat
Labels
type/enhancement The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages
Milestone
1.22.0-incubating

Comments

@merlimat
Copy link
Contributor

Expected behavior

When doing lookups, we are connecting to the endpoint represented by its serviceUrl. When using native Pulsar protocol, we should resolve the DNS at each attempt, in order to adapt when the DNS config changes.

Additionally, at each attempt, we need to try all the IP addresses returned by DNS, since in same cases DNS is used as a service discovery mechanism.

This is already working correctly on HTTP based lookup where the serviceUrl is passed to asyncHttpClient library.

Actual behavior

Currently, the client impl is resolving the serviceUrl name at startup at it is caching the 1st IP address:

https://github.com/apache/incubator-pulsar/blob/4288839fb82f2c64549fc8c6fb4f70db2b72b961/pulsar-client/src/main/java/org/apache/pulsar/client/impl/BinaryProtoLookupService.java#L52

  • We should defer resolving until we are actually connecting, so that new attempts might be leading to a different IP returned by DNS server.
  • We should try all the returned IP addresses until a successful connection can be established
  • Additionally, we should use Netty async DnsResolver class to do it asynchronously.
@merlimat merlimat added help wanted type/enhancement The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages labels Nov 30, 2017
@merlimat merlimat added this to the 1.22.0-incubating milestone Nov 30, 2017
@merlimat merlimat self-assigned this Nov 30, 2017
merlimat added a commit to merlimat/pulsar that referenced this issue Nov 30, 2017
@merlimat
Issue apache#929: Perform async DNS resolution each time attempting c…
019bfed 
…onnection to service Url

Also attempt to connecto to all IP addresses present in DNS record
@merlimat merlimat mentioned this issue Nov 30, 2017
Merged
merlimat added a commit that referenced this issue Jan 2, 2018
@merlimat
Issue #929: Perform async DNS resolution each time attempting connect… (
039932a 
#930)
@merlimat merlimat closed this as completed Jan 4, 2018
@rdhabalia rdhabalia mentioned this issue Jan 24, 2018
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@merlimat merlimat

Labels
type/enhancement The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages
Projects
None yet
Milestone
1.22.0-incubating
Development

No branches or pull requests
1 participant
@merlimat