Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Continue requesting block reports until all DataNodes have reported #53

Closed
xkrogen opened this issue Jul 31, 2018 · 0 comments
Closed

Comments

@xkrogen
Copy link
Collaborator

xkrogen commented Jul 31, 2018

Currently within waitForNameNodeReadiness, while waiting for the NameNode to have received enough block reports to be ready for use, the AppMaster will poll the NameNode to discover which DataNodes haven't sent block reports yet and trigger reports on those DataNodes. This can help when a DataNode sent its initial block report before all of its blocks were injected, in which case a better report wouldn't be sent until the block report interval expired (which can be very long). Right now it stops as soon as the block thresholds are met, but it would be better if it continued to do this even after the thresholds are met, until all DataNodes have actually reported.

xkrogen added a commit to xkrogen/dynamometer that referenced this issue Jul 31, 2018
…ing reports until there are no more DataNodes which match the criteria
@xkrogen xkrogen closed this as completed in a2d1385 Sep 7, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant