Skip to content
This repository has been archived by the owner on Mar 3, 2023. It is now read-only.

Remove scheduler state manager adaptor requirement for submission dry-run #1697

Merged

Conversation

objmagic
Copy link
Contributor

@objmagic objmagic commented Feb 7, 2017

Dry-run submission does not need to use adaptor to scheduler state manager. With this PR, state manager initialization will not happen, which reduces dry-run of submission to Aurora from 10 seconds to 1.7 seconds.

resolves #1692

@objmagic objmagic self-assigned this Feb 7, 2017
@objmagic objmagic requested a review from billonahill February 7, 2017 18:36
@objmagic objmagic added this to the 0.14.6 milestone Feb 7, 2017
@objmagic
Copy link
Contributor Author

objmagic commented Feb 7, 2017

Also addressed some comments from @mycFelix in 1e34ccc

@@ -381,6 +381,9 @@ public String createTable() {
}
}

private static final List<String> titleNames = Arrays.asList(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

psf should be TITLE_NAMES

} else {
// initialize the state manager
statemgr.initialize(config);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do you mean?

Copy link
Contributor

@huijunw huijunw Feb 8, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me. it is expected like what the PR description says.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can add a reply next time instead of making an inline comment.

@objmagic
Copy link
Contributor Author

objmagic commented Feb 8, 2017

build is fine on CI, unit tests passed locally. Website build breaks the CI because of some package issues, which will be solved in #1703.

merging now.

@objmagic objmagic merged commit 5b12434 into apache:master Feb 8, 2017
@objmagic objmagic deleted the no-statemgr-adaptor-submission-dry-run branch February 8, 2017 18:01
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Accelerate dry-run submission
3 participants