SPARK-4337. [YARN] Add ability to cancel pending requests #4141

sryza · 2015-01-21T19:38:33Z

No description provided.

SparkQA · 2015-01-21T19:42:58Z

Test build #25901 has started for PR 4141 at commit 472f944.

This patch merges cleanly.

SparkQA · 2015-01-21T21:06:48Z

Test build #25901 has finished for PR 4141 at commit 472f944.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-01-21T21:06:52Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25901/
Test FAILed.

vanzin · 2015-01-21T21:36:07Z

LGTM. Something seems funny with jenkins, lots of PRs are failing tests with seemingly no reason...

lianhuiwang · 2015-01-25T06:22:23Z

LGTM, i think @tdas you can take a look at this PR.

andrewor14 · 2015-01-25T23:23:13Z

retest this please

SparkQA · 2015-01-25T23:27:50Z

Test build #26068 has started for PR 4141 at commit 472f944.

This patch merges cleanly.

SparkQA · 2015-01-26T00:09:35Z

Test build #26068 has finished for PR 4141 at commit 472f944.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-01-26T00:09:39Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26068/
Test FAILed.

squito · 2015-01-26T19:38:49Z

yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala

how about making it private[yarn]? will still be visible in tests

YarnAllocator is already private[yarn], so this will end up private[yarn] already.

squito · 2015-01-26T19:39:25Z

just a minor comment, otherwise lgtm

pwendell · 2015-01-26T21:01:57Z

Jenkins, test this please.

SparkQA · 2015-01-26T21:02:44Z

Test build #26110 has started for PR 4141 at commit 472f944.

This patch merges cleanly.

SparkQA · 2015-01-26T22:18:51Z

Test build #26110 has finished for PR 4141 at commit 472f944.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-01-26T22:18:55Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26110/
Test PASSed.

tdas · 2015-01-27T05:58:36Z

@lianhuiwang I am not sure I can comment on this PR. I am unfamiliar with this piece of code.

andrewor14 · 2015-02-05T23:28:38Z

yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala

you can just make this $numToCancel

andrewor14 · 2015-02-05T23:43:54Z

@sryza this looks correct to me. My comments are mostly minor. Quick question, do we have to do any additional thing for this to work with dynamic allocation too or does it come for free with this patch?

andrewor14 · 2015-02-05T23:45:22Z

Never mind I just found #4168

sryza · 2015-02-06T17:28:48Z

Thanks for the review @andrewor14. Updated patch addresses your comments.

SparkQA · 2015-02-06T17:32:53Z

Test build #26921 has started for PR 4141 at commit a98bd20.

This patch merges cleanly.

SparkQA · 2015-02-06T18:45:08Z

Test build #26921 has finished for PR 4141 at commit a98bd20.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-02-06T18:45:12Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26921/
Test PASSed.

andrewor14 · 2015-02-06T18:53:07Z

Ok I'm merging this into master and 1.3 thanks

Author: Sandy Ryza <sandy@cloudera.com> Closes #4141 from sryza/sandy-spark-4337 and squashes the following commits: a98bd20 [Sandy Ryza] Andrew's comments cdaab7f [Sandy Ryza] SPARK-4337. Add ability to cancel pending requests to YARN (cherry picked from commit 1a88f20) Signed-off-by: Andrew Or <andrew@databricks.com>

sryza changed the title ~~SPARK-4337. Add ability to cancel pending requests to YARN~~ SPARK-4337. [YARN] Add ability to cancel pending requests Jan 22, 2015

squito reviewed Jan 26, 2015
View reviewed changes

andrewor14 reviewed Feb 5, 2015
View reviewed changes

yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala Outdated

Copy link

Contributor

andrewor14 Feb 5, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can just make this $numToCancel

sryza added 2 commits February 6, 2015 09:20

SPARK-4337. Add ability to cancel pending requests to YARN

cdaab7f

Andrew's comments

a98bd20

sryza force-pushed the sandy-spark-4337 branch from 472f944 to a98bd20 Compare February 6, 2015 17:28

asfgit closed this in 1a88f20 Feb 6, 2015

SPARK-4337. [YARN] Add ability to cancel pending requests #4141

SPARK-4337. [YARN] Add ability to cancel pending requests #4141

Uh oh!

Conversation

sryza commented Jan 21, 2015

Uh oh!

SparkQA commented Jan 21, 2015

Uh oh!

SparkQA commented Jan 21, 2015

Uh oh!

AmplabJenkins commented Jan 21, 2015

Uh oh!

vanzin commented Jan 21, 2015

Uh oh!

lianhuiwang commented Jan 25, 2015

Uh oh!

andrewor14 commented Jan 25, 2015

Uh oh!

SparkQA commented Jan 25, 2015

Uh oh!

SparkQA commented Jan 26, 2015

Uh oh!

AmplabJenkins commented Jan 26, 2015

Uh oh!

squito Jan 26, 2015

Choose a reason for hiding this comment

Uh oh!

sryza Jan 26, 2015

Choose a reason for hiding this comment

Uh oh!

squito commented Jan 26, 2015

Uh oh!

pwendell commented Jan 26, 2015

Uh oh!

SparkQA commented Jan 26, 2015

Uh oh!

SparkQA commented Jan 26, 2015

Uh oh!

AmplabJenkins commented Jan 26, 2015

Uh oh!

tdas commented Jan 27, 2015

Uh oh!

andrewor14 Feb 5, 2015

Choose a reason for hiding this comment

Uh oh!

andrewor14 commented Feb 5, 2015

Uh oh!

andrewor14 commented Feb 5, 2015

Uh oh!

sryza commented Feb 6, 2015

Uh oh!

SparkQA commented Feb 6, 2015

Uh oh!

SparkQA commented Feb 6, 2015

Uh oh!

AmplabJenkins commented Feb 6, 2015

Uh oh!

andrewor14 commented Feb 6, 2015

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants