Skip to content

SPARK-1933: Throw a more meaningful exception when a directory is passed to addJar/addFile. #888

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
rxin wants to merge 2 commits into from
Closed

SPARK-1933: Throw a more meaningful exception when a directory is passed to addJar/addFile. #888

rxin wants to merge 2 commits into from

Conversation

@rxin
Copy link
Contributor

@rxin rxin commented May 27, 2014

@AmplabJenkins
Copy link

AmplabJenkins commented May 27, 2014

Merged build triggered.

@AmplabJenkins
Copy link

AmplabJenkins commented May 27, 2014

Merged build started.

@AmplabJenkins
Copy link

AmplabJenkins commented May 27, 2014

Merged build finished. All automated tests passed.

@AmplabJenkins
Copy link

AmplabJenkins commented May 27, 2014

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15217/

pwendell
pwendell reviewed May 27, 2014
View reviewed changes
core/src/main/scala/org/apache/spark/HttpFileServer.scala
Copy link
Contributor

@pwendell pwendell May 27, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, but would it make more sense to put this check in SparkContext#addFile? We might make assumptions elsewhere in the code the user has supplied a file.

Copy link
Contributor Author

@rxin rxin May 27, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually originally put it in addFIle and addJar, and added a bunch of code to check whether the file is local (because the argument to addJar/addFile can be http, ftp, etc). Decided to move it here since all code paths go through this place.

@pwendell
Copy link
Contributor

pwendell commented May 27, 2014

I made a suggestion about putting this somewhere else, but I could see arguments for doing it either way, so LGTM.

@rxin
Copy link
Contributor Author

rxin commented May 27, 2014

Ok thanks for taking a look. Merging this in master & branch-1.0.

asfgit pushed a commit that referenced this pull request May 27, 2014
@rxin
SPARK-1933: Throw a more meaningful exception when a directory is pas…
fcb3750 
…sed to addJar/addFile.

https://issues.apache.org/jira/browse/SPARK-1933

Author: Reynold Xin <rxin@apache.org>

Closes #888 from rxin/addfile and squashes the following commits:

8c402a3 [Reynold Xin] Updated comment.
ff6c162 [Reynold Xin] SPARK-1933: Throw a more meaningful exception when a directory is passed to addJar/addFile.

(cherry picked from commit 90e281b)
Signed-off-by: Reynold Xin <rxin@apache.org>
@asfgit asfgit closed this in 90e281b May 27, 2014
@rxin rxin deleted the addfile branch May 27, 2014 05:07
pdeyhim pushed a commit to pdeyhim/spark-1 that referenced this pull request Jun 25, 2014
@rxin @pdeyhim
SPARK-1933: Throw a more meaningful exception when a directory is pas…
b3aa4be 
…sed to addJar/addFile.

https://issues.apache.org/jira/browse/SPARK-1933

Author: Reynold Xin <rxin@apache.org>

Closes apache#888 from rxin/addfile and squashes the following commits:

8c402a3 [Reynold Xin] Updated comment.
ff6c162 [Reynold Xin] SPARK-1933: Throw a more meaningful exception when a directory is passed to addJar/addFile.
xiliu82 pushed a commit to xiliu82/spark that referenced this pull request Sep 4, 2014
@rxin @conviva-zz
SPARK-1933: Throw a more meaningful exception when a directory is pas…
dca6ea6 
…sed to addJar/addFile.

https://issues.apache.org/jira/browse/SPARK-1933

Author: Reynold Xin <rxin@apache.org>

Closes apache#888 from rxin/addfile and squashes the following commits:

8c402a3 [Reynold Xin] Updated comment.
ff6c162 [Reynold Xin] SPARK-1933: Throw a more meaningful exception when a directory is passed to addJar/addFile.
wangyum added a commit that referenced this pull request May 26, 2023
@wangyum
[CARMEL-5878] Backport upstream changes #888 from hongyzhang/CARMEL-5878
d647e58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@rxin @AmplabJenkins @pwendell