Skip to content

Commit

Permalink
[SPARK-11056] Improve documentation of SBT build.
Browse files Browse the repository at this point in the history
This commit improves the documentation around building Spark to
(1) recommend using SBT interactive mode to avoid the overhead of
launching SBT and (2) refer to the wiki page that documents using
SPARK_PREPEND_CLASSES to avoid creating the assembly jar for each
compile.

cc srowen

Author: Kay Ousterhout <kayousterhout@gmail.com>

Closes apache#9068 from kayousterhout/SPARK-11056.
  • Loading branch information
kayousterhout committed Oct 12, 2015
1 parent 8a354be commit 091c2c3
Showing 1 changed file with 5 additions and 0 deletions.
5 changes: 5 additions & 0 deletions docs/building-spark.md
Original file line number Diff line number Diff line change
Expand Up @@ -216,6 +216,11 @@ can be set to control the SBT build. For example:

build/sbt -Pyarn -Phadoop-2.3 assembly

To avoid the overhead of launching sbt each time you need to re-compile, you can launch sbt
in interactive mode by running `build/sbt`, and then run all build commands at the command
prompt. For more recommendations on reducing build time, refer to the
[wiki page](https://cwiki.apache.org/confluence/display/SPARK/Useful+Developer+Tools#UsefulDeveloperTools-ReducingBuildTimes).

# Testing with SBT

Some of the tests require Spark to be packaged first, so always run `build/sbt assembly` the first time. The following is an example of a correct (build, test) sequence:
Expand Down

0 comments on commit 091c2c3

Please sign in to comment.