Skip to content

Commit

Permalink
Add blurb about jar distribution (or lack thereof).
Browse files Browse the repository at this point in the history
  • Loading branch information
Marcelo Vanzin committed Oct 29, 2019
1 parent 8352e28 commit 69600e6
Showing 1 changed file with 8 additions and 3 deletions.
11 changes: 8 additions & 3 deletions docs/monitoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -1156,6 +1156,11 @@ applications. There are two configuration keys available for loading plugins int
Both take a comma-separated list of class names that implement the
<code>org.apache.spark.api.plugin.SparkPlugin</code> interface. The two names exist so that it's
possible for one list to be placed in the Spark default config file, allowing users to
easily add other plugins from the command line without overwriting the config file's list.

Duplicate plugins are ignored.
easily add other plugins from the command line without overwriting the config file's list. Duplicate
plugins are ignored.

Distribution of the jar files containing the plugin code is currently not done by Spark. The user
or admin should make sure that the jar file is available to Spark applications, for example, by
including the plugin jar with the Spark distribution. The exception to this rule is the YARN
backend, where the <code>--jars</code> command line option (or equivalent config entry) can be
used to make the plugin code available to both executors and cluster-mode drivers.

0 comments on commit 69600e6

Please sign in to comment.