Skip to content

Commit 0b275cf

Browse files
committed
SPARK-25004: Update configuration docs.
1 parent bb8fecb commit 0b275cf

File tree

1 file changed

+4
-1
lines changed

1 file changed

+4
-1
lines changed

docs/configuration.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -185,7 +185,10 @@ of the most common options to set are:
185185
<td>
186186
The amount of memory to be allocated to PySpark in each executor, in MiB
187187
unless otherwise specified. If set, PySpark memory for an executor will be
188-
limited to this amount. If not set, Spark will not limit Python's memory use.
188+
limited to this amount. If not set, Spark will not limit Python's memory use
189+
and it is up to the application to avoid exceeding the overhead memory space
190+
shared with other non-JVM processes. When PySpark is run in YARN, this memory
191+
is added to executor resource requests.
189192
</td>
190193
</tr>
191194
<tr>

0 commit comments

Comments
 (0)