diff --git a/docs/configuration.md b/docs/configuration.md index a9753925407d7..2687f542b8bd3 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -639,6 +639,7 @@ Apart from these, the following properties are also available, and may be useful
spark.eventLog.enabled is true.
+ Compression will use spark.io.compression.codec.
spark.io.compression.codec.
spark.io.compression.codeclz4, lzf,
+ The codec used to compress internal data such as RDD partitions, event log, broadcast variables
+ and shuffle outputs. By default, Spark provides three codecs: lz4, lzf,
and snappy. You can also use fully qualified class names to specify the codec,
e.g.
org.apache.spark.io.LZ4CompressionCodec,
@@ -881,6 +883,7 @@ Apart from these, the following properties are also available, and may be useful
StorageLevel.MEMORY_ONLY_SER in Java
and Scala or StorageLevel.MEMORY_ONLY in Python).
Can save substantial space at the cost of some extra CPU time.
+ Compression will use spark.io.compression.codec.