Skip to content
This repository was archived by the owner on Mar 24, 2025. It is now read-only.
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ When writing files the API accepts several options:
* `nullValue`: The value to write `null` value. Default is string `null`. When this is `null`, it does not write attributes and elements for fields.
* `attributePrefix`: The prefix for attributes so that we can differentiating attributes and elements. This will be the prefix for field names. Default is `@`.
* `valueTag`: The tag used for the value when there are attributes in the element having no child. Default is `#VALUE`.
* `codec`: compression codec to use when saving to file. Should be the fully qualified name of a class implementing `org.apache.hadoop.io.compress.CompressionCodec` or one of case-insensitive shorten names (`bzip2`, `gzip`, `lz4`, and `snappy`). Defaults to no compression when a codec is not specified.
* `compression`: compression codec to use when saving to file. Should be the fully qualified name of a class implementing `org.apache.hadoop.io.compress.CompressionCodec` or one of case-insensitive shorten names (`bzip2`, `gzip`, `lz4`, and `snappy`). Defaults to no compression when a codec is not specified.

Currently it supports the shortened name usage. You can use just `xml` instead of `com.databricks.spark.xml` from Spark 1.5.0+

Expand Down
2 changes: 1 addition & 1 deletion src/main/scala/com/databricks/spark/xml/XmlOptions.scala
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ private[xml] class XmlOptions(
extends Serializable{

val charset = parameters.getOrElse("charset", XmlOptions.DEFAULT_CHARSET)
val codec = parameters.get("codec").orNull
val codec = parameters.get("compression").orElse(parameters.get("codec")).orNull
val rowTag = parameters.getOrElse("rowTag", XmlOptions.DEFAULT_ROW_TAG)
val rootTag = parameters.getOrElse("rootTag", XmlOptions.DEFAULT_ROOT_TAG)
val samplingRatio = parameters.get("samplingRatio").map(_.toDouble).getOrElse(1.0)
Expand Down
2 changes: 1 addition & 1 deletion src/test/scala/com/databricks/spark/xml/XmlSuite.scala
Original file line number Diff line number Diff line change
Expand Up @@ -353,7 +353,7 @@ class XmlSuite extends FunSuite with BeforeAndAfterAll {

val cars = sqlContext.xmlFile(carsFile)
cars.save("com.databricks.spark.xml", SaveMode.Overwrite,
Map("path" -> copyFilePath, "codec" -> "gZiP"))
Map("path" -> copyFilePath, "compression" -> "gZiP"))
val carsCopyPartFile = new File(copyFilePath, "part-00000.gz")
// Check that the part file has a .gz extension
assert(carsCopyPartFile.exists())
Expand Down