(DOCSP-21416) Update connector api, version constant, and release notes for v10 (#94)

zach-carr · zach-carr · commit 6ff7d1f15398 · 2022-04-07T10:52:37.000-04:00
* (DOCSP-21416) Finalize v10 docs
diff --git a/snooty.toml b/snooty.toml
@@ -6,7 +6,7 @@ intersphinx = ["https://www.mongodb.com/docs/manual/objects.inv"]
 toc_landing_pages = ["configuration"]
 
 [constants]
-current-version = "10.0"
+current-version = "10.0.0"
 spark-core-version = "3.0.1"
 spark-sql-version = "3.0.1"
 scala-version = "2.12"
diff --git a/source/includes/extracts-command-line.yaml b/source/includes/extracts-command-line.yaml
@@ -17,11 +17,11 @@ content: |
 ---
 ref: list-configuration-explanation
 content: |
-   - The :ref:`spark.mongodb.input.uri <spark-input-conf>` specifies the
+   - The :ref:`spark.mongodb.read.uri <spark-input-conf>` specifies the
      MongoDB server address (``127.0.0.1``), the database to connect
      (``test``), and the collection (``myCollection``) from which to read
      data, and the read preference.
-   - The :ref:`spark.mongodb.output.uri <spark-output-conf>` specifies the
+   - The :ref:`spark.mongodb.write.uri <spark-output-conf>` specifies the
      MongoDB server address (``127.0.0.1``), the database to connect
      (``test``), and the collection (``myCollection``) to which to write
      data. Connects to port ``27017`` by default.
@@ -37,8 +37,8 @@ content: |
 
    .. code-block:: sh
 
-      ./bin/spark-shell --conf "spark.mongodb.input.uri=mongodb://127.0.0.1/test.myCollection?readPreference=primaryPreferred" \
-                        --conf "spark.mongodb.output.uri=mongodb://127.0.0.1/test.myCollection" \
+      ./bin/spark-shell --conf "spark.mongodb.read.uri=mongodb://127.0.0.1/test.myCollection?readPreference=primaryPreferred" \
+                        --conf "spark.mongodb.write.uri=mongodb://127.0.0.1/test.myCollection" \
                         --packages org.mongodb.spark:mongo-spark-connector_{+scala-version+}:{+current-version+}
  
    .. include:: /includes/extracts/list-configuration-explanation.rst
@@ -54,8 +54,8 @@ content: |
 
    .. code-block:: sh
 
-      ./bin/pyspark --conf "spark.mongodb.input.uri=mongodb://127.0.0.1/test.myCollection?readPreference=primaryPreferred" \
-                    --conf "spark.mongodb.output.uri=mongodb://127.0.0.1/test.myCollection" \
+      ./bin/pyspark --conf "spark.mongodb.read.uri=mongodb://127.0.0.1/test.myCollection?readPreference=primaryPreferred" \
+                    --conf "spark.mongodb.write.uri=mongodb://127.0.0.1/test.myCollection" \
                     --packages org.mongodb.spark:mongo-spark-connector_{+scala-version+}:{+current-version+}
  
    .. include:: /includes/extracts/list-configuration-explanation.rst
@@ -71,8 +71,8 @@ content: |
 
    .. code-block:: sh
 
-      ./bin/sparkR  --conf "spark.mongodb.input.uri=mongodb://127.0.0.1/test.myCollection?readPreference=primaryPreferred" \
-                    --conf "spark.mongodb.output.uri=mongodb://127.0.0.1/test.myCollection" \
+      ./bin/sparkR  --conf "spark.mongodb.read.uri=mongodb://127.0.0.1/test.myCollection?readPreference=primaryPreferred" \
+                    --conf "spark.mongodb.write.uri=mongodb://127.0.0.1/test.myCollection" \
                     --packages org.mongodb.spark:mongo-spark-connector_{+scala-version+}:{+current-version+}
 
    .. include:: /includes/extracts/list-configuration-explanation.rst
diff --git a/source/includes/list-prerequisites.rst b/source/includes/list-prerequisites.rst
@@ -2,8 +2,6 @@
   :manual:`MongoDB documentation </>` and `Spark documentation
   <https://spark.apache.org/docs/latest/>`_ for more details.
 
-- Running MongoDB instance (version 2.6 or later).
+- Running MongoDB instance (version 3.6 or later).
 
-- Spark 2.4.x.
-
-- Scala 2.12.x
+- Spark 3.2.x.
diff --git a/source/includes/sparkconf-partitioner-options-note.rst b/source/includes/sparkconf-partitioner-options-note.rst
@@ -1,8 +1,8 @@
 
 .. note::
 
-   If you use ``SparkConf`` to set the connector's input 
+   If you use ``SparkConf`` to set the connector's read 
    configurations, prefix each property with 
-   ``spark.mongodb.input.partitionerOptions.`` 
+   ``spark.mongodb.read.partitionerOptions.`` 
    instead of ``partitioner.options.``.
    
diff --git a/source/index.txt b/source/index.txt
@@ -41,80 +41,13 @@ versions of Apache Spark and MongoDB:
      - MongoDB Version
 
    * - **{+current-version+}**
-     - **3.0.x**, **3.1.x**
-     - **2.6 or later**
+     - **3.2.x**
+     - **3.6 or later**
 
-   * - 2.4.4
-     - 2.4.x
-     - 2.6 or later
+Announcements
+-------------
 
-   * - 2.3.6
-     - 2.3.x
-     - 2.6 or later
-
-   * - 2.2.10
-     - 2.2.x
-     - 2.6 or later
-
-   * - 2.1.9
-     - 2.1.x
-     - 2.6 or later
-
-   * - 2.0.0
-     - 2.0.x
-     - 2.6 or later
-
-   * - 1.1.0
-     - 1.6.x
-     - 2.6 or later
-
-
-----
-
-.. note:: **Announcements**
-
-   - **August 17, 2020**, `MongoDB Connector for Spark version v3.0.0
-     <https://www.mongodb.com/products/spark-connector>`_ Released.
-
-   - **Jun 10, 2020**, `MongoDB Connector for Spark versions v2.4.2,
-     v2.3.4, v2.2.8, and v2.1.7
-     <https://www.mongodb.com/products/spark-connector>`_ Released.
-
-   - **Jun 06, 2019**, `MongoDB Connector for Spark versions v2.4.1,
-     v2.3.3, v2.2.7, and v2.1.6
-     <https://www.mongodb.com/products/spark-connector>`_ Released.
-
-   - **Dec 07, 2018**, `MongoDB Connector for Spark versions v2.4.0,
-     v2.3.2, v2.2.6, and v2.1.5
-     <https://www.mongodb.com/products/spark-connector>`_ Released.
-
-   - **Oct 08, 2018**, `MongoDB Connector for Spark versions v2.3.1,
-     v2.2.5, and v2.1.4
-     <https://www.mongodb.com/products/spark-connector>`_ Released.
-
-   - **July 30, 2018**, `MongoDB Connector for Spark versions v2.3.0,
-     v2.2.4, and v2.1.3
-     <https://www.mongodb.com/products/spark-connector>`_ Released.
-
-   - **June 19, 2018**, `MongoDB Connector for Spark versions v2.2.3 and
-     v2.1.2
-     <https://www.mongodb.com/products/spark-connector>`_ Released.
-
-   - **April 18, 2018**, `MongoDB Connector for Spark version v2.2.2
-     <https://www.mongodb.com/products/spark-connector>`_ Released.
-
-   - **Oct 31, 2017**, `MongoDB Connector for Spark version v2.2.1
-     <https://www.mongodb.com/products/spark-connector>`_ Released.
-
-   - **July 13, 2017**, `MongoDB Connector for Spark version v2.2.0
-     <https://www.mongodb.com/products/spark-connector>`_ Released.
-
-   - **July 12, 2017**, `MongoDB Connector for Spark versions v2.2.0 and
-     v2.1.0
-     <https://www.mongodb.com/products/spark-connector>`_ Released.
-
-   - **November 1, 2016**, `MongoDB Connector for Spark v2.0.0
-     <https://www.mongodb.com/products/spark-connector>`_ Released.
+- **March 31, 2022**, `MongoDB Connector for Spark version v10.0.0 <https://www.mongodb.com/products/spark-connector>`_ released.
 
 .. toctree::
    :titlesonly:
@@ -126,4 +59,4 @@ versions of Apache Spark and MongoDB:
    structured-streaming
    faq
    release-notes
-   API Docs <https://www.javadoc.io/doc/org.mongodb.spark/mongo-spark-connector_{+scala-version+}/{+current-version+}>
+   API Docs <https://www.javadoc.io/doc/org.mongodb.spark/mongo-spark-connector/latest/index.html>
diff --git a/source/java/aggregation.txt b/source/java/aggregation.txt
@@ -20,8 +20,8 @@
        SparkSession spark = SparkSession.builder()
          .master("local")
          .appName("MongoSparkConnectorIntro")
-         .config("spark.mongodb.input.uri", "mongodb://127.0.0.1/test.myCollection")
-         .config("spark.mongodb.output.uri", "mongodb://127.0.0.1/test.myCollection")
+         .config("spark.mongodb.read.uri", "mongodb://127.0.0.1/test.myCollection")
+         .config("spark.mongodb.write.uri", "mongodb://127.0.0.1/test.myCollection")
          .getOrCreate();
          
        // Create a JavaSparkContext using the SparkSession's SparkContext object
diff --git a/source/java/api.txt b/source/java/api.txt
@@ -59,8 +59,8 @@ The Java API provides a ``JavaSparkContext`` that takes a
        SparkSession spark = SparkSession.builder()
          .master("local")
          .appName("MongoSparkConnectorIntro")
-         .config("spark.mongodb.input.uri", "mongodb://127.0.0.1/test.myCollection")
-         .config("spark.mongodb.output.uri", "mongodb://127.0.0.1/test.myCollection")
+         .config("spark.mongodb.read.uri", "mongodb://127.0.0.1/test.myCollection")
+         .config("spark.mongodb.write.uri", "mongodb://127.0.0.1/test.myCollection")
          .getOrCreate();
          
        // Create a JavaSparkContext using the SparkSession's SparkContext object
@@ -73,12 +73,12 @@ The Java API provides a ``JavaSparkContext`` that takes a
      }
    }
    
-- The :ref:`spark.mongodb.input.uri <spark-input-conf>` specifies the
+- The :ref:`spark.mongodb.read.uri <spark-input-conf>` specifies the
   MongoDB server  address(``127.0.0.1``), the database to connect
   (``test``), and the collection (``myCollection``) from which to read
   data, and the read preference.
 
-- The :ref:`spark.mongodb.output.uri <spark-output-conf>` specifies the
+- The :ref:`spark.mongodb.write.uri <spark-output-conf>` specifies the
   MongoDB server address(``127.0.0.1``), the database to connect
   (``test``), and the collection (``myCollection``) to which to write
   data.
diff --git a/source/java/datasets-and-sql.txt b/source/java/datasets-and-sql.txt
@@ -46,8 +46,8 @@ Consider a collection named ``characters``:
        SparkSession spark = SparkSession.builder()
          .master("local")
          .appName("MongoSparkConnectorIntro")
-         .config("spark.mongodb.input.uri", "mongodb://127.0.0.1/test.myCollection")
-         .config("spark.mongodb.output.uri", "mongodb://127.0.0.1/test.myCollection")
+         .config("spark.mongodb.read.uri", "mongodb://127.0.0.1/test.myCollection")
+         .config("spark.mongodb.write.uri", "mongodb://127.0.0.1/test.myCollection")
          .getOrCreate();
          
        // Create a JavaSparkContext using the SparkSession's SparkContext object
diff --git a/source/java/read-from-mongodb.txt b/source/java/read-from-mongodb.txt
@@ -21,8 +21,8 @@ saved as part of the :ref:`write example <java-write>`.
        SparkSession spark = SparkSession.builder()
          .master("local")
          .appName("MongoSparkConnectorIntro")
-         .config("spark.mongodb.input.uri", "mongodb://127.0.0.1/test.myCollection")
-         .config("spark.mongodb.output.uri", "mongodb://127.0.0.1/test.myCollection")
+         .config("spark.mongodb.read.uri", "mongodb://127.0.0.1/test.myCollection")
+         .config("spark.mongodb.write.uri", "mongodb://127.0.0.1/test.myCollection")
          .getOrCreate();
          
        // Create a JavaSparkContext using the SparkSession's SparkContext object
@@ -69,8 +69,8 @@ Specify a ``ReadConfig``
        SparkSession spark = SparkSession.builder()
          .master("local")
          .appName("MongoSparkConnectorIntro")
-         .config("spark.mongodb.input.uri", "mongodb://127.0.0.1/test.myCollection")
-         .config("spark.mongodb.output.uri", "mongodb://127.0.0.1/test.myCollection")
+         .config("spark.mongodb.read.uri", "mongodb://127.0.0.1/test.myCollection")
+         .config("spark.mongodb.write.uri", "mongodb://127.0.0.1/test.myCollection")
          .getOrCreate();
          
        // Create a JavaSparkContext using the SparkSession's SparkContext object
diff --git a/source/java/write-to-mongodb.txt b/source/java/write-to-mongodb.txt
@@ -21,8 +21,8 @@
        SparkSession spark = SparkSession.builder()
        	 .master("local")
        	 .appName("MongoSparkConnectorIntro")
-       	 .config("spark.mongodb.input.uri", "mongodb://127.0.0.1/test.myCollection")
-       	 .config("spark.mongodb.output.uri", "mongodb://127.0.0.1/test.myCollection")
+       	 .config("spark.mongodb.read.uri", "mongodb://127.0.0.1/test.myCollection")
+       	 .config("spark.mongodb.write.uri", "mongodb://127.0.0.1/test.myCollection")
        	 .getOrCreate();
 	     
        JavaSparkContext jsc = new JavaSparkContext(spark.sparkContext());
@@ -77,8 +77,8 @@ Using a ``WriteConfig``
        SparkSession spark = SparkSession.builder()
          .master("local")
          .appName("MongoSparkConnectorIntro")
-         .config("spark.mongodb.input.uri", "mongodb://127.0.0.1/test.myCollection")
-         .config("spark.mongodb.output.uri", "mongodb://127.0.0.1/test.myCollection")
+         .config("spark.mongodb.read.uri", "mongodb://127.0.0.1/test.myCollection")
+         .config("spark.mongodb.write.uri", "mongodb://127.0.0.1/test.myCollection")
          .getOrCreate();
 	   
        JavaSparkContext jsc = new JavaSparkContext(spark.sparkContext());
diff --git a/source/python/api.txt b/source/python/api.txt
@@ -23,8 +23,8 @@ Create a ``SparkSession`` Object
    ``spark`` by default. In a standalone Python application, you need
    to create your ``SparkSession`` object explicitly, as show below.
    
-If you specified the ``spark.mongodb.input.uri``
-and ``spark.mongodb.output.uri`` configuration options when you
+If you specified the ``spark.mongodb.read.uri``
+and ``spark.mongodb.write.uri`` configuration options when you
 started ``pyspark``, the default ``SparkSession`` object uses them.
 If you'd rather create your own ``SparkSession`` object from within
 ``pyspark``, you can use ``SparkSession.builder`` and specify different
@@ -37,8 +37,8 @@ configuration options.
    my_spark = SparkSession \
        .builder \
        .appName("myApp") \
-       .config("spark.mongodb.input.uri", "mongodb://127.0.0.1/test.coll") \
-       .config("spark.mongodb.output.uri", "mongodb://127.0.0.1/test.coll") \
+       .config("spark.mongodb.read.uri", "mongodb://127.0.0.1/test.coll") \
+       .config("spark.mongodb.write.uri", "mongodb://127.0.0.1/test.coll") \
        .getOrCreate()
 
 You can use a ``SparkSession`` object to write data to MongoDB, read
diff --git a/source/python/read-from-mongodb.txt b/source/python/read-from-mongodb.txt
@@ -1,6 +1,6 @@
 You can create a Spark DataFrame to hold data from the MongoDB
 collection specified in the
-:ref:`spark.mongodb.input.uri<pyspark-shell>` option which your
+:ref:`spark.mongodb.read.uri<pyspark-shell>` option which your
 ``SparkSession`` option is using.
 
 .. include:: /includes/example-load-dataframe.rst
diff --git a/source/python/write-to-mongodb.txt b/source/python/write-to-mongodb.txt
@@ -9,15 +9,15 @@ a list of tuples containing names and ages, and a list of column names:
       ("Dwalin", 169), ("Oin", 167), ("Gloin", 158), ("Fili", 82), ("Bombur", None)], ["name", "age"])
 
 Write the ``people`` DataFrame to the MongoDB database and collection
-specified in the :ref:`spark.mongodb.output.uri<pyspark-shell>` option
+specified in the :ref:`spark.mongodb.write.uri<pyspark-shell>` option
 by using the ``write`` method:
 
 .. code-block:: python
 
    people.write.format("mongodb").mode("append").save()
 
 The above operation writes to the MongoDB database and collection
-specified in the :ref:`spark.mongodb.output.uri<pyspark-shell>` option
+specified in the :ref:`spark.mongodb.write.uri<pyspark-shell>` option
 when you connect to the ``pyspark`` shell.
 
 To read the contents of the DataFrame, use the ``show()`` method.
diff --git a/source/release-notes.txt b/source/release-notes.txt
@@ -4,59 +4,9 @@ Release Notes
 
 .. default-domain:: mongodb
 
-MongoDB Connector for Spark 3.0.0
----------------------------------
+MongoDB Connector for Spark 10.0.0
+----------------------------------
 
-*Released on August 17, 2020*
+*Released on March 31, 2022*
 
-- Support for Spark 3.0.0.
-- Updated ``mongodb-driver-sync`` version to 4.0.4
-- Updated dependency to ``mongodb-driver-sync``
-
-MongoDB Connector for Spark 2.4.2
----------------------------------
-
-*Released on June 10, 2020*
-
-- Updated `MongoDB Java Driver
-  <http://mongodb.github.io/mongo-java-driver/3.12/>`__
-  to 3.12.5.
-- Don't use Java SPI for the data source internally.
-- Fix ``BsonOrdering`` bug for strings of different lengths.
-
-MongoDB Connector for Spark `2.4.1`_
-------------------------------------
-
-*Released on June 6, 2019*
-
-- Ensures nullable fields or container types accept ``null`` values.
-- Added ``ReadConfig.batchSize`` property. For more information, see
-  :ref:`spark-input-conf`.
-- Renamed system property ``spark.mongodb.keep_alive_ms`` to
-  ``mongodb.keep_alive_ms``.
-- Added ``MongoDriverInformation`` to the default ``MongoClient``.
-- Updated to latest Java driver (3.10.+)
-- Updated ``PartitionerHelper.matchQuery`` to no longer include ``$ne``/``$exists``
-  checks.
-- Added logging support for partitioners and their queries.
-- Added ``WriteConfig.extendedBsonTypes`` setting so users can disable
-  extended BSON types when writing. For more information, see
-  :ref:`spark-output-conf`.
-- Added Java spi can now use short form: ``spark.read.format("mongo")``.
-- ``spark.read.format("mongo")`` can be used in place of
-  ``spark.read.format("com.mongodb.spark.sql")`` and
-  ``spark.read.format("com.mongodb.spark.sql.DefaultSource")``.
-
-MongoDB Connector for Spark `2.4.0`_
-------------------------------------
-
-*Released on December 7, 2018*
-
-- Support Spark 2.4.0. Updated Spark dependency to 2.4.0.
-- Ensure ``WriteConfig.ordered`` is applied to write operations.
-- Updated Mongo Java Driver to 3.9.0.
-- Added Scala 2.12 support.
-- Fixed ``MongoSpark.toDF()`` to use the provided ``MongoConnector``.
-
-.. _2.4.1: https://github.com/mongodb/mongo-spark/compare/2.4.0...r2.4.1
-.. _2.4.0: https://github.com/mongodb/mongo-spark/compare/2.3.x...r2.4.0
+- Support for Spark Structured Streaming.
diff --git a/source/scala/api.txt b/source/scala/api.txt
@@ -73,8 +73,8 @@ Configuration
        val spark = SparkSession.builder()
          .master("local")
          .appName("MongoSparkConnectorIntro")
-         .config("spark.mongodb.input.uri", "mongodb://127.0.0.1/test.myCollection")
-         .config("spark.mongodb.output.uri", "mongodb://127.0.0.1/test.myCollection")
+         .config("spark.mongodb.read.uri", "mongodb://127.0.0.1/test.myCollection")
+         .config("spark.mongodb.write.uri", "mongodb://127.0.0.1/test.myCollection")
          .getOrCreate()
          
      }
diff --git a/source/scala/datasets-and-sql.txt b/source/scala/datasets-and-sql.txt
@@ -27,8 +27,8 @@ Insert the following documents to the ``characters`` collection:
        val sparkSession = SparkSession.builder()
          .master("local")
          .appName("MongoSparkConnectorIntro")
-         .config("spark.mongodb.input.uri", "mongodb://127.0.0.1/test.characters")
-         .config("spark.mongodb.output.uri", "mongodb://127.0.0.1/test.characters")
+         .config("spark.mongodb.read.uri", "mongodb://127.0.0.1/test.characters")
+         .config("spark.mongodb.write.uri", "mongodb://127.0.0.1/test.characters")
          .getOrCreate()
 
        import com.mongodb.spark._
diff --git a/source/structured-streaming.txt b/source/structured-streaming.txt
