Add 4.0.0-preview2 news (#561)

![Screenshot 2024-09-26 at 08 48 34](https://github.com/user-attachments/assets/c021890f-049e-4da4-bead-a44c1129d5fb) ![Screenshot 2024-09-26 at 08 49 35](https://github.com/user-attachments/assets/bc6e9a98-6439-4d39-b680-4cd70f44f7e2)
apache · Sep 26, 2024 · 4c9d5c1 · 4c9d5c1
1 parent 21eca53
commit 4c9d5c1
Show file tree

Hide file tree

Showing 252 changed files with 1,037 additions and 742 deletions.
diff --git a/documentation.md b/documentation.md
@@ -85,6 +85,7 @@ navigation:
 <p>Documentation for preview releases:</p>
 
 <ul>
+  <li><a href="{{site.baseurl}}/docs/4.0.0-preview2/">Spark 4.0.0 preview2</a></li>
   <li><a href="{{site.baseurl}}/docs/4.0.0-preview1/">Spark 4.0.0 preview1</a></li>
   <li><a href="{{site.baseurl}}/docs/3.0.0-preview2/">Spark 3.0.0 preview2</a></li>
   <li><a href="{{site.baseurl}}/docs/3.0.0-preview/">Spark 3.0.0 preview</a></li>

diff --git a/news/_posts/2024-09-26-spark-4.0.0-preview2.md b/news/_posts/2024-09-26-spark-4.0.0-preview2.md
@@ -0,0 +1,28 @@
+---
+layout: post
+title: Preview release of Spark 4.0
+categories:
+- News
+tags: []
+status: publish
+type: post
+published: true
+meta:
+  _edit_last: '4'
+  _wpas_done_all: '1'
+---
+To enable wide-scale community testing of the upcoming Spark 4.0 release, the Apache Spark community has posted a
+<a href="https://archive.apache.org/dist/spark/spark-4.0.0-preview2/">Spark 4.0.0 preview2 release</a>.
+This preview is not a stable release in terms of either API or functionality, but it is meant to give the community early
+access to try the code that will become Spark 4.0. If you would like to test the release,
+please <a href="https://archive.apache.org/dist/spark/spark-4.0.0-preview2">download</a> it, and send feedback using either
+<a href="https://spark.apache.org/community.html">mailing lists</a> or
+<a href="https://issues.apache.org/jira/browse/SPARK/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel">JIRA</a>.
+The documentation is available at the <a href="https://spark.apache.org/docs/4.0.0-preview2/">link</a>.
+
+There are a lot of exciting new features added to Spark 4.0, including ANSI mode by default, Python data source, polymorphic
+Python UDTF, string collation support, new VARIANT data type, streaming state store data source, structured logging, Java 21 support, Java 17 by default
+and many more.
+
+We'd like to thank our contributors and users for their contributions and early feedback to this release. This release would not have
+been possible without you.
diff --git a/site/404.html b/site/404.html
@@ -179,6 +179,9 @@ <h2 id="oops">Oops!</h2>
         <h5>Latest News</h5>
         <ul class="list-unstyled">
 
+          <li><a href="/news/spark-4.0.0-preview2.html">Preview release of Spark 4.0</a>
+            <span class="small">(Sep 26, 2024)</span></li>
+
           <li><a href="/news/spark-3-5-3-released.html">Spark 3.5.3 released</a>
             <span class="small">(Sep 24, 2024)</span></li>
 
@@ -188,9 +191,6 @@ <h5>Latest News</h5>
           <li><a href="/news/spark-4.0.0-preview1.html">Preview release of Spark 4.0</a>
             <span class="small">(Jun 03, 2024)</span></li>
 
-          <li><a href="/news/spark-3-4-3-released.html">Spark 3.4.3 released</a>
-            <span class="small">(Apr 18, 2024)</span></li>
-
         </ul>
         <p class="small" style="text-align: right;"><a href="/news/index.html">Archive</a></p>
       </div>

diff --git a/site/committers.html b/site/committers.html
@@ -714,6 +714,9 @@ <h3>Policy on backporting bug fixes</h3>
         <h5>Latest News</h5>
         <ul class="list-unstyled">
 
+          <li><a href="/news/spark-4.0.0-preview2.html">Preview release of Spark 4.0</a>
+            <span class="small">(Sep 26, 2024)</span></li>
+
           <li><a href="/news/spark-3-5-3-released.html">Spark 3.5.3 released</a>
             <span class="small">(Sep 24, 2024)</span></li>
 
@@ -723,9 +726,6 @@ <h5>Latest News</h5>
           <li><a href="/news/spark-4.0.0-preview1.html">Preview release of Spark 4.0</a>
             <span class="small">(Jun 03, 2024)</span></li>
 
-          <li><a href="/news/spark-3-4-3-released.html">Spark 3.4.3 released</a>
-            <span class="small">(Apr 18, 2024)</span></li>
-
         </ul>
         <p class="small" style="text-align: right;"><a href="/news/index.html">Archive</a></p>
       </div>

diff --git a/site/community.html b/site/community.html
@@ -379,6 +379,9 @@ <h3>Powered by</h3>
         <h5>Latest News</h5>
         <ul class="list-unstyled">
 
+          <li><a href="/news/spark-4.0.0-preview2.html">Preview release of Spark 4.0</a>
+            <span class="small">(Sep 26, 2024)</span></li>
+
           <li><a href="/news/spark-3-5-3-released.html">Spark 3.5.3 released</a>
             <span class="small">(Sep 24, 2024)</span></li>
 
@@ -388,9 +391,6 @@ <h5>Latest News</h5>
           <li><a href="/news/spark-4.0.0-preview1.html">Preview release of Spark 4.0</a>
             <span class="small">(Jun 03, 2024)</span></li>
 
-          <li><a href="/news/spark-3-4-3-released.html">Spark 3.4.3 released</a>
-            <span class="small">(Apr 18, 2024)</span></li>
-
         </ul>
         <p class="small" style="text-align: right;"><a href="/news/index.html">Archive</a></p>
       </div>

diff --git a/site/contributing.html b/site/contributing.html
@@ -722,6 +722,9 @@ <h2>Code of conduct</h2>
         <h5>Latest News</h5>
         <ul class="list-unstyled">
 
+          <li><a href="/news/spark-4.0.0-preview2.html">Preview release of Spark 4.0</a>
+            <span class="small">(Sep 26, 2024)</span></li>
+
           <li><a href="/news/spark-3-5-3-released.html">Spark 3.5.3 released</a>
             <span class="small">(Sep 24, 2024)</span></li>
 
@@ -731,9 +734,6 @@ <h5>Latest News</h5>
           <li><a href="/news/spark-4.0.0-preview1.html">Preview release of Spark 4.0</a>
             <span class="small">(Jun 03, 2024)</span></li>
 
-          <li><a href="/news/spark-3-4-3-released.html">Spark 3.4.3 released</a>
-            <span class="small">(Apr 18, 2024)</span></li>
-
         </ul>
         <p class="small" style="text-align: right;"><a href="/news/index.html">Archive</a></p>
       </div>

diff --git a/site/developer-tools.html b/site/developer-tools.html
@@ -723,6 +723,9 @@ <h2>Generative tooling usage</h2>
         <h5>Latest News</h5>
         <ul class="list-unstyled">
 
+          <li><a href="/news/spark-4.0.0-preview2.html">Preview release of Spark 4.0</a>
+            <span class="small">(Sep 26, 2024)</span></li>
+
           <li><a href="/news/spark-3-5-3-released.html">Spark 3.5.3 released</a>
             <span class="small">(Sep 24, 2024)</span></li>
 
@@ -732,9 +735,6 @@ <h5>Latest News</h5>
           <li><a href="/news/spark-4.0.0-preview1.html">Preview release of Spark 4.0</a>
             <span class="small">(Jun 03, 2024)</span></li>
 
-          <li><a href="/news/spark-3-4-3-released.html">Spark 3.4.3 released</a>
-            <span class="small">(Apr 18, 2024)</span></li>
-
         </ul>
         <p class="small" style="text-align: right;"><a href="/news/index.html">Archive</a></p>
       </div>

diff --git a/site/docs/preview b/site/docs/preview
@@ -1 +1 @@
-4.0.0-preview1
+4.0.0-preview2
diff --git a/site/documentation.html b/site/documentation.html
@@ -231,6 +231,7 @@ <h2><span class="text-capitalize">Apache Spark<span class="tm">&trade;</span></s
 <p>Documentation for preview releases:</p>
 
 <ul>
+  <li><a href="/docs/4.0.0-preview2/">Spark 4.0.0 preview2</a></li>
   <li><a href="/docs/4.0.0-preview1/">Spark 4.0.0 preview1</a></li>
   <li><a href="/docs/3.0.0-preview2/">Spark 3.0.0 preview2</a></li>
   <li><a href="/docs/3.0.0-preview/">Spark 3.0.0 preview</a></li>
@@ -379,6 +380,9 @@ <h3>Research Papers</h3>
         <h5>Latest News</h5>
         <ul class="list-unstyled">
 
+          <li><a href="/news/spark-4.0.0-preview2.html">Preview release of Spark 4.0</a>
+            <span class="small">(Sep 26, 2024)</span></li>
+
           <li><a href="/news/spark-3-5-3-released.html">Spark 3.5.3 released</a>
             <span class="small">(Sep 24, 2024)</span></li>
 
@@ -388,9 +392,6 @@ <h5>Latest News</h5>
           <li><a href="/news/spark-4.0.0-preview1.html">Preview release of Spark 4.0</a>
             <span class="small">(Jun 03, 2024)</span></li>
 
-          <li><a href="/news/spark-3-4-3-released.html">Spark 3.4.3 released</a>
-            <span class="small">(Apr 18, 2024)</span></li>
-
         </ul>
         <p class="small" style="text-align: right;"><a href="/news/index.html">Archive</a></p>
       </div>

diff --git a/site/downloads.html b/site/downloads.html
@@ -219,6 +219,9 @@ <h3 id="archived-releases">Archived releases</h3>
         <h5>Latest News</h5>
         <ul class="list-unstyled">
 
+          <li><a href="/news/spark-4.0.0-preview2.html">Preview release of Spark 4.0</a>
+            <span class="small">(Sep 26, 2024)</span></li>
+
           <li><a href="/news/spark-3-5-3-released.html">Spark 3.5.3 released</a>
             <span class="small">(Sep 24, 2024)</span></li>
 
@@ -228,9 +231,6 @@ <h5>Latest News</h5>
           <li><a href="/news/spark-4.0.0-preview1.html">Preview release of Spark 4.0</a>
             <span class="small">(Jun 03, 2024)</span></li>
 
-          <li><a href="/news/spark-3-4-3-released.html">Spark 3.4.3 released</a>
-            <span class="small">(Apr 18, 2024)</span></li>
-
         </ul>
         <p class="small" style="text-align: right;"><a href="/news/index.html">Archive</a></p>
       </div>

diff --git a/site/error-message-guidelines.html b/site/error-message-guidelines.html
@@ -556,6 +556,9 @@ <h4 id="wording-guide">Wording guide</h4>
         <h5>Latest News</h5>
         <ul class="list-unstyled">
 
+          <li><a href="/news/spark-4.0.0-preview2.html">Preview release of Spark 4.0</a>
+            <span class="small">(Sep 26, 2024)</span></li>
+
           <li><a href="/news/spark-3-5-3-released.html">Spark 3.5.3 released</a>
             <span class="small">(Sep 24, 2024)</span></li>
 
@@ -565,9 +568,6 @@ <h5>Latest News</h5>
           <li><a href="/news/spark-4.0.0-preview1.html">Preview release of Spark 4.0</a>
             <span class="small">(Jun 03, 2024)</span></li>
 
-          <li><a href="/news/spark-3-4-3-released.html">Spark 3.4.3 released</a>
-            <span class="small">(Apr 18, 2024)</span></li>
-
         </ul>
         <p class="small" style="text-align: right;"><a href="/news/index.html">Archive</a></p>
       </div>

diff --git a/site/examples.html b/site/examples.html
@@ -504,6 +504,9 @@ <h2>Additional examples</h2>
         <h5>Latest News</h5>
         <ul class="list-unstyled">
 
+          <li><a href="/news/spark-4.0.0-preview2.html">Preview release of Spark 4.0</a>
+            <span class="small">(Sep 26, 2024)</span></li>
+
           <li><a href="/news/spark-3-5-3-released.html">Spark 3.5.3 released</a>
             <span class="small">(Sep 24, 2024)</span></li>
 
@@ -513,9 +516,6 @@ <h5>Latest News</h5>
           <li><a href="/news/spark-4.0.0-preview1.html">Preview release of Spark 4.0</a>
             <span class="small">(Jun 03, 2024)</span></li>
 
-          <li><a href="/news/spark-3-4-3-released.html">Spark 3.4.3 released</a>
-            <span class="small">(Apr 18, 2024)</span></li>
-
         </ul>
         <p class="small" style="text-align: right;"><a href="/news/index.html">Archive</a></p>
       </div>

diff --git a/site/faq.html b/site/faq.html
@@ -226,6 +226,9 @@ <h2><span class="text-capitalize">Apache Spark<span class="tm">&trade;</span></s
         <h5>Latest News</h5>
         <ul class="list-unstyled">
 
+          <li><a href="/news/spark-4.0.0-preview2.html">Preview release of Spark 4.0</a>
+            <span class="small">(Sep 26, 2024)</span></li>
+
           <li><a href="/news/spark-3-5-3-released.html">Spark 3.5.3 released</a>
             <span class="small">(Sep 24, 2024)</span></li>
 
@@ -235,9 +238,6 @@ <h5>Latest News</h5>
           <li><a href="/news/spark-4.0.0-preview1.html">Preview release of Spark 4.0</a>
             <span class="small">(Jun 03, 2024)</span></li>
 
-          <li><a href="/news/spark-3-4-3-released.html">Spark 3.4.3 released</a>
-            <span class="small">(Apr 18, 2024)</span></li>
-
         </ul>
         <p class="small" style="text-align: right;"><a href="/news/index.html">Archive</a></p>
       </div>

diff --git a/site/graphx/index.html b/site/graphx/index.html
@@ -271,6 +271,9 @@ <h3>Getting started</h3>
         <h5>Latest News</h5>
         <ul class="list-unstyled">
 
+          <li><a href="/news/spark-4.0.0-preview2.html">Preview release of Spark 4.0</a>
+            <span class="small">(Sep 26, 2024)</span></li>
+
           <li><a href="/news/spark-3-5-3-released.html">Spark 3.5.3 released</a>
             <span class="small">(Sep 24, 2024)</span></li>
 
@@ -280,9 +283,6 @@ <h5>Latest News</h5>
           <li><a href="/news/spark-4.0.0-preview1.html">Preview release of Spark 4.0</a>
             <span class="small">(Jun 03, 2024)</span></li>
 
-          <li><a href="/news/spark-3-4-3-released.html">Spark 3.4.3 released</a>
-            <span class="small">(Apr 18, 2024)</span></li>
-
         </ul>
         <p class="small" style="text-align: right;"><a href="/news/index.html">Archive</a></p>
       </div>

diff --git a/site/history.html b/site/history.html
@@ -180,6 +180,9 @@ <h2><span class="text-capitalize">Apache Spark<span class="tm">&trade;</span></s
         <h5>Latest News</h5>
         <ul class="list-unstyled">
 
+          <li><a href="/news/spark-4.0.0-preview2.html">Preview release of Spark 4.0</a>
+            <span class="small">(Sep 26, 2024)</span></li>
+
           <li><a href="/news/spark-3-5-3-released.html">Spark 3.5.3 released</a>
             <span class="small">(Sep 24, 2024)</span></li>
 
@@ -189,9 +192,6 @@ <h5>Latest News</h5>
           <li><a href="/news/spark-4.0.0-preview1.html">Preview release of Spark 4.0</a>
             <span class="small">(Jun 03, 2024)</span></li>
 
-          <li><a href="/news/spark-3-4-3-released.html">Spark 3.4.3 released</a>
-            <span class="small">(Apr 18, 2024)</span></li>
-
         </ul>
         <p class="small" style="text-align: right;"><a href="/news/index.html">Archive</a></p>
       </div>

diff --git a/site/improvement-proposals.html b/site/improvement-proposals.html
@@ -252,6 +252,9 @@ <h4>Implementing an SPIP</h4>
         <h5>Latest News</h5>
         <ul class="list-unstyled">
 
+          <li><a href="/news/spark-4.0.0-preview2.html">Preview release of Spark 4.0</a>
+            <span class="small">(Sep 26, 2024)</span></li>
+
           <li><a href="/news/spark-3-5-3-released.html">Spark 3.5.3 released</a>
             <span class="small">(Sep 24, 2024)</span></li>
 
@@ -261,9 +264,6 @@ <h5>Latest News</h5>
           <li><a href="/news/spark-4.0.0-preview1.html">Preview release of Spark 4.0</a>
             <span class="small">(Jun 03, 2024)</span></li>
 
-          <li><a href="/news/spark-3-4-3-released.html">Spark 3.4.3 released</a>
-            <span class="small">(Apr 18, 2024)</span></li>
-
         </ul>
         <p class="small" style="text-align: right;"><a href="/news/index.html">Archive</a></p>
       </div>

diff --git a/site/mailing-lists.html b/site/mailing-lists.html
@@ -164,6 +164,9 @@
         <h5>Latest News</h5>
         <ul class="list-unstyled">
 
+          <li><a href="/news/spark-4.0.0-preview2.html">Preview release of Spark 4.0</a>
+            <span class="small">(Sep 26, 2024)</span></li>
+
           <li><a href="/news/spark-3-5-3-released.html">Spark 3.5.3 released</a>
             <span class="small">(Sep 24, 2024)</span></li>
 
@@ -173,9 +176,6 @@ <h5>Latest News</h5>
           <li><a href="/news/spark-4.0.0-preview1.html">Preview release of Spark 4.0</a>
             <span class="small">(Jun 03, 2024)</span></li>
 
-          <li><a href="/news/spark-3-4-3-released.html">Spark 3.4.3 released</a>
-            <span class="small">(Apr 18, 2024)</span></li>
-
         </ul>
         <p class="small" style="text-align: right;"><a href="/news/index.html">Archive</a></p>
       </div>

diff --git a/site/mllib/index.html b/site/mllib/index.html
@@ -317,6 +317,9 @@ <h3>Getting started</h3>
         <h5>Latest News</h5>
         <ul class="list-unstyled">
 
+          <li><a href="/news/spark-4.0.0-preview2.html">Preview release of Spark 4.0</a>
+            <span class="small">(Sep 26, 2024)</span></li>
+
           <li><a href="/news/spark-3-5-3-released.html">Spark 3.5.3 released</a>
             <span class="small">(Sep 24, 2024)</span></li>
 
@@ -326,9 +329,6 @@ <h5>Latest News</h5>
           <li><a href="/news/spark-4.0.0-preview1.html">Preview release of Spark 4.0</a>
             <span class="small">(Jun 03, 2024)</span></li>
 
-          <li><a href="/news/spark-3-4-3-released.html">Spark 3.4.3 released</a>
-            <span class="small">(Apr 18, 2024)</span></li>
-
         </ul>
         <p class="small" style="text-align: right;"><a href="/news/index.html">Archive</a></p>
       </div>

diff --git a/site/news/3-1-3-released.html b/site/news/3-1-3-released.html
@@ -170,6 +170,9 @@ <h2>Spark 3.1.3 released</h2>
         <h5>Latest News</h5>
         <ul class="list-unstyled">
 
+          <li><a href="/news/spark-4.0.0-preview2.html">Preview release of Spark 4.0</a>
+            <span class="small">(Sep 26, 2024)</span></li>
+
           <li><a href="/news/spark-3-5-3-released.html">Spark 3.5.3 released</a>
             <span class="small">(Sep 24, 2024)</span></li>
 
@@ -179,9 +182,6 @@ <h5>Latest News</h5>
           <li><a href="/news/spark-4.0.0-preview1.html">Preview release of Spark 4.0</a>
             <span class="small">(Jun 03, 2024)</span></li>
 
-          <li><a href="/news/spark-3-4-3-released.html">Spark 3.4.3 released</a>
-            <span class="small">(Apr 18, 2024)</span></li>
-
         </ul>
         <p class="small" style="text-align: right;"><a href="/news/index.html">Archive</a></p>
       </div>

diff --git a/site/news/amp-camp-2013-registration-ope.html b/site/news/amp-camp-2013-registration-ope.html
@@ -170,6 +170,9 @@ <h2>Registration open for AMP Camp training camp in Berkeley</h2>
         <h5>Latest News</h5>
         <ul class="list-unstyled">
 
+          <li><a href="/news/spark-4.0.0-preview2.html">Preview release of Spark 4.0</a>
+            <span class="small">(Sep 26, 2024)</span></li>
+
           <li><a href="/news/spark-3-5-3-released.html">Spark 3.5.3 released</a>
             <span class="small">(Sep 24, 2024)</span></li>
 
@@ -179,9 +182,6 @@ <h5>Latest News</h5>
           <li><a href="/news/spark-4.0.0-preview1.html">Preview release of Spark 4.0</a>
             <span class="small">(Jun 03, 2024)</span></li>
 
-          <li><a href="/news/spark-3-4-3-released.html">Spark 3.4.3 released</a>
-            <span class="small">(Apr 18, 2024)</span></li>
-
         </ul>
         <p class="small" style="text-align: right;"><a href="/news/index.html">Archive</a></p>
       </div>