diff --git a/docs/_config.yml b/docs/_config.yml index 2f85659649..0013b8157e 100644 --- a/docs/_config.yml +++ b/docs/_config.yml @@ -15,7 +15,7 @@ # limitations under the License. # -title: Firestorm homepage +title: Uniffle homepage baseurl: "" url: "" diff --git a/docs/client_guide.md b/docs/client_guide.md index ba26f3c571..63b2f1e36a 100644 --- a/docs/client_guide.md +++ b/docs/client_guide.md @@ -1,8 +1,8 @@ --- layout: page -displayTitle: Firestorm Shuffle Client Guide -title: Firestorm Shuffle Client Guide -description: Firestorm Shuffle Client Guide +displayTitle: Uniffle Shuffle Client Guide +title: Uniffle Shuffle Client Guide +description: Uniffle Shuffle Client Guide license: | Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with @@ -19,13 +19,13 @@ license: | See the License for the specific language governing permissions and limitations under the License. --- -# Firestorm Shuffle Client Guide +# Uniffle Shuffle Client Guide -Firestorm is designed as a unified shuffle engine for multiple computing frameworks, including Apache Spark and Apache Hadoop. -Firestorm has provided pluggable client plugins to enable remote shuffle in Spark and MapReduce. +Uniffle is designed as a unified shuffle engine for multiple computing frameworks, including Apache Spark and Apache Hadoop. +Uniffle has provided pluggable client plugins to enable remote shuffle in Spark and MapReduce. ## Deploy -This document will introduce how to deploy Firestorm client plugins with Spark and MapReduce. +This document will introduce how to deploy Uniffle client plugins with Spark and MapReduce. ### Deploy Spark Client Plugin @@ -35,7 +35,7 @@ This document will introduce how to deploy Firestorm client plugins with Spark a The jar for Spark3 is located in /jars/client/spark3/rss-client-XXXXX-shaded.jar -2. Update Spark conf to enable Firestorm, eg, +2. Update Spark conf to enable Uniffle, eg, ``` spark.shuffle.manager org.apache.spark.shuffle.RssShuffleManager @@ -45,7 +45,7 @@ This document will introduce how to deploy Firestorm client plugins with Spark a ### Support Spark Dynamic Allocation -To support spark dynamic allocation with Firestorm, spark code should be updated. +To support spark dynamic allocation with Uniffle, spark code should be updated. There are 2 patches for spark-2.4.6 and spark-3.1.2 in spark-patches folder for reference. After apply the patch and rebuild spark, add following configuration in spark conf to enable dynamic allocation: @@ -60,7 +60,7 @@ After apply the patch and rebuild spark, add following configuration in spark co The jar for MapReduce is located in /jars/client/mr/rss-client-mr-XXXXX-shaded.jar -2. Update MapReduce conf to enable Firestorm, eg, +2. Update MapReduce conf to enable Uniffle, eg, ``` -Dmapreduce.rss.coordinator.quorum=:19999,:19999 @@ -95,7 +95,7 @@ Notice: ### Adaptive Remote Shuffle Enabling -To select build-in shuffle or remote shuffle in a smart manner, Firestorm support adaptive enabling. +To select build-in shuffle or remote shuffle in a smart manner, Uniffle support adaptive enabling. The client should use `DelegationRssShuffleManager` and provide its unique so that the coordinator could distinguish whether it should enable remote shuffle. ``` @@ -110,12 +110,12 @@ Other configuration: |Property Name|Default|Description| |---|---|---| -|spark.rss.access.timeout.ms|10000|The timeout to access Firestorm coordinator| +|spark.rss.access.timeout.ms|10000|The timeout to access Uniffle coordinator| ### Client Quorum Setting -Firestorm supports client-side quorum protocol to tolerant shuffle server crash. +Uniffle supports client-side quorum protocol to tolerant shuffle server crash. This feature is client-side behaviour, in which shuffle writer sends each block to multiple servers, and shuffle readers could fetch block data from one of server. Since sending multiple replicas of blocks can reduce the shuffle performance and resource consumption, we designed it as an optional feature. diff --git a/docs/coordinator_guide.md b/docs/coordinator_guide.md index 29fca32144..274c875ecd 100644 --- a/docs/coordinator_guide.md +++ b/docs/coordinator_guide.md @@ -1,8 +1,8 @@ --- layout: page -displayTitle: Firestorm Coordinator Guide -title: Firestorm Coordinator Guide -description: Firestorm Coordinator Guide +displayTitle: Uniffle Coordinator Guide +title: Uniffle Coordinator Guide +description: Uniffle Coordinator Guide license: | Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with @@ -20,4 +20,4 @@ license: | limitations under the License. --- -# Firestorm Coordinator Guide +# Uniffle Coordinator Guide diff --git a/docs/index.md b/docs/index.md index afe73eb482..b0bb847941 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,8 +1,8 @@ --- layout: page -displayTitle: Firestorm Overview +displayTitle: Uniffle Overview title: Overview -description: Firestorm documentation homepage +description: Uniffle documentation homepage license: | Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with @@ -20,22 +20,22 @@ license: | limitations under the License. --- -Firestorm is a Remote Shuffle Service, and provides the capability for Apache Spark applications +Uniffle is a Remote Shuffle Service, and provides the capability for Apache Spark applications to store shuffle data on remote servers. ![Rss Architecture](asset/rss_architecture.png) -More advanced details for Firestorm users are available in the following: +More advanced details for Uniffle users are available in the following: -- [Firestorm Coordinator Guide](coordinator_guide.html) +- [Uniffle Coordinator Guide](coordinator_guide.html) -- [Firestorm Shuffle Server Guide](server_guide.html) +- [Uniffle Shuffle Server Guide](server_guide.html) -- [Firestorm Shuffle Client Guide](client_guide.html) +- [Uniffle Shuffle Client Guide](client_guide.html) -Here you can read API docs for Firestorm along with its submodules. +Here you can read API docs for Uniffle along with its submodules. - [Java API (Javadoc)](apidocs/index.html) diff --git a/docs/server_guide.md b/docs/server_guide.md index abd86f08e7..e959839f04 100644 --- a/docs/server_guide.md +++ b/docs/server_guide.md @@ -1,8 +1,8 @@ --- layout: page -displayTitle: Firestorm Shuffle Server Guide -title: Firestorm Shuffle Server Guide -description: Firestorm Shuffle Server Guide +displayTitle: Uniffle Shuffle Server Guide +title: Uniffle Shuffle Server Guide +description: Uniffle Shuffle Server Guide license: | Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with @@ -19,4 +19,4 @@ license: | See the License for the specific language governing permissions and limitations under the License. --- -# Firestorm Shuffle Server Guide +# Uniffle Shuffle Server Guide