@@ -25,12 +25,12 @@ weight: 400
2525Apache Polaris now provides Catalog support for Generic Tables (non-iceberg tables), please check out
2626the [ Catalog API Spec] ({{% ref "polaris-catalog-service" %}}) for Generic Table API specs.
2727
28- Along with the Generic Table Catalog support, Polaris is also releasing a Spark Client, which help
29- providing an end-to-end solution for Apache Spark to manage Delta tables using Polaris.
28+ Along with the Generic Table Catalog support, Polaris is also releasing a Spark Client, which helps to
29+ provide an end-to-end solution for Apache Spark to manage Delta tables using Polaris.
3030
3131Note the Polaris Spark Client is able to handle both Iceberg and Delta tables, not just Delta.
3232
33- This pages documents how to build and use the Apache Polaris Spark Client before formal release .
33+ This page documents how to build and use the Polaris Spark Client directly with the source repo .
3434
3535## Prerequisite
36361 . Check out the polaris repo
@@ -59,14 +59,14 @@ The getting-started will start two containers:
5959
6060The notebook ` SparkPolaris.ipynb ` provided under ` plugins/spark/v3.5/getting-started/notebooks ` provides examples
6161with basic commands, includes:
62- 1 ) Connect to Polaris using Python client to create Catalog and Roles
62+ 1 ) Connect to Polaris using Python client to create a Catalog and Roles
63632 ) Start Spark session using the Polaris Spark Client
64643 ) Using Spark to perform table operations for both Delta and Iceberg
6565
6666## Start Spark against a deployed Polaris Service
67- If you want to start Spark with a deployed Polaris service, you can follow the following instructions.
67+ If you want to start Spark with a deployed Polaris service, you can follow the instructions below .
6868
69- Before start, Make sure the service deployed is up-to-date, and Spark 3.5 with at least version 3.5.3 is installed.
69+ Before starting, make sure the service deployed is up-to-date, and that Spark 3.5 with at least version 3.5.3 is installed.
7070
7171### Build Spark Client Jars
7272The polaris-spark project provides a task createPolarisSparkJar to help building jars for the Polaris Spark client,
@@ -83,7 +83,7 @@ cd ~/polaris
8383If you want to build a Scala 2.13 compatible jar, you can use the following command:
8484- ` ./gradlew :polaris-spark-3.5_2.13:createPolarisSparkJar `
8585
86- The result jar is located at plugins/spark/v3.5/build/<scala_version>/libs after the build. You can also copy the
86+ The result jar is located at ` plugins/spark/v3.5/build/<scala_version>/libs ` after the build. You can also copy the
8787corresponding jar to any location your Spark will have access.
8888
8989### Connecting with Spark Using the built jar
@@ -111,7 +111,7 @@ the same name. Replace the `polaris-service-uri`, `client-id` and `client-secret
111111[ Quick Start] ({{% ref "../0.9.0/quickstart" %}}) for more details about those fields.
112112
113113Or you can create a spark session start the connection, following is an example with pyspark
114- ``` shell
114+ ``` python
115115from pyspark.sql import SparkSession
116116
117117spark = SparkSession.builder
@@ -133,7 +133,7 @@ Similar as the CLI command, make sure the corresponding fields are replaced corr
133133### Create tables with Spark
134134After the spark is started, you can use it to create and access Iceberg and Delta table like what you are doing before,
135135for example:
136- ``` shell
136+ ``` python
137137spark.sql(" USE polaris" )
138138spark.sql(" CREATE NAMESPACE IF NOT EXISTS DELTA_NS" )
139139spark.sql(" CREATE NAMESPACE IF NOT EXISTS DELTA_NS.PUBLIC" )
@@ -151,5 +151,6 @@ The Polaris Spark client has the following functionality limitations:
1511512 ) Create a Delta table without explicit location is not supported.
1521523 ) Rename a Delta table is not supported.
1531534 ) ALTER TABLE ... SET LOCATION/SET FILEFORMAT/ADD PARTITION is not supported for DELTA table.
154- 5 ) For other non-iceberg tables like csv, there is no specific guarantee provided today.
155- 6 ) Role-based RBAC support for Delta table write is not available. Create, Drop and List RBAC support is available.
154+ 5 ) For other non-Iceberg tables like csv, there is no specific guarantee provided today.
155+ 6 ) TABLE_WRITE_DATA privileges is not supported for Delta Table.
156+ 7 ) Credential Vending is not supported for Delta Table.
0 commit comments