Skip to content

Commit f448b6d

Browse files
committed
address feedback
1 parent bf27cae commit f448b6d

File tree

2 files changed

+15
-13
lines changed

2 files changed

+15
-13
lines changed

plugins/spark/README.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -97,5 +97,6 @@ Following describes the current functionality limitations of the Polaris Spark c
9797
2) Create a Delta table without explicit location is not supported.
9898
3) Rename a Delta table is not supported.
9999
4) ALTER TABLE ... SET LOCATION/SET FILEFORMAT/ADD PARTITION is not supported for DELTA table.
100-
5) For other non-iceberg tables like csv, there is no specific guarantee provided today.
101-
6) Role-based RBAC support for Delta table write is not available. Create, Drop and List RBAC support is available.
100+
5) For other non-Iceberg tables like csv, there is no specific guarantee provided today.
101+
6) TABLE_WRITE_DATA privilege is not supported for Delta Table.
102+
7) Credential Vending is not supported for Delta Table.

site/content/in-dev/unreleased/polaris-spark-client.md

Lines changed: 12 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -25,12 +25,12 @@ weight: 400
2525
Apache Polaris now provides Catalog support for Generic Tables (non-iceberg tables), please check out
2626
the [Catalog API Spec]({{% ref "polaris-catalog-service" %}}) for Generic Table API specs.
2727

28-
Along with the Generic Table Catalog support, Polaris is also releasing a Spark Client, which help
29-
providing an end-to-end solution for Apache Spark to manage Delta tables using Polaris.
28+
Along with the Generic Table Catalog support, Polaris is also releasing a Spark Client, which helps to
29+
provide an end-to-end solution for Apache Spark to manage Delta tables using Polaris.
3030

3131
Note the Polaris Spark Client is able to handle both Iceberg and Delta tables, not just Delta.
3232

33-
This pages documents how to build and use the Apache Polaris Spark Client before formal release.
33+
This page documents how to build and use the Polaris Spark Client directly with the source repo.
3434

3535
## Prerequisite
3636
1. Check out the polaris repo
@@ -59,14 +59,14 @@ The getting-started will start two containers:
5959

6060
The notebook `SparkPolaris.ipynb` provided under `plugins/spark/v3.5/getting-started/notebooks` provides examples
6161
with basic commands, includes:
62-
1) Connect to Polaris using Python client to create Catalog and Roles
62+
1) Connect to Polaris using Python client to create a Catalog and Roles
6363
2) Start Spark session using the Polaris Spark Client
6464
3) Using Spark to perform table operations for both Delta and Iceberg
6565

6666
## Start Spark against a deployed Polaris Service
67-
If you want to start Spark with a deployed Polaris service, you can follow the following instructions.
67+
If you want to start Spark with a deployed Polaris service, you can follow the instructions below.
6868

69-
Before start, Make sure the service deployed is up-to-date, and Spark 3.5 with at least version 3.5.3 is installed.
69+
Before starting, make sure the service deployed is up-to-date, and that Spark 3.5 with at least version 3.5.3 is installed.
7070

7171
### Build Spark Client Jars
7272
The polaris-spark project provides a task createPolarisSparkJar to help building jars for the Polaris Spark client,
@@ -83,7 +83,7 @@ cd ~/polaris
8383
If you want to build a Scala 2.13 compatible jar, you can use the following command:
8484
- `./gradlew :polaris-spark-3.5_2.13:createPolarisSparkJar`
8585

86-
The result jar is located at plugins/spark/v3.5/build/<scala_version>/libs after the build. You can also copy the
86+
The result jar is located at `plugins/spark/v3.5/build/<scala_version>/libs` after the build. You can also copy the
8787
corresponding jar to any location your Spark will have access.
8888

8989
### Connecting with Spark Using the built jar
@@ -111,7 +111,7 @@ the same name. Replace the `polaris-service-uri`, `client-id` and `client-secret
111111
[Quick Start]({{% ref "../0.9.0/quickstart" %}}) for more details about those fields.
112112

113113
Or you can create a spark session start the connection, following is an example with pyspark
114-
```shell
114+
```python
115115
from pyspark.sql import SparkSession
116116

117117
spark = SparkSession.builder
@@ -133,7 +133,7 @@ Similar as the CLI command, make sure the corresponding fields are replaced corr
133133
### Create tables with Spark
134134
After the spark is started, you can use it to create and access Iceberg and Delta table like what you are doing before,
135135
for example:
136-
```shell
136+
```python
137137
spark.sql("USE polaris")
138138
spark.sql("CREATE NAMESPACE IF NOT EXISTS DELTA_NS")
139139
spark.sql("CREATE NAMESPACE IF NOT EXISTS DELTA_NS.PUBLIC")
@@ -151,5 +151,6 @@ The Polaris Spark client has the following functionality limitations:
151151
2) Create a Delta table without explicit location is not supported.
152152
3) Rename a Delta table is not supported.
153153
4) ALTER TABLE ... SET LOCATION/SET FILEFORMAT/ADD PARTITION is not supported for DELTA table.
154-
5) For other non-iceberg tables like csv, there is no specific guarantee provided today.
155-
6) Role-based RBAC support for Delta table write is not available. Create, Drop and List RBAC support is available.
154+
5) For other non-Iceberg tables like csv, there is no specific guarantee provided today.
155+
6) TABLE_WRITE_DATA privileges is not supported for Delta Table.
156+
7) Credential Vending is not supported for Delta Table.

0 commit comments

Comments
 (0)