Skip to content

Commit 919e9d8

Browse files
eric-maynardannafil
eric-maynard
and
annafil
authored
Implement missing doc "Configuring Polaris for Production" (#43)
* wip * checkin * check in * rebase * bootstrapping notes * emphasis * plug the quickstart in the readme, so there's a path to the other docs * ensure new links are rendered in web * regenerate web docs * regenerate docs * typofix * regen docs * bootstrap fix * regen docs * changes per review * regen docs * regen docs * refactor * regen docs * changes per review * regen docs * style * regen docs --------- Co-authored-by: annafil <7892219+annafil@users.noreply.github.com>
1 parent 353acea commit 919e9d8

File tree

4 files changed

+362
-2
lines changed

4 files changed

+362
-2
lines changed

‎docs/configuring-polaris-for-production.md‎

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
<!--
2+
Copyright (c) 2024 Snowflake Computing Inc.
3+
4+
Licensed under the Apache License, Version 2.0 (the "License");
5+
you may not use this file except in compliance with the License.
6+
You may obtain a copy of the License at
7+
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
10+
Unless required by applicable law or agreed to in writing, software
11+
distributed under the License is distributed on an "AS IS" BASIS,
12+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
See the License for the specific language governing permissions and
14+
limitations under the License.
15+
-->
16+
17+
18+
# Configuring Polaris for Production
19+
20+
The default `polaris-server.yml` configuration is intended for develoment and testing. When deploying Polaris in production, there are several best practices to keep in mind.
21+
22+
## Security
23+
24+
### Configurations
25+
26+
Notable configuration used to secure a Polaris deployment are outlined below.
27+
28+
#### oauth2
29+
30+
> [!WARNING]
31+
> Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below.
32+
33+
* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it.
34+
* Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/polaris-catalog/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/polaris-catalog/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys.
35+
36+
#### authenticator.tokenBroker
37+
38+
> [!WARNING]
39+
> Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above.
40+
41+
#### callContextResolver** & **realmContextResolver
42+
* Use these configurations to specify a service that can resolve a realm from bearer tokens.
43+
* The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/polaris-catalog/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/polaris-catalog/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
44+
45+
## Metastore Management
46+
47+
> [!IMPORTANT]
48+
> The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database.
49+
50+
A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/polaris-catalog/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted.
51+
52+
Be sure to secure your metastore backend since it will be storing credentials and catalog metadata.
53+
54+
### Configuring EclipseLink
55+
56+
To use [EclipseLink](https://eclipse.dev/eclipselink/) for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an [EclipseLink `persistence.xml` file](https://eclipse.dev/eclipselink/documentation/2.5/solutions/testingjpa002.htm). This file, local to the Polaris service, will contain information on what database to use for metastore management and how to connect to it.
57+
58+
### Bootstrapping
59+
60+
Before using Polaris when using a metastore manager other than `in-memory`, you must **bootstrap** the metastore manager. This is a manual operation that must be performed **only once** in order to prepare the metastore manager to integrate with Polaris. When the metastore manager is bootstrapped, any existing Polaris entities in the metastore manager may be **purged**.
61+
62+
To bootstrap Polaris, run:
63+
64+
```bash
65+
java -jar /path/to/jar/polaris-service-1.0.0-all.jar bootstrap polaris-server.yml
66+
```
67+
68+
Afterwards, Polaris can be launched normally:
69+
70+
```bash
71+
java -jar /path/to/jar/polaris-service-1.0.0-all.jar server polaris-server.yml
72+
```
73+
74+
## Other Configurations
75+
76+
When deploying Polaris in production, consider adjusting the following configurations:
77+
78+
#### featureConfiguration.SUPPORTED_CATALOG_STORAGE_TYPES
79+
- By default Polaris catalogs are allowed to be located in local filesystem with the `FILE` storage type. This should be disabled for production systems.
80+
- Use this configuration to additionally disable any other storage types that will not be in use.
81+
82+

‎docs/index.html‎

Lines changed: 118 additions & 2 deletions
Large diffs are not rendered by default.

‎spec/docs.yaml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,4 +24,7 @@ tags:
2424
- name: Access Control
2525
description:
2626
$ref: ../docs/access-control.md
27+
- name: Configuring Polaris for Production
28+
description:
29+
$ref: ../docs/configuring-polaris-for-production.md
2730

‎spec/index.yaml‎

Lines changed: 159 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -934,6 +934,164 @@ tags:
934934
![Diagram that shows an example of how RBAC works in Polaris
935935
Catalog.](./img/rbac-example.svg "Polaris Catalog RBAC example")
936936
x-displayName: Access Control
937+
- name: Configuring Polaris for Production
938+
description: >+
939+
<!--
940+
Copyright (c) 2024 Snowflake Computing Inc.
941+
942+
Licensed under the Apache License, Version 2.0 (the "License");
943+
you may not use this file except in compliance with the License.
944+
You may obtain a copy of the License at
945+
946+
http://www.apache.org/licenses/LICENSE-2.0
947+
948+
Unless required by applicable law or agreed to in writing, software
949+
distributed under the License is distributed on an "AS IS" BASIS,
950+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
951+
See the License for the specific language governing permissions and
952+
limitations under the License.
953+
-->
954+
955+
956+
957+
# Configuring Polaris for Production
958+
959+
960+
The default `polaris-server.yml` configuration is intended for develoment
961+
and testing. When deploying Polaris in production, there are several best
962+
practices to keep in mind.
963+
964+
965+
## Security
966+
967+
968+
### Configurations
969+
970+
971+
Notable configuration used to secure a Polaris deployment are outlined
972+
below.
973+
974+
975+
#### oauth2
976+
977+
978+
> [!WARNING]
979+
980+
> Ensure that the `tokenBroker` setting reflects the token broker
981+
specified in `authenticator` below.
982+
983+
984+
* Configure [OAuth](https://oauth.net/2/) with this setting. Remove the
985+
`TestInlineBearerTokenPolarisAuthenticator` option and uncomment the
986+
`DefaultPolarisAuthenticator` authenticator option beneath it.
987+
988+
* Then, configure the token broker. You can configure the token broker to
989+
use either
990+
[asymmetric](https://github.com/polaris-catalog/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24)
991+
or
992+
[symmetric](https://github.com/polaris-catalog/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23)
993+
keys.
994+
995+
996+
#### authenticator.tokenBroker
997+
998+
999+
> [!WARNING]
1000+
1001+
> Ensure that the `tokenBroker` setting reflects the token broker
1002+
specified in `oauth2` above.
1003+
1004+
1005+
#### callContextResolver** & **realmContextResolver
1006+
1007+
* Use these configurations to specify a service that can resolve a realm
1008+
from bearer tokens.
1009+
1010+
* The service(s) used here must implement the relevant interfaces (i.e.
1011+
[CallContextResolver](https://github.com/polaris-catalog/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27)
1012+
and
1013+
[RealmContextResolver](https://github.com/polaris-catalog/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)).
1014+
1015+
1016+
## Metastore Management
1017+
1018+
1019+
> [!IMPORTANT]
1020+
1021+
> The default `in-memory` implementation for `metastoreManager` is meant
1022+
for testing and not suitable for production usage. Instead, consider an
1023+
implementation such as `eclipse-link` which allows you to store metadata
1024+
in a remote database.
1025+
1026+
1027+
A Metastore Manger should be configured with an implementation that
1028+
durably persists Polaris entities. Use the configuration
1029+
`metaStoreManager` to configure a
1030+
[MetastoreManager](https://github.com/polaris-catalog/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47)
1031+
implementation where Polaris entities will be persisted.
1032+
1033+
1034+
Be sure to secure your metastore backend since it will be storing
1035+
credentials and catalog metadata.
1036+
1037+
1038+
### Configuring EclipseLink
1039+
1040+
1041+
To use [EclipseLink](https://eclipse.dev/eclipselink/) for metastore
1042+
management, specify the configuration `metaStoreManager.conf-file` to
1043+
point to an [EclipseLink `persistence.xml`
1044+
file](https://eclipse.dev/eclipselink/documentation/2.5/solutions/testingjpa002.htm).
1045+
This file, local to the Polaris service, will contain information on what
1046+
database to use for metastore management and how to connect to it.
1047+
1048+
1049+
### Bootstrapping
1050+
1051+
1052+
Before using Polaris when using a metastore manager other than
1053+
`in-memory`, you must **bootstrap** the metastore manager. This is a
1054+
manual operation that must be performed **only once** in order to prepare
1055+
the metastore manager to integrate with Polaris. When the metastore
1056+
manager is bootstrapped, any existing Polaris entities in the metastore
1057+
manager may be **purged**.
1058+
1059+
1060+
To bootstrap Polaris, run:
1061+
1062+
1063+
```bash
1064+
1065+
java -jar /path/to/jar/polaris-service-1.0.0-all.jar bootstrap
1066+
polaris-server.yml
1067+
1068+
```
1069+
1070+
1071+
Afterwards, Polaris can be launched normally:
1072+
1073+
1074+
```bash
1075+
1076+
java -jar /path/to/jar/polaris-service-1.0.0-all.jar server
1077+
polaris-server.yml
1078+
1079+
```
1080+
1081+
1082+
## Other Configurations
1083+
1084+
1085+
When deploying Polaris in production, consider adjusting the following
1086+
configurations:
1087+
1088+
1089+
#### featureConfiguration.SUPPORTED_CATALOG_STORAGE_TYPES
1090+
- By default Polaris catalogs are allowed to be located in local filesystem with the `FILE` storage type. This should be disabled for production systems.
1091+
- Use this configuration to additionally disable any other storage types that will not be in use.
1092+
1093+
1094+
x-displayName: Configuring Polaris for Production
9371095
- name: polaris-management-service_other
9381096
x-displayName: other
9391097
- name: Configuration API
@@ -6853,6 +7011,7 @@ x-tagGroups:
68537011
- Polaris Catalog Overview
68547012
- Polaris Catalog Entities
68557013
- Access Control
7014+
- Configuring Polaris for Production
68567015
- name: Polaris Management Service
68577016
tags:
68587017
- polaris-management-service_other

0 commit comments

Comments
 (0)