From b33141d265edb45913916b5bd8f4656ad17c237f Mon Sep 17 00:00:00 2001 From: Qi Yu Date: Wed, 13 Dec 2023 19:12:24 +0800 Subject: [PATCH] [Minor] improvement(docs): Optimized Trino related docs. (#1139) ### What changes were proposed in this pull request? Optimize the docs of Gravitino Trino connector. ### Why are the changes needed? Better user experience. ### Does this PR introduce _any_ user-facing change? N/A ### How was this patch tested? N/A --------- Co-authored-by: yuhui --- docs/lakehouse-iceberg-catalog.md | 6 ++-- docs/trino-connector/catalog-hive.md | 14 ++++++--- docs/trino-connector/catalog-iceberg.md | 14 ++++++--- docs/trino-connector/catalog-mysql.md | 14 ++++++--- docs/trino-connector/catalog-postgresql.md | 13 +++++---- docs/trino-connector/configuration.md | 9 +++--- docs/trino-connector/installation.md | 34 +++++++++------------- docs/trino-connector/supported-catalog.md | 5 ++-- docs/trino-connector/trino-connector.md | 8 +++-- 9 files changed, 67 insertions(+), 50 deletions(-) diff --git a/docs/lakehouse-iceberg-catalog.md b/docs/lakehouse-iceberg-catalog.md index 1f1ede4c16c..4ee38c3e1c5 100644 --- a/docs/lakehouse-iceberg-catalog.md +++ b/docs/lakehouse-iceberg-catalog.md @@ -55,7 +55,7 @@ You must download the corresponding JDBC driver to the `catalogs/lakehouse-icebe ### Catalog operations -Please refer to [Manage Metadata Using Gravitino](./manage-metadata-using-gravitino#catalogs-operations) for more details. +Please refer to [Manage Metadata Using Gravitino](./manage-metadata-using-gravitino.md#catalogs-operations) for more details. ## Schema @@ -69,7 +69,7 @@ You could put properties except `comment`. ### Schema operations -Please refer to [Manage Metadata Using Gravitino](./manage-metadata-using-gravitino#schemas-operations) for more details. +Please refer to [Manage Metadata Using Gravitino](./manage-metadata-using-gravitino.md#schemas-operations) for more details. ## Table @@ -161,7 +161,7 @@ The Gravitino server doesn't allow passing the following reserved fields. ### Table operations -Please refer to [Manage Metadata Using Gravitino](./manage-metadata-using-gravitino#tables-operations) for more details. +Please refer to [Manage Metadata Using Gravitino](./manage-metadata-using-gravitino.md#tables-operations) for more details. #### Alter table operations diff --git a/docs/trino-connector/catalog-hive.md b/docs/trino-connector/catalog-hive.md index d00997ae983..d8cb484942f 100644 --- a/docs/trino-connector/catalog-hive.md +++ b/docs/trino-connector/catalog-hive.md @@ -96,10 +96,10 @@ CREATE TABLE "metalake.catalog".dbname.tabname ## Basic usage examples -First, you need to create a metalake and catalog in Gravitino. -For example, create a new metalake named `test` and create a new catalog named `hive_test` using the `hive` provider. -And configure the Metalake `test` into the `Graviton connector`. -For More information about the Hive catalog, please refer to [Hive catalog](../apache-hive-catalog.md). +You need to do the following steps before you can use the Hive catalog in Trino through Gravitino. + +- Create a metalake and catalog in Gravitino. Assuming that the metalake name is `test` and the catalog name is `hive_test`, +then you can use the following code to create them in Gravitino: ```bash curl -X POST -H "Content-Type: application/json" \ @@ -122,6 +122,12 @@ curl -X POST \ }' http://gravitino-host:8090/api/metalakes/test/catalogs ``` +For More information about the Hive catalog, please refer to [Hive catalog](../apache-hive-catalog.md). + +- Set the value of configuration `gravitino.metalake` to the metalake you have created, named 'test', and start the Trino container. + +Use the Trino CLI to connect to the Trino container and run a query. + Listing all Gravitino managed catalogs: ```sql diff --git a/docs/trino-connector/catalog-iceberg.md b/docs/trino-connector/catalog-iceberg.md index 3692a1428d9..6064d34cc24 100644 --- a/docs/trino-connector/catalog-iceberg.md +++ b/docs/trino-connector/catalog-iceberg.md @@ -48,10 +48,10 @@ Iceberg's tables and schemas do not support properties. ## Basic usage examples -First, you need to create a metalake and catalog in Gravitino. -For example, create a new metalake named `test` and create a new catalog named `iceberg_test` using the `lakehouse-iceberg` provider. -And configure the Metalake `test` into the `Graviton connector`. -For More information about the Iceberg catalog, please refer to [Iceberg catalog](../lakehouse-iceberg-catalog.md). +You need to do the following steps before you can use the Iceberg catalog in Trino through Gravitino. + +- Create a metalake and catalog in Gravitino. Assuming that the metalake name is `test` and the catalog name is `iceberg_test`, +then you can use the following code to create them in Gravitino: ```bash curl -X POST -H "Content-Type: application/json" \ @@ -75,6 +75,12 @@ curl -X POST -H "Content-Type: application/json" \ }' http://gravitino-host:8090/api/metalakes/test/catalogs ``` +For More information about the Iceberg catalog, please refer to [Iceberg catalog](../lakehouse-iceberg-catalog.md). + +- Set the value of configuration `gravitino.metalake` to the metalake you have created, named 'test', and start the Trino container. + +Use the Trino CLI to connect to the Trino container and run a query. + Listing all Gravitino managed catalogs: ```sql diff --git a/docs/trino-connector/catalog-mysql.md b/docs/trino-connector/catalog-mysql.md index 6aa8568df22..964539461d7 100644 --- a/docs/trino-connector/catalog-mysql.md +++ b/docs/trino-connector/catalog-mysql.md @@ -42,10 +42,10 @@ MySQL's tables and schemas cannot support properties. ## Basic usage examples -First, you need to create a metalake and catalog in Gravitino. -For example, create a new metalake named `test` and create a new catalog named `mysql_test` using the `jdbc-mysql` provider. -And configure the Metalake `test` into the `Graviton connector`. -For More information about the MySQL catalog, please refer to [MySQL catalog](../jdbc-mysql-catalog.md). +You need to do the following steps before you can use the MySQL catalog in Trino through Gravitino. + +- Create a metalake and catalog in Gravitino. Assuming that the metalake name is `test` and the catalog name is `mysql_test`, +then you can use the following code to create them in Gravitino: ```bash curl -X POST -H "Content-Type: application/json" \ @@ -70,6 +70,12 @@ curl -X POST -H "Content-Type: application/json" \ }' http://gravitino-host:8090/api/metalakes/test/catalogs ``` +For More information about the MySQL catalog, please refer to [MySQL catalog](../jdbc-mysql-catalog.md). + +- Set the value of configuration `gravitino.metalake` to the metalake you have created, named 'test', and start the Trino container. + +Use the Trino CLI to connect to the Trino container and run a query. + Listing all Gravitino managed catalogs: ```sql diff --git a/docs/trino-connector/catalog-postgresql.md b/docs/trino-connector/catalog-postgresql.md index e841b1e7962..3a1e96fb7c7 100644 --- a/docs/trino-connector/catalog-postgresql.md +++ b/docs/trino-connector/catalog-postgresql.md @@ -43,10 +43,9 @@ PostgreSQL's tables and schemas cannot support properties. ## Basic usage examples -First, you need to create a metalake and catalog in Gravitino. -For example, create a new metalake named `test` and create a new catalog named `postgresql_test` using the `jdbc-postgresql` provider. -And configure the Metalake `test` into the `Graviton connector`. -For More information about the PostgreSQL catalog, please refer to [PostgreSQL catalog](../jdbc-postgresql-catalog.md). +You need to do the following steps before you can use the PostgreSQL catalog in Trino through Gravitino. + +- Create a metalake and catalog in Gravitino. Assuming that the metalake name is `test` and the catalog name is `postgresql_test`, then you can use the following code to create them in Gravitino: ```bash curl -X POST -H "Content-Type: application/json" \ @@ -70,8 +69,12 @@ curl -X POST -H "Content-Type: application/json" \ "jdbc-driver": "org.postgresql.Driver" } }' http://gravitino-host:8090/api/metalakes/test/catalogs - ``` +For more information about the PostgreSQL catalog, please refer to [PostgreSQL catalog](../jdbc-postgresql-catalog.md). + +- Set the value of configuration `gravitino.metalake` to the metalake you have created, named 'test', and start the Trino container. + +Use the Trino CLI to connect to the Trino container and run a query. Listing all Gravitino managed catalogs: diff --git a/docs/trino-connector/configuration.md b/docs/trino-connector/configuration.md index 8141c9ba533..471723aa8cf 100644 --- a/docs/trino-connector/configuration.md +++ b/docs/trino-connector/configuration.md @@ -6,7 +6,8 @@ license: "Copyright 2023 Datastrato Pvt Ltd. This software is licensed under the Apache License version 2." --- -| Property | Type | Default Value | Description | Required | Since Version | -|--------------------|--------|-----------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|---------------| -| gravitino.url | string | http://localhost:8090 | The `gravitino.url` defines the connection URL for the Gravitino server. If not set, the default value is `http://localhost:8090`. If the Gravitino server is not starting, Trino can still start normally. Once the Gravitino server is up and running, the Gravitino connector will automatically connect. You can find error details in the Trino logs. | Yes | 0.2.0 | -| gravitino.metalake | string | (none) | The `gravitino.metalake` defines the metalake used. You can create it beforehand or later on. If not set, Trino might throw an error upon startup. It must be set to a valid metalake name. If it's not created, the Gravitino connector continues checking until it's created. Once created, it will load the catalogs, schemas, and tables into Trino and maintain synchronization. | Yes | 0.2.0 | \ No newline at end of file +| Property | Type | Default Value | Description | Required | Since Version | +|--------------------|--------|-------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|---------------| +| connector.name | string | gravitino | The `connector.name` defines the name of Gravitino connector, this value is always 'gravitino'. | Yes | 0.2.0 | +| gravitino.metalake | string | (none) | The `gravitino.metalake` defines the metalake used. You can create it beforehand or later on. If not set, Trino might throw an error upon startup. It must be set to a valid metalake name. If it's not created, the Gravitino connector continues checking until it's created. Once created, it will load the catalogs, schemas, and tables into Trino and maintain synchronization. | Yes | 0.2.0 | +| gravitino.url | string | http://localhost:8090 | The `gravitino.url` defines the connection URL for the Gravitino server. If not set, the default value is `http://localhost:8090`. If the Gravitino server is not starting, Trino can still start normally. Once the Gravitino server is up and running, the Gravitino connector will automatically connect. You can find error details in the Trino logs. | Yes | 0.2.0 | diff --git a/docs/trino-connector/installation.md b/docs/trino-connector/installation.md index 1fd897ba281..02a1d2d65b5 100644 --- a/docs/trino-connector/installation.md +++ b/docs/trino-connector/installation.md @@ -6,13 +6,12 @@ license: "Copyright 2023 Datastrato Pvt Ltd. This software is licensed under the Apache License version 2." --- -To install the Gravitino connector, first deploy the Trino environment, and then install the Gravitino connector plugin into Trino. -Please refer to the [Deploying Trino documentation](https://trino.io/docs/current/installation/deployment.html). +To install the Gravitino connector, you should first deploy the Trino environment, and then install the Gravitino connector plugin into Trino. +Please refer to the [Deploying Trino documentation](https://trino.io/docs/current/installation/deployment.html) and do the following steps: -1. Download the Gravitino connector tarball and unpack it. +1. [Download](https://github.com/datastrato/gravitino/releases) the Gravitino connector tarball and unpack it. The tarball contains a single top-level directory `gravitino-trino-connector-`, which called the connector directory. - [Download the gravitino-connector](https://github.com/datastrato/gravitino/releases). 2. Copy the connector directory to the Trino's plugin directory. Normally, the directory location is `Trino-server-/plugin`, and the directory contains other catalogs used by Trino. 3. Add Trino JVM arguments `-Dlog4j.configurationFile=file:////etc/trino/log4j2.properties` to enable logging for the Gravitino connector. @@ -67,10 +66,9 @@ Now you can see the Gravitino connector directory in the plugin directory. ### Configuring the Gravitino connector -Assuming you have now started the Gravitino server on the host `gravitino-server-host` and can access it properly. -And you have created a metalake named `test`. If not, please refer to the [Gravitino Getting-started](../getting-started.md) +Assuming you have now started the Gravitino server on the host `gravitino-server-host` and already created a metalake named `test`, if those have not been prepared, please refer to the [Gravitino Getting-started](../getting-started.md). -Add catalog configuration to the Trino configuration file `/etc/trino/catalog/gravitino.properties`. +To configure Gravitino connector correctly, you need to put the following configurations to the Trino configuration file `/etc/trino/catalog/gravitino.properties`. ```text connector.name=gravitino @@ -78,19 +76,19 @@ gravitino.url=http://gravitino-server-host:8090 gravitino.metalake=test ``` -The `gravitino.name` defines which Gravitino connector is used. It must be `gravitino`. -The `gravitino.metalake` defines which metalake are used. It should exist in the Gravitino server. -The `gravitino.uri` defines the connection information about Gravitino server. Make sure your container can access the Gravitino server. +- The `gravitino.name` defines which Gravitino connector is used. It must be `gravitino`. +- The `gravitino.metalake` defines which metalake are used. It should exist in the Gravitino server. +- The `gravitino.uri` defines the connection information about Gravitino server. Make sure your container can access the Gravitino server. -If you don't have the `test` metalake. You can create a new metalake named `test`. +Full configurations for Gravitino connector can be seen [here](configuration.md) -Create a new metalake named `test` by the following command. +If you haven't created the metalake named `test`, you can use the following command to create it. ```shell curl -X POST -H "Content-Type: application/json" -d '{"name":"test","comment":"comment","properties":{}}' http://gravitino-server-host:8090/api/metalakes ``` -Restart the Trino container to load the Gravitino connector. +And then restart the Trino container to load the Gravitino connector. ```shell docker restart trino-gravitino @@ -113,12 +111,9 @@ tpch system ``` -You can see the `gravitino` catalog in the result. This signifies the successful installation of the Gravitino connector. +You can see the `gravitino` catalog in the result set. This signifies the successful installation of the Gravitino connector. -Assuming you have created a catalog named `test.jdbc-mysql` in the Gravitino server. -If you don't have it, please refer to [Create a Catalog](../manage-metadata-using-gravitino.md#create-a-catalog) - -Then you can use the Trino CLI to connect to the Trino container and run a query. +Assuming you have created a catalog named `test.jdbc-mysql` in the Gravitino server, or please refer to [Create a Catalog](../manage-metadata-using-gravitino#create-a-catalog). Then you can use the Trino CLI to connect to the Trino container and run a query like this. ```text docker exec -it trino trino @@ -134,5 +129,4 @@ system test.jdbc-mysql ``` -The catalog named 'test.jdbc-mysql' is your created catalog by gravitino server. -You can use it to access the mysql database like other Trino catalogs. \ No newline at end of file +The catalog named 'test.jdbc-mysql' is the catalog that you created by gravitino server, and you can use it to access the mysql database like other Trino catalogs. \ No newline at end of file diff --git a/docs/trino-connector/supported-catalog.md b/docs/trino-connector/supported-catalog.md index 4b3f0e7068b..b2602f809e0 100644 --- a/docs/trino-connector/supported-catalog.md +++ b/docs/trino-connector/supported-catalog.md @@ -19,10 +19,9 @@ Trino currently does not support creating Gravitino managed catalogs. If you need to create a catalog, please refer to: - [Create a Catalog](../manage-metadata-using-gravitino.md#create-a-catalog) -## Data type mapping +## Data type mapping between Trino and Gravitino -All types of catalogs support the following data type conversions. -Data Type Conversion between Trino and Gravitino is as following: +All types of catalogs support the following data type conversions between Trino and Gravitino: | Gravitino Type | Trino Type | |----------------|------------| diff --git a/docs/trino-connector/trino-connector.md b/docs/trino-connector/trino-connector.md index efea55cba0e..6a5fbca9262 100644 --- a/docs/trino-connector/trino-connector.md +++ b/docs/trino-connector/trino-connector.md @@ -7,11 +7,13 @@ This software is licensed under the Apache License version 2." --- Trino can manage and access data using the Trino connector provided by `Gravitino`, commonly referred to as the `Gravitino connector`. -After configuring the Gravitino connector in Trino, it can automatically load catalog metadata from Gravitino, allowing users to directly access these catalogs in Trino. +After configuring the Gravitino connector in Trino, Trino can automatically load catalog metadata from Gravitino, allowing users to directly access these catalogs in Trino. Once integrated with Gravitino, Trino can operate on all Gravitino data without requiring additional configuration. -When metadata such as catalogs, schemas, or tables change in Gravitino, Trino can also update itself through Gravitino. - +:::node +Once metadata such as catalogs, schemas, or tables are changed in Gravitino, Trino can update itself through Gravitino, this process usually takes +about 3~10 seconds. +::: The loading of Gravitino's catalogs into Trino follows the naming convention: