Skip to content

Commit

Permalink
[Minor] improvement(docs): Optimized Trino related docs. (apache#1139)
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?

Optimize the docs of Gravitino Trino connector. 

### Why are the changes needed?

Better user experience. 

### Does this PR introduce _any_ user-facing change?

N/A

### How was this patch tested?

N/A

---------

Co-authored-by: yuhui <hui@datastrato.com>
  • Loading branch information
yuqi1129 and diqiu50 authored Dec 13, 2023
1 parent aab86ed commit b33141d
Show file tree
Hide file tree
Showing 9 changed files with 67 additions and 50 deletions.
6 changes: 3 additions & 3 deletions docs/lakehouse-iceberg-catalog.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ You must download the corresponding JDBC driver to the `catalogs/lakehouse-icebe

### Catalog operations

Please refer to [Manage Metadata Using Gravitino](./manage-metadata-using-gravitino#catalogs-operations) for more details.
Please refer to [Manage Metadata Using Gravitino](./manage-metadata-using-gravitino.md#catalogs-operations) for more details.

## Schema

Expand All @@ -69,7 +69,7 @@ You could put properties except `comment`.

### Schema operations

Please refer to [Manage Metadata Using Gravitino](./manage-metadata-using-gravitino#schemas-operations) for more details.
Please refer to [Manage Metadata Using Gravitino](./manage-metadata-using-gravitino.md#schemas-operations) for more details.

## Table

Expand Down Expand Up @@ -161,7 +161,7 @@ The Gravitino server doesn't allow passing the following reserved fields.

### Table operations

Please refer to [Manage Metadata Using Gravitino](./manage-metadata-using-gravitino#tables-operations) for more details.
Please refer to [Manage Metadata Using Gravitino](./manage-metadata-using-gravitino.md#tables-operations) for more details.

#### Alter table operations

Expand Down
14 changes: 10 additions & 4 deletions docs/trino-connector/catalog-hive.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,10 +96,10 @@ CREATE TABLE "metalake.catalog".dbname.tabname

## Basic usage examples

First, you need to create a metalake and catalog in Gravitino.
For example, create a new metalake named `test` and create a new catalog named `hive_test` using the `hive` provider.
And configure the Metalake `test` into the `Graviton connector`.
For More information about the Hive catalog, please refer to [Hive catalog](../apache-hive-catalog.md).
You need to do the following steps before you can use the Hive catalog in Trino through Gravitino.

- Create a metalake and catalog in Gravitino. Assuming that the metalake name is `test` and the catalog name is `hive_test`,
then you can use the following code to create them in Gravitino:

```bash
curl -X POST -H "Content-Type: application/json" \
Expand All @@ -122,6 +122,12 @@ curl -X POST \
}' http://gravitino-host:8090/api/metalakes/test/catalogs
```

For More information about the Hive catalog, please refer to [Hive catalog](../apache-hive-catalog.md).

- Set the value of configuration `gravitino.metalake` to the metalake you have created, named 'test', and start the Trino container.

Use the Trino CLI to connect to the Trino container and run a query.

Listing all Gravitino managed catalogs:

```sql
Expand Down
14 changes: 10 additions & 4 deletions docs/trino-connector/catalog-iceberg.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,10 +48,10 @@ Iceberg's tables and schemas do not support properties.

## Basic usage examples

First, you need to create a metalake and catalog in Gravitino.
For example, create a new metalake named `test` and create a new catalog named `iceberg_test` using the `lakehouse-iceberg` provider.
And configure the Metalake `test` into the `Graviton connector`.
For More information about the Iceberg catalog, please refer to [Iceberg catalog](../lakehouse-iceberg-catalog.md).
You need to do the following steps before you can use the Iceberg catalog in Trino through Gravitino.

- Create a metalake and catalog in Gravitino. Assuming that the metalake name is `test` and the catalog name is `iceberg_test`,
then you can use the following code to create them in Gravitino:

```bash
curl -X POST -H "Content-Type: application/json" \
Expand All @@ -75,6 +75,12 @@ curl -X POST -H "Content-Type: application/json" \
}' http://gravitino-host:8090/api/metalakes/test/catalogs
```

For More information about the Iceberg catalog, please refer to [Iceberg catalog](../lakehouse-iceberg-catalog.md).

- Set the value of configuration `gravitino.metalake` to the metalake you have created, named 'test', and start the Trino container.

Use the Trino CLI to connect to the Trino container and run a query.

Listing all Gravitino managed catalogs:

```sql
Expand Down
14 changes: 10 additions & 4 deletions docs/trino-connector/catalog-mysql.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,10 +42,10 @@ MySQL's tables and schemas cannot support properties.

## Basic usage examples

First, you need to create a metalake and catalog in Gravitino.
For example, create a new metalake named `test` and create a new catalog named `mysql_test` using the `jdbc-mysql` provider.
And configure the Metalake `test` into the `Graviton connector`.
For More information about the MySQL catalog, please refer to [MySQL catalog](../jdbc-mysql-catalog.md).
You need to do the following steps before you can use the MySQL catalog in Trino through Gravitino.

- Create a metalake and catalog in Gravitino. Assuming that the metalake name is `test` and the catalog name is `mysql_test`,
then you can use the following code to create them in Gravitino:

```bash
curl -X POST -H "Content-Type: application/json" \
Expand All @@ -70,6 +70,12 @@ curl -X POST -H "Content-Type: application/json" \
}' http://gravitino-host:8090/api/metalakes/test/catalogs
```

For More information about the MySQL catalog, please refer to [MySQL catalog](../jdbc-mysql-catalog.md).

- Set the value of configuration `gravitino.metalake` to the metalake you have created, named 'test', and start the Trino container.

Use the Trino CLI to connect to the Trino container and run a query.

Listing all Gravitino managed catalogs:

```sql
Expand Down
13 changes: 8 additions & 5 deletions docs/trino-connector/catalog-postgresql.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,10 +43,9 @@ PostgreSQL's tables and schemas cannot support properties.

## Basic usage examples

First, you need to create a metalake and catalog in Gravitino.
For example, create a new metalake named `test` and create a new catalog named `postgresql_test` using the `jdbc-postgresql` provider.
And configure the Metalake `test` into the `Graviton connector`.
For More information about the PostgreSQL catalog, please refer to [PostgreSQL catalog](../jdbc-postgresql-catalog.md).
You need to do the following steps before you can use the PostgreSQL catalog in Trino through Gravitino.

- Create a metalake and catalog in Gravitino. Assuming that the metalake name is `test` and the catalog name is `postgresql_test`, then you can use the following code to create them in Gravitino:

```bash
curl -X POST -H "Content-Type: application/json" \
Expand All @@ -70,8 +69,12 @@ curl -X POST -H "Content-Type: application/json" \
"jdbc-driver": "org.postgresql.Driver"
}
}' http://gravitino-host:8090/api/metalakes/test/catalogs

```
For more information about the PostgreSQL catalog, please refer to [PostgreSQL catalog](../jdbc-postgresql-catalog.md).

- Set the value of configuration `gravitino.metalake` to the metalake you have created, named 'test', and start the Trino container.

Use the Trino CLI to connect to the Trino container and run a query.

Listing all Gravitino managed catalogs:

Expand Down
9 changes: 5 additions & 4 deletions docs/trino-connector/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@ license: "Copyright 2023 Datastrato Pvt Ltd.
This software is licensed under the Apache License version 2."
---

| Property | Type | Default Value | Description | Required | Since Version |
|--------------------|--------|-----------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|---------------|
| gravitino.url | string | http://localhost:8090 | The `gravitino.url` defines the connection URL for the Gravitino server. If not set, the default value is `http://localhost:8090`. If the Gravitino server is not starting, Trino can still start normally. Once the Gravitino server is up and running, the Gravitino connector will automatically connect. You can find error details in the Trino logs. | Yes | 0.2.0 |
| gravitino.metalake | string | (none) | The `gravitino.metalake` defines the metalake used. You can create it beforehand or later on. If not set, Trino might throw an error upon startup. It must be set to a valid metalake name. If it's not created, the Gravitino connector continues checking until it's created. Once created, it will load the catalogs, schemas, and tables into Trino and maintain synchronization. | Yes | 0.2.0 |
| Property | Type | Default Value | Description | Required | Since Version |
|--------------------|--------|-------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|---------------|
| connector.name | string | gravitino | The `connector.name` defines the name of Gravitino connector, this value is always 'gravitino'. | Yes | 0.2.0 |
| gravitino.metalake | string | (none) | The `gravitino.metalake` defines the metalake used. You can create it beforehand or later on. If not set, Trino might throw an error upon startup. It must be set to a valid metalake name. If it's not created, the Gravitino connector continues checking until it's created. Once created, it will load the catalogs, schemas, and tables into Trino and maintain synchronization. | Yes | 0.2.0 |
| gravitino.url | string | http://localhost:8090 | The `gravitino.url` defines the connection URL for the Gravitino server. If not set, the default value is `http://localhost:8090`. If the Gravitino server is not starting, Trino can still start normally. Once the Gravitino server is up and running, the Gravitino connector will automatically connect. You can find error details in the Trino logs. | Yes | 0.2.0 |
34 changes: 14 additions & 20 deletions docs/trino-connector/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,12 @@ license: "Copyright 2023 Datastrato Pvt Ltd.
This software is licensed under the Apache License version 2."
---

To install the Gravitino connector, first deploy the Trino environment, and then install the Gravitino connector plugin into Trino.
Please refer to the [Deploying Trino documentation](https://trino.io/docs/current/installation/deployment.html).
To install the Gravitino connector, you should first deploy the Trino environment, and then install the Gravitino connector plugin into Trino.
Please refer to the [Deploying Trino documentation](https://trino.io/docs/current/installation/deployment.html) and do the following steps:

1. Download the Gravitino connector tarball and unpack it.
1. [Download](https://github.com/datastrato/gravitino/releases) the Gravitino connector tarball and unpack it.
The tarball contains a single top-level directory `gravitino-trino-connector-<version>`,
which called the connector directory.
[Download the gravitino-connector](https://github.com/datastrato/gravitino/releases).
2. Copy the connector directory to the Trino's plugin directory.
Normally, the directory location is `Trino-server-<version>/plugin`, and the directory contains other catalogs used by Trino.
3. Add Trino JVM arguments `-Dlog4j.configurationFile=file:////etc/trino/log4j2.properties` to enable logging for the Gravitino connector.
Expand Down Expand Up @@ -67,30 +66,29 @@ Now you can see the Gravitino connector directory in the plugin directory.

### Configuring the Gravitino connector

Assuming you have now started the Gravitino server on the host `gravitino-server-host` and can access it properly.
And you have created a metalake named `test`. If not, please refer to the [Gravitino Getting-started](../getting-started.md)
Assuming you have now started the Gravitino server on the host `gravitino-server-host` and already created a metalake named `test`, if those have not been prepared, please refer to the [Gravitino Getting-started](../getting-started.md).

Add catalog configuration to the Trino configuration file `/etc/trino/catalog/gravitino.properties`.
To configure Gravitino connector correctly, you need to put the following configurations to the Trino configuration file `/etc/trino/catalog/gravitino.properties`.

```text
connector.name=gravitino
gravitino.url=http://gravitino-server-host:8090
gravitino.metalake=test
```

The `gravitino.name` defines which Gravitino connector is used. It must be `gravitino`.
The `gravitino.metalake` defines which metalake are used. It should exist in the Gravitino server.
The `gravitino.uri` defines the connection information about Gravitino server. Make sure your container can access the Gravitino server.
- The `gravitino.name` defines which Gravitino connector is used. It must be `gravitino`.
- The `gravitino.metalake` defines which metalake are used. It should exist in the Gravitino server.
- The `gravitino.uri` defines the connection information about Gravitino server. Make sure your container can access the Gravitino server.

If you don't have the `test` metalake. You can create a new metalake named `test`.
Full configurations for Gravitino connector can be seen [here](configuration.md)

Create a new metalake named `test` by the following command.
If you haven't created the metalake named `test`, you can use the following command to create it.

```shell
curl -X POST -H "Content-Type: application/json" -d '{"name":"test","comment":"comment","properties":{}}' http://gravitino-server-host:8090/api/metalakes
```

Restart the Trino container to load the Gravitino connector.
And then restart the Trino container to load the Gravitino connector.

```shell
docker restart trino-gravitino
Expand All @@ -113,12 +111,9 @@ tpch
system
```

You can see the `gravitino` catalog in the result. This signifies the successful installation of the Gravitino connector.
You can see the `gravitino` catalog in the result set. This signifies the successful installation of the Gravitino connector.

Assuming you have created a catalog named `test.jdbc-mysql` in the Gravitino server.
If you don't have it, please refer to [Create a Catalog](../manage-metadata-using-gravitino.md#create-a-catalog)

Then you can use the Trino CLI to connect to the Trino container and run a query.
Assuming you have created a catalog named `test.jdbc-mysql` in the Gravitino server, or please refer to [Create a Catalog](../manage-metadata-using-gravitino#create-a-catalog). Then you can use the Trino CLI to connect to the Trino container and run a query like this.

```text
docker exec -it trino trino
Expand All @@ -134,5 +129,4 @@ system
test.jdbc-mysql
```

The catalog named 'test.jdbc-mysql' is your created catalog by gravitino server.
You can use it to access the mysql database like other Trino catalogs.
The catalog named 'test.jdbc-mysql' is the catalog that you created by gravitino server, and you can use it to access the mysql database like other Trino catalogs.
5 changes: 2 additions & 3 deletions docs/trino-connector/supported-catalog.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,9 @@ Trino currently does not support creating Gravitino managed catalogs.
If you need to create a catalog, please refer to:
- [Create a Catalog](../manage-metadata-using-gravitino.md#create-a-catalog)

## Data type mapping
## Data type mapping between Trino and Gravitino

All types of catalogs support the following data type conversions.
Data Type Conversion between Trino and Gravitino is as following:
All types of catalogs support the following data type conversions between Trino and Gravitino:

| Gravitino Type | Trino Type |
|----------------|------------|
Expand Down
8 changes: 5 additions & 3 deletions docs/trino-connector/trino-connector.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,13 @@ This software is licensed under the Apache License version 2."
---

Trino can manage and access data using the Trino connector provided by `Gravitino`, commonly referred to as the `Gravitino connector`.
After configuring the Gravitino connector in Trino, it can automatically load catalog metadata from Gravitino, allowing users to directly access these catalogs in Trino.
After configuring the Gravitino connector in Trino, Trino can automatically load catalog metadata from Gravitino, allowing users to directly access these catalogs in Trino.
Once integrated with Gravitino, Trino can operate on all Gravitino data without requiring additional configuration.

When metadata such as catalogs, schemas, or tables change in Gravitino, Trino can also update itself through Gravitino.

:::node
Once metadata such as catalogs, schemas, or tables are changed in Gravitino, Trino can update itself through Gravitino, this process usually takes
about 3~10 seconds.
:::

The loading of Gravitino's catalogs into Trino follows the naming convention:

Expand Down

0 comments on commit b33141d

Please sign in to comment.