Skip to content

Latest commit

 

History

History
355 lines (222 loc) · 15 KB

webui.md

File metadata and controls

355 lines (222 loc) · 15 KB
Raw
title slug keyword last_update license
Apache Gravitino web UI
/webui
webui
date author
2024-04-12
ch3yne
This software is licensed under the Apache License version 2.

This document primarily outlines how users can manage metadata within Apache Gravitino using the web UI, the graphical interface is accessible through a web browser as an alternative to writing code or using the REST interface.

Currently, you can integrate OAuth settings to view, add, modify, and delete metalakes, create catalogs, and view catalogs, schemas, and tables, among other functions.

Build and deploy the Gravitino Web UI and open it in a browser at http://<gravitino-host>:<gravitino-port>, by default is http://localhost:8090.

Initial page

The web UI homepage displayed in Gravitino depends on the configuration parameter for OAuth mode, see the details in Security.

Set parameter for gravitino.authenticators, simple or oauth. Simple mode is the default authentication option. If multiple authenticators are set, the first one is taken by default.

:::tip After changing the configuration, make sure to restart the Gravitino server.

<path-to-gravitino>/bin/gravitino.sh restart :::

Simple mode

gravitino.authenticators = simple

Set the configuration parameter gravitino.authenticators to simple, and the web UI displays the homepage (Metalakes).

webui-metalakes-simple

At the top-right, the UI displays the current Gravitino version.

The main content displays the existing metalake list.

Oauth mode

gravitino.authenticators = oauth

Set the configuration parameter gravitino.authenticators to oauth, and the web UI displays the login page.

:::caution If both OAuth and HTTPS are set, due to the different security permission rules of various browsers, to avoid cross-domain errors, it is recommended to use the Chrome browser for access and operation.

Such as Safari need to enable the developer menu, and select Disable Cross-Origin Restrictions from the develop menu. :::

webui-login-with-oauth

  1. Enter the values corresponding to your specific configuration. For detailed instructions, please refer to Security.

  2. Clicking on the LOGIN button takes you to the homepage.

webui-metalakes-oauth

At the top-right, there is an icon button that takes you to the login page when clicked.

Manage metadata

All the manage actions are performed by using the REST API

Metalake

Create metalake

On the homepage, clicking on the CREATE METALAKE button displays a dialog to create a metalake.

create-metalake-dialog

Creating a metalake needs these fields:

  1. Name(required): the name of the metalake.
  2. Comment(optional): the comment of the metalake.
  3. Properties(optional): Click on the ADD PROPERTY button to add custom properties.

metalake-list

There are 3 actions you can perform on a metalake.

metalake-actions

Show metalake details

Clicking on the action icon in the table cell.

You can see the detailed information of this metalake in the drawer component on the right.

metalake-details

Edit metalake

Clicking on the action icon in the table cell.

Displays the dialog for modifying fields of the selected metalake.

create-metalake-dialog

Delete metalake

Clicking on the action icon in the table cell.

Displays a confirmation dialog, clicking on the SUBMIT button deletes this metalake.

delete-metalake

Catalog

Clicking on a metalake name in the table views catalogs in a metalake.

If this is the first time, it shows no data until after creating a catalog.

Clicking on the left arrow icon button takes you to the metalake page.

metalake-catalogs

Clicking on the Tab - DETAILS views the details of the catalog on the metalake catalogs page.

metalake-catalogs-details

On the left side of the page is a tree list, and the icons of the catalog correspond to their type and provider.

  • Catalog
  • Schema
  • Table

tree-view

Hover your mouse over the corresponding icon to the data changes to a reload icon . Click on this icon to reload the currently selected data.

tree-view-reload-catalog

Create catalog

Clicking on the CREATE CATALOG button displays the dialog to create a catalog.

create-catalog

Creating a catalog requires these fields:

  1. Catalog name(required): the name of the catalog
  2. Type(required): relational/fileset/messaging, the default value is relational
  3. Provider(required):
    1. Type relational - hive/iceberg/mysql/postgresql/doris
    2. Type fileset - hadoop
    3. Type messaging - kafka
  4. Comment(optional): the comment of this catalog
  5. Properties(each provider must fill in the required property fields specifically)
Providers

Required properties in various providers

1. Type relational
Follow the [Apache Hive catalog](./apache-hive-catalog.md) document. 
<Image img={require('./assets/webui/props-hive.png')} style={{ width: 480 }} />

|Key           |Description                                           |
|--------------|------------------------------------------------------|
|metastore.uris|The Hive metastore URIs e.g. `thrift://127.0.0.1:9083`|
Follow the [Lakehouse Iceberg catalog](./lakehouse-iceberg-catalog.md) document. 
the parameter `catalog-backend` provides two values: `hive`, and `jdbc`.

|Key            |Description      |
|---------------|-----------------|
|catalog-backend|`hive`, or `jdbc`|

- `hive`

<Image img={require('./assets/webui/props-iceberg-hive.png')} style={{ width: 480 }} />

|Key      |Description                     |
|---------|--------------------------------|
|uri      |Iceberg catalog URI config      |
|warehouse|Iceberg catalog warehouse config|

- `jdbc`

<Image img={require('./assets/webui/props-iceberg-jdbc.png')} style={{ width: 480 }} />

|Key          |Description                                                                                            |
|-------------|-------------------------------------------------------------------------------------------------------|
|uri          |Iceberg catalog URI config                                                                             |
|warehouse    |Iceberg catalog warehouse config                                                                       |
|jdbc-driver  |"com.mysql.jdbc.Driver" or "com.mysql.cj.jdbc.Driver" for MySQL, "org.postgresql.Driver" for PostgreSQL|
|jdbc-user    |jdbc username                                                                                          |
|jdbc-password|jdbc password                                                                                          |
Follow the [JDBC MySQL catalog](./jdbc-mysql-catalog.md) document. 
<Image img={require('./assets/webui/props-mysql.png')} style={{ width: 480 }} />

|Key          |Description                                                                                        |
|-------------|---------------------------------------------------------------------------------------------------|
|jdbc-driver  |JDBC URL for connecting to the database. e.g. `com.mysql.jdbc.Driver` or `com.mysql.cj.jdbc.Driver`|
|jdbc-url     |e.g. `jdbc:mysql://localhost:3306`                                                                 |
|jdbc-user    |The JDBC user name                                                                                 |
|jdbc-password|The JDBC password                                                                                  |
Follow the [JDBC PostgreSQL catalog](./jdbc-postgresql-catalog) document. 
<Image img={require('./assets/webui/props-pg.png')} style={{ width: 480 }} />

|Key          |Description                                          |
|-------------|-----------------------------------------------------|
|jdbc-driver  |e.g. `org.postgresql.Driver`                    |
|jdbc-url     |e.g. `jdbc:postgresql://localhost:5432/your_database`|
|jdbc-user    |The JDBC user name                                   |
|jdbc-password|The JDBC password                                    |
|jdbc-database|e.g. `pg_database`                                   |
Follow the [JDBC Doris catalog](./jdbc-doris-catalog.md) document. 
<Image img={require('./assets/webui/props-doris.png')} style={{ width: 480 }} />

|Key          |Description                                                          |
|-------------|---------------------------------------------------------------------|
|jdbc-driver  |JDBC URL for connecting to the database. e.g. `com.mysql.jdbc.Driver`|
|jdbc-url     |e.g. `jdbc:mysql://localhost:9030`                                   |
|jdbc-user    |The JDBC user name                                                   |
|jdbc-password|The JDBC password                                                    |

:::tip Due to the current limitation of the web interface, which only allows for viewing, the functionality to create or modify schema, tables, or filesets is not available. Please refer to the documentation to use the REST API for these operations. :::

2. Type fileset
Follow the [Hadoop catalog](./hadoop-catalog.md) document. 
<Image img={require('./assets/webui/create-fileset-hadoop-catalog-dialog.png')} style={{ width: 480 }} />
3. Type messaging
Follow the [Kafka catalog](./kafka-catalog.md) document. 
<Image img={require('./assets/webui/create-messaging-kafka-catalog-dialog.png')} style={{ width: 480 }} />

| Key               | Description                                                                               |
| ----------------- | ----------------------------------------------------------------------------------------- |
| bootstrap.servers | The Kafka broker(s) to connect to, allowing for multiple brokers by comma-separating them |

After verifying the values of these fields, clicking on the CREATE button creates a catalog.

created-catalog

Show catalog details

Clicking on the action icon in the table cell.

You can see the detailed information of this catalog in the drawer component on the right.

show-catalog-details

Edit catalog

Clicking on the action icon in the table cell.

Displays the dialog for modifying fields of the selected catalog.

update-catalog

Only the name, comment, and custom fields in properties can be modified, other fields such as type, provider, and default fields in properties cannot be modified.

The fields that are not allowed to be modified cannot be selected and modified in the web UI.

Delete catalog

Clicking on the action icon in the table cell.

Displays a confirmation dialog, clicking on the SUBMIT button deletes this catalog.

delete-catalog

Schema

demo-tables

Table

demo-columns

Fileset

fileset-details

Topic

topic-details

Feature capabilities

Page Capabilities
Metalake View ✔ / Create ✔ / Edit ✔ / Delete ✔
Catalog View ✔ / Create ✔ / Edit ✔ / Delete ✔
Schema View ✔ / Create ✘ / Edit ✘ / Delete ✘
Table View ✔ / Create ✘ / Edit ✘ / Delete ✘
Fileset View ✔ / Create ✘ / Edit ✘ / Delete ✘
Topic View ✔ / Create ✘ / Edit ✘ / Delete ✘

E2E test

End-to-end testing for web frontends is conducted using the Selenium testing framework, which is Java-based.

Test cases can be found in the project directory: integration-test/src/test/java/org/apache/gravitino/integration/test/web/ui, where the pages directory is designated for storing definitions of frontend elements, among others. The root directory contains the actual steps for the test cases.

:::tip While writing test cases, running them in a local environment may not pose any issues.

However, due to the limited performance capabilities of GitHub Actions, scenarios involving delayed DOM loading—such as the time taken for a popup animation to open—can result in test failures.

To circumvent this issue, it is necessary to manually insert a delay operation, for instance, by adding such as Thread.sleep(sleepTimeMillis).

This ensures that the test waits for the completion of the delay animation before proceeding with the next operation, thereby avoiding the problem.

It is advisable to utilize the waits methods inherent to Selenium as a substitute for Thread.sleep(). :::