Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature]: Enrich Catalog types #507

Closed
2 tasks done
XBaith opened this issue Oct 24, 2022 · 5 comments
Closed
2 tasks done

[Feature]: Enrich Catalog types #507

XBaith opened this issue Oct 24, 2022 · 5 comments
Labels
type:feature Feature Requests

Comments

@XBaith
Copy link
Contributor

XBaith commented Oct 24, 2022

Description

Now, Arctic only completely support hadoop catalog type.But there are some types of catalog that need to be implemented or refined: such as AWS GlueCatalog.

Use case/motivation

Supports which arctic tables need to be stored on AWS.

Describe the solution

  1. Implement an ArcticFileIO named ArcticS3FileIO to support writing data to S3.
  2. Implement an ArcticCatalog which called ArcticGlueCatalog to support managing Glue tables.

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@XBaith XBaith added the type:feature Feature Requests label Oct 24, 2022
@XBaith XBaith changed the title [Feature]: Need to improve the type of catalog [Feature]: Enrich Catalog types Oct 24, 2022
@XBaith
Copy link
Contributor Author

XBaith commented Oct 24, 2022

Hi @wangtaohz plz check this issue and give some suggestions

@baiyangtx
Copy link
Contributor

We are doing a Catalog and Mixed-format code refactor, this maybe solve your problem.

You can track the process in this #1061

@XBaith
Copy link
Contributor Author

XBaith commented Apr 25, 2023

We are doing a Catalog and Mixed-format code refactor, this maybe solve your problem.

You can track the process in this #1061

Thanks

@zhoujinsong
Copy link
Contributor

@XBaith
As you are using the Iceberg format tables, you can manage iceberg tables with all kinds of catalog type now:

  • Choose meta store type to Hive if you are using Iceberg HiveCatalog
  • Choose meta store type to Hadoop if you are using Iceberg HadoopCatalog
  • Choose meta store type to Custom if you are using other Iceberg Catalogs like AWS/JDBC/REST and remember set the catalog-impl property in your catalog setting.

@zhoujinsong
Copy link
Contributor

Closed as have been supported.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:feature Feature Requests
Projects
None yet
Development

No branches or pull requests

3 participants