Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BitSail][Connector] Doris support deducing partitions. #149

Open
BlockLiu opened this issue Nov 9, 2022 · 4 comments
Open

[BitSail][Connector] Doris support deducing partitions. #149

BlockLiu opened this issue Nov 9, 2022 · 4 comments
Assignees
Labels
difficulty-easy Easy difficulty to fix this issue help wanted Extra attention is needed

Comments

@BlockLiu
Copy link
Collaborator

BlockLiu commented Nov 9, 2022

Is your feature request related to a problem? Please describe

Current Doris connector supports write one or more partitions in one job.

To write N partitions, users have to determine the details of the N partitions, including name, start and end range, which is a burden for users.
Details can be seen at com.bytedance.bitsail.connector.doris.option.DorisWriterOptions.PARTITIONS.

In some cases, these partitions do match some patterns, like:

{
   "name": "p2022-11-01",
   "start_range": "2022-11-01",
   "end_range": "2022-11-02",
},
{
   "name": "p2022-11-02",
   "start_range": "2022-11-02",
   "end_range": "2022-11-03",
}

So we may want to just give a template and the connector can recognize the pattern and automatically know what the partitions are.

Describe the solution you'd like

Describe alternatives you've considered

Additional context

@garyli1019 garyli1019 added help wanted Extra attention is needed difficulty-easy Easy difficulty to fix this issue labels Nov 9, 2022
@beyond-up
Copy link
Contributor

Hi,I am interested in bitsail and doris, can you assign this issue to me? thx!

beyond-up added a commit to beyond-up/bitsail that referenced this issue Dec 26, 2022
@BlockLiu
Copy link
Collaborator Author

Hi @beyond-up, thanks for your contribution to the BitSail community. Would you like to join contributor chat group? Here is the QR code. If it expires, you can leave a message here. Look forward to your coming!
ContributorGroup

@yucongcong654
Copy link
Collaborator

如果上面群聊二维码过期了,也可添加微信:13651994967,拉您入群 @beyond-up

@beyond-up
Copy link
Contributor

Sorry, I closed by mistake, please assign this issue to me again thx ! I will continue。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
difficulty-easy Easy difficulty to fix this issue help wanted Extra attention is needed
Projects
Status: No status
Development

No branches or pull requests

4 participants