Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use OpenDAL to access object store #608

Closed
jiacai2050 opened this issue Feb 3, 2023 · 4 comments · Fixed by #720
Closed

Use OpenDAL to access object store #608

jiacai2050 opened this issue Feb 3, 2023 · 4 comments · Fixed by #720
Labels
dependencies Pull requests that update a dependency file feature New feature or request good first issue Good for newcomers

Comments

@jiacai2050
Copy link
Contributor

jiacai2050 commented Feb 3, 2023 •
edited
Loading

Describe This Problem

OpenDAL is widely adopted in Rust community to access files in many backends, such as S3, OSS, etc...

By integrate with it, we can gain more features from community.

Proposal

Replace AliyunOSS implementation with OpenDAL

Additional Context

No response
@jiacai2050 jiacai2050 added the feature New feature or request label Feb 3, 2023
@jiacai2050 jiacai2050 changed the title Use OpenDAL to access AliyunOSS Use OpenDAL to access object store Feb 3, 2023
@jiacai2050 jiacai2050 added good first issue Good for newcomers dependencies Pull requests that update a dependency file labels Feb 3, 2023
@tustvold
Copy link

FWIW DataFusion has first party support for object_store, which is part of the Apache Arrow project. Just thought I would mention it

@jiacai2050
Copy link
Contributor Author

@tustvold We already use object_store in our codebase, opendal here is meant to implement object_store trait for aliyun OSS.

@tustvold
Copy link

I think aliyun OSS might be supported by object_store natively - apache/arrow-rs#2777

@jiacai2050
Copy link
Contributor Author

That sounds reasonable to me, we will try implement OSS directly in object_store.

@jiacai2050 jiacai2050 mentioned this issue Mar 9, 2023
Merged
@tanruixiang tanruixiang mentioned this issue Oct 21, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
dependencies Pull requests that update a dependency file feature New feature or request good first issue Good for newcomers
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

refactor: remove custom oss impl jiacai2050/horaedb
2 participants
@tustvold @jiacai2050