Investigate OpenDAL features #2367
Description
What is the problem the feature request solves?
We would like to learn more about OpenDAL for supporting HDFS, S3, and other file stores.
Some questions to answer:
- Do we still need to pursue the fs-hdfs approach or can we replace it with OpenDAL
- How does OpenDAL support s3? Does it use a Rust-native solution using the AWS Rust SDK, or does it use the same approach as HDFS support where it uses hadoop-aws in JVM?
- Can OpenDAL support all of our requirements around custom authentication?
- Does OpenDAL perform well?
Describe the potential solution
No response
Additional context
No response