Skip to content

Allow passing fsspec options to WebHDFS to enable Kerberos and https #6935

@gudmundur-heimisson

Description

@gudmundur-heimisson

After #6662 it is no longer possible to use custom hdfscli clients with WebHDFS, as a result we are no longer able to connect using Kerberos or HTTPS.

fsspec allows passing in options to enable kerberos and https, we simply need to pass them through and expose them in the DVC config file:
https://filesystem-spec.readthedocs.io/en/latest/_modules/fsspec/implementations/webhdfs.html

This would also be an opportunity to remove the no longer used hdfscli options.

Metadata

Metadata

Assignees

No one assigned

    Labels

    fs: webhdfsRelated to the WebHDFS filesystemregressionOhh, we broke something :-(

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions