Update WebHDFS docs pending https://github.com/iterative/dvc/pull/6936 #3009
Conversation
|
This has now been released with 2.8.3. |
shcheklein
left a comment
shcheklein
left a comment
There was a problem hiding this comment.
Thanks @gudmundur-heimisson !
| - `user` - user name to access the remote, can be empty in case of using `token` | ||
| or if using a `HdfsCLI` cfg file. May only be used when Hadoop security is | ||
| off. Defaults to current user as determined by `whoami`. | ||
| Only provide the `user` parameter if you are not using `kerberos` or `token` | ||
| authentication, since those authentication methods already contain the user | ||
| information. |
There was a problem hiding this comment.
A bit confused. So there's no more user parameter? This removes that bullet completely.
I see it's removed from the explicit schema in https://github.com/iterative/dvc/pull/6936/files but it's still used on the test so it must be somewhere I think.
@gudmundur-heimisson @efiop thanks
There was a problem hiding this comment.
In the test it is read out of the URL string. It feels confusing to me to be able to provide the user in both the URL string and also as an explicit parameter so I just removed it as a parameter, but I don't feel that strongly about it.
As a side-note for WebHDFS if the cluster is secured you will never be using this parameter anyway because you must use kerberos or the token (unless you have a Cloudera cluster which can allow LDAP, but that is not part of the Apache standard).
If my understanding is correct, because of this line if we allow it as an explicit parameter as well it would override whatever is in the URL string, which we should probably document somewhere:
There was a problem hiding this comment.
Thanks. Still confused though, so is the user param deprecated currently? @efiop
* ref: copy edit Kerberos info in `remote add` (WebHDFS) rel. #3009 * ref: re-explain Kerberos info in `remote modify` (WebHDFS) * typos & CE * ref: separate HDFS and WebHDFS config notes per #3019 (review)
4 participants
Updates to WebHDFS remote documentation to reflect changes in:
treeverse/dvc#6936
Also see dvc issue:
treeverse/dvc#6935