Add rclone to conda-requirements.txt

[annefou]

No description provided.

[annefou]

@minrk I also would like to add the associated configuration files but I am not sure where to add them.

[minrk]

what files should go where? That probably belongs in the Dockerfile

[annefou]

what files should go where? That probably belongs in the Dockerfile

that is what I thought but we also need to have the user credentials for our buckets.

Below is what I have put in $HOME/.config/rclone/rclone.conf (manually and with correct access_key_id and secret_access_key):

[cmarine]
type = s3
provider = Other
endpoint = https://s3.waw3-1.cloudferro.com
acl = public-read

[gfts]
type = s3
provider = Other
env_auth = true
access_key_id = XXX
secret_access_key = YYY
endpoint = https://s3.gra.perf.cloud.ovh.net
profile = gfts
region = gra

[minrk]

it shouldn't need the access key or profile with env_auth = true, so I think the same file works for everyone. This config worked for me:

[gfts]
type = s3
provider = Other
env_auth = true
region = gra
endpoint = https://s3.gra.perf.cloud.ovh.net

[annefou]

then I can add it in the repo and docker file, right?

[minrk]

We'll still need to copy it into home in an initContainer here, so might as well create it there, rather than needing to update the image.

I can do that, if you want.

[annefou]

We'll still need to copy it into home in an initContainer here, so might as well create it there, rather than needing to update the image.

I can do that, if you want.

Out of curiosity, why can't we copy it in the docker file (and have the config file in https://github.com/destination-earth/DestinE_ESA_GFTS/tree/main/jupyterhub/images/user) e.g. updating the current Dockerfile with something like:

WORKDIR /home/$NB_USER
RUN mkdir -p ./.config/rclone
COPY rclone.conf ./.config/rclone

[minrk]

$HOME is a mounted volume in the container, it doesn't come from the image. Files in $HOME in the image will not be available in the container.

That would work if we had no persistence of your home directory.

[annefou]

Ok! I got it. just to understand: in here, would you generate the config file on the fly?

[minrk]

yes, that's right

[annefou]

Ok. then let me try...