Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document design goals for S3Path #77

Open
nickrobinson251 opened this issue Apr 1, 2020 · 2 comments
Open

Document design goals for S3Path #77

nickrobinson251 opened this issue Apr 1, 2020 · 2 comments

Comments

@nickrobinson251
Copy link
Contributor

Had a really helpful offline conversation with @rofinn about this, in the context of #76 and differing from other interfaces such as aws-cli.

Soem points I took away:

  • We try to be consistent with Julia base first and foremost.
  • We try to be consistent with the FilePathsBase API.
  • We are trying to provide a filepath-like interface.
  • This means we want to avoid special-casing the behaviour of methods on S3Paths in order to match aws-cli or any other interface for interacting with S3.

As Rory put it:

My guess is that the cli is doing a bunch of special casing to make that work nicely with S3. The goal is to avoid that as much as possible with the path API.
From the S3 docs:

The Amazon S3 console treats all objects that have a forward slash (“/”) character as the last (trailing) character in the key name as a folder, for example examplekeyname/. You can’t upload an object that has a key name with a trailing “/” character using the Amazon S3 console. However, you can upload objects that are named with a trailing “/” with the Amazon S3 API by using the AWS CLI, AWS SDKs, or REST API.
An object that is named with a trailing “/” appears as a folder in the Amazon S3 console. The Amazon S3 console does not display the content and metadata for such an object. When you use the console to copy an object named with a trailing “/”, a new folder is created in the destination location, but the object’s data and metadata are not copied.

I don’t think the the goal of an S3Path should be to emulate a completely different command line tool over the parent type and existing filepaths API

It's particularly important to be watchful / consistent with S3Paths, given the weirdness of S3 "directories".

https://docs.aws.amazon.com/AmazonS3/latest/user-guide/using-folders.html

@nickrobinson251
Copy link
Contributor Author

For actually adding something like this to hosted docs, we first need #49

@nickrobinson251
Copy link
Contributor Author

Xref #226

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant