[SUGGESTION] Organize uploaded files in different folders

[markmssd]

Hey all,

This is not an issue but a suggestion. If people seem to like it, I will implement it and PR.

I have the plugin installed and working perfectly. And the baseUrl set so my files get served through AWS CloudFront instead of through S3 directly (speed is crucial in my case).

However, all the files are getting uploaded to the same (root) directly. For example:

my-s3-bucket
 - file1.png
 - file2.jpeg
 - file3.pdf

I think it would be much better if everything was saved in its own folder, based on which Parse Class saved it. For example, assuming a class Car and a class House:

my-s3-bucket
 - Car <-- folder
   - car1.png
   - car2.jpeg
   - specs.pdf
- house <-- folder
   - house1.png
   - house2.png
   - outline.pdf

It could be broken even further based on which column saved them. For example, assuming the class Car has a column TopView and SideView:

my-s3-bucket
 - Car <-- folder
   - TopView <-- subfolder
       - car1.png
       - car2.jpeg
       - specs.pdf
   - SideView <-- subfolder
       - car1.png
       - car2.jpeg
       - specs.pdf

I personally think it will make files in the bucket much easier to navigate, especially when we have hundreds or thousands of them.

Let me know what you think!

[gateway]

This would be an awesome feature, while currently everything gets dumped into the root , I think people like to organize things as pointed out above.. any thoughts to maybe adding this?

[acinader]

my $.02 would be that this should be another, alternate file adapter. The simple s3 adapter does its job. A class based hierarchy could have its own adapter and a time based hierarchy (which we've used in other applications) could be another.

[flovilmart]

Could be additional adapter options as well

[super-fantastic]

Any updates on this? Awesome suggestion btw!

[flovilmart]

That would require a change in parse-server as well, which can be problematic as impacting all adapters. but why not.

[thedistricts]

This would be lovely. But even if you didn't go as far and simply made sure any filename strings passed through are prepended rather then appended to the hash that's generated by the adaptor.

An example:
here I'm passing through track-dNORNUbt9h as the trackId
PFFile(name:"track-\(trackId).json", data:data)

the file name saved to s3 is:
0001bcab9b405d09fe93d3d16cfd401f_track-dNORNUbt9h.json

S3 only allows you to search for file names via the prefix. Having the hash at the front essentially renders this feature unusable - unless you know the entire string.

[acinader]

I certainly see why that would be advantageous.

I think the problem here is that we shouldn't change the existing file name convention.

So there could be an option to switch the order?

@thedistricts are you interested in giving it a shot? I'd be glad to review.

[flovilmart]

What we can perhaps explore is allow / as a character in the filenames, and so, in parse-server, apply the uuid to the fileName not the full passed path. This way, one would be able to specify a path, like uploads/myFile.png which will be turned into uploads/UNIQUESTRING-myFile.png

Then it’s up to the adapters to support such sub paths.

That would be reasonable I believe. What do you think?

[acinader]

@flovilmart, much better suggestion.

[flovilmart]

Alright’ there’s a similar issue open on the server; I don’t have time now to implement it :/

[sariabuzahra]

@flovilmart, why do we need "UNIQUESTRING-" before filename? is there a way to remove that and keep the file name as is?

[flovilmart]

Because of potential collisions on the clients and retry mechanisms. Also it helps prevent a potential attacker guessing the name of the ‘next’ file

[sariabuzahra]

What if I'm controlling all of these from my side. Is there a way to make "adding unique prefix string" configurable? Or a way to override it?

[flovilmart]

@sariabuzahra There's a similar issue for this particular case opened on the server that you can track here: parse-community/parse-server#4899

@markmssd this issue is 100% parse-server. As I mentioned previously, I don't have time to implement it at this time, however, I velieve it's reasonable to allow the / separator in the file name if you wish to store the file in a subpath. This will require changes on parse-server. As you mentioned you were ready to PR / implement it, I'm all willing to assist you in there. Remember that this 'feature' should not impact all users and should be an 'opt-in'. I believe the reasonable path is the one described above (letting the file to have a name that has separators), yet I'm opened to other suggestions.

[markmssd]

Thanks @flovilmart ,

It's been a while I haven't touched Parse, it's just running smoothly in an old prototype I had back then, and I've moved to a Graph DB for the new app.

I'm willing to give it a try but having some time constraints for now. I will let you know when I start :)

[mpatnode]

The above PR pushes the directory decision to the parse client, and allows the S3 adapter to decide whether to allow directories or not. The current behavior is preserved if desired, or you can disable it via config on the FilesController and enable directories in the S3 adapter. IMHO: Unique filename guarantee belongs in the adapter. I suppose that desire could be expressed in the controller, then passed along to the currently loaded adapter for it to implement as it sees fit.

[dplewis]

Closing via #76