Add view for all of a user's S3 files

[catarak]

No description provided.

[PaliwalSparsh]

I guess i will be making changes here. Here the project array contains the projects and the project contains all the files in it. So we can iterate over the projects and list the files with url.

Other than this bug.
I felt it would be great if we would manage S3 like this -
http://docs.aws.amazon.com/AmazonS3/latest/dev/ListingKeysHierarchy.html
So that we can make queries to S3 more efficiently.

[catarak]

I definitely think there is room to improve the S3 querying/storage, since when I implemented the S3 uploading I wasn't thinking about querying. Do you think it makes sense to store files on S3 by username and project id? Then, for example, say you wanted to upload a file dog.jog to your project with the id abc123, and your username is catarak:
catarak/abc123/dog.jpg
Then, we could use this prefix querying to get all files for a specific user just by querying with catarak.

[catarak]

I guess this means we'd also have to write a script to migrate all of the existing files to the new naming scheme, but I don't think that would be too hard. We'd just loop through all of the projects stored in MongoDB, look at the files that have a URL, change the URL, and then update the file with the new URL.

[PaliwalSparsh]

Exactly i thought about it similarly.
<username>/<project_id>/<filename>

Outcome of our investment in establishing hierarchy -
Doing this will be really helpful for listing objects present in S3 with a query, this was the place where I saw the use of having hierarchy could help. Though the same listing operations can be done by us on the server side only and we need not move to S3 ie by just looking at files with url's(as we plan to do in this issue). I don't think things like deleting or downloading files will be affected on introducing hierarchy much as they require directly the file key.

But surely more structured S3 won't hurt.

[catarak]

I just realized that if a user uploads a file and the project has never been saved, then the project doesn't have an id. So I think we'll have to structure these like <username>/<filename>. I think this is fine anyway since we want to enforce a limit on the total size of the files a user can upload, not just the files per project.