Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement recursive uploads #415

Merged
merged 3 commits into from
Dec 5, 2023

Conversation

joereuss12
Copy link
Contributor

First draft of recursive uploads. This will go through a directory and upload every file to the destination. The only problem is that it will not maintain the same directory structure since I could not find a way to have PUT work with directories.

@joereuss12 joereuss12 linked an issue Nov 28, 2023 that may be closed by this pull request
@joereuss12
Copy link
Contributor Author

@bbockelm Would you like me to look more into somehow keeping the directory structure on the server side? As of now, it only spits all of the files into the specified directory on the origin because I could not figure out how to do a client side PUT with directories.

@joereuss12 joereuss12 force-pushed the recursive-uploads-branch branch from 21cd896 to 64e4794 Compare December 1, 2023 20:42
@joereuss12 joereuss12 marked this pull request as ready for review December 1, 2023 20:51
@joereuss12
Copy link
Contributor Author

Marking this ready for review. Cannot create a unit test because it requires a director to be spinning so it can find the namespace.DirListHost. Therefore, I will just wait to create an end-to-end test for it when Emma completes her "federation in a box".

@bbockelm
Copy link
Collaborator

bbockelm commented Dec 5, 2023

@joereuss12 - there are some conflicts to fix up here.

Copy link
Collaborator

@turetske turetske left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved when merge conflicts are resolved

First draft of recursive uploads. This will go through a directory and
upload every file to the destination. The only problem is that it will
not maintain the same directory structure since I could not find a way
to have PUT work with directories.
With the use of gowebdav.Mkdir() the UploadDirectory function now
maintains directory structure on the server side.
Also fixes issues with progress bars for recursive uploads. It is at the
same state as recursive downloads so it looks fine without debug logging
but is wonky with it. Functionality should work fine.
@joereuss12 joereuss12 force-pushed the recursive-uploads-branch branch from 64e4794 to bf4ba6d Compare December 5, 2023 22:41
@turetske turetske merged commit cafecbb into PelicanPlatform:main Dec 5, 2023
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Recursive upload of files
3 participants