Fix GCSToGCSOperator copying list of objects without wildcard #28111
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes:
GCSToGCSOperator
in case copying list of objects without wildcardAforementioned changes of
GCSToGCSOperator
cover the following cases:The previous implementation didn't actually copy files - it was just creating an empty destination folder. The following fix solves this problem and performs actual copying of the listed files into the specified destination folder.
For example, we have a folder
test_folder/
and a filetest_folder/file.txt
inside of it. If we miss a trailing slash at the source folder name, then the previous implementation instead of copying the filefile.txt
were creating two filestest_folder
andnew_folderfile.txt
. It seems that there are two bugs here:a) a file
new_folder
created instead of a foldernew_folder/
;b) a wrong path
new_folderfile.txt
for the copied file was generated instead ofnew_folder/file.txt
.The following fix resolves these problems.