Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Issue While doing Multi-part Upload on GCP (google cloud) [Just Verify] #112

Open
click2cloud-alpha opened this issue Apr 9, 2019 · 8 comments
Open

Issue While doing Multi-part Upload on GCP (google cloud) [Just Verify] #112

click2cloud-alpha opened this issue Apr 9, 2019 · 8 comments
Assignees
@merothh
Labels
backlog Backlog items - consider for next planning always SODACODE2022 SODACODE2022 Hackathon Label SODALOW SODACODE2022 Hackathon Low Priority Label
Milestone
Daito SODA v1.0

Comments

@click2cloud-alpha
Copy link
Contributor

Test Scenario: Performing migration of large data from AWS to GCP
File Size : 1.0 GB
File name : CentOs6.vhd
Error log
ss

Reason : GCP doesn’t provide Multipart Upload.
Blocking : we can achieve it with GCP parallel upload but OpenSDS multi-cloud common controller architecture supports multi-part API not composite and parallel upload proposed by GCP .
Limitation : GCP has limitation for multipart upload.

Reference link: https://stackoverflow.com/questions/27830432/google-cloud-storage-support-of-s3-multipart-upload
https://cloud.google.com/storage/docs/composite-objects

suggested Solution : if we need to resolve this , we would need your suggestion like how we can manage this parallel and composite upload of GCP with OpenSDS Multi-cloud Controller architechture.
@sfzeng
Copy link
Collaborator

sfzeng commented Apr 19, 2019 •
edited
Loading

It seems gcp do support multipart upload, and our multi-cloud architecture is compatible with it, but we need more time to fix it. See:
QQ图片20190419152616

@skdwriting
Copy link
Collaborator

@nguptaopensds / @sfzeng can we consider this for capri ? Please comment.

@skdwriting skdwriting added the CAPRI To be handled for capri 18th June label Jun 11, 2019
@kumarashit
Copy link
Collaborator

https://cloud.google.com/storage/docs/json_api/v1/how-tos/multipart-upload

@kumarashit
Copy link
Collaborator

https://www.zenko.io/blog/four-differences-google-amazon-s3-api/

@skdwriting skdwriting added backlog Backlog items - consider for next planning always and removed CAPRI To be handled for capri 18th June labels Jul 2, 2019
@skdwriting skdwriting added this to the Daito SODA v1.0 milestone Oct 25, 2019
This was referenced Dec 30, 2019
Closed
Merged
Click2cloud-Pallas pushed a commit to Click2Cloud/multi-cloud that referenced this issue Apr 8, 2021
@leonwanghui
Merge pull request sodafoundation#112 from jimccfun/isssue140
6e2d9df 
Fix interface issues with fileshare
@PravinRanjan10 PravinRanjan10 changed the title Isssue While doing Multi-part Upload on GCP (google cloud) Isssue While doing Multi-part Upload on GCP (google cloud) [Just Verify] Feb 28, 2022
@PravinRanjan10 PravinRanjan10 added SODACODE2022 SODACODE2022 Hackathon Label SODALOW SODACODE2022 Hackathon Low Priority Label labels Feb 28, 2022
@shelar1423
Copy link

is this issue closed>
? @PravinRanjan10

@PravinRanjan10
Copy link
Collaborator

@shelar1423 not yet. please take it and verify

@anvithks anvithks changed the title Isssue While doing Multi-part Upload on GCP (google cloud) [Just Verify] Issue While doing Multi-part Upload on GCP (google cloud) [Just Verify] Apr 2, 2022
@merothh
Copy link

merothh commented Apr 25, 2022

WIll verify

@merothh
Copy link

merothh commented Apr 26, 2022 •
edited
Loading

WIll verify

Uploading a 512M file stills fails.

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@merothh merothh

Labels
backlog Backlog items - consider for next planning always SODACODE2022 SODACODE2022 Hackathon Label SODALOW SODACODE2022 Hackathon Low Priority Label
Projects
None yet
Milestone
Daito SODA v1.0
Development

No branches or pull requests
8 participants
@skdwriting @merothh @sfzeng @PravinRanjan10 @click2cloud-alpha @kumarashit @nguptaopensds @shelar1423