Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Initial Classes and Interface for Transfer Manager #1874

Merged
merged 16 commits into from
Feb 2, 2023

Conversation

sydney-munro
Copy link
Collaborator

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> ☕️

If you write sample code, please follow the samples format.

@sydney-munro sydney-munro requested a review from a team as a code owner January 31, 2023 19:05
@product-auto-label product-auto-label bot added size: l Pull request size is large. api: storage Issues related to the googleapis/java-storage API. labels Jan 31, 2023
Copy link
Collaborator

@BenWhitehead BenWhitehead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few minor things to change, and a question or two.

public DownloadResult build() {
checkNotNull(input);
checkNotNull(status);
return new DownloadResult(input, outputDestination, status, exception);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: validate either outputDestination or exception are non-null relative to the value of status.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will add this in a follow up. I know its only a few lines but was trying to get through the other comments first.


class TransferManagerConfig {
@NonNull private final int maxWorkers;
@NonNull private final int perWorkerBufferSize;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thoughts on changing this to maxTotalBufferSize which could leave us wiggle room on how much per worker we use if we're able to determine scaling up buffer size can increase throughput.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like we are moving further from the original design which was chunkSize if we continue in this direction. I don't think this is necessarily a bad thing but node and python are also out already with chunkSize.

@BenWhitehead BenWhitehead added this to the Transfer Manager milestone Feb 1, 2023
@sydney-munro sydney-munro merged commit 57fa6d5 into feat/transfer-manager Feb 2, 2023
@sydney-munro sydney-munro deleted the transfer-proto branch February 2, 2023 20:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: storage Issues related to the googleapis/java-storage API. size: l Pull request size is large.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants