Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use gaul/s3proxy #1113

Merged
merged 7 commits into from
Dec 28, 2020
Merged

use gaul/s3proxy #1113

merged 7 commits into from
Dec 28, 2020

Conversation

johnnyaug
Copy link
Contributor

No description provided.

@johnnyaug johnnyaug requested a review from ozkatz December 27, 2020 17:10
@@ -269,7 +275,16 @@ func RepoOperationHandler(sc *ServerContext, repoID string, handler operations.R
repo, err := authOp.Cataloger.GetRepository(sc.ctx, repoID)
if errors.Is(err, db.ErrNotFound) {
authOp.Log().WithField("repository", repoID).Warn("the specified repo does not exist")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning in both cases will be probably bad on the performance

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should warn on invalid requests or missing resources

Copy link
Collaborator

@ozkatz ozkatz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!! elegant and simple solution.
I would strongly suggest changing the configuration setting name

config/config.go Outdated Show resolved Hide resolved
@@ -269,7 +275,16 @@ func RepoOperationHandler(sc *ServerContext, repoID string, handler operations.R
repo, err := authOp.Cataloger.GetRepository(sc.ctx, repoID)
if errors.Is(err, db.ErrNotFound) {
authOp.Log().WithField("repository", repoID).Warn("the specified repo does not exist")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should warn on invalid requests or missing resources

gateway/handler.go Outdated Show resolved Hide resolved
gateway/handler.go Outdated Show resolved Hide resolved
gateway/handler.go Outdated Show resolved Hide resolved
@johnnyaug johnnyaug requested review from nopcoder and ozkatz December 28, 2020 11:26
@codecov-io
Copy link

Codecov Report

Merging #1113 (11ceb65) into master (07e7b31) will decrease coverage by 0.00%.
The diff coverage is 17.64%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1113      +/-   ##
==========================================
- Coverage   46.98%   46.98%   -0.01%     
==========================================
  Files         174      174              
  Lines       14016    14028      +12     
==========================================
+ Hits         6586     6591       +5     
- Misses       6576     6583       +7     
  Partials      854      854              
Impacted Files Coverage Δ
config/config.go 33.33% <0.00%> (-0.22%) ⬇️
gateway/handler.go 56.21% <18.75%> (-1.03%) ⬇️
catalog/mvcc/cataloger_create_repository.go 62.96% <0.00%> (ø)
catalog/mvcc/cataloger_commit.go 80.00% <0.00%> (+2.85%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 07e7b31...11ceb65. Read the comment docs.

Copy link
Collaborator

@ozkatz ozkatz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! I'm fine with merging this, only a couple of small comments

authOp.Log().WithField("repository", repoID).Warn("the specified repo does not exist")
authOp.EncodeError(gatewayerrors.ErrNoSuchBucket.ToAPIErr())
if sc.fallbackProxy == nil {
authOp.Log().WithField("repository", repoID).Warn("the specified repo does not exist")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For anything that is the user's fault I recommend using Debug

@@ -74,6 +74,7 @@ This reference uses `.` to denote the nesting of values.
(`*.s3.local.lakefs.io` always resolves to 127.0.0.1, useful for
local development
* `gateways.s3.region` `(string : "us-east-1")` - AWS region we're pretending to be. Should match the region configuration used in AWS SDK clients
* `gateways.s3.fallback_url` `(string)` - If specified, requests with a non-existing repository will be forwarded to this url.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add a note about s3 proxy and the use case?

@johnnyaug johnnyaug merged commit 8731b60 into master Dec 28, 2020
@johnnyaug johnnyaug deleted the feature/s3_passthrough branch December 28, 2020 13:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants