Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Azure adapter #1444

Merged
merged 16 commits into from
Feb 16, 2021
Merged

Add Azure adapter #1444

merged 16 commits into from
Feb 16, 2021

Conversation

guy-har
Copy link
Contributor

@guy-har guy-har commented Feb 10, 2021

No description provided.

block/azure/chunkwriting.go Outdated Show resolved Hide resolved
@@ -5,6 +5,10 @@ import (
"errors"
"fmt"

"github.com/Azure/azure-storage-blob-go/azblob"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/dd

block/factory/build.go Outdated Show resolved Hide resolved
block/azure/adapter.go Outdated Show resolved Hide resolved
block/azure/adapter.go Show resolved Hide resolved
}
// write block sizes
sd := strconv.Itoa(int(cp.reader.CopiedSize)) + "\n"
_, err = cp.toSizes.StageBlock(cp.ctx, base64Etag, strings.NewReader(sd), cp.o.AccessConditions.LeaseAccessConditions, nil, cp.o.ClientProvidedKeyOptions)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion:
Use a struct:

type PartInfo struct {
  blockIDs []string `json:"block_ids"`
  size int `json:"size"`
}

Then, have just one file for all the blockInfos, and read it with one method and not two.

@codecov-io
Copy link

codecov-io commented Feb 11, 2021

Codecov Report

Merging #1444 (1ce074b) into master (a60d3a4) will decrease coverage by 0.80%.
The diff coverage is 10.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1444      +/-   ##
==========================================
- Coverage   39.86%   39.05%   -0.81%     
==========================================
  Files         156      159       +3     
  Lines       12478    12828     +350     
==========================================
+ Hits         4974     5010      +36     
- Misses       6795     7101     +306     
- Partials      709      717       +8     
Impacted Files Coverage Δ
block/namespace.go 62.12% <0.00%> (-4.01%) ⬇️
config/config.go 33.51% <16.66%> (-0.58%) ⬇️
graveler/graveler.go 32.16% <0.00%> (-4.02%) ⬇️
pyramid/tier_fs.go 58.77% <0.00%> (-3.95%) ⬇️
catalog/entry_catalog.go 13.68% <0.00%> (-2.11%) ⬇️
api/controller.go 36.53% <0.00%> (-1.28%) ⬇️
graveler/ref/manager.go 70.74% <0.00%> (-1.24%) ⬇️
catalog/rocks_cataloger.go 24.22% <0.00%> (-0.84%) ⬇️
api/client.go 3.15% <0.00%> (-0.09%) ⬇️
catalog/action.go
... and 7 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a60d3a4...1ce074b. Read the comment docs.

Copy link
Contributor

@nopcoder nopcoder left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A partial review looking into the adapter.go now

block/azure/adapter.go Show resolved Hide resolved
config/config.go Outdated Show resolved Hide resolved
config/config.go Outdated Show resolved Hide resolved
config/config.go Outdated Show resolved Hide resolved
block/factory/build.go Show resolved Hide resolved
block/factory/build.go Outdated Show resolved Hide resolved
block/factory/build.go Show resolved Hide resolved
block/azure/adapter.go Outdated Show resolved Hide resolved
block/azure/adapter.go Outdated Show resolved Hide resolved
block/azure/adapter.go Outdated Show resolved Hide resolved
block/azure/adapter.go Show resolved Hide resolved
block/azure/adapter.go Outdated Show resolved Hide resolved
block/azure/adapter.go Outdated Show resolved Hide resolved
block/azure/adapter.go Outdated Show resolved Hide resolved
block/azure/adapter.go Outdated Show resolved Hide resolved
block/azure/adapter.go Outdated Show resolved Hide resolved
block/azure/adapter.go Outdated Show resolved Hide resolved
block/azure/adapter.go Outdated Show resolved Hide resolved
block/azure/adapter.go Outdated Show resolved Hide resolved
block/azure/adapter.go Outdated Show resolved Hide resolved
block/azure/adapter.go Outdated Show resolved Hide resolved
block/azure/adapter.go Outdated Show resolved Hide resolved
block/azure/adapter.go Outdated Show resolved Hide resolved
block/azure/adapter.go Outdated Show resolved Hide resolved
block/azure/adapter.go Outdated Show resolved Hide resolved
block/azure/chunkwriting.go Outdated Show resolved Hide resolved
block/azure/chunkwriting.go Outdated Show resolved Hide resolved
@guy-har guy-har requested a review from nopcoder February 16, 2021 06:19
@guy-har guy-har marked this pull request as ready for review February 16, 2021 08:18
@guy-har guy-har merged commit 484738c into master Feb 16, 2021
@guy-har guy-har deleted the feature/azure-blob-store branch February 16, 2021 17:06
@guy-har guy-har linked an issue Feb 16, 2021 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support Azure Blob store
4 participants