Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(oci-layout): support in fetch and push for manifest and blob commands #752

Merged
merged 49 commits into from
Jan 18, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
83c4af1
feat: support oci image layout in `oras cp`
qweeah Jan 13, 2023
f8d4a27
add example
qweeah Jan 13, 2023
2026d9d
add example
qweeah Jan 13, 2023
071c792
add go mod
qweeah Jan 13, 2023
c244cd5
rename
qweeah Jan 13, 2023
a5fd875
doc clean
qweeah Jan 13, 2023
d4f96d3
update license header
qweeah Jan 13, 2023
b0e8e28
add interface for printer
qweeah Jan 13, 2023
3df7915
doc clean
qweeah Jan 13, 2023
ec2f556
rename
qweeah Jan 13, 2023
07dba15
Merge remote-tracking branch 'origin_src/main' into oci-copy
qweeah Jan 13, 2023
f5bb892
update examples
qweeah Jan 13, 2023
6dbd0a6
update doc
qweeah Jan 13, 2023
d71860c
doc clean
qweeah Jan 13, 2023
6d11cef
resolve comments
qweeah Jan 13, 2023
01c5674
add wrapper type for tag status printer
qweeah Jan 13, 2023
c20e948
remove long oci image layout flag
qweeah Jan 13, 2023
3962a55
doc clean
qweeah Jan 13, 2023
33f6561
resolve comments
qweeah Jan 14, 2023
fca9db1
Merge remote-tracking branch 'origin_src/main' into oci-copy
qweeah Jan 14, 2023
2a70519
fix unit test
qweeah Jan 14, 2023
6036d0d
rename target type
qweeah Jan 16, 2023
1e463d4
simplify examples
qweeah Jan 16, 2023
c83b54b
fix bug for windows path parsing and add tests
qweeah Jan 16, 2023
b4b2a6c
fix unit test
qweeah Jan 16, 2023
1585f5a
resolve comments
qweeah Jan 16, 2023
12fc8ea
doc clean
qweeah Jan 16, 2023
87771b6
doc clean
qweeah Jan 16, 2023
6af8f50
rename parsing arguments
qweeah Jan 16, 2023
56e29f9
doc clean
qweeah Jan 16, 2023
b9f5df3
resolve comments
qweeah Jan 16, 2023
440731e
fix doc
qweeah Jan 17, 2023
7543bd0
support `manifest push`
qweeah Jan 17, 2023
8e2bbc1
add doc
qweeah Jan 17, 2023
fe4fa9e
code clean
qweeah Jan 17, 2023
4faeac9
add blob fetch
qweeah Jan 17, 2023
a8785e1
support blob fetch
qweeah Jan 17, 2023
73dbbc0
Merge branch 'main' into oci-manifest-blob
qweeah Jan 17, 2023
612675d
doc clean
qweeah Jan 17, 2023
1911818
doc clean
qweeah Jan 17, 2023
a92d99d
doc clean
qweeah Jan 17, 2023
816d955
alignment
qweeah Jan 17, 2023
ce77cd8
bug fix
qweeah Jan 18, 2023
3ef5416
Merge branch 'main' into oci-manifest-blob
qweeah Jan 18, 2023
9e38fea
resolve comments
qweeah Jan 18, 2023
6c0df68
resolve comments
qweeah Jan 18, 2023
6e63a12
doc clean
qweeah Jan 18, 2023
dd9e27d
doc clean
qweeah Jan 18, 2023
dc3e52a
fix doc bug
qweeah Jan 18, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions cmd/oras/attach.go
Original file line number Diff line number Diff line change
Expand Up @@ -50,28 +50,28 @@ func attachCmd() *cobra.Command {

** This command is in preview and under development. **

Example - Attach file 'hi.txt' with type 'doc/example' to manifest 'hello:test' in registry 'localhost:5000':
oras attach --artifact-type doc/example localhost:5000/hello:test hi.txt
Example - Attach file 'hi.txt' with type 'doc/example' to manifest 'hello:v1' in registry 'localhost:5000':
oras attach --artifact-type doc/example localhost:5000/hello:v1 hi.txt

Example - Attach file "hi.txt" with specific media type when building the manifest:
oras attach --artifact-type doc/example --image-spec v1.1-image localhost:5000/hello:test hi.txt # OCI image
oras attach --artifact-type doc/example --image-spec v1.1-artifact localhost:5000/hello:test hi.txt # OCI artifact
oras attach --artifact-type doc/example --image-spec v1.1-image localhost:5000/hello:v1 hi.txt # OCI image
oras attach --artifact-type doc/example --image-spec v1.1-artifact localhost:5000/hello:v1 hi.txt # OCI artifact

Example - Attach file "hi.txt" using a specific method for the Referrers API:
oras attach --artifact-type doc/example --distribution-spec v1.1-referrers-api localhost:5000/hello:test hi.txt # via API
oras attach --artifact-type doc/example --distribution-spec v1.1-referrers-tag localhost:5000/hello:test hi.txt # via tag scheme
oras attach --artifact-type doc/example --distribution-spec v1.1-referrers-api localhost:5000/hello:v1 hi.txt # via API
oras attach --artifact-type doc/example --distribution-spec v1.1-referrers-tag localhost:5000/hello:v1 hi.txt # via tag scheme

Example - Attach file 'hi.txt' and add annotations from file 'annotation.json':
oras attach --artifact-type doc/example --annotation-file annotation.json localhost:5000/hello:latest hi.txt
oras attach --artifact-type doc/example --annotation-file annotation.json localhost:5000/hello:v1 hi.txt

Example - Attach an artifact with manifest annotations:
oras attach --artifact-type doc/example --annotation "key1=val1" --annotation "key2=val2" localhost:5000/hello:latest
oras attach --artifact-type doc/example --annotation "key1=val1" --annotation "key2=val2" localhost:5000/hello:v1

Example - Attach file 'hi.txt' and add manifest annotations:
oras attach --artifact-type doc/example --annotation "key=val" localhost:5000/hello:latest hi.txt
oras attach --artifact-type doc/example --annotation "key=val" localhost:5000/hello:v1 hi.txt

Example - Attach file 'hi.txt' and export the pushed manifest to 'manifest.json':
oras attach --artifact-type doc/example --export-manifest manifest.json localhost:5000/hello:latest hi.txt
oras attach --artifact-type doc/example --export-manifest manifest.json localhost:5000/hello:v1 hi.txt
`,
Args: cobra.MinimumNArgs(1),
PreRunE: func(cmd *cobra.Command, args []string) error {
Expand Down
37 changes: 23 additions & 14 deletions cmd/oras/blob/fetch.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,12 @@ import (
"io"
"os"

"github.com/opencontainers/go-digest"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/spf13/cobra"
"oras.land/oras-go/v2"
"oras.land/oras-go/v2/content"
"oras.land/oras-go/v2/registry/remote"
"oras.land/oras/cmd/oras/internal/option"
)

Expand All @@ -33,10 +35,9 @@ type fetchBlobOptions struct {
option.Common
option.Descriptor
option.Pretty
option.Remote
option.Target

outputPath string
targetRef string
}

func fetchCmd() *cobra.Command {
Expand All @@ -48,17 +49,23 @@ func fetchCmd() *cobra.Command {

** This command is in preview and under development. **

Example - Fetch the blob and save it to a local file:
Example - Fetch a blob from registry and save it to a local file:
oras blob fetch --output blob.tar.gz localhost:5000/hello@sha256:9a201d228ebd966211f7d1131be19f152be428bd373a92071c71d8deaf83b3e5

Example - Fetch the blob and print the raw blob content:
Example - Fetch a blob from registry and print the raw blob content:
oras blob fetch --output - localhost:5000/hello@sha256:9a201d228ebd966211f7d1131be19f152be428bd373a92071c71d8deaf83b3e5

Example - Fetch and print the descriptor of a blob:
oras blob fetch --descriptor localhost:5000/hello@sha256:9a201d228ebd966211f7d1131be19f152be428bd373a92071c71d8deaf83b3e5

Example - Fetch the blob, save it to a local file and print the descriptor:
Example - Fetch a blob, save it to a local file and print the descriptor:
oras blob fetch --output blob.tar.gz --descriptor localhost:5000/hello@sha256:9a201d228ebd966211f7d1131be19f152be428bd373a92071c71d8deaf83b3e5

Example - Fetch and print a blob from OCI image layout folder 'layout-dir':
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it exact to use layout-dir@sha256:9a201d228ebd966211f7d1131be19f152be428bd373a92071c71d8deaf83b3e5 to represent an OCI layout folder?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

layout-dir@sha256:9a201d228ebd966211f7d1131be19f152be428bd373a92071c71d8deaf83b3e5 means fetch a blob with digest sha256:9a201d228ebd966211f7d1131be19f152be428bd373a92071c71d8deaf83b3e5 from an OCI image layout folder 'layout-dir'

oras blob fetch --oci-layout --output - layout-dir@sha256:9a201d228ebd966211f7d1131be19f152be428bd373a92071c71d8deaf83b3e5

Example - Fetch and print a blob from OCI image layout archive file 'layout.tar':
oras blob fetch --oci-layout --output - layout.tar@sha256:9a201d228ebd966211f7d1131be19f152be428bd373a92071c71d8deaf83b3e5
`,
Args: cobra.ExactArgs(1),
PreRunE: func(cmd *cobra.Command, args []string) error {
Expand All @@ -69,12 +76,11 @@ Example - Fetch the blob, save it to a local file and print the descriptor:
if opts.outputPath == "-" && opts.OutputDescriptor {
return errors.New("`--output -` cannot be used with `--descriptor` at the same time")
}

opts.RawReference = args[0]
return option.Parse(&opts)
},
Aliases: []string{"get"},
RunE: func(cmd *cobra.Command, args []string) error {
opts.targetRef = args[0]
return fetchBlob(opts)
},
}
Expand All @@ -86,32 +92,35 @@ Example - Fetch the blob, save it to a local file and print the descriptor:

func fetchBlob(opts fetchBlobOptions) (fetchErr error) {
ctx, _ := opts.SetLoggerLevel()

repo, err := opts.NewRepository(opts.targetRef, opts.Common)
var target oras.ReadOnlyTarget
target, err := opts.NewReadonlyTarget(ctx, opts.Common)
if err != nil {
return err
}

if _, err = repo.Reference.Digest(); err != nil {
return fmt.Errorf("%s: blob reference must be of the form <name@digest>", opts.targetRef)
if _, err = digest.Parse(opts.Reference); err != nil {
return fmt.Errorf("%s: blob reference must be of the form <name@digest>", opts.RawReference)
}

src, err := opts.CachedTarget(repo.Blobs())
if repo, ok := target.(*remote.Repository); ok {
target = repo.Blobs()
}
src, err := opts.CachedTarget(target)
if err != nil {
return err
}

var desc ocispec.Descriptor
if opts.outputPath == "" {
// fetch blob descriptor only
desc, err = oras.Resolve(ctx, src, opts.targetRef, oras.DefaultResolveOptions)
desc, err = oras.Resolve(ctx, src, opts.Reference, oras.DefaultResolveOptions)
if err != nil {
return err
}
} else {
// fetch blob content
var rc io.ReadCloser
desc, rc, err = oras.Fetch(ctx, src, opts.targetRef, oras.DefaultFetchOptions)
desc, rc, err = oras.Fetch(ctx, src, opts.Reference, oras.DefaultFetchOptions)
if err != nil {
return err
}
Expand Down
24 changes: 13 additions & 11 deletions cmd/oras/blob/push.go
Original file line number Diff line number Diff line change
Expand Up @@ -31,12 +31,11 @@ type pushBlobOptions struct {
option.Common
option.Descriptor
option.Pretty
option.Remote
option.Target

fileRef string
mediaType string
size int64
targetRef string
}

func pushCmd() *cobra.Command {
Expand All @@ -48,30 +47,33 @@ func pushCmd() *cobra.Command {

** This command is in preview and under development. **

Example - Push blob "hi.txt":
Example - Push blob 'hi.txt' to a registry:
oras blob push localhost:5000/hello hi.txt

Example - Push blob "hi.txt" with the specific digest:
Example - Push blob 'hi.txt' with the specific digest:
oras blob push localhost:5000/hello@sha256:9a201d228ebd966211f7d1131be19f152be428bd373a92071c71d8deaf83b3e5 hi.txt

Example - Push blob from stdin with blob size and digest:
oras blob push --size 12 localhost:5000/hello@sha256:9a201d228ebd966211f7d1131be19f152be428bd373a92071c71d8deaf83b3e5 -

Example - Push blob "hi.txt" and output the descriptor:
Example - Push blob 'hi.txt' and output the descriptor:
oras blob push --descriptor localhost:5000/hello hi.txt

Example - Push blob "hi.txt" with the specific returned media type in the descriptor:
Example - Push blob 'hi.txt' with the specific returned media type in the descriptor:
oras blob push --media-type application/vnd.oci.image.config.v1+json --descriptor localhost:5000/hello hi.txt

Example - Push blob "hi.txt" and output the prettified descriptor:
Example - Push blob 'hi.txt' and output the prettified descriptor:
oras blob push --descriptor --pretty localhost:5000/hello hi.txt

Example - Push blob without TLS:
oras blob push --insecure localhost:5000/hello hi.txt

Example - Push blob 'hi.txt' into an OCI layout folder 'layout-dir':
oras blob push --oci-layout layout-dir hi.txt
`,
Args: cobra.ExactArgs(2),
PreRunE: func(cmd *cobra.Command, args []string) error {
opts.targetRef = args[0]
opts.RawReference = args[0]
opts.fileRef = args[1]
if opts.fileRef == "-" {
if opts.PasswordFromStdin {
Expand All @@ -97,13 +99,13 @@ Example - Push blob without TLS:
func pushBlob(opts pushBlobOptions) (err error) {
ctx, _ := opts.SetLoggerLevel()

repo, err := opts.NewRepository(opts.targetRef, opts.Common)
repo, err := opts.NewTarget(opts.Common)
if err != nil {
return err
}

// prepare blob content
desc, rc, err := file.PrepareBlobContent(opts.fileRef, opts.mediaType, repo.Reference.Reference, opts.size)
desc, rc, err := file.PrepareBlobContent(opts.fileRef, opts.mediaType, opts.Reference, opts.size)
if err != nil {
return err
}
Expand Down Expand Up @@ -139,7 +141,7 @@ func pushBlob(opts pushBlobOptions) (err error) {
return opts.Output(os.Stdout, descJSON)
}

fmt.Println("Pushed", opts.targetRef)
fmt.Println("Pushed", opts.AnnotatedReference())
fmt.Println("Digest:", desc.Digest)

return nil
Expand Down
5 changes: 2 additions & 3 deletions cmd/oras/cp.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,6 @@ import (
"github.com/spf13/cobra"
"oras.land/oras-go/v2"
"oras.land/oras/cmd/oras/internal/display"
"oras.land/oras/cmd/oras/internal/errors"
"oras.land/oras/cmd/oras/internal/option"
)

Expand Down Expand Up @@ -99,8 +98,8 @@ func runCopy(opts copyOptions) error {
if err != nil {
return err
}
if opts.From.Reference == "" {
return errors.NewErrInvalidReferenceStr(opts.From.RawReference)
if err := opts.From.EnsureReferenceNotEmpty(); err != nil {
return err
}

// Prepare destination
Expand Down
15 changes: 9 additions & 6 deletions cmd/oras/discover.go
Original file line number Diff line number Diff line change
Expand Up @@ -53,14 +53,17 @@ func discoverCmd() *cobra.Command {

** This command is in preview and under development. **

Example - Discover direct referrers of manifest 'hello:latest' in registry 'localhost:5000':
oras discover localhost:5000/hello
Example - Discover direct referrers of manifest 'hello:v1' in registry 'localhost:5000':
oras discover localhost:5000/hello:v1

Example - Discover all the referrers of manifest 'hello:latest' in registry 'localhost:5000' and display in a tree view:
oras discover -o tree localhost:5000/hello
Example - Discover all the referrers of manifest 'hello:v1' in registry 'localhost:5000', displayed in a tree view:
oras discover -o tree localhost:5000/hello:v1

Example - Discover referrers with type 'test-artifact' of manifest 'hello:latest' in registry 'localhost:5000':
oras discover --artifact-type test-artifact localhost:5000/hello
Example - Discover all the referrers of manifest with annotations, displayed in a tree view:
oras discover -v -o tree localhost:5000/hello:v1

Example - Discover referrers with type 'test-artifact' of manifest 'hello:v1' in registry 'localhost:5000':
oras discover --artifact-type test-artifact localhost:5000/hello:v1
`,
Args: cobra.ExactArgs(1),
PreRunE: func(cmd *cobra.Command, args []string) error {
Expand Down
9 changes: 9 additions & 0 deletions cmd/oras/internal/option/target.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ import (
"oras.land/oras-go/v2"
"oras.land/oras-go/v2/content/oci"
"oras.land/oras-go/v2/registry"
"oras.land/oras/cmd/oras/internal/errors"
"oras.land/oras/cmd/oras/internal/fileref"
)

Expand Down Expand Up @@ -165,6 +166,14 @@ func (opts *Target) NewReadonlyTarget(ctx context.Context, common Common) (ReadO
return nil, fmt.Errorf("unknown target type: %q", opts.Type)
}

// EnsureReferenceNotEmpty ensures whether the tag or digest is empty.
func (opts *Target) EnsureReferenceNotEmpty() error {
if opts.Reference == "" {
return errors.NewErrInvalidReferenceStr(opts.RawReference)
}
return nil
}

// BinaryTarget struct contains flags and arguments specifying two registries or
// image layouts.
type BinaryTarget struct {
Expand Down
8 changes: 4 additions & 4 deletions cmd/oras/manifest/delete.go
Original file line number Diff line number Diff line change
Expand Up @@ -46,14 +46,14 @@ func deleteCmd() *cobra.Command {

** This command is in preview and under development. **

Example - Delete a manifest tagged with 'latest' from repository 'localhost:5000/hello':
oras manifest delete localhost:5000/hello:latest
Example - Delete a manifest tagged with 'v1' from repository 'localhost:5000/hello':
oras manifest delete localhost:5000/hello:v1

Example - Delete a manifest without prompting confirmation:
oras manifest delete --force localhost:5000/hello:latest
oras manifest delete --force localhost:5000/hello:v1

Example - Delete a manifest and print its descriptor:
oras manifest delete --descriptor localhost:5000/hello:latest
oras manifest delete --descriptor localhost:5000/hello:v1

Example - Delete a manifest by digest 'sha256:99e4703fbf30916f549cd6bfa9cdbab614b5392fbe64fdee971359a77073cdf9' from repository 'localhost:5000/hello':
oras manifest delete localhost:5000/hello@sha:99e4703fbf30916f549cd6bfa9cdbab614b5392fbe64fdee971359a77073cdf9
Expand Down
Loading