Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support oci layout target #692

Closed
wants to merge 68 commits into from
Closed

Conversation

qweeah
Copy link
Contributor

@qweeah qweeah commented Nov 17, 2022

This PR added a new flag --oci to support oci layout in all commands EXCEPT below:

  • oras manifest delete, oras blob delete: oras-go doesn't support Garbage Collection for OCI targets.
  • oras discover: OCI layout referrer is not well designed in oras-go yet and graph predecessors won't contain artifact type.
  • oras login, oras logout and oras repo ls: OCI layout itself represents a local repository, not a remote registry and won't need to support those.

The reference format of a OCI layout reference is <path>[:<tag>|@<digest>]
You can:

  1. Pull an image in oci layout to local folder ghcr.io/oras-project/oras with tag v0.16.0
oras cp ghcr.io/oras-project/oras:v0.16.0 ghcr.io/oras-project/oras:v0.16.0 --to-oci

you can inspect the pulled image via

oras manifest fetch  ghcr.io/oras-project/oras:v0.16.0 --oci
oras manifest fetch  ghcr.io/oras-project/oras:v0.16.0 --platform linux/amd64 --oci #inspect certain platform
  1. Push the image into a remote repository oci-layout in registry some.registry with tag from-folder, you can simply do
$IMAGE="some.registry/oci-layout:from-folder"
oras cp ghcr.io/oras-project/oras:v0.16.0 $IMAGE --from-oci 

or push from a tarball

cd ghcr.io/oras-project/oras
tar -cvf pulled.tar *
oras cp pulled.tar:test $IMAGE --from-oci 
  1. Attach a signature towards it
echo sig > sig
oras attach ghcr.io/oras-project/oras:v0.16.0 sig --artifact-type test.signature --oci

Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
@codecov-commenter
Copy link

codecov-commenter commented Nov 17, 2022

Codecov Report

Merging #692 (e161136) into main (09e997a) will decrease coverage by 9.94%.
The diff coverage is 18.54%.

@@            Coverage Diff             @@
##             main     #692      +/-   ##
==========================================
- Coverage   73.17%   63.22%   -9.95%     
==========================================
  Files          16       18       +2     
  Lines         574      688     +114     
==========================================
+ Hits          420      435      +15     
- Misses        123      220      +97     
- Partials       31       33       +2     
Impacted Files Coverage Δ
cmd/oras/internal/option/parser.go 0.00% <0.00%> (ø)
cmd/oras/internal/option/target.go 11.76% <11.76%> (ø)
cmd/oras/internal/option/platform.go 100.00% <100.00%> (ø)
cmd/oras/internal/option/remote.go 69.51% <100.00%> (+1.21%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Comment on lines 31 to 35
// Target option struct.
type BinaryTarget struct {
From Target
To Target
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not a big fan of this target parameter. It seems like the type could be encoded in the source and destination like URIs. Create a factory method like you that creates a destination and sources based on the string. E.g.: oras cp file://image.tgz oci://harbor.mine.demo:30003/hello:latest

The default could be an oci registry so oras cp file://image.tgz harbor.mine.demo:30003/hello:latest

Copy link
Contributor Author

@qweeah qweeah Dec 5, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @TerryHowe for reviewing this. Indeed, the OCI layout type could be encoded in the source and destination like URIs(and it's really a good user experience), but there are still other configurable options that requires user input:

  1. what is the source tag/digest for copying?
    The example scenario you are giving should be copying from a local oci layout file image.tgz towards a remote registry harbor.mine.demo:30003/hello with tag latest. We can use oci:// to specify the source type and ignore http(s):// in the dest, so the command looks like below

oras cp oci://image.tgz harbor.mine.demo:30003/hello:latest

In the command, only the path of an oci-layout is provided but what should be copied from image.tgz is not specified, either we create a new flag for that or we use a delimitor so user can specify both path and org.opencontainers.image.ref.name at the same time.

  1. what is the type of oci-layout, is it a tarball or folder?
    ORAS is able to speculate those if an oci layout target is the copy source, but what if the copy destination is an oci-layout?
    Say thethe command is: oras cp harbor.mine.demo:30003/hello:latest oci://image. Without dedicated flags,
    the user can possibly want pull harbor.mine.demo:30003/hello:latest and add it to 1) a local oci layout folder named image; 2) a local oci layout tarball named image;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW I am still working on the design spec for supporting oci layout, will share it to you if once it's ready for review

Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Signed-off-by: Billy Zha <jinzha1@microsoft.com>
@qweeah qweeah changed the title (WIP) feat: support oci layout target feat: support oci layout target Jan 6, 2023
@qweeah qweeah marked this pull request as ready for review January 6, 2023 03:04
@qweeah qweeah marked this pull request as draft January 6, 2023 04:34
@qweeah
Copy link
Contributor Author

qweeah commented Jan 6, 2023

Will separate changes to smaller PRs

@qweeah
Copy link
Contributor Author

qweeah commented May 16, 2023

close as it's merged

@qweeah qweeah closed this May 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants