Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

copy.Options: add an InstancePlatforms field #1938

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

nalind
Copy link
Member

@nalind nalind commented May 2, 2023

Add an InstancePlatforms field to the copy.Options structure. When asked to copy only a specific subset of instances, combine the Instances list with the instances which match the platforms given in the InstancePlatforms list, returning an error if we're unable to find an instance which matches any of the InstancePlatforms. I think this is a necessary step in implementing RUN-1803 in skopeo cleanly.

Copy link
Collaborator

@mtrmac mtrmac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

The implementation looks good overall.

WRT the API:

Looking at the example in containers/skopeo#1935 , and the code in https://github.com/osbuild/osbuild/blob/b92168bc16d8251256ec34df938d616bca99cd22/sources/org.osbuild.skopeo#L105 , it seems that the primary use case revolves around the CopySystemImage choice, what containers/skopeo#1935 calls --multi-arch=sparse:[system]. (Cc: @achilleas-k to keep me honest.)

How is that going to be represented? I’m a bit torn.

  • One approach is to model it in this API explicitly (and therefore inherit the way platform.WantedPlatforms consumes types.SystemContext): seems more consistent, OTOH the API could be more complex, either with the members of InstancePlatforms some kind of sum type (enum{ case system; case platform(platform) }), or with an extra InstanceForSystemPlatform field. Not quite elegant.
  • Another is to do this just in the Skopeo CLI. Probably simpler all around, OTOH I think it’s valuable to be fairly strict about Skopeo features all being available to other callers of c/image.

Ideally I think some kind of c/image API this would be better, assuming we can design a practical one that doesn’t grow to an absurd size.

This PR doesn’t say, so I assume it plans to use the Skopeo approach? Or a separate c/image PR?


(For the record, and mostly because I wrote this for the initial implementation which was destructive. Not a concern for the code as is.

  • There are two kinds of users in this area
    • Some want to keep the original manifest list, and create a sparse copy (with some instances missing), so that the digest references are not broken — that’s what RUN-1803 wants, some registries reject such lists.
    • Some want to strip the modify the manifest list, so that the copy is accepted by all registries. That’s e.g. Add copy option to strip sparse manifests #1707 .

The API to choose instances should (for the future where the “strip” variant exists) allow both, with a single orthogonal switch between the two behaviors. That’s probably going to happen “by default” in many implementations (the API of EditInstances kind of forces that on us :) ), just something to keep an eye on.)


Cc: @flouthoc for the interaction with the Zstd work — I think this should be mostly orthogonal, just a heads-up about potential merge conflicts.

copy/multiple.go Outdated Show resolved Hide resolved
copy/multiple.go Outdated Show resolved Hide resolved
copy/multiple.go Outdated Show resolved Hide resolved
copy/multiple.go Outdated Show resolved Hide resolved
copy/multiple.go Show resolved Hide resolved
@TomSweeneyRedHat
Copy link
Member

The VM img.keepalive test had failed, it look like a google instance issue. I've restarted.

@mtrmac
Copy link
Collaborator

mtrmac commented May 2, 2023

This PR doesn’t say, so I assume it plans to use the Skopeo approach?

containers/skopeo#1987 implements that in Skopeo.

@mtrmac
Copy link
Collaborator

mtrmac commented May 2, 2023

Note to self: the --override-arch=amd64 --multi-arch=sparse:system case in containers/skopeo#1987 needs a more detailed look.

@mtrmac
Copy link
Collaborator

mtrmac commented May 2, 2023

Oh… The decision on what to filter on is actually non-obvious. Do we want platform triplets, or separately OSes and architectures? All three? containers/skopeo#1987 (comment) for a starting point, to continue tomorrow…

@nalind nalind force-pushed the instance-platforms branch 2 times, most recently from 474b858 to d2f3505 Compare May 3, 2023 13:43
Copy link
Collaborator

@mtrmac mtrmac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implementation of the API LGTM.

Outstanding design questions:

  • How to represent system
  • Do we want the user to specify whole platforms? Or just arch[+variant]?

copy/multiple_test.go Outdated Show resolved Hide resolved
@nalind nalind force-pushed the instance-platforms branch 4 times, most recently from 60d1598 to f275417 Compare May 3, 2023 20:26
Copy link
Contributor

@flouthoc flouthoc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -44,3 +44,7 @@ func (s *Set[E]) Empty() bool {
func (s *Set[E]) Values() []E {
return maps.Keys(s.m)
}

func (s Set[E]) Size() int {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
func (s Set[E]) Size() int {
func (s *Set[E]) Size() int {

please.

The current version works just fine, but I managed to inattentively screw up the Add/Delete methods (#1951 ), so using a pointer receiver everywhere would hopefully make it harder for me to make the same mistake again.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed it.

Add an InstancePlatforms field to the copy.Options structure.  When
asked to copy only a specific subset of instances, combine the Instances
list with the instances which match the platforms given in the
InstancePlatforms list, returning an error if we're unable to find an
instance which matches any of the InstancePlatforms.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
@mtrmac mtrmac added the kind/feature A request for, or a PR adding, new functionality label Jun 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature A request for, or a PR adding, new functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants