Skip to content

image-layout: move refs to refs/tags #219

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

image-layout: move refs to refs/tags #219

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 12 additions & 12 deletions image-layout.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
## Open Container Initiative Image Layout Specification

The OCI Image Layout is a slash separated layout of OCI content-addressable blobs and [location-addressable](https://en.wikipedia.org/wiki/Content-addressable_storage#Content-addressed_vs._location-addressed) references (refs).
The OCI Image Layout is a slash separated layout of OCI content-addressable blobs and [location-addressable](https://en.wikipedia.org/wiki/Content-addressable_storage#Content-addressed_vs._location-addressed) reference tags (tags).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean we are dropping the concept of generic refs? I thought we were just calling out a "Reserved" structure within refs?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Mon, Aug 29, 2016 at 02:07:10PM -0700, Stephen Day wrote:

@@ -1,13 +1,13 @@

Open Container Initiative Image Layout Specification

-The OCI Image Layout is a slash separated layout of OCI content-addressable blobs and location-addressable references (refs).
+The OCI Image Layout is a slash separated layout of OCI content-addressable blobs and location-addressable reference tags (tags).

Does this mean we are dropping the concept of generic refs? I
thought we were just calling out a "Reserved" structure within
refs?

Ugh. In order of my decreasing preference:

a. Keep our current generic refs 1.
b. Replace generic refs with refs that have some required tag
semantics (this PR?).
c. Support both generic refs and refs with required tag semantics
(what you seem to be suggesting with the “Reserved structure”
comment).

Any type information about refs should go in the descriptor media
type field. I see no reason to bake it into the bucket that holds the
descriptors.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the use case of "generic refs"? I thought we were just trying to clear up the use case of the refs directory by making the tags use case explicit through naming, as git does.

The alternative proposal we discussed was just renaming "refs/" to "tags/".

Copy link
Contributor

@wking wking Aug 29, 2016 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Mon, Aug 29, 2016 at 02:24:21PM -0700, Brandon Philips wrote:

What is the use case of "generic refs"?

It allows image-layout users to link link other content if they see a
use case for it (vs. requiring refs to point to a manifest(-list) with
some standard name mapping). For example, Git has a tagged blob 1,
because that was a convenient way to distribute Junio's public key.

Having multiple image-names in one image-layout is a similar use-case
that I think we want to allow at the image-layout level.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is calling this a "tag" stopping you from having multiple image-names? I view this as similar to how GitHub has the gh-pages branch. We aren't restricting the use cases but trying to clearup the intended use case.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Mon, Aug 29, 2016 at 03:02:46PM -0700, Brandon Philips wrote:

I don't know. I simply don't understand why having a directory
called refs/tags is confusing.

You'll have people talking about refs and people talking about tags
and people wondering if there's some distinction between them (which
it sounds like there isn't).

What is your use case for having lots of refs pointing to different
images in a single image-layout?

I'm publishing a bunch of images on example.com, and don't want to be
bothered with maintaining multiple image-layout instances 1. Or I
want a convenient way to distribute my public key along with the
images it signs 2. And supporting generic refs protects
image-layout implementations from having to worry about scope creep
and such if a more opinionated refs spec decided that it needed to
support something different from the narrow use case it initially
picked.

rkt image import --name mystuff.com/foo/bar /path/to/oci-image-layout
docker import --name mystuff.com/foo/bar /path/to/oci-image-layout

In these cases the name and tags make sense. What is your use case?

I'd rather keep both explicit:

$ my-tool import LAYOUT-PATH REF [NAME]

unless we decide to support something like HEAD 1, in which case REF
would also be optional. If name was not specified on the command
line, the name used for the imported object would come from some
verifiable name (e.g. using the Merkle objects discussed in #176, see
3).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Mon, Aug 29, 2016 at 3:16 PM W. Trevor King notifications@github.com
wrote:

I'd rather keep both explicit: $ my-tool import LAYOUT-PATH REF [NAME]
unless we decide to support something like HEAD [1], in which case REF
would also be optional. If name was not specified on the command line, the
name used for the imported object would come from some verifiable name
(e.g. using the Merkle objects discussed in #176
#176, see [3]). [1]: #219
(comment)
#219 (comment)
[2]: #219 (comment)
#219 (comment)
[3]: #173 (comment)
#173 (comment)

Eh, I think that it is going to be more common for someone to download "an
entire image named foo" then wanting to import the whole thing and expect
the tags to show back up rather than forcing the user to specify every ref.
For example you would actually need to do:

my-tool import /path/to/layout my-img-foo-bar-v1.0.0 my.img/foo/bar v1.0.0
my-tool run my.img/foo/bar:v1.0.0

vs

my-tool import /path/to/layout my.img/foo/bar
my-tool run my.img/foo/bar:v1.0.0

I feel there is a usability cost to supporting lots of imgs in a single
layout on this common import UX.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Mon, Aug 29, 2016 at 03:24:32PM -0700, Brandon Philips wrote:

Eh, I think that it is going to be more common for someone to download "an
entire image named foo" then wanting to import the whole thing and expect
the tags to show back up rather than forcing the user to specify every ref.
For example you would actually need to do:

my-tool import /path/to/layout my-img-foo-bar-v1.0.0 my.img/foo/bar v1.0.0
my-tool run my.img/foo/bar:v1.0.0

vs

my-tool import /path/to/layout my.img/foo/bar
my-tool run my.img/foo/bar:v1.0.0

I feel there is a usability cost to supporting lots of imgs in a single
layout on this common import UX.

So add an:

$ my-tool import-all LAYOUT-PATH

that imports all the refs (still getting names from the CAS blobs
1). Or, without a HEAD spec, make:

$ my-tool import LAYOUT-PATH

(without an explicit REF) import all the refs.

Specifying an unverified name on the command line or extracting
information from the ref name (vs. the referenced CAS content)
should not be part of your main UX.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How would you get a name from a CAS blob?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Mon, Aug 29, 2016 at 04:19:33PM -0700, Brandon Philips wrote:

How would you get a name from a CAS blob?

See #176 and 1.

This layout MAY be used in a variety of different transport mechanisms: archive formats (e.g. tar, zip), shared filesystem environments (e.g. nfs), or networked file fetching (e.g. http, ftp, rsync).
Given an image layout a tool can convert a given ref into a runnable OCI Image Format by finding an appopriate manifest from the manifest list, unpacking the filesystem serializations in the correct order, and then converting the image configuration into an OCI Runtime config.json.

The image layout has two top level directories:
The image layout has two directory structures:

- "blobs" contains content-addressable blobs. A blob has no schema and should be considered opaque.
- "refs" contains [descriptors][descriptors]. Commonly pointing to an image manifest.
- "refs/tags" contains [descriptors][descriptors]. Commonly pointing to an image manifest. Other subdirectories under `refs/` are reserved for future use.


It also contains a file that is used to identify the layout version:
Expand All @@ -26,9 +26,9 @@ $ find .
./blobs/sha256-5b0bcabd1ed22e9fb1310cf6c2dec7cdef19f0ad69efa1f392e94a4333501270
./blobs/sha256-e692418e4cbaf90ca69d05a66403747baa33ee08806650b51fab815ad7fc331f
./oci-layout
./refs
./refs/v1.0
./refs/stable-release
./refs/tags
./refs/tags/v1.0
./refs/tags/stable-release
```

Blobs are named by their contents:
Expand All @@ -38,21 +38,21 @@ $ shasum -a 256 ./blobs/sha256-afff3924849e458c5ef237db5f89539274d5e609db5db935e
afff3924849e458c5ef237db5f89539274d5e609db5db935ed3959c90f1f2d51 ./blobs/sha256-afff3924849e458c5ef237db5f89539274d5e609db5db935ed3959c90f1f2d51
```

Object names in the `refs` subdirectories MUST NOT include characters outside of the set of "A" to "Z", "a" to "z", "0" to "9", the hyphen `-`, the dot `.`, and the underscore `_`.
Object names in the `blobs` subdirectories are composed of hash algorithm, hash digest and separator. Hash digest MUST NOT include characters outside of the set of "A" to "F", "a" to "f", "0" to "9". Separator MUST NOT include characters outside of the hyphen `-`, the dot `.`, and the underscore `_`.
Object names in the `refs/tags` directory MUST NOT include characters outside of the set of "A" to "Z", "a" to "z", "0" to "9", the hyphen `-`, the dot `.`, and the underscore `_`.
Object names in the `blobs` directory are composed of hash algorithm, hash digest and separator. Hash digest MUST NOT include characters outside of the set of "A" to "F", "a" to "f", "0" to "9". Separator MUST NOT include characters outside of the hyphen `-`, the dot `.`, and the underscore `_`.
Hash algorithm identifiers containing the colon `:` will be converted to the hyphen `-`.
For example `sha256:5b` will map to the layout `blobs/sha256-5b`.
The blobs directory MAY contain blobs which are not referenced by any of the refs.
The blobs directory MAY contain blobs which are not referenced by any of the tags.
The blobs directory MAY be missing referenced blobs, in which case the missing blobs SHOULD be fulfilled by an external blob store.

No semantic restriction is given for object names in the `refs` subdirectory.
Each object in the `refs` subdirectory MUST be of type `application/vnd.oci.descriptor.v1+json`.
No semantic restriction is given for object names in the `refs/tags` subdirectory.
Each object in the `refs/tags` subdirectory MUST be of type `application/vnd.oci.descriptor.v1+json`.
In general the `mediatype` of this [descriptor][descriptors] object will be either `application/vnd.oci.image.manifest.list.v1+json` or `application/vnd.oci.image.manifest.v1+json` although future versions of the spec MAY use a different mediatype.

This illustrates the expected contents of a given ref, the manifest list it points to and the blobs the manifest references.

```
$ cat ./refs/v1.0
$ cat ./refs/tags/v1.0
{"size": 4096, "digest": "sha256:e692418e4cbaf90ca69d05a66403747baa33ee08806650b51fab815ad7fc331f", "mediatype": "application/vnd.oci.image.manifest.list.v1+json"}
```
```
Expand Down
2 changes: 1 addition & 1 deletion image/descriptor.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ func (d *descriptor) normalizeDigest() string {

func findDescriptor(w walker, name string) (*descriptor, error) {
var d descriptor
dpath := filepath.Join("refs", name)
dpath := filepath.Join("refs", "tags", name)

f := func(path string, info os.FileInfo, r io.Reader) error {
if info.IsDir() {
Expand Down