image-layout: OPTIONAL index.json

From the 2017-01-25 call we agreed to keep the manifest list as the
entry point for accessing objects. The index concept is only a more
generic use of the manifest-list.
opencontainers#438 (comment)

This change uses the manifest-list as an index that allows
implementations to work around needing to walk/traverse the `./refs/` or
`./blobs/` directories.

This includes validating the image-layout example.

This also makes `validate-examples` automatically update `schema-fs` if
there have been changes to the JSON schema.

Obsoletes opencontainers#438

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>

Makefile

@@ -76,12 +76,14 @@ $(OUTPUT_DIRNAME)/$(DOC_FILENAME).html: $(DOC_FILES) $(FIGURE_FILES)
 	ls -sh $(shell readlink -f $@)
 endif
 
-validate-examples:
+validate-examples: schema/fs.go
 	go test -run TestValidate ./schema
 
-schema-fs:
+schema/fs.go: $(wildcard schema/*.json)
+	cd schema && printf "%s\n\n%s\n" "$$(cat ../.header)" "$$(go generate)" > fs.go
+
+schema-fs: schema/fs.go
 	@echo "generating schema fs"
-	@cd schema && printf "%s\n\n%s\n" "$$(cat ../.header)" "$$(go generate)" > fs.go
 
 check-license:
 	@echo "checking license headers"

config.md

@@ -229,4 +229,4 @@ Here is an example image configuration JSON document:
 ```
 
 [rfc3339-s5.6]: https://tools.ietf.org/html/rfc3339#section-5.6
-[runtime-platform]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc2/config.md#platform
+[runtime-platform]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc3/config.md#platform

image-layout.md

@@ -3,22 +3,24 @@
 The OCI Image Layout is a slash separated layout of OCI content-addressable blobs and [location-addressable](https://en.wikipedia.org/wiki/Content-addressable_storage#Content-addressed_vs._location-addressed) references (refs).
 This layout MAY be used in a variety of different transport mechanisms: archive formats (e.g. tar, zip), shared filesystem environments (e.g. nfs), or networked file fetching (e.g. http, ftp, rsync).
 
-Given an image layout and a ref, a tool can create an [OCI Runtime Specification bundle](https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc2/bundle.md) by:
+Given an image layout and a ref, a tool can create an [OCI Runtime Specification bundle](https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc3/bundle.md) by:
 
 * Following the ref to find a [manifest](manifest.md#image-manifest), possibly via a [manifest list](manifest-list.md#manifest-list)
 * [Applying the filesystem layers](layer.md#applying) in the specified order
-* Converting the [image configuration](config.md) into an [OCI Runtime Specification `config.json`](https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc2/config.md)
+* Converting the [image configuration](config.md) into an [OCI Runtime Specification `config.json`](https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc3/config.md)
 
 # Content
 
 The image layout MUST contain two top level directories:
 
 - `blobs` contains content-addressable blobs.
-  A blob has no schema and should be considered opaque.
+    - A blob has no schema and should be considered opaque.
+    - Directory MAY be empty.
+    - See [blobs](#blobs) section
 - `refs` contains [descriptors][descriptors].
-  Commonly pointing to an [image manifest](manifest.md#image-manifest) or an [image manifest list](manifest-list.md#oci-image-manifest-list-specification).
-
-Both `blobs` and `refs` MAY be empty.
+    - Commonly pointing to an [image manifest](manifest.md#image-manifest) or an [image manifest list](manifest-list.md#oci-image-manifest-list-specification).
+    - Directory MAY be empty.
+    - See [refs](#refs) section
 
 The image layout MUST also contain an `oci-layout` file:
 
@@ -27,23 +29,26 @@ The image layout MUST also contain an `oci-layout` file:
     - The `imageLayoutVersion` value will align with the OCI Image Specification version at the time changes to the layout are made, and will pin a given version until changes to the layout are required
 - It MAY include additional fields
 
+The image layout MAY also contain an `index.json` file:
+
+- It MUST be a JSON object
+- It MUST have the base properties of [manifest-list](manifest-list.md).
+- See [index.json](#index.json) section
+
 ## Example Layout
 
 This is an example image layout:
 
 ```
 $ cd example.com/app/
-$ find .
-.
-./blobs
-./blobs/sha256/e692418e4cbaf90ca69d05a66403747baa33ee08806650b51fab815ad7fc331f
-./blobs/sha256/afff3924849e458c5ef237db5f89539274d5e609db5db935ed3959c90f1f2d51
-./blobs/sha256/9b97579de92b1c195b85bb42a11011378ee549b02d7fe9c17bf2a6b35d5cb079
-./blobs/sha256/5b0bcabd1ed22e9fb1310cf6c2dec7cdef19f0ad69efa1f392e94a4333501270
-./oci-layout
-./refs
+$ find . -type f
+./index.json
 ./refs/v1.0
 ./refs/stable-release
+./oci-layout
+./blobs/sha256/3588d02542238316759cbf24502f4344ffcc8a60c803870022f335d1390c13b4
+./blobs/sha256/4b0bc1c4050b03c95ef2a8e36e25feac42fd31283e8c30b3ee5df6b043155d3c
+./blobs/sha256/7968321274dc6b6171697c33df7815310468e694ac5be0ec03ff053bb135e768
 ```
 
 Blobs are named by their contents:
@@ -156,4 +161,43 @@ $ cat ./blobs/sha256/e692418e4cbaf90ca69d05a66403747baa33ee08806650b51fab815ad7f
 [tar stream]
 ```
 
+## index.json
+
+This OPTIONAL file is for use indexing references and elements of the image-layout, where walking the `refs` and `blobs` is sub-optimal.
+It is intended to be generated from the content in an image-layout after any changes.
+
+From the base properties of the [manifest-list](manifest-list.md), this object will us
+
+### Index Example
+
+```json,title=Manifest%20List&mediatype=application/vnd.oci.image.manifest.list.v1%2Bjson
+{
+  "schemaVersion": 2,
+  "manifests": [
+    {
+      "mediaType": "application/vnd.oci.image.manifest.v1+json",
+      "size": 7143,
+      "digest": "sha256:e692418e4cbaf90ca69d05a66403747baa33ee08806650b51fab815ad7fc331f",
+      "annotations": {
+        "org.opencontainers.ref.name": "v1.0"
+      }
+    }
+  ],
+  "elements": [
+    {
+      "mediaType": "application/xml",
+      "size": 7143,
+      "digest": "sha256:b3d63d132d21c3ff4c35a061adf23cf43da8ae054247e32faa95494d904a007e",
+      "annotations": {
+        "org.freedesktop.specifications.metainfo.version": "1.0",
+        "org.freedesktop.specifications.metainfo.type": "AppStream"
+      }
+    }
+  ],
+  "annotations": {
+    "com.example.index.revision": "r124356"
+  }
+}
+```
+
 [descriptors]: ./descriptor.md

manifest-list.md

@@ -11,7 +11,9 @@ For the media type(s) that this document is compatible with, see the [matrix][ma
 - **`schemaVersion`** *int*
 
   This REQUIRED property specifies the image manifest schema version.
-  For this version of the specification, this MUST be `2` to ensure backward compatibility with older versions of Docker. The value of this field will not change. This field MAY be removed in a future version of the specification.
+  For this version of the specification, this MUST be `2` to ensure backward compatibility with older versions of Docker.
+  The value of this field will not change
+  This field MAY be removed in a future version of the specification.
 
 - **`mediaType`** *string*
 
@@ -114,5 +116,5 @@ For the media type(s) that this document is compatible with, see the [matrix][ma
 }
 ```
 
-[runtime-platform2]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc2/config.md#platform
+[runtime-platform2]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc3/config.md#platform
 [matrix]: media-types.md#compatibility-matrix

schema/content-descriptor.json

@@ -23,6 +23,10 @@
         "type": "string",
         "format": "uri"
       }
+    },
+    "annotations": {
+      "id": "https://opencontainers.org/schema/image/descriptor/annotations",
+      "$ref": "defs-image.json#/definitions/annotations"
     }
   },
   "required": [

schema/defs-image.json

@@ -17,8 +17,7 @@
       "required": [
         "mediaType",
         "size",
-        "digest",
-        "platform"
+        "digest"
       ],
       "properties": {
         "mediaType": {
@@ -78,6 +77,17 @@
               }
             }
           }
+        },
+        "annotations": {
+          "id": "https://opencontainers.org/schema/image/annotations",
+          "oneOf": [
+            {
+              "$ref": "defs.json#/definitions/mapStringString"
+            },
+            {
+              "type": "null"
+            }
+          ]
         }
       }
     },