Proliferation of manifests at W3C

[tantek]

Here at #w3cTPAC 2019 I’m learning that there are quite a few different manifest file format efforts by different groups for different specifications with different but often similar use-cases. Filing this design issue per @hober’s request to track the growing number of manifest formats and approaches to hopefully collapsing and reducing them.

Here is a few I know of along with their individual TAG review issues.

• Web App Manifest (W3CTAG w3ctag/design-reviews#30)
• Payment Method Manifest (W3CTAG w3ctag/design-reviews#162, Payment Method w3ctag/design-reviews#27, W3CTAG w3ctag/design-reviews#231 Payment Handler)
• Audio Book Manifest (W3CTAG w3ctag/design-reviews#345)
• Publication Manifest (W3CTAG w3ctag/design-reviews#344)

And one apparent need for a manifest: W3CTAG w3ctag/design-reviews#218. There are likely others, please add more below.

From a web author, developer, publisher perspective, it would be great if we could simplify the number of manifests, ideally one, perhaps with core properties, optional properties for specific use-cases, and a method of expanding each of those by separate working groups for maximum re-use and avoiding collisions / duplication of slightly different properties.

This issue is intended as a meta-issue just to track the growing number of manifests.

For specific proposals to provide guidelines to both spec and web authors about how to create some data and provide it in a manifest file, see: W3CTAG w3ctag/design-reviews#95 (please do not mark as duplicate, I think these serve two different purposes). Thanks for your consideration!

(Originally published at: https://tantek.com/2019/261/b1/proliferation-manifests-w3c)

[kevinmarks]

There have also been multiple gadget manifests for embedding small web apps in pages or elsewhere - iGoogle/OpenSocial, Microsoft's sidebar gadgets and Apple's Dashboard widgets.
These seem to have fallen out of favour as mobile browsers took off, but are worth looking at while reviewing these.
The w3c previously tried to converge some of these https://www.w3.org/TR/widgets-reqs/ but it looks like that effort was obsoleted

[llemeurfr]

The W3C Publishing WG did grab this issue before and its conclusion was that the relationship between a Web Application and a Web Publication is an "association": a reading application is able to read multiple publications and a publication can be loaded by multiple apps. A publisher does not want to publish an application with each publication he releases. A user will install a preferred app (maybe a highly accessible app), then use it to import many publications (usually organized as a "bookshelf" inside the app). Some reading customizations are made at the level of the app (e.g. dark mode), others at the level of a specific publication (e.g. font size).

Therefore, even if Web Application Manifest and Publication Manifest share a lot (both are expressed in JSON-LD, the processing model for obtaining a publication manifest is a sibling of the web app manifest process etc.), conflating the two concepts would end up in creating a weird aggregation of properties that would not make web developers happier.

What SHOULD certainly converge between different JSON-LD based manifests is i18n plumbing -> e.g. the use of "dir" and "lang" for property values in the Web App Manifest vs @direction and @language in the Publication Manifest.

Ah, also, the Audio Book Manifest is simply a specialization of the more generic Publication Manifest (an Audio Book IS a Publication), which goes in the direction of the initial comment of this issue.

[lknik]

Seems there are already some overlaps or fragmentation-like aspects. For PWA manifests the file used today is manifest.json, but the (new) spec above mentions file.webmanifest prominently, though alllowing .json

The official file extension for the manifest is .webmanifest. Some web servers recognize this extension and transfer the file using the standardized media type for a manifest (application/manifest+json). Developers can also choose a different extension (e.g. .json)

Web Payment is another manifest.json, but the file location is indicated via HTTP header (not in a tag).

Any option to put them all ink a tag (e.g. differentiated via an attribute)?

[kenchris]

@agektmr for question about payment handler

[agektmr]

It's only available through HTTP Link header AFAIK. I'd defer further questions to @ianbjacobs and @adrianhopebailie as I was not involved in the discussion.

[adrianhopebailie]

Restating my comment from w3c/payment-method-manifest#27

Merging the [Web App and Payment Method] manifests doesn't make sense because they are for very different things.

A Web App is a user application whereas a payment method is a sub-protocol (or specifically a request/response format) of the Payment Request API. The designer of a payment method may wish to provide meta-data about the method to browsers to:

1. limit the origins that are permitted to handle payments using this method
2. indicate which web apps (and also non-Web apps) are preferred for processing payments using this method (to facilitate Just In Time installation for example)

Said differently, and in a similar vein to @llemeurfr 's comment above about the relationship between an App and Publication:

1. A payment method is NOT a payment app (where a payment app is a web app that uses the payment handler API).
2. Many payment apps (at different origins) might support payments using the same payment method
3. A single payment app can support multiple payment methods (described by manifests at various origins)

[adrianhopebailie]

@lknik said:

Web Payment is another manifest.json, but the file location is indicated via HTTP header (not in a tag).

This is not entirely true. The browser finds out about the manifest first and foremost when a website calls Payment Request API and provides one or more payment methods it supports identified by their Payment Method Identifier (a URL).

Historically the browser used a HEAD to that URL to discover the manifest (URL of which was returned in a Location header in the response) but the spec has been updated to support simply performing a GET to that URL which MAY return the actual manifest content. See w3c/payment-method-manifest#37

i.e. There is no convention for the payment method manifest file name

[kenchris]

Existing TAG guidance on casing rules:

https://w3ctag.github.io/design-principles/#casing-rules

[kenchris]

For PWA manifests the file used today is manifest.json, but the (new) spec above mentions file.webmanifest prominently, though alllowing .json

The reason is that the Media type of manifest is "application/manifest+json" and servers can handle these differently, but sometime that is easier to do by using a different extensions.

[kenchris]

FYI: ImageResource is being split out of web app manifest: https://github.com/w3c/image-resource/

[jeffjaffe]

Taking @tantek 's comment a different way.

Perhaps instead of collapsing and reducing the number of Manifests, maybe we should recognize that there are different use cases and requirements and we should have a structured taxonomy of Manifests. Each Manifest would be for a specific set of use cases and we would explain in the taxonomy why we need multiples (e.g. the size of the objects might be different, the frequency that snippets are sent around might be different). Notionally, it could look something like:

Manifest Level 1. For WebApps.... description of canonical use case.
Manifest Level 2. For Payments
Manifest Level 3. For AudioBooks
Manifest Level 4. For Publications.

[hadleybeeman]

Reviewed at our TAG face-to-face in Cupertino.

We want to take this forward, as a big chunk of work. We are moving it to the design principles repo, where we'll catch it at the right time.

[jeffjaffe]

Link to the design principles repo?

[annevk]

https://w3ctag.github.io/design-principles/ / https://github.com/w3ctag/design-principles

[torgo]

We've suggested adding a section 6.3 to the document "Manifests" with some text to the effect that specs should make use of existing manifests:

"Don't make new manifests with incompatible designs. Webapp manifest is extensible."

Is there more we can say here?

[marcoscaceres]

We (manifest editors) screwed up in converting the spec to use WebIDL. We should caution new manifests not to use WebIDL as a data format.

I’m hoping to convert the spec to use infra instead. The TAG should recommend extensions using infra types instead... and to discuss proposed extensions with WebApps WG! Don’t add random things. Only use WebIDL for APIs, not JSON things.

[domenic]

"Don't make new manifests with incompatible designs. Webapp manifest is extensible."

I think this kind of blanket advice is a bit tricky. Webapp manifest has a specific design which is not suitable for all cases. See for example https://github.com/WICG/origin-policy/blob/master/policy-format.md#appendix-relation-to-web-app-manifest

[torgo]

We discussed in call March-30 and agreed to link this and #95 and work towards adding some text into the document itself which provides guidance to spec authors, including the info from this issue.

[torgo]

We agreed that the addition of PR #189 allows us to close this.

[kewisch]

As discussed with @tantek, I wanted to note there are also WebExtensions that have a manifest.json at their root. At current this is merely a de-facto standard as it is implemented in multiple browsers. Note I am not asking for this issue to be reopened, this is just to say there continue to be more manifests related to browsers and the web possibly worth tracking.