What is Portability.md saying about APIs?

[sunfishcode]

Portability.md currently has this paragraph:

Developer-exposed APIs (such as POSIX) are expected to be portable at a
source-code level through WebAssembly libraries which will use
feature detection. These libraries aren't necessarily
standardized: WebAssembly will follow the
extensible web manifesto's lead and expose
low-level capabilities that expose the possibilities of the underlying platform
as closely as possible. WebAssembly therefore standardizes a lower-level
abstraction layer, and expects libraries to offer portable APIs.

I'm not clear on what this paragraph is saying. What is this lower-level abstraction layer that WebAssembly will be standardizing here? Is POSIX meant to be an example of a higher-level API or a lower-level API, in Extensible Web Manifesto terms?

[jfbastien]

The key takeaway for developers is: your code will just compile, it's the library author's job to ensure it does. This is very close to e.g. what Emscripten does with C/C++ standard libraries and with SDL, or what @flagxor's does for NaCl/PNaCl do with naclports.

IIUC your question doesn't see to disagree with this?

Your question rather seems to be about the lower-level API layer. Yes, I'm presenting POSIX as a high-level API. I'd say that libc and libc++ also are high-level APIs.

What's probably very confusing is: what's the lower level API? The quote you present indeed doesn't say anything about these APIs! I think this is still wide open for discussion, so the omission was on purpose. I think @kg touches on this in #119, and @ncbray has been playing around with some API questions especially w.r.t. WebIDL.

IMO we should create a new document that explains these low-level APIs and how they work. We just haven't agreed on anything yet. We could update Portability.md to make this clear?

[sunfishcode]

Perhaps one reason I'm confused is that up until this paragraph, the page is talking about "portable" meaning what kind of hardware/OS platforms WebAssembly can run on. It sounds like you're saying the last paragraph is using "portable" to mean what application developers need to port their applications to WebAssembly. This shift in meaning is abrupt to me. My first inclination is to say that this is an entirely different topic that belongs on a different page.

[jfbastien]

From experience developers don't make the distinction of why their code doesn't compile and run properly, that's why I had put it on the same page, and then explained that this wasn't the portable they were looking for. It could be on a separate page if you think that would be better.

[sunfishcode]

In my mind, developers and implementors are distinct audiences (even if some people belong to both). Things like the as-if paragraph earlier in the page is of interest to implementors. It's actively unhelpful to talk about in introductory material for application developers. If we want to write documentation for application developers, let's do that in a new page.

Perhaps clarifying the last sentence would also help me:

WebAssembly therefore standardizes a lower-level abstraction layer, and expects libraries to offer portable APIs.

"standardizes" is present-tense, making it seem like we've already standardized something, or that this document anticipates it happening by the time most people read it. Should this be future tense?

And, the position of the word "portable" here permits multiple interpretations. Is WebAssembly's lower-level abstraction layer not going to be portable? Or is "portable" being used here to mean "higher-level"?

[lukewagner]

In the context of this discussion, the sentence "WebAssembly therefore standardizes a lower-level abstraction layer, and expects libraries to offer portable APIs." now seems to contradict everything else we've said and agreed on: a lower-level abstraction layer implies that WebAssembly would come up with a single, shared low-level abstraction layer, but that's exactly what we've said we're not doing. Maybe what I think you're saying is that the lower-level abstraction isn't the APIs, but rather the low-level features of dynamic loading, polyfilling and feature testing which allow you to probe your host environment, see what APIs it has, and then implement a higher-level portable API (like POSIX), all from within user-space.

How about: "WebAssembly does not attempt to itself provide a set of portable APIs. In a Web environment, functionality is accessed through the Web APIs defined by the Web Platform. Non-Web environments can choose to implement standard Web APIs, standard non-Web APIs (e.g. POSIX), or invent their own. Portability at the C/C++ level can, however, still be achieved by programming to a standard API (e.g., POSIX) and relying on standard libraries to map the standard interface to the host environment's APIs either at compile-time (via #ifdef) or run-time (via feature detection and dynamic loading)."

[jfbastien]

@lukewagner's rewrite sounds good. I'd like @davidsehr / @flagxor / @ncbray / @kg to comment though.

[sunfishcode]

Since the rest of this page is implementation-oriented, how about the following:

"WebAssembly itself does not currently define a set of APIs. In a Web environment, implementations provide the Web APIs defined by the Web Platform. Non-Web environments can choose to provide standard Web APIs, non-Web APIs specified in other standards (e.g. POSIX), or invent their own. Applications can determine which APIs are available through feature detection."

Helping developers understand portability at the C/C++ level is a great thing to do, but it seems like it's getting into different-target-audience territory.

[kg]

As I understand things, we basically aren't standardizing anything other than the tiny set of opcodes built into asm. No syscalls, no math stdlib (outside anything you import from Math or the built in ops), etc. So in that sense 'low level capabilities' in this context is confusing, since it makes it sound like we're specifying a low level set of platform APIs or syscalls when what we're doing is closer to building an abstract virtual machine with no standard APIs.

Alongside that there is a standard we're specifying for how that virtual machine would specifically interact with JavaScript, at which point you can use JS APIs. No new APIs there either, just a way to pull existing ones in. In a native environment likewise we could expose a mechanism to pull in C cdecl functions or COM interfaces, etc, but that also wouldn't require exposing a custom API - we're just exposing existing ones.

I think Luke's rewrite expresses all this pretty well. Maybe replace does not itself attempt to provide a set of portable APIs with does not specify any APIs or syscalls. We're not specifying any unportable APIs either, as far as I know :-)

[lukewagner]

@sunfishcode The doc is "portability" and it is logical to expect devs to ask "so can i write portable POSIX" code and the answer is "yes, but you'll need a library's help". I think this is valuable to have in the doc.

@kg Good suggestion.

[sunfishcode]

@lukewagner I agree devs will ask that, and it'd be great to have an answer. My objection is that we're bundling a bunch of unrelated topics together on one page and it's confusing me. I can submit a separate PR to address my concerns though.

[titzer]

On Thu, Jun 11, 2015 at 3:54 PM, K. Gadd notifications@github.com wrote:

As I understand things, we basically aren't standardizing anything other
than the tiny set of opcodes built into asm. No syscalls, no math stdlib
(outside anything you import from Math or the built in ops), etc. So in
that sense 'low level capabilities' in this context is confusing, since it
makes it sound like we're specifying a low level set of platform APIs or
syscalls when what we're doing is closer to building an abstract virtual
machine with no standard APIs.

Alongside that there is a standard we're specifying for how that virtual
machine would specifically interact with JavaScript, at which point you can
use JS APIs. No new APIs there either, just a way to pull existing ones in.
In a native environment likewise we could expose a mechanism to pull in C
cdecl functions or COM interfaces, etc, but that also wouldn't require
exposing a custom API - we're just exposing existing ones.

I think Luke's rewrite expresses all this pretty well. Maybe replace does
not itself attempt to provide a set of portable APIs with does not
specify any APIs or syscalls. We're not specifying any unportable APIs
either, as far as I know :-)

I agreed broadly with the above; we should find a way to separate the
specification from what the data-crunching abilities are (engine) and the
APIs/interaction model (transmission). In particular this engine piece that
seems to be hiding inside should have very few points of contact with the
outside world--preferably no defined API calls, just enough to do
code/data, and then another layer should add "standard" API calls and a
broader framework for specifying synchronicity.

—
Reply to this email directly or view it on GitHub
#146 (comment).

[sunfishcode]

With the wording now checked in, and the addition of CAndC++.md addressing C/C++ developers, I'm calling this done.