Load event insufficient

[getify]

I brought up this topic in #10 and #20 and it was discussed. But I'm reopening the discussion because in my current use-case experience implementing a script loader, I'm finding the existing behavior is concretely insufficient.

The problem can be easiest illustrated with the inherent race condition of this code:

var elem = document.createElement( "link" );
elem.setAttribute( "href", href );
elem.setAttribute( "as", "script" );
elem.setAttribute( "rel", "preload" );
document.head.appendChild( elem );

// do some other stuff

elem.addEventListener( "load", function(){
   console.log( "preloaded" );
});

Now, that's not actually what I'm doing in code. But that race condition is deeply concerning. I know there's lots of places that such a race condition exists, but I believe this use-case I'm about to describe makes the pain acute enough that it should be revisited for some sort of special handling.

What I'm doing with my script loader (3.0 rewrite of LABjs) is to have it create and inject <link rel=preload> elements, listen to their load events, then re-request then with <script async> tags in desired order of execution. That part works fine with behavior as-is.

But where it falls apart is the desire, pushed especially by the recent pressure on web perf for PWAs, etc, to push as much of resource loading to declarative markup as possible.

I have a server-side component for LABjs which analyzes your resource tree to determine what the client will need to load and in what order. This tool can produce the client code to load these resources with the $LAB API fine. But it also produces the markup of the <link rel=preload ..> tags, for insertion into the initial HTML response.

This is considered "best practice" from a web perf perspective, to get critical resources loading as possible.

Unfortunately, we now have a race condition. If LABjs finds the <link rel=preload> element in the DOM, and subscribes a load event, I can't guarantee if I'm going to get that event or if it already passed. That's clearly broken.

The only option then is for LABjs to create and insert another <link rel=preload> element so that I can hear the event (which if it's already loaded, will essentially fire "on the next tick").

That's deeply troubling to have to churn DOM modifications unnecessarily just to get assurances of an event notification. It violates another set of performance best practices to force that extraneous DOM modification.

The unenviable position LABjs finds itself in is: either suck for modifying the DOM unnecessarily, or suck for not getting resources markup-discoverable early enough.

Presumably the engine has some deterministic way to ask if the resource is "already loaded" and not actually make a second request. It just finds that it's already in some-whatever-cache and then fires the load event (async).

This use case shows that we really need to expose some mechanism by which the code can determine if the thing is already loaded (ie, the load event has already passed), to avoid needing to mutate the DOM to figure that out.

The deterministic logic would then be (something like):

if (!elem.ready) {
   elem.addEventListener( "load", scriptReady );
}
else {
   scriptReady();
}

I don't care what the property or mechanism for determination is (but simplest is best). It could be something like the above (the document.readyState property would be precedent). Or it could be like "always fire the load event if if the element has already finished loading" (precedent is jQuery's ubiquitous .ready(..) event handling).

I just don't want it to involve having to mutate the DOM or make unnecessary network requests to figure it out. Doing so is performance counter-productive, for a feature that is explicitly performance focused.

In light of this use-case, can we please revisit how to determine if a resource is preloaded?

[yoavweiss]

I have a server-side component for LABjs which analyzes your resource tree to determine what the client will need to load and in what order. This tool can produce the client code to load these resources with the $LAB API fine. But it also produces the markup of the <link rel=preload ..> tags, for insertion into the initial HTML response.

Can that markup also include onload attributes that would register for you that this particular resource is loaded?

[getify]

Yes but that wouldn't really help if LABjs isn't present yet (which I can't guarantee). A related hack (hopefill/prollyfill) I'm presently experimenting with is to have an onload that sets the 'readyState' onto the element itself so LABjs can later inspect it. That's pretty ugly and it's really only something I'd want to do for real if it seemed the standard might actually do that eventually.

[yoavweiss]

You can use the onload attibute to register loaded URLs to, say, window.LAB_loadedURLs and then collect them once LAB boots up.

Alternatively, you can use !!window.performance.getEntriesByName(absolute_url_which_might_have_been_loaded).length to conclude if the load event already fired for a specific URL (at least if its response had a status code of 200)

[getify]

The other big problem with any onload solution is CSP. Feels like a deal breaker.

[yoavweiss]

True. No such problem with the getEntriesByName one.

[getify]

Is there any CORS related issues to that if you're checking on URLs that are from a different domain or anything like that?

[getify]

Looks like this approach may work but is a bit brittle, in that getEntriesByName(..) requires an absolute canonical URL to match by, but the href in the markup may be relative. :/

[yoavweiss]

Looks like this approach may work but is a bit brittle, in that getEntriesByName(..) requires an absolute canonical URL to match by, but the href in the markup may be relative. :/

URL() or its a.href hacky equivalent is your friend.

[yoavweiss]

Is there any CORS related issues to that if you're checking on URLs that are from a different domain or anything like that?

A Timing-Allow-Origin header is required for detailed timing, but the entry will be present even in its absence.

[getify]

Sure, doable but tougher to get right if there's a <base href=..> in the page.

[getify]

All in all, these are still hacky workarounds for what I claim should be a supported use-case. If the browser engine can tell "immediately" if a resource is preloaded -- it obviously can because it fires a another load event if you create an extra <link rel=preload> element for the same URL -- why wouldn't it be reasonable to expose this detection capability in some way to user code?

[yoavweiss]

This request is out of scope for preload - you could make a similar claim for e.g. <img>.

If you feel the current solutions are not sufficient, I suggest you start up a thread on the WICG's discourse and see if others share your pain.

[getify]

I will happily start a thread in that wider discussion group.

But I just have to say that I'm quite disappointed with the repeated and spurious claim of "out of scope". I believe it is 100% and entirely in scope, or I wouldn't have spent the time to bring it up here.

When I see "out of scope", what I'm actually reading is, "a problem I don't want to deal with", not "a problem that doesn't apply to this area." That's reinforced by the fact that we have nearly a dozen messages in this thread where the focus was on trying to invent some other hack rather than addressing the merits of the use-case itself.

Name any type of web content: images, stylesheets, video/audio, scripts, etc. In all of these cases, the use case of "resource preloading" applies. And in all of these cases, <link rel=preload> is the mechanism for preloading. There's no other part of the whole web platform where this discussion could possibly be more in scope than the preloading mechanism itself.

This isn't at all like <img> or <video> or <script> or anything like that, because those containers are all designed to actually apply/render/run the content once loaded. So, there's a clear mechanism for discovering if the thing has already loaded, because it must have loaded if its content actually "exists".

For resource preloading however, the whole point is that you're trying to load it in a container that doesn't apply it to the page, and therefore there's no other way to tell if it loaded other than a robust feature built into the container itself. This is the primary distinguishing feature from <link rel=prefetch> which is just a suggestion to load. <link rel=preload> is an explicit preload, which means the use of it implies the need to verify and rely on this guarantee. If not, there'd be no point in <link rel=preload> at all, prefetch would have been sufficient.

The load event is a good start for such a mechanism, but it's not enough.

The reason it's not enough is because "best practice" is to preload via markup (early parser discovery) and then progressively enhance your resource loading with some script logic (like a resource loader). That creates the inherent race condition I complained about in my OP, between what the markup does and what the code can detect.

If you're preloading something with markup, the code side needs to be able to fully determine the status of that mechanism. To suggest that this gap in preloading functionality should be handled by some other part of the web platform (or not at all) is just painfully myopic.

[yoavweiss]

The reason I claim this is out of scope is that such a mechanism, if indeed needed, should not apply only to preloaded resources.

For example, we already discuss similar concepts that can answer the "was this resource loaded yet?" question as part of whatwg/fetch#65.

In any case, I suggest you specify why your use-case is not addressed by something like

function wasThisLoaded(url) {
    absoluteURL = new URL(url, document.baseURI).href;
    return !!window.performance.getEntriesByName(absoluteURL).length;
}

[getify]

After re-reading my previous message, I apologize for my aggressive tone. That was not intentional. I've been strenuously advocating (on and off) for resource preloading for about 8 years now, with so many discussion threads and proposals and debates back and forth, I've lost count.

I was thrilled when <link rel=preload> came about, as it was a HUGE step forward in this years-long effort. It wasn't what I envisioned originally, but it seemed like it was designed with that spirit in mind.

That's why it's so frustrating to be so close to having a workable system but be told that the last missing piece -- which is critical IMO -- is just "out of scope".

Fundamentally, whether spec folks agree or not (for years there was disagreement even philosophically on this point), there will always be some of us out here that want full script-control over resource preloading, for a variety of use-cases that markup-alone will never solve well.

[getify]

For example, we already discuss similar concepts that can answer the "was this resource loaded yet?" question as part of whatwg/fetch#65.

Unless I've missed something and we're talking about a markup-oriented version of fetch, I think this is a very different kind of use-case. Script-controlled APIs (as opposed to reflective properties from DOM elements) have a whole world of different capabilities available to them to eliminate the race conditions.

But markup-to-script bridges have this inherent race condition with no clear way to solve it.

If there are other markup containers besides <link rel=preload> that fetch and contain content, but do not otherwise apply it to the document in a way that is observable, then those containers would indeed benefit from what I'm asking for here. I'm just not aware of any. This seems mostly or entirely a special characteristic of <link rel=preload> given its load-but-dont-apply semantics.

[getify]

Take the design of the Promises API, for example. This mechanism does not allow an outside observer to tell whether or not the value has been resolved. But they solved the inherent race condition by saying that no matter when you register a then(..) -- before or after value resolution -- you always get the value.

So, you don't need a way to tell if the value is there, you just declaratively state what you want to do when it is there and leave the timing up to the system.

The equivalent would be a load event that fires at time of attaching to the <link rel=preload> element, even if the resource loading has already completed. That's one possible way to very simply address this race condition problem. As stated earlier, this precedent exists with jQuery's document.ready(..) handling.

They knew, exactly as I'm saying now, that designing a system with an inherent race condition in it is a bad idea. So they just eliminated the possibility.

[getify]

In any case, I suggest you specify why your use-case is not addressed by something like

It is not the case that I'm suggesting there's no hacky workarounds that (at least partially) address my need. There clearly are multiple ways to hack at this, as this thread demonstrates.

But this thread isn't like a support ticket asking "how do I do something?" I'm still trying to make the principled case that this missing functionality is a gap that should be addressed.

So far, the only concrete reason I've been given for why it shouldn't be addressed here is that there may be other places in the web platform that would also like it addressed. It's not at all clear to me that a single system would serve all those use cases.

But even if it was, this thread could (and should) be used to bolster support for that. Spec authors always want concrete use cases for features, and this is one such case. However, the tone here feels a bit more skeptical and hands-off'ish, with phrases like "if indeed needed".

I have I believe demonstrated a concrete reason why the feature needs to exist, and so far haven't seen a concrete reason why it shouldn't exist. That's frustrating.

[getify]

Take the design of the Promises API, for example.

Heck, for that matter, I know I've seen somewhere in spec discussions the idea of a .loaded property exposing a promise that's resolved once it is such. That's yet another entirely valid way this could be addressed, and could easily be precedent for other mechanisms that need similar behavior.

But why wouldn't we take the opportunity to use this current concrete use case as the impetus to push for something, and use that as a proof-point for other potential consumers of such behavior? Why kick the can down the road to wait for somewhere else in the platform to finally address it? Why not let this be the test case to appeal to the supreme court, if you will?

[getify]

If you feel the current solutions are not sufficient, I suggest you start up a thread on the WICG's discourse and see if others share your pain.

Done: https://discourse.wicg.io/t/finding-a-better-mechanism-for-detecting-loaded-status-of-link-rel-preload

[getify]

@yoavweiss

BTW, I made a testing resource related to <link rel=preload>. It's a mock of the relevant parts of the DOM API related to dynamic resource loading behavior. I explain more here:

https://discourse.wicg.io/t/mocking-resource-loading-behavior/2104