Create in-browser Live Development infrastructure based on postMessage

[humphd]

Brackets recently started to evolve its LiveDevelopment feature to work in multiple browsers. The new architecture allows the editor to talk to a browser over a transport layer (currently WebSocket based) and communicate bi-directionally between the preview and editor document(s).

Our goal with Mozilla's editor for Thimble is to leverage this new architecture in order to create an in-browser editor with an in-browser livedev preview which substitute an iframe-based browser, a postMessage-based transport layer, and a nohost-based Blob URL Object server.

Accomplishing this will involve a bunch of things. I've started to make a list below, and I'm CC'ing @busykai for additional thoughts. This list is incomplete, but should get us started.

• LiveDev needs to default to multi browser (i.e., the livedev.multibrowser pref should be true):

https://github.com/humphd/brackets/blob/bramble/src/LiveDevelopment/main.js#L77-L80

We can probably use PreferencesManager.set() in an extension to flip this to true at startup. We just need to make sure it happens early enough (i.e., before AppInit.appReady) so as to not miss the multibrowser init here:

https://github.com/humphd/brackets/blob/bramble/src/LiveDevelopment/main.js#L303-L314

• Launch/open a browser to display the preview need to be altered:

https://github.com/humphd/brackets/blob/bramble/src/LiveDevelopment/LiveDevMultiBrowser.js#L517-L527

https://github.com/humphd/brackets/blob/bramble/src/LiveDevelopment/MultiBrowserImpl/launchers/Launcher.js#L47

We need to change that to not use Node, but to start the iframe-based browser. Maybe we should override the launcher, or maybe we should implement the various “open browser” bits of the appshell:

https://github.com/humphd/brackets/blob/bramble/src/thirdparty/browser-appshell.js#L74-L79

• We need to create a LiveDevServer that implements enough of the LiveDevelopment/Servers/BaseServer base class. The FileServer is an example:

https://github.com/humphd/brackets/blob/bramble/src/LiveDevelopment/Servers/FileServer.js

We can do this in an extension. We then need to register this server with the LiveDevServerManager such that it has a high enough priority to always get used (e.g., 9001, which is higher than 0 or 99):

https://github.com/adobe/brackets/blob/14370aedfb6fdb7416a53cb0b7dc8d1a7bd7a5d4/src/LiveDevelopment/LiveDevelopment.js#L1482-L1484

• We should be able to alter how URLs are created to our server, and instead create Blob URL Objects (i.e., override the pathToUrl method):

https://github.com/humphd/brackets/blob/bramble/src/LiveDevelopment/Servers/BaseServer.js#L92-L112

One issue to ponder is that we'll need to do this async (i.e., read path from fs), and pathToUrl is sync. The URL gets used with the launcher here:

https://github.com/adobe/brackets/blob/14370aedfb6fdb7416a53cb0b7dc8d1a7bd7a5d4/src/LiveDevelopment/LiveDevMultiBrowser.js#L537-L541

We can steal code from nohost to create the Blob URLs:

https://github.com/humphd/nohost/blob/master/src/handlers.js#L510-L530

• We need to switch the MultBrowser LiveDev from a web socket transport to postMessage. This will require changes to both the editor side and remote (i.e., preview) side:
• Editor side - https://github.com/humphd/brackets/blob/bramble/src/LiveDevelopment/MultiBrowserImpl/transports/NodeSocketTransport.js
• Remote/Preview side - https://github.com/humphd/brackets/blob/bramble/src/LiveDevelopment/MultiBrowserImpl/transports/remote/NodeSocketTransportRemote.js

[humphd]

@busykai if you have pointers or thoughts on how we should approach this, we'd appreciate any guidance. Especially if our work can be generalized so it can be upstreamed to Brackets proper vs. just used in our editor fork.

[humphd]

One question I have is how best to abstract the transport and launcher, such that we can use postMessage and iframe versions instead of ws and default browser.

In https://github.com/humphd/brackets/blob/bramble/src/LiveDevelopment/LiveDevMultiBrowser.js#L803-L817 the default transport is assumed to be ws-based. With the protocol I can later call setTransport on the protocol and swap it out; but it would be nice if we could pass it in via the config options.

The launcher is more difficult, since we'll have no choice but to override it in code: https://github.com/humphd/brackets/blob/bramble/src/LiveDevelopment/LiveDevMultiBrowser.js#L88. It would be nice if this was configurable.

[busykai]

@humphd, thanks for cc'ing me. I'll need to read carefully through this, collect some context and comment in details later (soon), but, at a glance, we had very similar ideas on how it Live Development should evolve further, so upstreaming support for your use case is the right path.

[busykai]

CC: @sebaslv

[humphd]

Started in on this here https://github.com/humphd/brackets-browser-livedev. Anything that doesn't need to live in Brackets proper can go in this extension, while we experiment.

I've got the basics of the Blob URL server done (untested, ported from my nohost work). I'll do more this week to integrate the iframe browser, postMessage transport, etc.

[busykai]

@humphd, current version of Thimble seems to use plain CodeMirror, right? So are planning to move it to Brackets?

Anyways, I assumed a couple of things, here are my thoughts which I hope you find helpful.

LiveDev needs to default to multi browser (i.e., the livedev.multibrowser pref should be true):

We can probably use PreferencesManager.set() in an extension to flip this to true at startup. We just need to make sure it happens early enough (i.e., before AppInit.appReady) so as to not miss the multibrowser init here:

There's a AppInit.extensionsLoaded() which seems the right moment to flip it. Additionally, you could define it in the "session" scope of the PreferencesManager to make it is of highest priority.

Launch/open a browser to display the preview need to be altered:

We need to change that to not use Node, but to start the iframe-based browser. Maybe we should override the launcher, or maybe we should implement the various “open browser” bits of the appshell:

It seem like (from looking at Thimble) your LivePreview would always be there, i.e. the lifecycle of the editor is tied to the lifecycle of the LivePreview. If that assumption is correct, you may consider running LivePreview server always, right from the start-up. Then your Launcher would be a postMessage telling the iframe to open the LivePreview server end-point.

We need to create a LiveDevServer that implements enough of the LiveDevelopment/Servers/BaseServer base class. The FileServer is an example:

https://github.com/humphd/brackets/blob/bramble/src/LiveDevelopment/Servers/FileServer.js

We can do this in an extension. We then need to register this server with the LiveDevServerManager such that it has a high enough priority to always get used (e.g., 9001, which is higher than 0 or 99):

I believe for your purposes StaticServer (in src/extensions/default/StaticServer) is a much better example and you could leverage great deal of it (replacing express with your nohost server). StaticServer is a better example because it serves the content of currently opened documented. It is fundamental to be able to apply Live Preview to html documents. You will need to serve in-memory (in-editor) representation of the document rather the from a file.

Notes on the transport:

1. you'll need to create your postMessage-based implementation for the remote transport as well. transport has a getRemoteScript() which should return plain remote script in the textual form to be injected in the main html document.
2. transport load could be done using RequireJS paths definitions in a the same way it is done for the filesystem implementation here. it does not seem that (at least so far) two transport implementation need to live together. If we'd load it with require-configured path, no further changes would be needed.

Other notes:

1. We have plans for decoupling launch a browser from launch LivePreview. At the moment, Live Preview is launched with the launch of the first browser and is closed when the last one disconnects. Another model is needed to properly support multiple browsers. Currently they are coupled and it is unfortunate. However, like I pointed out before, you could launch LivePreview right at the beginning and then your iframe-based LivePreview would be connected to it permanently (given I'm assuming correctly) for the entire duration of the editing session.
2. Your comments are duly noted and we will take them into account and plan on them. Especially, ability to specify the transport implementation by configuration.

Let me know if I missed any of your questions. We'd be happy to stay up-to-date on your advancements and keep the implementations in-sync. Hope it all makes sense.

CC: @sebaslv

[humphd]

Correct, the current Thimble uses a plain CodeMirror. Our plan is to rip out the raw CodeMirror instance and separate preview iframe, and do it all via a single Brackets based iframe with this modified live dev stuff. We already have Brackets working in the browser with a filesystem based on IndexedDB.

Thanks for all your thoughts, they align with guesses we made as we planned things out. Our current thinking is to try the following flow:

1. Flip the pref for MultiBrowser LiveDev once extensions are loaded (AppInit.extensionsLoaded()), changing AppInit.appInit() here - https://github.com/humphd/brackets-browser-livedev/blob/master/main.js#L23
2. Listen once for the statusChange event with a ”STATUS_INACTIVE" on the LiveDevMultiBrowser object in our extension, and start our work here: https://github.com/humphd/brackets/blob/bramble/src/LiveDevelopment/LiveDevMultiBrowser.js#L816 As soon as this happens, wire the transport layer using setTransport(). Probably this means showing the iframe browser now, so transport has a ref.
3. Wait for our document to be loaded form Thimble in an editor. Maybe listen one-time for projectOpen on ProjectManager? https://github.com/humphd/brackets/blob/bramble/src/project/ProjectManager.js#L895. At this point we can trigger LiveDevMultiBrowser.open(), which will start our server and launch the browser.
4. We have to hack Launcher.launch(url) to do something like this:

function launch(url) {
    // For this to work, you might need to expose some "globals" (e.g., server/nohost) on appshell
    // We have server and LiveDoc
    // Get LiveDoc contents, feed through nohost.serveHTML to get Blob URL object
    // Give iframe the new URL to display
}

This will connect the preview back to the live dev instance via postMessage based protocol messages.

For sure we will stay in touch. My goal is for us to have a version of this working by the end of the week, which should help us figure out what we still don’t know, what could be done through better abstraction, etc.

I’ll report back when we have something.

[sedge]

@humphd We have this working now (huzzah!). Should we close?

[humphd]

Let's leave open while we track adobe#10558 and adobe#10634 upstream.

[sedge]

@humphd Since we aren't waiting on upstream anymore, I'm going to close this.