Idea of runtime mode

[zcbenz]

Currently developers using atom-shell have to ship the whole atom-shell binaries when distributing their apps, but since now we have asar as atom-shell's app format, we may add runtime mode to atom-shell like Adobe Air or Chrome Hosted Apps that developers only need to distribute packaged app in asar format as long as end users have the runtime mode of atom-shell installed.

The runtime mode can just be another app based on atom-shell (let's call it atom-runtime for now), and has deep platform integrations:

• On Windows, we can provide installer and auto updater for atom-runtime, and register the .asar file type to be opened with it.
• On Linux, we provide official software sources of atom-runtime for some popular distributions, and also register atom-runtime as handler of .asar files.
• On Mac, we need to provide a tool to help users package their apps into Mac bundles. We can reference how Steam created bundles for downloaded games, or how Parallel created bundles for applications in hosted OS.

We can even provide a tool or service to help generating installers for developers' apps which can automatically download atom-runtime if it is not installed on user's machine, just like many .NET applications.

[Subash]

+100, That would be very very helpful to reduce the size of app.

[ghost]

Can the .asar extension be changed? It sounds really weird in hungarian. Basically it would mean the shit in a bad way (link).

[YurySolovyov]

.apak? with all seriousness, there are tons of languages, so almost any extension can sound weird in some of them.

IMO runtime installation and updating should be as easy as possible. Many runtimes have very annoying notifications. Also, if you are going to do this, some additional manifest metadata should be provided, because newer app versions may not work on old runtime, so you need to compare current runtime version with version in app manifest( possibly even versions range as in node packages ), and then if runtime needs to be updated, make this process as easy as possible. I don't know what easy is for most people, but for me it is just window with progress bar or status messages ("Downloading... 57%", "Updating...")
I'm not sure but background updates may also be considered. It may be some update service or just runtime download while app is running and update when app closes.

[malcolmalex]

+1 for this. I'm working on an application that I have to deliver for mac and windows, and this would simplify the delivery considerably.

[joshuawarner32]

This would be awesome!

A few thoughts:

• As @YuriSolovyov said, we need a way to declare what versions of the atom-runtime a particular asar is compatible with. Preferably, this should also allow specifying an exact version, since this lessens the burden on QA when releasing an asar-based app.
  • To support that, we'd need to allow having multiple versions of the atom-runtime installed side-by-side, potentially invisibly to the user.
  • Note that this is somewhat at odds with the ideals of limiting download sizes. However, a company producing multiple asar-based products could standardize internally so all their apps still use the same version.
• You didn't mention support for auto-updating atom-runtime on Mac, but that would be very useful.
• Rather than having source-only installations on linux, it would be good to support installing / updating binaries, both through .deb/.rpm packages as well as some method that doesn't require sudo rights to install (i.e. per-user). For runtimes installed per-user, auto-update would also be very useful.
• Would there be any support for keeping installed asar files up-to-date - or is that something that'd have to be implemented independently in each app?

Lastly, what's the timeline on this effort? What would be the best way to get started, if I wanted to begin implementing parts of this? I'm open to contributing.

[zcbenz]

@joshuawarner32 Currently this is just a rough idea, I plan to work on it after solving some major issues of atom-shell, but it could be months later.

[YurySolovyov]

I think it is also worth looking at existing runtimes and how they are working/updated, how disturbing they are and what other pros and cons.

[steelbrain]

Currently this is just a rough idea, I plan to work on it after solving some major issues of atom-shell, but it could be months later.

It's been about 5-6 months. It'd be great to have this implemented.

[joshuawarner32]

We've built up an internal implementation of this that fits our particular use-case very well, and we might be able to upstream it. The danger would be that it's actually too specific to our use-case, and nobody else would be interested in the extra complexity.

Here are the highlights of our design:

• Deploy .asar.gz files for several different apps on a server
• Distribute a single build of atom-shell to users, which doesn't bundle any code for specific apps
• On app startup, download the appropriate .asar.gz file (if it's not there, or there's a more recent one on the server). Extract it and run the contained app.
• Our atom-shell build accepts a --app argument to specify which app to download/run

Is that sort-of what atom-shell folk had in mind? Would something like that be useful to other people?

[tonypee]

Like the JRE it could be the ERE (electron runtime environment), with .eapp files (electron app)

[steelbrain]

-1 for .eapp

[sergeysova]

+1 I need it!

[anaisbetts]

On Windows, we can provide installer and auto updater for atom-runtime, and register the .asar file type to be opened with it.

Install Steam on Windows and check out how they create game shortcuts, they actually use a protocol handler, it's relatively clever

[zcbenz]

👍 on the Steam model.

There is no technical obstacle on implementing this, the whole thing can just be an Electron app itself. But the problem is it is not easy to implement neither, taking all the details into account, I think it requires a small team working full time on it, I'm not sure whether it would ever happen.

[anaisbetts]

But the problem is it is not easy to implement neither, taking all the details into account, I think it requires a small team working full time on it, I'm not sure whether it would ever happen.

There are a lot of questions in my mind that, while certainly aren't insurmountable, make this plan Difficult. For example, say "App 1" requires 0.30.x and cannot run on 0.31.x, and "App 2" requires the opposite. Who wins in this scenario?

It seems like a lot of effort and infrastructure / trickiness to optimize for something that isn't our biggest problem - disk space and bandwidth, at the expense of developer complexity (i.e. now developers have to create this crazy chain-installer to first install/verify Electron, then install their app)

I don't mean to be hatin' on an idea! I just think that with the time we'd take to do this, we could instead spend it on our big problem - making it super dead easy for new devs to create and publish Electron apps, from their first line of code all the way to shipping it to users

[steelbrain]

@paulcbetts But hey, it's benefits are worth it, aren't they? When updating an app, users will have to download less than an mb instead of ~70mb. Which means we won't even need auto-updating frameworks and all, we will just have to download and extract a micro-zip archive with a couple of files.

I've spent the last three months in a remote area where the speed was less than 128kbps and trust me, the upgrade size is a huge pain in the arse, the worst part is that there's still tons of people still in similar situations

[anaisbetts]

Which means we won't even need auto-updating frameworks and all, we will just have to download and extract a micro-zip archieve with a couple of files.

I don't think that's true though, because almost certainly at some point you'll want to move in an update to a newer version of Electron and then kerplowie, you're in a similar sitch (or at least one that is similarly ugly)

[RIAEvangelist]

Basic apps now are between 30 and 50 Megs unless you have a crap ton of videos inside your package. I don't think that's a stretch for most people to download. I believe the average youtube video is between 30 and 40 mb...

[JoshTheDerf]

I'd just like to make a note here, in China, the connection speed averages about 20kB/s (Granted I've got a connection slower than ~92% of my city.) Updating Atom/electron usually requires a full day of wget -c'ing. unless the taobao mirrors are used.

For version compatibility, electron development moves too fast to have a catch-all version either, perhaps some sort of wrapper/helper could be used to use/download the requested major version when the app starts?

[RIAEvangelist]

20 kb/s? how can you survive? I can't imagine how difficult that must make things!

[RIAEvangelist]

@steelbrain not my intention

[JoshTheDerf]

@RIAEvangelist Getting on at the fast times and lots of patience. The speed gets really fast late at night, so just leave downloads running or scheduled while you're sleeping. :3

Anyways, I think it wouldn't be too much work to write a (basic) third-party wrapper, anyone want to try?

.asar files, however, can't easily declare their electron version. Perhaps another program descriptor system would be needed?

[YurySolovyov]

How about some kind of runtime version dispatcher that (downloads if needed and) launchers required version of runtime for your app? We can add an entry to package.json for that.

[howdev]

Electron is owned by Microsoft now. ...

It is? ...

Not exactly. I think what is meant by this is that Microsoft acquired GitHub in June 2018 and thus has considerable influence over GitHub's developers, who originally developed atom then extracted atom-shell and later renamed it electron.

For what it's worth, Edge is now using Chromium under the hood. Also the puppeteer (-core) project provides an alternative to electron that, if I understand correctly, covers many use cases of a "shared electron runtime".

Github is a company with the service only Microsoft bought. There were already software hosted on it. But nobody owns Open Source even VSCode is under Open Source
It is complete misunderstanding that teaches Open Source wrong. Nobody owns Open Source. Please read the Open Source philosphy
https://www.gnu.org/philosophy/free-sw.html
Everyone has the same freedom in Open source to modify, distribute, or even to sell or ask for donation. No developer is under control. They all have freedom under Open Source license

[howdev]

Runtime does not solve the huge memory consumption of Electron apps. This case is worst than Java. There are not many small apps built with Java that needs to run by users. You only need to run one or two Java desktop apps at the most usually such as database editor and eclipse. But with electron framework, everybody try to build desktop apps to run on a system. Each of those apps runs chrome regardless of the size of the app unlike Java. Atom uses 600MB RAM, a messenger uses 300MB. People start running electron apps. OS, a browser and one atom uses 4GB. couple more apps uses 8GB. Is not just size, the electron apps are slow

[howdev]

@jobukkit Electron is developed by OpenJs Foundation. Microsoft does not own it, they are one of the members of the foundation.

[sentialx]

electron-global is not actually suitable for the current Electron release strategy. There are too many breaking changes in the API in too short periods of time and even minor bumps can break something, because of the amount of changes in Chromium version bump. In order to make this library useful, Electron should introduce LTS releasing strategy. It is not an easy task, though.

@howdev It's worth mentioning that RAM usage depends on how an app is optimized. Unfortunately, most Electron apps are really badly optimized.

[martpie]

Runtime does not solve the huge memory consumption of Electron apps. This case is worst than Java

Please, we all know the weaknesses (and advantages) of Electron as a framework. There are 63 participants to this issue, and probably hundreds of subscribers. The notifications noise is extremely high.

Please only comment if you have something useful regarding how to make a shared runtime with Electron (thank you 🙏)

[baconbrad]

To extend upon @martpie's comment I think the project has outgrown an issue thread. This is a project that will take a multiple contributors, have multiple sub issues arise, and will require planning and organization a thread like this cannot provide. The runtime will be doomed or greatly delayed by these limitations. As Electron's popularity grows I can see a much bigger need for a runtime. Perhaps an official repo for a runtime should be started on either /electron or /electron-userland?

[skapoor8]

+1 for this. Would be open to contributing as well.

[heylix]

I think the project has outgrown an issue thread. This is a project that will take a multiple contributors, have multiple sub issues arise, and will require planning and organization a thread like this cannot provide.

Then start planning and create linked Issues? It's not like project management is a new concept to software development.

For starters, we can list what is needed for this here.

• LTS version to minimise spread
  • maybe do it like Debian and identify major versions through features, not upstream version numbers or arbitrary points in time?
• revive electron-global to use those LTS versions
• shared install directory for all versions
  • possibly adhering to Linux FHS, macOS and Windows path specs as well
• interface for Electron apps to know which versions of Electron they support (like package.json)
• properly integrate this into NPM
• packaging for distributions
  • maybe look at Flatpak to see how runtimes can be bundled
    • which may still not be the best approach

→ is what I can think of off the top of my head. Maybe others can chime in so we can create a list of Issues to create and actually work on instead of complaining and spamming this thread?

Please stop "+1"-ing the comments, there's an emoji reaction button at the top right of every comment where you can just do this:

– which shows your support but is way less spammy.

[sapioit]

I wish for this thread to remain here, at least to link to those other threads/issues suggested to be made in other places, and maybe also to offer updates on their progress.

[alixsep]

Just out of curiosity, Why haven't this been implemented yet? Are there any serious limitations?
Bundling a full browser for each app is absolutely not a good idea but that is the Electron's biggest weakness. If this issue gets fixed and all Electron apps use 1 global browser bundle, I dare to say Electron will be the absolute best way to make cross-platform apps for years.

[martpie]

Just out of curiosity, Why haven't this been implemented yet? Are there any serious limitations? Bundling a full browser for each app is absolutely not a good idea but that is the Electron's biggest weakness. If this issue gets fixed and all Electron apps use 1 global browser bundle, I dare to say Electron will be the absolute best way to make cross-platform apps for years.

The thing is distributing a shared runtime has serious impact on framework development:

• Make sure all APIs are backward compatible for a LONG time, means new features and heavy refactoring can heavily be impacted, by fear of breaking existing apps
• If the first point was not respected, looking at the current release pace of Electron, it would mean you would have a lot of different versions of Electron installed and running, reducing the benefits gains a lot
• Suddenly, the Electron team is in charge of updating its own framework, rather than letting the developers do it, that's additional infra and monitoring work, and I can pretty easily understand why they would not want to do that

Having a shared runtime is not a "perfect" solution, it's "one" solution, with its own pros and cons, advantages and drawbacks.

[memeplex]

Make sure all APIs are backward compatible for a LONG time, means new features and heavy refactoring can heavily be impacted, by fear of breaking existing apps

For the chromium part, how this compares to PWA? PWA is seen as a viable strategy nowadays, even though it has this same problem AFAICS. transpilers and polyfill are usually mentioned in this context.

Of course electron also ships node.

[sapioit]

• If the first point was not respected, looking at the current release pace of Electron, it would mean you would have a lot of different versions of Electron installed and running, reducing the benefits gains a lot

Sequential (diff) "updates" can be added to a base version, to reduce the size on disk for having more than one version of Electron installed.

For example, you need versions 1,4, 1.5, and 1.8 (random relevant numbers). You would have the v1.4 downloaded, then you would have the v1.5 downloaded (by the app managing them), then the v1.5 would be saved as a difference-file to be applied to v1.4, then the v1.8 would be downloaded, and a diff would be made for the previous available version (less storage-friendly than checking for every other version present, but a lot better than saving all versions in full). That's how they would be stored.

Then, you would simply download an archive which would specify the needed version of Electron, a copy of that would be made from the available versions, or a new version would be downloaded, extracted for immediate use, while in the background a diff would be made (with the origin of the diff specified, i.e. v1.4-diff-v1.6), so you can use the same version of electron again without re-downloading it. After you got a pristine (brand new, untouched) version of electron, you then the app would copy over the pristine files the relevant files from the app you want to launch, overwriting the pristine files when needed, then before closing, the list of to-be-saved files and folders (i.e. config files and folders) would be copied back into the archive, and the archive would get updated with the latest changes. That way, the old "executable" archive would get have it's older files replaced by the newer modified files. Then the local copy made for this app would get deleted.

That way, the space on disk would be significantly lowered, when not running an app. But you would have an instance for each new app. Though there are ways to optimize that even more, I think.

[xinkun-mini]

maybe, it can be use like gradle wrapper, electron can make sure every bugfix versions are backward compatible. for example: all 1.1.x are compatible, all 1.2.x are compatible
developers can define their app's electron feature version like '1.2.x'
then all electron apps compatible with same version on users computer only has one electron runtime, like ：$userhome/.electron/wrapper/1.2.x/

[foxxy432]

IT"S A PROBLEM WITH electron.run.32 Therefore these should be merged first in the appropriate order: #39581 #39581 #39726 #39732 #39696

[foxxy432]

While for a decade you guys could not even decide whether to make a runtime. Microsoft apparently released a webview runtime in 2020; capitalizing on users frustrated with installing dozens of electron-based apps with duplicated resources and eating away disk space; and developers annoyance to packaging electron with every new release.
Microsoft Edge WebView has two type of release

Evergreen - Runtime edition. User install this to run their desired webview app.
Fixed - this edition is to be packaged together with the Webview app, much like current Electron. Helps with app, which may lose stability with newer webview version.

More information on Microsoft's announcement.
Many popular apps; like PDF24, ImageGlass etc. already started using Microsoft WebView. Electron is rapidly loosing market share, especially for windows only apps. You guys should at least start an effort to test out some of the ideas.

[foxxy432]

While for a decade you guys could not even decide whether to make a runtime. Microsoft apparently released a webview runtime in 2020; capitalizing on users frustrated with installing dozens of electron-based apps with duplicated resources and eating away disk space; and developers annoyance to packaging electron with every new release.
Microsoft Edge WebView has two type of release
Evergreen - Runtime edition. User install this to run their desired webview app.
Fixed - this edition is to be packaged together with the Webview app, much like current Electron. Helps with app, which may lose stability with newer webview version.

More information on Microsoft's announcement.
Many popular apps; like PDF24, ImageGlass etc. already started using Microsoft WebView. Electron is rapidly loosing market share, especially for windows only apps. You guys should at least start an effort to test out some of the ideas.