Move Lute to ESM modules

[etherealite]

Moving Lute to ESM Modules

Proof of concept for the proposal to incrementally introduce ECMAScript Modules (ESM) into Lute.

Motivation:

At present, Lute has some blocks of script tags with manually tracked dependencies between each script. Certain HTML templates make use of different scripts, so a large number of script tags have to be painstakingly re-defined, with each script dependency hand-tracked and accounted for in the template code.

When ESM modules are combined with import maps, the need for manually maintaining dependencies between scripts and across different template files is alleviated or eliminated. This comes with the added bonus of ESM providing all the benefits of module isolation and an easier syntax to link program components than globals and objects alone.

Approach:

An incremental migration from script tags is preferred. This will involve keeping most assets defined as script tags in place, at least until their dependents can be migrated. Each library dependency that currently exists as a script tag can be upgraded to a current or minimal version supporting ESM.

New ESM module library dependencies can co-exist with their script tag counterparts, but they will be separated through the isolation provided by ESM. A site-wide import map has been placed in the root-most HTML template; this is where all ESM module dependencies should reside.

Cautions and Limitations:

Having parallel versions of core libraries in use across the codebase carries risks even with the isolation of ESM modules. Within your ESM module, you might be using a reference to a script tag library without noticing it.

You might do this:

import DataTable from 'datatables.net';

new DataTable('#booktable', {
...
   return $('#datatables_config_widget');

When you meant to do this:

import DataTable from 'datatables.net';
import $ from 'jquery'; /** <-woops, forgot this */

new DataTable('#booktable', {
 ...
   return $('#datatables_config_widget');

When the script tag for jQuery is ultimately removed, this code will stop working in the former case, and it will often be tricky to determine why. Consequently, it would be best to ensure that any script converted into an ESM module is free of global references to names declared in script tags when possible.

Dependencies that Ship with Support for ESM:

Some of the library scripts that Lute currently ships with do not support ESM modules, at least not with the versions present in Lute. However, many of the scripts could be upgraded to versions where ESM is supported with a medium level of effort.

Dependencies that Do Not Ship with Support for ESM:

For libraries that don’t support ESM, there is the option of making one-off custom builds. Scripts that use CommonJS can sometimes be converted to ESM with minimal edits and a compilation step or even automatically. For now, the issue is being sidestepped by using a CDN that converts CommonJS to ESM in the case of jQuery and DataTables. Finding a solution to this particular issue is ongoing.

[jzohrab]

Belated thanks, @etherealite .

Maybe a good place to start with this would be Lute's modules themselves. There's only a few of them (in lute/static/js), but it's enough to occasionally make things annoying. e.g., I now want to add a "lute_shortcuts.js" helper module with a simple function get_shortcut_string(e) to be used in a few places (in lute.js, as well as on a user form), and I don't like having it just be a function floating in space. Do you have any suggestions as to how to start such a thing?