Scala Cli supporting remapping JS import at link time?

[Quafadas]

@armanbilge Please chime in, should this already be better explained elsewhere. I'm opening a discussion here, because I've been thinking on and off for a couple of days about this, and I think this could hold transformational potential for scalaJS, particularly if done in scala-cli...

Here is the best description I'm aware of;
scala-js/scala-js#4837

In that issue, @sjrd suggests scalaJS core might not be the right place. I will claim, scala-cli is.

A first cut on a solution, produced on an astonishingly short timescale;
https://github.com/armanbilge/scalajs-importmap

What I'd propose to avoid;
#297

Why would this is worthy of its own discussion / important / worth spamming people over?
An example; the other day, I wanted to cache an http request persistently through a page reload. Because whatever.

I had a good key for a simple "GET", so I wanted I a simple key -> val store. I'm not aware of one in scala JS. So hit up google, discovered indexedDB, and after experimenting scalaJSdom, struggling a little, experimenting with scalablytyped on a couple of libraries, research, try fail, eventually hit on;
https://www.npmjs.com/package/idb-keyval

So cool, works.

Now, I have a promise runtime, an IO runtime to make the request I want to cache (love smithy) and laminar (love that too) EventStream to manage the UI. 3 runtimes! As an aside, It is a marvel of engineering and testament to the robustness of scala JS, that actually works (!).

Not so ideal about that solution
It took, quite some time, and quite a bit of headspace to get it to all work. Also, now I have scalably typed in the build. Scalably typed is awesome, but does add some complexity to the build, and certainly after clean, some non-trivial amount of time to the application feedback loop.

I was basically solving a "POC" problem, with "project" infrastructure. I see two issues;

1. That kind of POC should ideally be done in a "clean room" scala-cli setting where it's much faster to explore try fail. Not an application setting with associated overheads.
2. The time investment I made cannot be subsequently leveraged by the community. I wonder, how much headspace is lost to re-inventing similar things?

Maybe it was an idiosyncratic need, but what I really wanted to do, was publish a tiny helper library, that married this idb-keyval facade to a thin Laminar skin in a clean room.

Why didn't I
Because I have only a fuzzy idea of how to go about doing it. I think I'd need a new sbt project, the vite config, the scalable typed plugin plus its library incantations, bundler setup, dev server setup etc - the toolchain is actually... kind of long. I concluded, that it was an investment that would cost me more time, than was justifiable.

How would this help?
If in fact it was possible to just skip the entire bundling thing, which I believe this would allow, then scala-cli already has everything else it needs to experiment with, test and publish that tiny helper library. I could have solved my POC problem in the small, and then sucked it into my "actual project". Plus anyone else who hit the same problem, might then have stumbled across an idiosyncratic interface, rather than re-inventing it themselves in isolation.

What I'm going to claim here, (on a small sample size of me) is that this is actually quite a general pattern in scala JS.

Interfacing into existing javascript / typescript libraries is a killer feature - but right now, I suspect lots of collective headspace is lost, to "skinning" or squeezing those libraries into idiomatic scala JS inside application code. Unless I'm missing something, there are remarkably few "higher level" libraries built on top of, say Laminar - I'm not aware of a CachedEventStream for example (hoping I don't come away red faced here).

I make three claims;

• the reason for the paucity of such libraries is that the current toolchain required to marry a scalaJS library to a typescript library, is too intimidating for "me" and those at a similar skill level.
• scala-cli + this feature has the potential to sweep that entire toolchain away - making this sort of scalaJS library publishing practicable for a larger body of (non-expert) people
• it feels "philosophically" in line with what scala-cli is trying to achieve

I could well have missed something in this thought experiment, but I think it has exciting potential...

[armanbilge]

Thank you so much for starting this discussion! 💯

I completely concur. It would be excellent if scala-cli would support this feature. I recently attempted to motivate this exact idea in #297 (comment).

Link-time "import mapping" would enable scala-cli to create standalone, ready-to-deploy applications with JS dependencies out-of-the-box, without having to rely on additional tooling and bundlers, like npm and vite.

The key idea is to take advantage of the fact that:

1. browsers are already able to load JavaScript ES modules from a URL
2. JavaScript CDNs such as unpkg and jsDelivr already serve NPM packages as ES modules from URLs

All we have to do to make this work is:

1. provide a way to configure a mapping from NPM packages to CDN URLs
2. rewrite the @JSImport statements in a published Scala.js library according to that mapping

This mapping can and should happen at user application linking time, because they may have their own preferred CDNs (or even self-host). Published libraries can and should continue to reference the NPM packages for their declared @JSImports since this is standard practice.

Example

Consider this popular web component library for Laminar:

• https://sherpal.github.io/laminar-ui5-demo/
• https://github.com/sherpal/LaminarSAPUI5Bindings

That library depends on the @ui5/webcomponents npm package. So, out-of-the-box, it is impossible to use that library with sbt or scala-cli without involving a JavaScript bundler. Indeed, the demo uses vite.

However, with import mapping, we can rewrite that library's @JSImport statements in our application at linktime to direct the browser to load the ui5 library from the unpkg CDN.

The configuration using sbt-scalajs-importmap looks like this:

scalaJSImportMap := { (rawImport: String) =>
  "https://unpkg.com/" + rawImport + "?module"
}

Concretely, it makes this change in the generated JS:

- import * as $i_$0040ui5$002fwebcomponents$002fdist$002fInput$002ejs from "@ui5/webcomponents/dist/Input.js";
+ import * as $i_https$003a$002f$002funpkg$002ecom$002f$0040ui5$002fwebcomponents$002fdist$002fInput$002ejs$003fmodule from "https://unpkg.com/@ui5/webcomponents/dist/Input.js?module";

Full project configuration below.

project/plugins.sbt

addSbtPlugin("com.armanbilge" % "sbt-scalajs-importmap" % "0.1.0")

build.sbt

scalaVersion := "3.2.2"
enablePlugins(ScalaJSPlugin, ScalaJSImportMapPlugin)
scalaJSLinkerConfig ~= (_.withModuleKind(ModuleKind.ESModule))
scalaJSUseMainModuleInitializer := true
libraryDependencies ++= List(
  "be.doeraene" %%% "web-components-ui5" % "1.9.2",
  "com.raquo" %%% "laminar" % "15.0.0"
)
scalaJSImportMap := { (rawImport: String) =>
  "https://unpkg.com/" + rawImport + "?module"
}

demo.scala

import be.doeraene.webcomponents.ui5.*
import be.doeraene.webcomponents.ui5.configkeys.*
import com.raquo.laminar.api.L.{*, given}
import com.raquo.airstream.core.*
import org.scalajs.dom

@main def main = render(
  dom.document.querySelector("#app"),
  div(
    Input(
      _.required := true,
      _.valueState := ValueState.Information,
      _.placeholder := "Enter your name",
      onChange.mapToValue --> Observer(println)
    )
  )
)

index.html

<!DOCTYPE html>
<html>
  <head></head>
  <body>
    <div id="app"></div>
    <script type="module" src="target/scala-3.2.2/sandbox-fastopt/main.js"></script>
  </body>
</html>

That works well enough for sbt. But as Simon points out this feature could be even more valuable in scala-cli for fast prototyping and deploying.

[tgodzik]

I am not really well versed in ScalaJS or JS worlds themselves, but is it even possible using the current using directives? How could that look like?

[armanbilge]

I am not really well versed in ScalaJS or JS worlds themselves, but is it even possible using the current using directives? How could that look like?

Good question. Yes, we'll have to find a different configuration format rather than an arbitrary String => String function. Actually, Seb just pointed out to me that to upstream this feature to Scala.js core we'd have to change that anyway, so that the config is serializable.

For example a similar mapping configuration is based on regex.
https://github.com/scala-js/scala-js/blob/ede78b394ac77ac2d2404d45a9a246e7e3a567bb/linker-interface/shared/src/main/scala/org/scalajs/linker/interface/Semantics.scala#L180-L192

[Quafadas]

@armanbilge I was thinking about this ... scala3 has a fairly strong type system, right?

Wacky idea, but could something like

val from: "jsLib" = "jsLib"
val to: "somewhere/in/cdn" = "somewhere/in/cdn"

"jsLib" => "somewhere/in/cdn"

Help with that at all?