Is it possible to use modules located for example on github?

[DeadMozay]

Is it possible to use modules located for example on github?
The module is loaded and packaged in node_modules, but when it is built, it still tries to load over the Internet.
For example, the package-lock.json file contains a module

"eslint-plugin-mattermost": "github:mattermost/eslint-plugin-mattermost#46ad99355644a719bf32082f472048f526605181",
...
"node_modules/eslint-plugin-mattermost": {
      "resolved": "git+ssh://git@github.com/mattermost/eslint-plugin-mattermost.git#46ad99355644a719bf32082f472048f526605181",
      "dev": true
    },
....
"eslint-plugin-mattermost": {
      "version": "git+ssh://git@github.com/mattermost/eslint-plugin-mattermost.git#46ad99355644a719bf32082f472048f526605181",
      "dev": true,
      "from": "eslint-plugin-mattermost@github:mattermost/eslint-plugin-mattermost#46ad99355644a719bf32082f472048f526605181"
    },

it is loaded and packaged with all modules
but when building it throws an error

npm ERR! request to https://codeload.github.com/mattermost/eslint-plugin-mattermost/tar.gz/46ad99355644a719bf32082f472048f526605181 failed, reason: getaddrinfo EAI_AGAIN codeload.github.com

[SchoolGuy]

I have a similar/equal problem:

npm ERR! command git --no-replace-objects ls-remote ssh://git@github.com/jitsi/eslint-config-jitsi.git

In my case it is above visible dependency which is successfully included in the tree but it appears npm still wants to access the internet.

[AdamMajer]

The respective package.json file(s) would have to be patched to request these things locally with the available version.

The tricky bit is some of the packages that are published upstream on the npm registry include these ssh:// and related non-registry locations.

So, solution would need to be 2 step,

1. the node_modules OBS service would need to create some sort of a map file between these special locations and local names and versions. Then,
2. this map would have to be read by this service and applied at runtime and provide patched package.json to npm.

[SchoolGuy]

@AdamMajer I don't know how but something is already there. The registry stil just checks for another version. I am not sure however why because in my case it would have been a sufficient one.

[AdamMajer]

"Registry" doesn't check for anything. It's just a dumb piece of software serving package.json from inside the packages. These are then used by npm to resolve all dependencies recursively. All intelligence on this lies within npm

What we have done with parts of cockpit is to replace the dependency with a version. So instead of ssh://git@github.com/jitsi/eslint-config-jitsi.git, this can become just some version. This version is then injected as additional tarball for npm-localhost-proxy and npm resolves the version from it.

The nasty problem is this also appears in packages fetched from upstream npm registry.

This service will download the packages from git or https and make a tarball. But content of package.json is not replaced with just version numbers. For this we should ideally have a map like this git -> package + version which then the npm-localhost-proxy could use. It's related to

openSUSE/npm-localhost-proxy#2

[SchoolGuy]

@AdamMajer Is it realistic that this gets solved soon? Or what can be done to help? Packaging Jitsi for example would get much easier with this service...

[AdamMajer]

This is now resolved via 1daa6a2

There are some caveats that are related to npm itself. For example, a dependency with https:// referring to git repo, needs to be changed to git+https://git@... due to npm/cli#2610 -- resolved deps will get your local git+ssh:// resolution and then things break. So, for example,

-    "react-data-grid": "https://github.com/pgadmin-org/react-data-grid.git#200d2f5e02de694e3e9ffbe177c279bc40240fb8",
+    "react-data-grid": "git+https://git@github.com/pgadmin-org/react-data-grid.git#200d2f5e02de694e3e9ffbe177c279bc40240fb8",

then run the node_modules service. THEN you need to patch the package.json with actual version you get from the git repo for npm to resolve it in the build VM to the ones that are available locally (including the github pulled version) to the npm-localhost-registry program.

-    "react-data-grid": "git+https://git@github.com/pgadmin-org/react-data-grid.git#200d2f5e02de694e3e9ffbe177c279bc40240fb8",
+    "react-data-grid": "7.0.0-beta.14",

You can look in OBS at the server:database:postgresql/pgadmin4 package for an example.