flatpak-node-generator Throws KeyError: 'integrity'

[dginovker]

flatpak-builder version

1.2.2

Linux distribution and version

Xubuntu 22.04 LTS x86_64

Affected flatpak-builder tool

node/flatpak-node-generator.py

flatpak-builder tool cli args

npm package-lock.json

Source repository URL

https://github.com/piskelapp/piskel

Flatpak-builder manifest URL

(Not yet created)

Description

Hello fantastic maintainers -

I'm trying to package the electron piskel app (requested on the forums) but running into trouble. I'm sure this is something silly like using the tool wrong, but I'm also hoping the tool can be improved to not crash and leave a user confused.

Here are the steps I went through with the logic:

• Build flatpak-node-generator so I can generate the electron sources
• Clone the piskel app so I can get the the lockfile
• Run npm install to get the package-lock.json
• The above command fails since piskel is an old app with deprecated dependencies, so run npm install phantomjs-prebuilt@2.1.14 --ignore-scripts to get a required dependency
• Run npm install again
• Run flatpak-node-generator npm package-lock.json

$ flatpak-node-generator npm package-lock.json 
Reading packages from lockfiles...
Traceback (most recent call last):
  File "/home/wacket/.local/bin/flatpak-node-generator", line 8, in <module>
    sys.exit(main())
  File "/home/wacket/.local/pipx/venvs/flatpak-node-generator/lib/python3.10/site-packages/flatpak_node_generator/main.py", line 277, in main
    asyncio.run(_async_main())
  File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/home/wacket/.local/pipx/venvs/flatpak-node-generator/lib/python3.10/site-packages/flatpak_node_generator/main.py", line 198, in _async_main
    packages.update(lockfile_provider.process_lockfile(lockfile))
  File "/home/wacket/.local/pipx/venvs/flatpak-node-generator/lib/python3.10/site-packages/flatpak_node_generator/providers/npm.py", line 97, in process_lockfile
    yield from self.process_dependencies(lockfile, data.get('dependencies', {}))
  File "/home/wacket/.local/pipx/venvs/flatpak-node-generator/lib/python3.10/site-packages/flatpak_node_generator/providers/npm.py", line 78, in process_dependencies
    integrity = Integrity.parse(info['integrity'])
KeyError: 'integrity'

More potentially relevant info:

• Node version v16.14.2

Thanks!

[dginovker]

Looking into the issue more, I strongly suspect this has to do with running npm install phantomjs-prebuilt@2.1.14 --ignore-scripts. It adds an object to package-lock.json that is missing the 'integrity' child

[dginovker]

Closing issue since the package I'm trying to make is basically unbuildable

[gasinvein]

Can you please provide the package-lock.json file that caused the error?

[threema-danilo]

I have a case where this happens.

I depend on a local library (node-argon2) through a file path:

"argon2": "file:libs/node-argon2",

In the lockfile (which uses lockfileVersion 2), in the packages key, the entry looks like this:

     "node_modules/argon2": {
       "version": "0.28.3",
       "resolved": "file:libs/node-argon2",
       "hasInstallScript": true,
       "license": "MIT",
       "dependencies": {
         "@phc/format": "^1.0.0",
         "node-addon-api": "^4.2.0"
       },
       "engines": {
         "node": ">=12.0.0"
       }
     },

This contains a "resolved" URI that starts with file: so it can be handled accordingly.

However, under the dependencies key, the entry looks like this:

      "argon2": {
        "version": "0.28.3",
        "requires": {
          "@phc/format": "^1.0.0",
          "node-addon-api": "^4.2.0"
        }
      },

It contains neither a resolved key nor integrity.

I think the problem is that - for compatibility reasons - the self._process_packages_v1 function is used even for v2 packages:

If I force the use of self._process_packages_v2 it works.

So we probably either need a fix for the v1 function, or we should start using the v2 function for lockfile version 2.

[threema-danilo]

Hah, interesting, the issue above isn't present for all v2 lockfiles with Node 18, only with 18.14 due to this: nodejs/node#46542

When downgrading to a version >=18,<18.14 it works.

The problem is that node 18.14 changed the way local dependencies are installed. Besides the missing integrity field, there's also the problem that transitive dependencies don't end up in the lockfile at all anymore.

I guess in this case flatpak-node-generator doesn't need to do anything about it.

[gasinvein]

If I force the use of self._process_packages_v2 it works.

So we probably either need a fix for the v1 function, or we should start using the v2 function for lockfile version 2.

Yeah, we should default to v2 syntax for lockfiles v2. Eventually we will, once the git source support is complete for v2. But for now, maybe we should add a CLI switch like --npm-force-lockfile-syntax=v{1,2}? CC @refi64