Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ANE-2235] Add support for PNPM v9 lockfile format #1501

Draft
wants to merge 10 commits into
base: master
Choose a base branch
from
Draft

Conversation

ryanlink
Copy link
Contributor

@ryanlink ryanlink commented Jan 31, 2025

Overview

Changes

This PR improves the PNPM v9 lockfile parsing to correctly handle:

  1. Package keys without leading slashes in v9 format
  2. Catalog versions (e.g., workspace:* or workspace:^1.0.0)
  3. Proper version normalization for workspace dependencies

Implementation Details

  • Added catalogs field to PnpmLockfile type to parse the new catalog section
  • Created CatalogMap and CatalogEntry types to properly handle catalog entries
  • Added getPackageVersion helper to resolve catalog version references
  • Updated version parsing to properly handle v7-v9 lockfiles
  • Simplified lockfile version handling by consolidating v7+ into PnpmLockGt6

Fixes

This addresses several issues where:

  • Package names and versions were incorrectly concatenated in the UI
  • Dependencies with catalog versions weren't properly resolved
  • Dev dependencies under the importers field weren't correctly identified

Motivation

PNPM v9 introduced a new lockfile format that uses a catalog system for version management. This change ensures we can correctly parse these lockfiles and extract accurate dependency information.

Acceptance criteria

Using pnpm 9 with a v9 lockfile, all npm dependencies are correctly resolved with proper locators in the FOSSA UI.

Testing plan

Tested against PNPM's own repository (which uses v9 lockfile) to verify correct handling of:

  • Catalog version references
  • Package keys without leading slashes
  • Workspace dependencies

Risks

Highlight any areas that you're unsure of, want feedback on, or want reviewers to pay particular attention to.

Example: I'm not sure I did X correctly, can reviewers please double-check that for me?

Metrics

Is this change something that can or should be tracked? If so, can we do it today? And how? If its easy, do it

References

https://fossa.atlassian.net/browse/ANE-2235
https://fossa.atlassian.net/browse/ANE-2177
https://teamfossa.slack.com/archives/C022BQWCR45/p1733495478363879

Checklist

  • I added tests for this PR's change (or explained in the PR description why tests don't make sense).
  • If this PR introduced a user-visible change, I added documentation into docs/.
  • If this PR added docs, I added links as appropriate to the user manual's ToC in docs/README.ms and gave consideration to how discoverable or not my documentation is.
  • If this change is externally visible, I updated Changelog.md. If this PR did not mark a release, I added my changes into an ## Unreleased section at the top.
  • If I made changes to .fossa.yml or fossa-deps.{json.yml}, I updated docs/references/files/*.schema.json AND I have updated example files used by fossa init command. You may also need to update these if you have added/removed new dependency type (e.g. pip) or analysis target type (e.g. poetry).
  • If I made changes to a subcommand's options, I updated docs/references/subcommands/<subcommand>.md.

csasarak and others added 4 commits January 3, 2025 23:08
1. Handle package keys without leading slash in v9 format
2. Add support for catalog versions (workspace:*) in v9 format
3. Improve version normalization and handling
4. Add more test cases and documentation
@ryanlink
Copy link
Contributor Author

ryanlink commented Jan 31, 2025

Integration test is consistently failing for me in GH and locally. TL;DR it’s Failed to fetch and copy dep: {git,"https://github.com/ninenines/cowlib", "2.11.0"} -- see https://fossa.atlassian.net/browse/ANE-2236?focusedCommentId=21909

- Add catalogs field to PnpmLockfile type
- Create CatalogMap and CatalogEntry types for parsing catalog data
- Add getPackageVersion helper to resolve catalog versions
- Update version parsing to properly handle v7-v9 lockfiles
@ryanlink ryanlink changed the title fix: improve pnpm v9 lockfile parsing and catalog version support (ANE-2235) [ANE-2235] fix: improve pnpm v9 lockfile parsing and catalog version support Jan 31, 2025
@ryanlink ryanlink changed the title [ANE-2235] fix: improve pnpm v9 lockfile parsing and catalog version support [ANE-2235] Add support for PNPM v9 lockfile format Jan 31, 2025
- Track dev dependencies from importers section
- Consider both importers and packages sections when determining dev status
- Pass dev dependency status through dependency resolution chain
- Update toDependency to combine isDev flags from both sources
- Rename version field in CatalogEntry to catalogVersion to avoid collision with ProjectMapDepMetadata
- Use parseJSON instead of undefined parseCatalogEntry
- Use fully qualified Yaml.parseJSON to fix compilation error
- Add KeyMap import
- Convert Object to Map Text using KeyMap.toMap
@ryanlink
Copy link
Contributor Author

ryanlink commented Feb 4, 2025

@csasarak Sorry I've made a big mess of this branch. Considering reverting my commits to restore to where you were, as I've been creating cascading issues. I'll start another branch for my experiments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants