Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support private registries for Maven #1045

Open
cuixq opened this issue Jun 17, 2024 · 4 comments
Open

Support private registries for Maven #1045

cuixq opened this issue Jun 17, 2024 · 4 comments
Assignees
@cuixq
Labels
enhancement New feature or request guided remediation Related to guided remediation / osv-scanner fix

Comments

@cuixq
Copy link
Contributor

cuixq commented Jun 17, 2024

Now by default, deps.dev API is queried for package versions and requirements when resolving dependencies.

We should also support fetching these data from private registries as well.
@cuixq cuixq mentioned this issue Jun 17, 2024
Merged
@jenspopp
Copy link

That would be a highly appreciated feature ;-) Right now it already fails for my most simple projects with:

failed extracting pom.xml: failed to merge parents: failed to get Maven project my.maven:super-pom:1.7.2: API query failed: Maven registry query status: 4
04 Not Found

@cuixq cuixq mentioned this issue Jul 18, 2024
Closed
@cuixq cuixq added the enhancement New feature or request label Jul 18, 2024
@cuixq cuixq self-assigned this Jul 30, 2024
@cuixq cuixq mentioned this issue Aug 2, 2024
Merged
@cuixq cuixq added the guided remediation Related to guided remediation / osv-scanner fix label Aug 8, 2024
@cuixq cuixq mentioned this issue Aug 21, 2024
Merged
cuixq added a commit that referenced this issue Aug 22, 2024
@cuixq
refactor: move Maven utility to a separate package (#1193)
35b36a3 
#1045

Considering that we want to support native Maven registry, we need
`MergeMavenParents` from `internal/manifest` in
`internal/resolution/client`, however `internal/manifest` imports
`internal/resolution/client` for `DependencyClient` for dependency
resolution, and this causes an import cycle.

This PR moves the Maven utility in `internal/manifest` to a separate
package `internal/utility/maven`.
@cuixq cuixq mentioned this issue Aug 27, 2024
Merged
cuixq added a commit that referenced this issue Sep 4, 2024
@cuixq
feat(internal): add Maven native dependency client (#1207)
a4e1386 
#1045

This PR adds a dependency client fetching from Maven Central repository:
 - the client is based on the Maven registry API client
 - fetched projects and metadata are cached
@cuixq
Copy link
Contributor Author

cuixq commented Sep 19, 2024

Some updates regarding this issue:

  • There is a MavenRegistryAPIClient to talk to a Maven registry directly for metadata. Currently the URL is hard-coded to Maven Central, and we would like to make this configurable.
  • There is also MavenRegistryClient which can be used as a DependencyClient in dependency resolution. More work is needed to make it work with multiple-registry resolution which may involve working on deps.dev's Maven resolver.

@cuixq cuixq mentioned this issue Sep 30, 2024
Merged
cuixq added a commit that referenced this issue Oct 21, 2024
@cuixq
feat: fetch Maven metadata from specified repositories (#1286)
0c596bf 
#1045

There are [repositories](https://maven.apache.org/pom.html#Repositories)
defined in a Maven pom.xml. When looking for an artifact, these
repositories are searched one by one until the artifact is found. Maven
Central is the default registry to try at the last.

To support this behaviour, this PR:
- makes `MavenRegistryAPIClient` host a list of registries besides the
default registry
- adds `UpdateRegistries` to `DependencyClient` to update the registries
 - adds a new flag to specify the default maven registry for `fix`
- add new experimental options to `scan` to align with what we have for
`fix`

TODO: 
 - still need to update documentation for new options/flags
 - update deps.dev Maven resolver for mutil-registry resolution
 - record not found requests to optimize performance
@another-rex
Copy link
Collaborator

Should this issue be closed with #1286 ?

@cuixq
Copy link
Contributor Author

cuixq commented Oct 30, 2024

There is one optimisation PR in progress and this can be closed after that.

@BrewTestBot BrewTestBot mentioned this issue Oct 31, 2024
Merged
@cuixq cuixq mentioned this issue Nov 21, 2024
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@cuixq cuixq

Labels
enhancement New feature or request guided remediation Related to guided remediation / osv-scanner fix
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jenspopp @cuixq @another-rex