Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Paths everywhere #430

Merged
merged 1 commit into from
Feb 20, 2024
Merged

Paths everywhere #430

merged 1 commit into from
Feb 20, 2024

Conversation

cstamas
Copy link
Member

@cstamas cstamas commented Feb 14, 2024

Internally, stop using java.io.File and use NIO2 Paths instead. From outside, this should not make a lot of difference (and is enforced by japicmp as documented here).

But, as a consequence: if Resolver used on FileSystem like Google JIMFS, whatever code touches File method (that will do Path.toFile() under the hood), it will explode. Hence, we need to make sure this call never happens internally, and leave it to consumer projects. Resolver alone should never invoke it.

Most notable changes:

  • change internals to use NIO2 Paths.
  • in resolver now everything is "resolved" from local repository basePath, hence, if you put your local repository onto non-default FileSystem, it will work just fine (as proved by Demo Snippets). Naturally, if anything touches File returning method, in case of non-default FileSystem, it explodes (Google JIMFs Path.toFile() simply throws).
  • change transport-file to support any (and not only default) FileSystem, covered by tests. This makes possible to use any FileSystem hosted repositories as "remote file repositories". Covered by UTs that now runs as parameterized two times: using default and using JIMFS FileSystem (as it is simple to set up, very handy for testing).
  • change file locking to support any (and not only default) FileSystem. Before this change the implicit requirement that "lock name is fully qualified absolute file system path", but now it changes to "lock name is proper string representation of an URI". The key change is in BasedirNameMapper, where one can now use custom Path (for example created with JIMFS) and use that to name locks (use file: for default FS or any other URIs, ie jimfs: ones).

If you build this locally, and then related Maven PR (w/ japicmp disabled if not fixed), and then uncomment Google FS in demo Booter class, you will end up with Resolver DEMOs running on in-memory FS.


https://issues.apache.org/jira/browse/MRESOLVER-496

WIP
@cstamas
Copy link
Member Author

cstamas commented Feb 18, 2024

@gnodet @michael-o @slawekjaranowski both PRs (this and Maven apache/maven#1413) are green, please eyeballs

Copy link
Contributor

@gnodet gnodet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@cstamas cstamas merged commit aafe7e7 into apache:master Feb 20, 2024
4 checks passed
@cstamas cstamas deleted the paths-everywhere branch February 20, 2024 16:35
Copy link
Member

@michael-o michael-o left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In future, when you request someone to review such a huge PR give people enough time to do it otherwise it does not make sense.

// for now do exactly same as happened before, but FileProcessor is a component and can be replaced
String checksum = "";
try (BufferedReader br = new BufferedReader(
new InputStreamReader(Files.newInputStream(checksumPath), StandardCharsets.UTF_8), 512)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cstamas
Copy link
Member Author

cstamas commented Feb 20, 2024

In future, when you request someone to review such a huge PR give people enough time to do it otherwise it does not make sense.

Sure, and sorry for that. I assumed a week is enough.

OTOH, the longer the PR is held back, the new PRs are accumulating and as may saw. requires quite effort to sort out conflicts... So there are two sides of the coin.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants