Coursier throws a FileNotFoundException

[lefou]

While running mill mill.scalalib.Dependency.updates Mill aborted abnormally with the following exception, which is most likely a concurrency issue in coursier, seen before but deemed to be fixed.

> millw mill.scalalib.Dependency/showUpdates --format PerDependency
[2/2] ========================================================================================= mill.scalalib.Dependency/showUpdates --format PerDependency ====================================================================================== 251s
1 tasks failed
mill.scalalib.Dependency.updates java.lang.Exception: Failure during task evaluation: mill.define.Task$TraverseCtx@64f79891 java.io.FileNotFoundException: /home/lefou/.cache/coursier/v1/http/localhost%3A8099/repository/iba-nexus/com/fasterxml/jackson/module/jackson-module-parameter-names/maven-metadata.xml (Datei oder Verzeichnis nicht gefunden)
mill.define.Task$TraverseCtx@5d34d543 java.io.FileNotFoundException: /home/lefou/.cache/coursier/v1/http/localhost%3A8099/repository/iba-nexus/com/fasterxml/jackson/module/jackson-module-parameter-names/maven-metadata.xml (Datei oder Verzeichnis nicht gefunden)
    mill.eval.Evaluator.$anonfun$evalOrThrow$default$1$1(Evaluator.scala:68)
    mill.eval.EvaluatorImpl$EvalOrThrow.apply(EvaluatorImpl.scala:85)
    mill.scalalib.dependency.versions.VersionsFinder$.findVersions(VersionsFinder.scala:29)
    mill.scalalib.dependency.DependencyUpdatesImpl$.apply(DependencyUpdatesImpl.scala:25)
    mill.scalalib.Dependency$.$anonfun$updates$2(Dependency.scala:22)
error: Recipe `mill-updates` failed on line 215 with exit code 1

A subsequent run succeeded, so this is indeed on of the issues we previously handled in Mill with retry-logic and removed it later in #3505.

[lefou]

>  mill --version
Mill Build Tool version 0.12.1-7-d8bbe5
Java version: 21.0.4, vendor: Eclipse Adoptium, runtime: /opt/openjdk-bin-21.0.4_p7
Default locale: de_DE, platform encoding: UTF-8
OS name: "Linux", version: 6.1.90-gentoo-x86_64, arch: amd64

[lihaoyi]

CC @alexarchambault

[alexarchambault]

I'm not sure coursier is to blame here. The stack trace originates from the dependency updates plugin. It seems it tries to read a file that's been removed in the mean time. Maybe a stale cached path?

[lihaoyi]

@alexarchambault the error should be coming from the coursier resolution within the dependency updates task. @lefou is there more to the stack trace? If not we might have to adjust the error reporting code to preserve the internal stack trace when re-throwing

[lefou]

Unfortunately not. I posted the full output.

[alexarchambault]

It seems the DependencyUpdatesImpl / VersionsFinder code attempts to fetch many version listings in parallel, right?

One hypothesis I have is the following: some tasks running concurrently try to fetch a same version listing whose copy in cache is about to expire (because the last check for it is almost TTL-old). One task gets a file path from coursier right before it expires, and the other sees it as expired, and removes the now stale copy from the cache, that the first task attempts to read (and fails).

[alexarchambault]

A possible workaround could consist in having all checks use the exact same cutoff date in the past (computed as now - TTL, but computed beforehand only once now). That way, no task would remove a file that another was about to read. This needs some small changes in coursier, allowing coursier users to do that.

[lihaoyi]

@alexarchambault I'm not familiar with coursier internals but that sounds reasonable