Generics - recursive bound results in UnresolvedTypeVariable

[mkouba]

If a class declares a recursive bound (which should be legal and often used e.g. in builders) such as interface Score<S extends Score<S>> {} the type hierarchy contains an UnresolvedTypeVariable. And according to the javadoc "This type will only occur as a result of a bug, or a non-compliant Java class file".

A test can be found here:
mkouba@2141db3

In fact, I'm not quite sure whether it's a bug or how to handle this correctly.

[mkouba]

CC @n1hility

[ge0ffrey]

This bug blocks an improvement in the OptaPlanner quarkus extension to put it feature wise on par with the spring-boot starter.
https://github.com/kiegroup/optaplanner/blob/master/optaplanner-quarkus-integration/optaplanner-quarkus/runtime/src/main/java/org/optaplanner/quarkus/OptaPlannerBeanProvider.java#L59

[n1hility]

recursive types should be allowed, so I am calling this a bug. The unresolved type is for cases where you have undefined type parameters which at one point or another some bytecode frameworks were generating.

[ge0ffrey]

Is there a workaround for this issue?

[n1hility]

@ge0ffrey the only workaround at the moment is to do this on the caller side by tracking identifiers.

Sorry this one is taking longer for me to get to. There are some fundamental problems with implementing a solution that are tricky to get right. There are two major issues at play.

The first is that we don't want to introduce a non-terminating hang (or stack overflow) for existing code out there that is currently expecting a finite scan. So we need to retrofit the API in some way to address that.

The second issue is that the internals of jandex are designed to recycle instances for memory usage due to the amount of state involved (for example the exact same method signature, or generic type definition in multiple places in the index (different classes etc is reused via an intern pool). This requires immutable acyclic type nodes, and in the process of analysis these are rebuilt in a copy on write fashion. Altogether this makes it difficult to represent cyclical references and in a nice clean API.

Solving this will likely involve some form of external cycle linkage table with unique identifiers (in a way that doesn't bloat memory) and probably a new Type like (e.g. CyclicTypeVariable)

.

[Felk]

Hello, is there an ETA on when this would likely be fixed? This issue is unfortunately blocking me right now and I'd need to know whether it's better to wait for a fix or to start searching for an alternative.

EDIT: I avoided the issue by not having the class I rely on be discovered through CDI, but to specialize the class that injects it and hardcode it there instead.

[n1hility]

@Felk Glad you found a workaround. I am sorry for the long delay on this issue. It requires basically a rewrite of the generics handling, enforced inner class ordering, and additional APIs. Due to the garage scope I have prioritized the other items for 3.0. I will prioritize this for a 3.1 immediately after.

[Ladicek]

Assigning to myself, as this is the last thing I wanna do for Jandex 3.0.

[Ladicek]

I think I have a pretty decent implementation of this. It adds a new kind of type, a TypeVariableReference, which is used in a class/method list of type parameters to represent a type variable that wasn't fully defined yet. For simple recursive type parameters, such as T extends Comparable<T>, this naturally means that type variables occurring in their own definitions are represented as references. For mutually recursive type parameters, type variables that occur in their definitions or before their definitions are represented as references. A type variable reference can be follow()-ed to obtain the underlying type variable.

Implementation-wise, this requires a patching pass both when indexing a class (to resolve the unresolved type variables and propagate annotated type variables) and when deserializing an index (types are serialized in topological order, that is, if a type is composite, it is serialized after its constituent types, and references go against that), but nothing special.

I'm currently finishing the work to support type annotations on mutually recursive type parameters, everything else is done, so hopefully next week.

[n1hility]

@Ladicek awesome, sounds like a good approach. Did you look at the interplay with type annotations? When those are applied they have to replace an immutable reference tree and this approach will conflict with cycles, switching to a reference type indirection as you are doing should solve it.

[Ladicek]

The approach for type annotations I have is pretty sensible I think: for @Ann1 T extends Comparable<@Ann2 T>, the 2nd T is represented as a reference which has the @Ann2 type annotation. When the reference is followed, you get the type variable for 1st T, which has the @Ann1 type annotation. The same holds for mutually recursive type parameters, though it requires a little more complexity implementation-wise.

[Ladicek]

OK so type annotations on mutually recursive type parameters are quite a bit more complex than I imagined. Basically, when I have

class Graph<G extends Graph<G, E, V>, E extends Edge<G, E, V>, V extends Vertex<G, E, V>> {
}

then the bound of G extends Graph<G, E, V> contains 3 type variable references (G, E, V), but the bound of E extends Edge<G, E, V> contains 1 type variable (G) and 2 type variable references (E, V), and the bound of V extends Vertex<G, E, V> contains 2 type variables (G, E) and 1 type variable reference (V).

If I add type annotations to the type variable occurences, they are not propagated correctly to the "nested" type variables, because those are different instances. Which probably happens due to the copy-on-write mechanism of attaching type annotations to types.

I'll need to spend more time on this.

[Ladicek]

I believe I finally nailed it: #221 I want to do some tests with parameterized types that have an owner, but other than that, I think I'm done. Yay!

[Ladicek]

Done in #221.