Error when integrating with the Kotlin API

[breandan]

Following #105, I encountered the error below while integrating with the Kotlin API:

[ERROR] Failed to load library integration class 'edu.umontreal.kotlingrad.notebook.Integration'
java.lang.NullPointerException: clazz must not be null
	at org.jetbrains.kotlinx.jupyter.libraries.LibrariesScanner.instantiate(LibrariesScanner.kt:99)
	at org.jetbrains.kotlinx.jupyter.libraries.LibrariesScanner.access$instantiate(LibrariesScanner.kt:20)
	at org.jetbrains.kotlinx.jupyter.libraries.LibrariesScanner$instantiateLibraries$$inlined$forEach$lambda$1.invoke(LibrariesScanner.kt:88)
	at org.jetbrains.kotlinx.jupyter.libraries.LibrariesScanner$instantiateLibraries$$inlined$forEach$lambda$1.invoke(LibrariesScanner.kt:20)
	at org.jetbrains.kotlinx.jupyter.libraries.LibrariesScanner$instantiateLibraries$1.invoke(LibrariesScanner.kt:72)
	at org.jetbrains.kotlinx.jupyter.libraries.LibrariesScanner.instantiateLibraries(LibrariesScanner.kt:87)
	at org.jetbrains.kotlinx.jupyter.libraries.LibrariesScanner.addLibrariesFromClassLoader(LibrariesScanner.kt:31)
	at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl$execute$1$3.invoke(CellExecutorImpl.kt:79)
	at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl$execute$1$3.invoke(CellExecutorImpl.kt:28)
	at org.jetbrains.kotlinx.jupyter.config.LoggingKt.catchAll(logging.kt:24)
	at org.jetbrains.kotlinx.jupyter.config.LoggingKt.catchAll$default(logging.kt:23)
	at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl.execute(CellExecutorImpl.kt:78)
	at org.jetbrains.kotlinx.jupyter.repl.CellExecutor$DefaultImpls.execute$default(CellExecutor.kt:20)
	at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl$eval$1.invoke(repl.kt:357)
	at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl$eval$1.invoke(repl.kt:144)
	at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl.withEvalContext(repl.kt:335)
	at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl.eval(repl.kt:350)
	at org.jetbrains.kotlinx.jupyter.ProtocolKt$shellMessagesHandler$res$1.invoke(protocol.kt:287)
	at org.jetbrains.kotlinx.jupyter.ProtocolKt$shellMessagesHandler$res$1.invoke(protocol.kt)
	at org.jetbrains.kotlinx.jupyter.ProtocolKt.evalWithIO(protocol.kt:443)
	at org.jetbrains.kotlinx.jupyter.ProtocolKt.shellMessagesHandler(protocol.kt:286)
	at org.jetbrains.kotlinx.jupyter.IkotlinKt.kernelServer(ikotlin.kt:133)
	at org.jetbrains.kotlinx.jupyter.IkotlinKt.kernelServer$default(ikotlin.kt:101)
	at org.jetbrains.kotlinx.jupyter.IkotlinKt.main(ikotlin.kt:74)
[ERROR] 
org.jetbrains.kotlinx.jupyter.compiler.util.ReplException: Failed to load library integration class 'edu.umontreal.kotlingrad.notebook.Integration'
	at org.jetbrains.kotlinx.jupyter.libraries.LibrariesScanner$instantiateLibraries$1.invoke(LibrariesScanner.kt:76)
	at org.jetbrains.kotlinx.jupyter.libraries.LibrariesScanner.instantiateLibraries(LibrariesScanner.kt:87)
	at org.jetbrains.kotlinx.jupyter.libraries.LibrariesScanner.addLibrariesFromClassLoader(LibrariesScanner.kt:31)
	at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl$execute$1$3.invoke(CellExecutorImpl.kt:79)
	at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl$execute$1$3.invoke(CellExecutorImpl.kt:28)
	at org.jetbrains.kotlinx.jupyter.config.LoggingKt.catchAll(logging.kt:24)
	at org.jetbrains.kotlinx.jupyter.config.LoggingKt.catchAll$default(logging.kt:23)
	at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl.execute(CellExecutorImpl.kt:78)
	at org.jetbrains.kotlinx.jupyter.repl.CellExecutor$DefaultImpls.execute$default(CellExecutor.kt:20)
	at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl$eval$1.invoke(repl.kt:357)
	at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl$eval$1.invoke(repl.kt:144)
	at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl.withEvalContext(repl.kt:335)
	at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl.eval(repl.kt:350)
	at org.jetbrains.kotlinx.jupyter.ProtocolKt$shellMessagesHandler$res$1.invoke(protocol.kt:287)
	at org.jetbrains.kotlinx.jupyter.ProtocolKt$shellMessagesHandler$res$1.invoke(protocol.kt)
	at org.jetbrains.kotlinx.jupyter.ProtocolKt.evalWithIO(protocol.kt:443)
	at org.jetbrains.kotlinx.jupyter.ProtocolKt.shellMessagesHandler(protocol.kt:286)
	at org.jetbrains.kotlinx.jupyter.IkotlinKt.kernelServer(ikotlin.kt:133)
	at org.jetbrains.kotlinx.jupyter.IkotlinKt.kernelServer$default(ikotlin.kt:101)
	at org.jetbrains.kotlinx.jupyter.IkotlinKt.main(ikotlin.kt:74)
Caused by: java.lang.NullPointerException: clazz must not be null
	at org.jetbrains.kotlinx.jupyter.libraries.LibrariesScanner.instantiate(LibrariesScanner.kt:99)
	at org.jetbrains.kotlinx.jupyter.libraries.LibrariesScanner.access$instantiate(LibrariesScanner.kt:20)
	at org.jetbrains.kotlinx.jupyter.libraries.LibrariesScanner$instantiateLibraries$$inlined$forEach$lambda$1.invoke(LibrariesScanner.kt:88)
	at org.jetbrains.kotlinx.jupyter.libraries.LibrariesScanner$instantiateLibraries$$inlined$forEach$lambda$1.invoke(LibrariesScanner.kt:20)
	at org.jetbrains.kotlinx.jupyter.libraries.LibrariesScanner$instantiateLibraries$1.invoke(LibrariesScanner.kt:72)
	... 19 more

Not sure what might be happening. This is my integration class:

package edu.umontreal.kotlingrad.api

import org.jetbrains.kotlinx.jupyter.api.annotations.JupyterLibrary
import org.jetbrains.kotlinx.jupyter.api.*
import org.jetbrains.kotlinx.jupyter.api.libraries.*

@JupyterLibrary
class Integration: JupyterIntegration() {
  override fun Builder.onLoaded() {
    import("edu.umontreal.kotlingrad.api.*")
  }
}

I am using Kotlin Jupyter built from the source as of 6cc5a93 with version 0.8.3.258 of the org.jetbrains.kotlin.jupyter.api plugin.

[ileasile]

It seems that edu.umontreal.kotlingrad.notebook.Integration which is in the exception message and edu.umontreal.kotlingrad.api.Integration which is in your code are different ones. May you do the full rebuild and check the result?

[breandan]

Yeah, I've tried quite a few variations of the version pasted above, all with the same result. The strange thing is, adding notebook integration worked fine with on another library, but I haven't been able to get around this error.

One thing I noticed during this process is that in addition to doing a full-rebuild and restarting the kernel, you also need to delete the caches via rm -rf ~/.ivy2/cache/<ARTIFACT> ~/.m2/repository/<ARTIFACT>, then redeploy via ./gradlew build publishToMavenLocal, otherwise the @file:Repository("*mavenLocal") resolver will use the cached version instead of fetching the latest version in ~/.m2/repository/<ARTIFACT>. I thought rm -rf ~/.m2/repository/<ARTIFACT> then redeploying would overwrite the existing version, but there is also a separate IVY cache which must be flushed. This caused a few hours of confusion.

[altavir]

@breandan I've actually reported the problem with ivy in #121. I think that if it is possible, the ivy cache should be bypassed for mavenLocal. Clearing m2 is not necessary, it is always considered to be snapshot repository. Also, I found out that kapt scanner does not clear caches properly in gradle. If you change the path to the integration module, you should do a clean build after that.

[ileasile]

Regarding initial plugin-related problem, may you try with 0.8.3.268?

[breandan]

@ileasile That seems to have worked, thank you for the fix!