Exception in thread "main" ai.djl.engine.EngineException: Failed to load MXNet native library

[AngshumanChakraborty]

I am new to DJL and gradle. I wanted to try out djl. So I got 0.16.0-SNAPSHOT. I imported it into eclipse version rcp-2019-03. I tried running /examples/src/main/java/ai/djl/examples/training/TrainMnist.java and received the following error message.

[WARN ] - Failed to detect GPU count: CUDA driver version is insufficient for CUDA runtime version (35)
Exception in thread "main" ai.djl.engine.EngineException: Failed to load MXNet native library
at ai.djl.mxnet.engine.MxEngine.newInstance(MxEngine.java:78)
at ai.djl.mxnet.engine.MxEngineProvider.getEngine(MxEngineProvider.java:40)
at ai.djl.engine.Engine.getEngine(Engine.java:177)
at ai.djl.engine.Engine.getInstance(Engine.java:132)
at ai.djl.examples.training.util.Arguments.initialize(Arguments.java:41)
at ai.djl.examples.training.util.Arguments.parseArgs(Arguments.java:78)
at ai.djl.examples.training.TrainMnist.runExample(TrainMnist.java:53)
at ai.djl.examples.training.TrainMnist.main(TrainMnist.java:49)
Caused by: java.lang.AssertionError: No mxnet version found in property file.
at ai.djl.util.Platform.detectPlatform(Platform.java:84)
at ai.djl.mxnet.jna.LibUtils.findLibraryInClasspath(LibUtils.java:110)
at ai.djl.mxnet.jna.LibUtils.getLibName(LibUtils.java:79)
at ai.djl.mxnet.jna.LibUtils.loadLibrary(LibUtils.java:63)
at ai.djl.mxnet.jna.JnaUtils.(JnaUtils.java:69)
at ai.djl.mxnet.engine.MxEngine.newInstance(MxEngine.java:52)
... 7 more

But when I run the same example from the command line by executing
./gradlew run -Dmain=ai.djl.examples.training.TrainMnist
I got 96.93 percent validation accuracy at the end of the second epoch. It was success. But I cannot do the same from within eclipse. I am using eclipse 16.04 x86_64. Although I have a CUDA GPU and CUDA 10.2 installed but CUDA driver not properly installed. Here I just want to run djl on CPU only.

[frankfliu]

@AngshumanChakraborty
The message is just a warning:

[WARN ] - Failed to detect GPU count: CUDA driver version is insufficient for CUDA runtime version (35)

The message means the CUDA driver is not matching the CUDA API, DJL will fallback to use CPU.

If the gradle command line works, that means eclipse didn't import the gradle project properly. I'm not familiar with eclipse. You might want to consult eclipse experts.

Based on the exception, it seems mxnet-engine.properties file is not included in the classpath. This file is created during the gradle build. If eclipse didn't execute gradle tasks, or it didn't include engines/mxnet/mxnet-engine/build/classes/java/main/ as class path, you might see this exception

Can you check:

1. example project contains both maven and gradle build script. Can you check if eclipse imported maven instead of gradle?
2. If you are using maven, can you try to manually publish API, engine/mxnet/mxnet-engine to your local maven repo:

./gradlew api:pTML
./gradlew engines:mxnet:mxnet-engine:pTML

[AngshumanChakraborty]

The example project within DJL gradle project fails with the message when I am running TrainMnist.java as java application.
Exception in thread "main" ai.djl.engine.EngineException: Failed to load MXNet native library at ai.djl.mxnet.engine.MxEngine.newInstance(MxEngine.java:78) at ai.djl.mxnet.engine.MxEngineProvider.getEngine(MxEngineProvider.java:40) at ai.djl.engine.Engine.getEngine(Engine.java:177) at ai.djl.engine.Engine.getInstance(Engine.java:132) at ai.djl.examples.training.util.Arguments.initialize(Arguments.java:41) at ai.djl.examples.training.util.Arguments.parseArgs(Arguments.java:78) at ai.djl.examples.training.TrainMnist.runExample(TrainMnist.java:58) at ai.djl.examples.training.TrainMnist.main(TrainMnist.java:54) Caused by: java.lang.AssertionError: No mxnet version found in property file. at ai.djl.util.Platform.detectPlatform(Platform.java:84) at ai.djl.mxnet.jna.LibUtils.findLibraryInClasspath(LibUtils.java:110) at ai.djl.mxnet.jna.LibUtils.getLibName(LibUtils.java:79) at ai.djl.mxnet.jna.LibUtils.loadLibrary(LibUtils.java:63) at ai.djl.mxnet.jna.JnaUtils.<clinit>(JnaUtils.java:69) at ai.djl.mxnet.engine.MxEngine.newInstance(MxEngine.java:52) ... 7 more
And here is the screenshot of the DJL projects that has been imported into eclipse.

The 'engine' project has no artifact visible in the eclipse. The 'engine' project does have the file mxnet-engine.properties in the above path.

1. Yes, example project contains both maven and gradle build script as is visible from the screenshot. I checked and found eclipse imported the entire djl as gradle because it has gradle nature. I think example is accessing djl/engines/mxnet/mxnet-engine/bin/main/ as it loads resources from this path.

gradle is itself using java 11 and for example and all other djl projects eclipse configuration is using java 8.

As a workaround, I created a new maven project and pulled in the required dependencies. Then I pulled in TrainMnist.java and was able to successfully train in eclipse.

When I run the djl/examples/application/run gradle task I get the following error:

Task :examples:run FAILED

Error: Could not create the Java Virtual Machine.
-Djava.endorsed.dirs=/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/endorsed is not supported. Endorsed standards and standalone APIs
Error: A fatal exception has occurred. Program will exit.
in modular form will be supported via the concept of upgradeable modules.

FAILURE: Build failed with an exception.

• What went wrong:

Execution failed for task ':examples:run'.

Process 'command '/usr/lib/jvm/java-11-openjdk-amd64/bin/java'' finished with non-zero exit value 1

• Try:
  Run with --info or --debug option to get more log output. Run with --scan to get full insights.

• Exception is:
  org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':examples:run'.

[frankfliu]

@AngshumanChakraborty

DJL compile target is set to java 8, but you should be able to run/build DJL using JDK 11.
I'm not sure where: -Djava.endorsed.dirs=/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/endorsed is coming from.

Are you using gradlew to run the gradle task? gradlew will download the supported gradle version for your project.