Check whether JNI plugin is broken due to JNA Gradle migration [JNA bindings] #3758

tucek · 2021-04-08T01:17:52Z

There might still be a reference to Maven in regard to using the JNI plugin.

check on which builds the test are running

markus2330 · 2021-04-08T09:09:24Z

Thank you for reporting the problem! Yes, good if you double-check if maven is gone everywhere (except of the external tests, I think we should support also external Maven projects).

mpranj · 2021-04-29T20:25:26Z

The JNI plugin uses Maven to add the unit tests. Since Maven was removed from most/all Dockerfiles etc. the JNI tests are not executed anywhere to my knowledge.

Can you update JNI to use Gradle?

Edit: You can find the relevant CMake somewhere here

libelektra/src/plugins/jni/CMakeLists.txt

Lines 38 to 41 in 7e9ac3a

    
           if (ADDTESTING_PHASE) 
        
           	check_binding_was_added ("jna" BINDING_WAS_ADDED) 
        
           	find_program (MAVEN_EXECUTABLE mvn)

markus2330 · 2021-05-02T11:15:46Z

Yes, it would be important to get these tests running. They might reveal problems in the JNI plugin.

tucek · 2021-05-03T07:57:29Z

I will have look at it today...

tucek · 2021-05-03T08:21:34Z

check on which builds the test are running

markus2330 · 2021-05-03T09:45:15Z

You can add TODOs with - [ ] in the top post, then GitHub shows you how many of them were already resolved.

tucek · 2021-05-03T19:38:51Z

The JNI plugin uses Maven to add the unit tests. Since Maven was removed from most/all Dockerfiles etc. the JNI tests are not executed anywhere to my knowledge.

Can you update JNI to use Gradle?

Edit: You can find the relevant CMake somewhere here

libelektra/src/plugins/jni/CMakeLists.txt

Lines 38 to 41 in 7e9ac3a

if (ADDTESTING_PHASE)

check_binding_was_added ("jna" BINDING_WAS_ADDED)

find_program (MAVEN_EXECUTABLE mvn)

I'm still trying to get it to work, but to be clear, the tests have been explicitly disabled before because of L42:

libelektra/src/plugins/jni/CMakeLists.txt

Lines 38 to 46 in 7e9ac3a

    
           if (ADDTESTING_PHASE) 
        
           	check_binding_was_added ("jna" BINDING_WAS_ADDED) 
        
           	find_program (MAVEN_EXECUTABLE mvn) 
        
           	if (ENABLE_BROKEN_TESTS) 
        
           		if (BUILD_TESTING 
        
           		    AND BINDING_WAS_ADDED 
        
           		    AND MAVEN_EXECUTABLE) 
        
           			add_plugintest (jni MEMLEAK)

From the git history, it seems to be disabled for years... or does someone build Elektra with ENABLE_BROKEN_TESTS somewhere?

mpranj · 2021-05-04T06:00:32Z

the tests have been explicitly disabled

Good that you found that. I think it was temporarily disabled and then forgotten.

does someone build Elektra with ENABLE_BROKEN_TESTS somewhere?

It is not tested in our CI, which is what matters. Please remove the ENABLE_BROKEN_TESTS part such that the tests are enabled again.

- removed maven dependency for JNI test - re-enabled test - TODO fix test

tucek · 2021-05-04T10:07:16Z

I've created PR #3825. Test is enabled and maven dependency has been removed. How can i execute the single "testmod_jni" test to see what is the problem?

markus2330 · 2021-05-04T10:13:44Z

ctest -V -R testmod_jni

mpranj · 2021-05-04T11:02:09Z

On Cirrus CI, Fedora 34 it says:

160/268 Test #110: testmod_jni ...................................Subprocess aborted***Exception:   2.10 sec
JNI          TESTS
==================
free(): double free detected in tcache 2

tucek · 2021-05-04T17:23:54Z

getting the same thing:

$ ctest -V -R testmod_jni
UpdateCTestConfiguration  from :/mnt/c/Data/libelektra-tucek-next/build/DartConfiguration.tcl
Parse Config file:/mnt/c/Data/libelektra-tucek-next/build/DartConfiguration.tcl
UpdateCTestConfiguration  from :/mnt/c/Data/libelektra-tucek-next/build/DartConfiguration.tcl
Parse Config file:/mnt/c/Data/libelektra-tucek-next/build/DartConfiguration.tcl
Test project /mnt/c/Data/libelektra-tucek-next/build
Constructing a list of tests
Done constructing a list of tests
Updating test list for fixtures
Added 0 tests to meet fixture requirements
Checking test dependency graph...
Checking test dependency graph end
test 66
    Start 66: testmod_jni

66: Test command: /mnt/c/Data/libelektra-tucek-next/build/bin/testmod_jni "/mnt/c/Data/libelektra-tucek-next/src/plugins/jni"
66: Environment variables:
66:  LD_LIBRARY_PATH=/mnt/c/Data/libelektra-tucek-next/build/lib
66:  KDB_TEST_BIN_DIR=/mnt/c/Data/libelektra-tucek-next/build/bin
66: Test timeout computed to be: 1500
66: JNI          TESTS
66: ==================
66:
66: free(): double free detected in tcache 2
1/1 Test #66: testmod_jni ......................Child aborted***Exception:   0.47 sec

0% tests passed, 1 tests failed out of 1

Label Time Summary:
memleak    =   0.47 sec*proc (1 test)

Total Test time (real) =   2.76 sec

The following tests FAILED:
         66 - testmod_jni (Child aborted)
Errors while running CTest

Does someone with more C experience has time to debug the test?

markus2330 · 2021-05-04T18:28:53Z

@tucek can you try to reduce which test case causes the double free? Can you run it with valgrind and post the log? (remove MEMLEAK in line 49 and run make run_memcheck).

tucek · 2021-05-05T12:19:04Z

make run_memcheck

I've changed add_plugintest (jni MEMLEAK) to add_plugintest (jni), but get the following error:

tucek@DESKTOP-FIV77AF:/mnt/c/Data/libelektra-tucek-next/build$ make run_memcheck
   Site: DESKTOP-FIV77AF
   Build name: Linux-c++
Memory check project /mnt/c/Data/libelektra-tucek-next/build
Memory checker (MemoryCheckCommand) not set, or cannot find the specified program.
Errors while running CTest
   Site: DESKTOP-FIV77AF
   Build name: Linux-c++
Memory check project /mnt/c/Data/libelektra-tucek-next/build
Memory checker (MemoryCheckCommand) not set, or cannot find the specified program.
Errors while running CTest
make[3]: *** [tests/CMakeFiles/run_memcheck.dir/build.make:57: tests/CMakeFiles/run_memcheck] Error 16
make[2]: *** [CMakeFiles/Makefile2:16337: tests/CMakeFiles/run_memcheck.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:16344: tests/CMakeFiles/run_memcheck.dir/rule] Error 2
make: *** [Makefile:5839: run_memcheck] Error 2

am I missing a dependency?

Reproduction:
check out branch PR #3825 is based on
enabled plugin jni
make
execute ctest -V -R testmod_jni

mpranj · 2021-05-05T12:56:42Z

You need valgrind installed, to execute the memcheck.

tucek · 2021-05-05T18:39:30Z

still getting the same output after successfully installing valgrind:

$ sudo apt-get install valgrind
[sudo] password for tucek:
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  gdb gdbserver libbabeltrace1 libc6-dbg libdw1
Suggested packages:
  gdb-doc valgrind-dbg valgrind-mpi kcachegrind alleyoop valkyrie
The following NEW packages will be installed:
  gdb gdbserver libbabeltrace1 libc6-dbg libdw1 valgrind
0 upgraded, 6 newly installed, 0 to remove and 0 not upgraded.
Need to get 34.6 MB of archives.
After this operation, 179 MB of additional disk space will be used.
Do you want to continue? [Y/n]
Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 libdw1 amd64 0.176-1.1build1 [226 kB]
Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 libbabeltrace1 amd64 1.5.8-1build1 [156 kB]
Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 gdb amd64 9.2-0ubuntu1~20.04 [3221 kB]
Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 gdbserver amd64 9.2-0ubuntu1~20.04 [222 kB]
Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libc6-dbg amd64 2.31-0ubuntu9.2 [10.5 MB]
Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 valgrind amd64 1:3.15.0-1ubuntu9.1 [20.3 MB]
Fetched 34.6 MB in 3s (10.4 MB/s)
Selecting previously unselected package libdw1:amd64.
(Reading database ... 49975 files and directories currently installed.)
Preparing to unpack .../0-libdw1_0.176-1.1build1_amd64.deb ...
Unpacking libdw1:amd64 (0.176-1.1build1) ...
Selecting previously unselected package libbabeltrace1:amd64.
Preparing to unpack .../1-libbabeltrace1_1.5.8-1build1_amd64.deb ...
Unpacking libbabeltrace1:amd64 (1.5.8-1build1) ...
Selecting previously unselected package gdb.
Preparing to unpack .../2-gdb_9.2-0ubuntu1~20.04_amd64.deb ...
Unpacking gdb (9.2-0ubuntu1~20.04) ...
Selecting previously unselected package gdbserver.
Preparing to unpack .../3-gdbserver_9.2-0ubuntu1~20.04_amd64.deb ...
Unpacking gdbserver (9.2-0ubuntu1~20.04) ...
Selecting previously unselected package libc6-dbg:amd64.
Preparing to unpack .../4-libc6-dbg_2.31-0ubuntu9.2_amd64.deb ...
Unpacking libc6-dbg:amd64 (2.31-0ubuntu9.2) ...
Selecting previously unselected package valgrind.
Preparing to unpack .../5-valgrind_1%3a3.15.0-1ubuntu9.1_amd64.deb ...
Unpacking valgrind (1:3.15.0-1ubuntu9.1) ...
Setting up libdw1:amd64 (0.176-1.1build1) ...
Setting up libc6-dbg:amd64 (2.31-0ubuntu9.2) ...
Setting up libbabeltrace1:amd64 (1.5.8-1build1) ...
Setting up gdbserver (9.2-0ubuntu1~20.04) ...
Setting up valgrind (1:3.15.0-1ubuntu9.1) ...
Setting up gdb (9.2-0ubuntu1~20.04) ...
Processing triggers for man-db (2.9.1-1) ...
Processing triggers for libc-bin (2.31-0ubuntu9.2) ...
$ valgrind
valgrind: no program specified
valgrind: Use --help for more information.

markus2330 · 2021-05-06T12:44:48Z

@tucek did you try to completely remove the build directory? I don't think cmake checks for valgrind again later.

Btw. if you do this in a PR, we should be also be able to download the artifacts from the buildserver.

tucek · 2021-05-06T15:06:39Z

run_memcheck.out.txt
LastDynamicAnalysis_20210506-1502.log.txt
DynamicAnalysis.xml.txt
MemoryChecker.66.log.txt
Does this help to pinpoint the problem?

markus2330 · 2021-05-07T07:57:59Z

I answered in #3825

* updated gradle to 7.0 * removed debug output

- removed maven dependency for JNI test - re-enabled test - TODO fix test

* updated gradle to 7.0 * removed debug output

- removed maven dependency for JNI test - re-enabled test - TODO fix test

* updated gradle to 7.0 * removed debug output

tucek changed the title ~~Check whether JNI plugin is broken due to #3~~ Check whether JNI plugin is broken due to JNA Gradle migration [JNA bindings] Apr 8, 2021

tucek self-assigned this Apr 8, 2021

tucek added the lang/java label Apr 8, 2021

tucek mentioned this issue Apr 8, 2021

0.9.5 release #3746

Closed

6 tasks

markus2330 added this to the 0.9.5 milestone Apr 8, 2021

mpranj modified the milestones: 0.9.5, 0.9.6 Apr 12, 2021

markus2330 added the urgent label May 2, 2021

tucek pushed a commit to tucek/libelektra that referenced this issue May 4, 2021

ElektraInitiative#3758

ef4715b

- removed maven dependency for JNI test - re-enabled test - TODO fix test

tucek pushed a commit to tucek/libelektra that referenced this issue May 9, 2021

ElektraInitiative#3758

f5d2cf4

* updated gradle to 7.0 * removed debug output

tucek linked a pull request May 13, 2021 that will close this issue

JNI Test Fix #3825

Merged

20 tasks

tucek pushed a commit to tucek/libelektra that referenced this issue May 14, 2021

ElektraInitiative#3758

665faee

- removed maven dependency for JNI test - re-enabled test - TODO fix test

tucek pushed a commit to tucek/libelektra that referenced this issue May 14, 2021

ElektraInitiative#3758

b7b7fa1

* updated gradle to 7.0 * removed debug output

tucek pushed a commit to tucek/libelektra that referenced this issue May 19, 2021

ElektraInitiative#3758

a5c7494

- removed maven dependency for JNI test - re-enabled test - TODO fix test

tucek pushed a commit to tucek/libelektra that referenced this issue May 19, 2021

ElektraInitiative#3758

50f2162

* updated gradle to 7.0 * removed debug output

markus2330 closed this as completed in #3825 May 24, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Check whether JNI plugin is broken due to JNA Gradle migration [JNA bindings] #3758

Check whether JNI plugin is broken due to JNA Gradle migration [JNA bindings] #3758

tucek commented Apr 8, 2021 •

edited by mpranj

Loading

markus2330 commented Apr 8, 2021

mpranj commented Apr 29, 2021 •

edited

Loading

markus2330 commented May 2, 2021

tucek commented May 3, 2021

tucek commented May 3, 2021

markus2330 commented May 3, 2021 •

edited

Loading

tucek commented May 3, 2021 •

edited

Loading

mpranj commented May 4, 2021

tucek commented May 4, 2021

markus2330 commented May 4, 2021

mpranj commented May 4, 2021

tucek commented May 4, 2021

markus2330 commented May 4, 2021

tucek commented May 5, 2021

mpranj commented May 5, 2021

tucek commented May 5, 2021

markus2330 commented May 6, 2021

tucek commented May 6, 2021 •

edited

Loading

markus2330 commented May 7, 2021

Check whether JNI plugin is broken due to JNA Gradle migration [JNA bindings] #3758

Check whether JNI plugin is broken due to JNA Gradle migration [JNA bindings] #3758

Comments

tucek commented Apr 8, 2021 • edited by mpranj Loading

markus2330 commented Apr 8, 2021

mpranj commented Apr 29, 2021 • edited Loading

markus2330 commented May 2, 2021

tucek commented May 3, 2021

tucek commented May 3, 2021

markus2330 commented May 3, 2021 • edited Loading

tucek commented May 3, 2021 • edited Loading

mpranj commented May 4, 2021

tucek commented May 4, 2021

markus2330 commented May 4, 2021

mpranj commented May 4, 2021

tucek commented May 4, 2021

markus2330 commented May 4, 2021

tucek commented May 5, 2021

mpranj commented May 5, 2021

tucek commented May 5, 2021

markus2330 commented May 6, 2021

tucek commented May 6, 2021 • edited Loading

markus2330 commented May 7, 2021

tucek commented Apr 8, 2021 •

edited by mpranj

Loading

mpranj commented Apr 29, 2021 •

edited

Loading

markus2330 commented May 3, 2021 •

edited

Loading

tucek commented May 3, 2021 •

edited

Loading

tucek commented May 6, 2021 •

edited

Loading