Implement arkworks wrapper #1

AllFi · 2023-08-30T14:43:53Z

This PR introduces LibarkworksWrapper that uses rust implementation from arkworks to provide necessary functions for BN128Addition, BN128Multiplication, and BN128Pairing precompiles.

Caveats:

The --allow-multiple-definition flag was added to the CMakeLists.txt to allow linking both Rust libraries. It is a bit risky. One potential solution is to introduce a wrapper Rust crate that depends on the previous two and compile it as a static library.
libzksnarkjni.so was rebuilt only for linux64

rust/libarkworks/Cargo.toml

Federico2014 · 2023-09-14T09:55:45Z

Hi, when I compile the code on Mac, it shows the error:

[ 87%] Building CXX object CMakeFiles/zksnarkjni.dir/src/LibrustzcashJNIImpl.cpp.o
[ 90%] Building CXX object CMakeFiles/zksnarkjni.dir/src/LibsodiumJNIImpl.cpp.o
[ 93%] Building CXX object CMakeFiles/zksnarkjni.dir/src/LibarkworksJNIImpl.cpp.o
[ 96%] Building C object CMakeFiles/zksnarkjni.dir/src/memcpy.c.o
[100%] Linking CXX shared library libzksnarkjni.dylib
duplicate symbol 'bool2jboolean(bool)' in:
    CMakeFiles/zksnarkjni.dir/src/LibrustzcashJNIImpl.cpp.o
    CMakeFiles/zksnarkjni.dir/src/LibarkworksJNIImpl.cpp.o
duplicate symbol '_rust_eh_personality' in:
    ../../rust/libarkworks/target/release/libarkworks.a(arkworks.arkworks.4d7a3a6372ab5791-cgu.0.rcgu.o)
    ../../rust/librustzcash/target/release/librustzcash.a(rustzcash.rustzcash.adafdb4df2d4979e-cgu.0.rcgu.o)
ld: 2 duplicate symbols for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [libzksnarkjni.dylib] Error 1
make[1]: *** [CMakeFiles/zksnarkjni.dir/all] Error 2
make: *** [all] Error 2

After I change bool2boolean to bool2Boolean in LibarkworksJNIImpl.cpp, it still has the error:

[ 87%] Building CXX object CMakeFiles/zksnarkjni.dir/src/LibrustzcashJNIImpl.cpp.o
[ 90%] Building CXX object CMakeFiles/zksnarkjni.dir/src/LibsodiumJNIImpl.cpp.o
[ 93%] Building CXX object CMakeFiles/zksnarkjni.dir/src/LibarkworksJNIImpl.cpp.o
[ 96%] Building C object CMakeFiles/zksnarkjni.dir/src/memcpy.c.o
[100%] Linking CXX shared library libzksnarkjni.dylib
duplicate symbol '_rust_eh_personality' in:
    ../../rust/libarkworks/target/release/libarkworks.a(arkworks.arkworks.4d7a3a6372ab5791-cgu.0.rcgu.o)
    ../../rust/librustzcash/target/release/librustzcash.a(rustzcash.rustzcash.adafdb4df2d4979e-cgu.0.rcgu.o)
ld: 1 duplicate symbol for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [libzksnarkjni.dylib] Error 1
make[1]: *** [CMakeFiles/zksnarkjni.dir/all] Error 2
make: *** [all] Error 2``

To solve the problem, it shoub set lto = false in Cargo.toml according to reference

r0wdy1 · 2023-09-14T12:45:19Z

To solve the problem, it shoub set lto = false in Cargo.toml according to reference

Did you try that solution? I'm sorry , we have deliberately tested the build on linux and missed this problem with mac, and it seems that intended flag --allow-multiple-definition doesn't help here either.

I have tried to set lto=false but it still fails

looking into it

3for · 2023-09-14T14:38:51Z

To solve the problem, it shoub set lto = false in Cargo.toml according to reference

Did you try that solution? I'm sorry , we have deliberately tested the build on linux and missed this problem with mac, and it seems that intended flag --allow-multiple-definition doesn't help here either.

I have tried to set lto=false but it still fails

looking into it

Set lto=false in both libarkworks and librustzcash will fix this issue.

AllFi · 2023-09-14T14:45:05Z

Hi, @Federico2014! Thank you for your feedback. I missed that LINK_FLAGS aren't used on macos. Moreover, it seems that --allow-multiple-definition flag doesn't work on macos either. That's why we replaced linking libarkworks.a with libarkworks.so (.dylib). Now it should work properly. I've checked bn128 benchmarks and didn't find any performance degradation with this solution. Could you please check if it works now?

r0wdy1 · 2023-09-14T15:24:01Z

Set lto=false in both libarkworks and librustzcash will fix this issue.

@3for
We were hesitant to change librustzcash. Is that a preferable solution?
If so we'll make a new pr right away

3for · 2023-09-14T15:34:30Z

Set lto=false in both libarkworks and librustzcash will fix this issue.

@3for We were hesitant to change librustzcash. Is that a preferable solution? If so we'll make a new pr right away

@AllFi 's solution is okay.

Federico2014 · 2023-09-14T15:35:53Z

Hi, @Federico2014! Thank you for your feedback. I missed that LINK_FLAGS aren't used on macos. Moreover, it seems that --allow-multiple-definition flag doesn't work on macos either. That's why we replaced linking libarkworks.a with libarkworks.so (.dylib). Now it should work properly. I've checked bn128 benchmarks and didn't find any performance degradation with this solution. Could you please check if it works now?

Excellent, it works now

3for · 2023-09-15T10:12:32Z

1)The lib in this pr does not work on MAC OS. So we fork and rebuild one in this repo.
2)When run PrecompiledContractsTest test case in java-tron on my MacBook, it fails with:

java.lang.UnsatisfiedLinkError: Could not load library. Reasons: [no zksnarkjni64-1.0.0 in java.library.path, no zksnarkjni-1.0.0 in java.library.path, no zksnarkjni in java.library.path, /private/var/folders/p5/bmmxtx5x14j_6znwywzj09w40000gn/T/libzksnarkjni-64-1-6569806775278865574.0: dlopen(/private/var/folders/p5/bmmxtx5x14j_6znwywzj09w40000gn/T/libzksnarkjni-64-1-6569806775278865574.0, 1): Library not loaded: /Users/tron/test/zkbob/zksnark-java-sdk-zkBob/rust/libarkworks/target/release/deps/libarkworks.dylib
  Referenced from: /private/var/folders/p5/bmmxtx5x14j_6znwywzj09w40000gn/T/libzksnarkjni-64-1-6569806775278865574.0
  Reason: image not found]

where /Users/tron/test/zkbob/zksnark-java-sdk-zkBob/rust/libarkworks/target/release/deps/libarkworks.dylib is an absolute address that doesn't exist on my machine.
3) We try back the set lto=false in both libarkworks and librustzcash, the same error as 2) occurs again.
4) Seems something wrong with the cmake scripts below?

if(${CMAKE_SYSTEM_NAME} MATCHES "Darwin")
 target_link_libraries(zksnarkjni
   ${CMAKE_THREAD_LIBS_INIT}
   "${CMAKE_CURRENT_SOURCE_DIR}/../rust/librustzcash/target/release/librustzcash.a"
   "${CMAKE_CURRENT_SOURCE_DIR}/../rust/libarkworks/target/release/libarkworks.dylib"
   ${NACL_DIR}/src/libsodium/src/libsodium/.libs/libsodium.a
  )
else()

AllFi · 2023-09-15T10:45:19Z

Hello, @3for! Yes, it seems that a solution with lto=false is better in terms of support. I think to make it work we need to:

Set lto=false in the libarkworks and librustzcash
Return crate-type = ["staticlib", "cdylib"] in the libarkworks/Cargo.toml
Move back to static linking by replacing libarkworks.dylib (.so) with libarkworks.a in the cmake script

We can also deal with dynamic linking but it is probably not worth it if the solution above is valid. I'll try this solution and come back with the results.

Federico2014 · 2023-09-15T10:52:39Z

@AllFi I have tried as you said. It seems that it only works when compiled by myself. Others will show the errors.

AllFi · 2023-09-15T10:55:10Z

@Federico2014 could you please share the error?

Federico2014 · 2023-09-15T11:02:35Z

@Federico2014 could you please share the error?

When I run the PrecompiledContractsTest, it shows:

Could not load library. Reasons: [no zksnarkjni64-1.0.0 in java.library.path, no zksnarkjni-1.0.0 in java.library.path, no zksnarkjni in java.library.path, /private/var/folders/j9/hkds7drd1479w9rg_x3qwn_c0000gn/T/libzksnarkjni-64-1-1424589431046675095.0: dlopen(/private/var/folders/j9/hkds7drd1479w9rg_x3qwn_c0000gn/T/libzksnarkjni-64-1-1424589431046675095.0, 1): Library not loaded: /Users/tron/test/zkbob/zksnark-java-sdk-zkBob/rust/libarkworks/target/release/deps/libarkworks.dylib
  Referenced from: /private/var/folders/j9/hkds7drd1479w9rg_x3qwn_c0000gn/T/libzksnarkjni-64-1-1424589431046675095.0
  Reason: image not found]
java.lang.UnsatisfiedLinkError: Could not load library. Reasons: [no zksnarkjni64-1.0.0 in java.library.path, no zksnarkjni-1.0.0 in java.library.path, no zksnarkjni in java.library.path, /private/var/folders/j9/hkds7drd1479w9rg_x3qwn_c0000gn/T/libzksnarkjni-64-1-1424589431046675095.0: dlopen(/private/var/folders/j9/hkds7drd1479w9rg_x3qwn_c0000gn/T/libzksnarkjni-64-1-1424589431046675095.0, 1): Library not loaded: /Users/tron/test/zkbob/zksnark-java-sdk-zkBob/rust/libarkworks/target/release/deps/libarkworks.dylib
  Referenced from: /private/var/folders/j9/hkds7drd1479w9rg_x3qwn_c0000gn/T/libzksnarkjni-64-1-1424589431046675095.0
  Reason: image not found]
	at org.fusesource.hawtjni.runtime.Library.doLoad(Library.java:182)
	at org.fusesource.hawtjni.runtime.Library.load(Library.java:140)
	at org.tron.common.zksnark.LibarkworksWrapper.<clinit>(LibarkworksWrapper.java:22)
	at org.tron.common.zksnark.JLibarkworks.libarkworksG1IsValid(JLibarkworks.java:9)
	at org.tron.core.vm.PrecompiledContracts$BN128Pairing.decodePair(PrecompiledContracts.java:888)
	at org.tron.core.vm.PrecompiledContracts$BN128Pairing.execute(PrecompiledContracts.java:866)
	at org.tron.common.runtime.vm.PrecompiledContractsTest.bn128PairingTest(PrecompiledContractsTest.java:1186)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.springframework.test.context.junit4.statements.RunBeforeTestExecutionCallbacks.evaluate(RunBeforeTestExecutionCallbacks.java:74)
	at org.springframework.test.context.junit4.statements.RunAfterTestExecutionCallbacks.evaluate(RunAfterTestExecutionCallbacks.java:84)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:75)
	at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:86)
	at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:84)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:251)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:97)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:190)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
	at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
	at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:118)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
	at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:412)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
	at java.lang.Thread.run(Thread.java:748)

I don't kwow why the above error includes the abosute path: 1): Library not loaded: /Users/tron/test/zkbob/zksnark-java-sdk-zkBob/rust/libarkworks/target/release/deps/libarkworks.dylib, so strange？

ps: in order to show the error, I have changed the path /Users/tron/test/zkbob/zksnark-java-sdk-zkBob on my Mac to /Users/tron/test/zkbob/zksnark-java-sdk-zkBob-bak, otherwise, it will work normally.

AllFi · 2023-09-15T11:08:48Z

I am not sure but it looks like it tries to use the old version of the library with dynamic linking. Could you please push a new version of the library in the https://github.com/Federico2014/zksnark-java-sdk-zkBob and try to build java-tron with --no-build-cache flag?

p.s. It looks like you've already tried this solution even before I mentioned it. I guess the root of the current problem is gradle cache but if it is not the case I'll investigate further.

…into arkworks

halibobo1205 · 2023-09-18T09:06:01Z

otool -L osx64/libzksnarkjni.jnilib

file osx64/libzksnarkjni.jnilib

@AllFi @Federico2014 , It's caused by an incorrect lib， just rebulid it for ok.
file osx64/aarch64/libzksnarkjni.jnilib

otool -L osx64/libzksnarkjni.jnilib

r0wdy1 · 2023-09-18T14:11:45Z

otool -L osx64/libzksnarkjni.jnilib file osx64/libzksnarkjni.jnilib @AllFi @Federico2014 , It's caused by an incorrect lib， just rebulid it for ok. file osx64/aarch64/libzksnarkjni.jnilib otool -L osx64/libzksnarkjni.jnilib

Hi, @halibobo1205 , I've added built libraries for both x86 and arm on OSX.
I have successfully used x86 to build tron-java on M2 with x86 JDK, all test have completed succesfuly

3for · 2023-09-18T14:37:05Z

I re-download the https://github.com/zkBob/java-tron/, checkout the arkworks-crypto branch, and run the PrecompiledContractsTest test case, get errors:

java.lang.UnsatisfiedLinkError: Could not load library. Reasons: [no zksnarkjni64-1.0.0 in java.library.path, no zksnarkjni-1.0.0 in java.library.path, no zksnarkjni in java.library.path, /private/var/folders/p5/bmmxtx5x14j_6znwywzj09w40000gn/T/libzksnarkjni-64-1-8155989276746936682.0: dlopen(/private/var/folders/p5/bmmxtx5x14j_6znwywzj09w40000gn/T/libzksnarkjni-64-1-8155989276746936682.0, 1): no suitable image found.  Did find:
	/private/var/folders/p5/bmmxtx5x14j_6znwywzj09w40000gn/T/libzksnarkjni-64-1-8155989276746936682.0: mach-o, but wrong architecture
	/private/var/folders/p5/bmmxtx5x14j_6znwywzj09w40000gn/T/libzksnarkjni-64-1-8155989276746936682.0: mach-o, but wrong architecture]
	at org.fusesource.hawtjni.runtime.Library.doLoad(Library.java:182)
	at org.fusesource.hawtjni.runtime.Library.load(Library.java:140)
	at org.tron.common.zksnark.LibarkworksWrapper.<clinit>(LibarkworksWrapper.java:22)
	at org.tron.common.zksnark.JLibarkworks.libarkworksG1IsValid(JLibarkworks.java:9)
	at org.tron.core.vm.PrecompiledContracts$BN128Pairing.decodePair(PrecompiledContracts.java:888)
	at org.tron.core.vm.PrecompiledContracts$BN128Pairing.execute(PrecompiledContracts.java:866)
	at org.tron.common.runtime.vm.PrecompiledContractsTest.bn128PairingTest(PrecompiledContractsTest.java:1186)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.springframework.test.context.junit4.statements.RunBeforeTestExecutionCallbacks.evaluate(RunBeforeTestExecutionCallbacks.java:74)
	at org.springframework.test.context.junit4.statements.RunAfterTestExecutionCallbacks.evaluate(RunAfterTestExecutionCallbacks.java:84)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:75)
	at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:86)
	at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:84)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:251)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:97)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:190)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
	at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
	at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:118)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
	at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:412)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
	at java.lang.Thread.run(Thread.java:750)

AllFi · 2023-09-18T16:13:04Z

Hello @3for! I've checked libraries again and everything seems to be correct.

Could you please try removing the cache in the ~/.gradle/caches and run the tests again? If it doesn't work it would be helpful if you build libsnarkjni on your device and share it so we can compare it with the current library in the repo.

p.s. I guess ./gradlew clean build test --refresh-dependencies should work too.

3for · 2023-09-19T02:34:18Z

Hello @3for! I've checked libraries again and everything seems to be correct. Could you please try removing the cache in the ~/.gradle/caches and run the tests again? If it doesn't work it would be helpful if you build libsnarkjni on your device and share it so we can compare it with the current library in the repo.

p.s. I guess ./gradlew clean build test --refresh-dependencies should work too.

I clean the cache by rm -rf ~/.gradle/caches, and rerun the PrecompiledContractsTest test case, get the same error as above. I re-download https://github.com/zkBob/zksnark-java-sdk, checkout arkworks branch, rebuild a libzksnarkjni.dylib :

$ otool -L libzksnarkjni.dylib 
libzksnarkjni.dylib:
	@rpath/libzksnarkjni.dylib (compatibility version 0.0.0, current version 0.0.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 400.9.4)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
$ file libzksnarkjni.dylib 
libzksnarkjni.dylib: Mach-O 64-bit dynamically linked shared library x86_64

Seems different from the lib:

$ otool -L libzksnarkjni.jnilib 
libzksnarkjni.jnilib:
        @rpath/libzksnarkjni.dylib (compatibility version 0.0.0, current version 0.0.0)
        /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.36.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
$ file libzksnarkjni.jnilib 
libzksnarkjni.jnilib: Mach-O 64-bit dynamically linked shared library x86_64
$ pwd
/Users/lanyu/zyd/zkBob/zksnark-java-sdk/src/main/resources/META-INF/native/osx64
lanyudeMacBook-Pro-5:osx64 lanyu$

r0wdy1 · 2023-09-19T07:09:18Z

@3for, sorry that the solution didn't work for you. i've built x86 and Intel mac and tested java-tron on M2 and everything seemed to be working fine. Don't have any ideas rn.
Can you try to rust java-tron tests with your built library?
If it works could you please push it to some branch from so we could test it ourself also?
May be we could just leave your version if everything is fine

AllFi · 2023-09-19T07:32:14Z

Hi @r0wdy1! It seems that the built library is already pushed in the https://github.com/Federico2014/zksnark-java-sdk-zkBob/tree/arkworks. Could you please check the arkworks-crypto branch in the https://github.com/Federico2014/java-tron-zkBob repository?

r0wdy1 · 2023-09-19T08:43:43Z

@3for , we have pushed a new version libzksnarkjni.jnilib . This time it's the one from @Federico2014 's repo
otool shows something new , I'm not really sure what to do with this info

../zksnark-java-sdk-federico/src/main/resources/META-INF/native/osx64/libzksnarkjni.jnilib:
   @rpath/libzksnarkjni.dylib (compatibility version 0.0.0, current version 0.0.0)
   /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1200.3.0)
   /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1311.0.0)

Java-tron works with this library , please test on your side

Federico2014 · 2023-09-19T10:03:34Z

Thanks so much for your help. We have passed tests now. @AllFi @r0wdy1 @halibobo1205
For the Mac version, it is the config set(LINK_FLAGS "-Wl,--allow-multiple-definition") which caused the problem. After I removed it, it is ok.
For the linux version, the java-tron tests still failed on our servers. The GLICB_2.29 version is not compatible with our servers.

The libc version on our servers is 2.17, so I rebuilt the libzksnarkjni.so. Then the java-tron tests passed.

AllFi · 2023-09-19T12:07:06Z

Glad to be of help! Sorry for the inconvenience and thank you for your efforts in investigating the issues. I've updated the linux64 library in this branch too. What should be our next course of action?

3for · 2023-09-20T03:17:36Z

It works for me now. The tests passed.

tomatoishealthy · 2023-09-20T04:21:28Z

Glad to be of help! Sorry for the inconvenience and thank you for your efforts in investigating the issues. I've updated the linux64 library in this branch too. What should be our next course of action?

Great collaboration!

As these PRs involve great performance improvements, maybe some upgrade compatibility and more tests should be considered, and it is better to merge them in a mandatory upgrade. The next mandatory upgrade of TRON is planned at the end of this year. Wondering if this date is suitable for you, but of course we can also make some additional attempts if needed.

Moving these PRs or submitting news to the develop branch of TRON is also appreciated, here is the dev doc. More people can see and discuss them directly.

r0wdy1 · 2023-09-20T08:49:50Z

Moving these PRs or submitting news to the develop branch of TRON is also appreciated, here is the dev doc. More people can see and discuss them directly.

@tomatoishealthy , this repo ( zksnark-java-sdk) doesn't have develop branch
The PR that changes node code is this one zkBob/java-tron#1 and it has been indeed created for develop branch
We can move this PR with it's description to the repo in tronprotocol organization (
tronprotocol/zksnark-java-sdk@master...zkBob:zksnark-java-sdk:arkworks). Is that what you meant?

In regards of the timeline of course we would be really interested in delivering this patches to production as early as possible, since it's pretty much the only thing that stops us from deploying on Tron network, so we will do everything possible on our end for that.

tomatoishealthy · 2023-09-20T09:11:43Z

We can move this PR with it's description to the repo in tronprotocol organization ( tronprotocol/zksnark-java-sdk@master...zkBob:zksnark-java-sdk:arkworks). Is that what you meant?

@r0wdy1 @AllFi , Sry, my mistake. That's right, develop branch for java-tron and master branch for zksnark-java-sdk is OK.

As this patch involves a relatively large performance improvement, if validators and fullnodes are not upgraded at the same time, it may cause a network fork. Therefore, the follow-up will also include some community coordination work and compatibility testing. Let us continue to push it to be online as soon as possible.

r0wdy1 · 2023-09-20T09:18:23Z

@r0wdy1 @AllFi , Sry, my mistake. That's right, develop branch for java-tron and master branch for zksnark-java-sdk is OK.

So do you want both PRs moved to the corresponding tronprotocol organization repositories for better visibility?

Therefore, the follow-up will also include some community coordination work and compatibility testing. Let us continue to push it to be online as soon as possible.

Great, please let us know if we can help somehow.
In the meantime we'll try to come up with additional tests

tomatoishealthy · 2023-09-20T09:21:26Z

So do you want both PRs moved to the corresponding tronprotocol organization repositories for better visibility?

Yes.

In the meantime we'll try to come up with additional tests

Great!

AllFi · 2023-09-20T15:55:44Z

Hello, @tomatoishealthy! I've created new PRs in tronprotocol organization repositories:

AllFi added 8 commits August 30, 2023 17:41

Add draft of arkworks wrapper

29d8aba

Refactor code, add more test cases

ad750bd

Minor refactoring

ffd196e

Check field elements sizes

47d6f15

Add left padding

9106c32

Remove padding

12176c4

Refactor serialize.rs, add pairing test

dc3b797

Reduce lib size

a0fe0c3

AllFi mentioned this pull request Sep 4, 2023

Optimize bn128 precompiled contracts zkBob/java-tron#1

Draft

AllFi changed the title ~~Add arkworks wrapper~~ Implement arkworks wrapper Sep 6, 2023

AllFi requested review from EvgenKor, akolotov, lok52, k1rill-fedoseev and r0wdy1 September 6, 2023 10:40

r0wdy1 mentioned this pull request Sep 13, 2023

Slow precompiles for alt_bn128 curve tronprotocol/java-tron#5492

Open

r0wdy1 reviewed Sep 13, 2023

View reviewed changes

rust/libarkworks/Cargo.toml Show resolved Hide resolved

r0wdy1 approved these changes Sep 13, 2023

View reviewed changes

EvgenKor approved these changes Sep 13, 2023

View reviewed changes

Link libarkworks.so (.dylib) instead of .a

f421d25

adds missing compiled libzksnarkjni.jnilib for Darwin

158d5d4

AllFi added 2 commits September 15, 2023 15:20

Set lto = false, return static linking

66b9766

Merge branch 'arkworks' of https://github.com/zkBob/zksnark-java-sdk …

8ebb6a8

…into arkworks

r0wdy1 and others added 2 commits September 18, 2023 16:01

adds build for arm64

022efa9

adds x86_64 build

8a9e724

copy built jnilib from Federico2014

e326fda

Update linux64 library

1a09f91

AllFi mentioned this pull request Sep 20, 2023

feat: add arkworks wrapper for bn128 operations tronprotocol/zksnark-java-sdk#9

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement arkworks wrapper #1

Implement arkworks wrapper #1

AllFi commented Aug 30, 2023 •

edited

Loading

Federico2014 commented Sep 14, 2023 •

edited

Loading

r0wdy1 commented Sep 14, 2023

3for commented Sep 14, 2023

AllFi commented Sep 14, 2023

r0wdy1 commented Sep 14, 2023 •

edited

Loading

3for commented Sep 14, 2023

Federico2014 commented Sep 14, 2023

3for commented Sep 15, 2023

AllFi commented Sep 15, 2023

Federico2014 commented Sep 15, 2023

AllFi commented Sep 15, 2023

Federico2014 commented Sep 15, 2023 •

edited

Loading

AllFi commented Sep 15, 2023 •

edited

Loading

halibobo1205 commented Sep 18, 2023

r0wdy1 commented Sep 18, 2023

3for commented Sep 18, 2023

AllFi commented Sep 18, 2023 •

edited

Loading

3for commented Sep 19, 2023

r0wdy1 commented Sep 19, 2023

AllFi commented Sep 19, 2023

r0wdy1 commented Sep 19, 2023

Federico2014 commented Sep 19, 2023 •

edited

Loading

AllFi commented Sep 19, 2023

3for commented Sep 20, 2023

tomatoishealthy commented Sep 20, 2023

r0wdy1 commented Sep 20, 2023

tomatoishealthy commented Sep 20, 2023 •

edited

Loading

r0wdy1 commented Sep 20, 2023

tomatoishealthy commented Sep 20, 2023

AllFi commented Sep 20, 2023

Implement arkworks wrapper #1

Are you sure you want to change the base?

Implement arkworks wrapper #1

Conversation

AllFi commented Aug 30, 2023 • edited Loading

Federico2014 commented Sep 14, 2023 • edited Loading

r0wdy1 commented Sep 14, 2023

3for commented Sep 14, 2023

AllFi commented Sep 14, 2023

r0wdy1 commented Sep 14, 2023 • edited Loading

3for commented Sep 14, 2023

Federico2014 commented Sep 14, 2023

3for commented Sep 15, 2023

AllFi commented Sep 15, 2023

Federico2014 commented Sep 15, 2023

AllFi commented Sep 15, 2023

Federico2014 commented Sep 15, 2023 • edited Loading

AllFi commented Sep 15, 2023 • edited Loading

halibobo1205 commented Sep 18, 2023

r0wdy1 commented Sep 18, 2023

3for commented Sep 18, 2023

AllFi commented Sep 18, 2023 • edited Loading

3for commented Sep 19, 2023

r0wdy1 commented Sep 19, 2023

AllFi commented Sep 19, 2023

r0wdy1 commented Sep 19, 2023

Federico2014 commented Sep 19, 2023 • edited Loading

AllFi commented Sep 19, 2023

3for commented Sep 20, 2023

tomatoishealthy commented Sep 20, 2023

r0wdy1 commented Sep 20, 2023

tomatoishealthy commented Sep 20, 2023 • edited Loading

r0wdy1 commented Sep 20, 2023

tomatoishealthy commented Sep 20, 2023

AllFi commented Sep 20, 2023

AllFi commented Aug 30, 2023 •

edited

Loading

Federico2014 commented Sep 14, 2023 •

edited

Loading

r0wdy1 commented Sep 14, 2023 •

edited

Loading

Federico2014 commented Sep 15, 2023 •

edited

Loading

AllFi commented Sep 15, 2023 •

edited

Loading

AllFi commented Sep 18, 2023 •

edited

Loading

Federico2014 commented Sep 19, 2023 •

edited

Loading

tomatoishealthy commented Sep 20, 2023 •

edited

Loading