Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feat][Java] Initialize JAVA SDK: add INFO implementation #212

Merged
merged 16 commits into from
Aug 16, 2023

Conversation

Thespica
Copy link
Contributor

@Thespica Thespica commented Jul 25, 2023

Proposed changes

This PR aims to implement grpahinfo in Java with fastFFI.

Known problems

  • Can't implement function which return StdMap
  • Don't konw how to get generic factory like StdVctor<Property>
  • Test is too simple.

Plan

Before 7.30

Make graphinfo's test better

7.31-8.13

  • CI
  • Readers and test.

8.14-8.28

Writers and test.

After 8.28

  • Generate JavaDoc
  • README in java
  • Examples

Related issue

#72

@Thespica
Copy link
Contributor Author

StdMap error log:

[INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ gar-java ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 23 source files to /home/john/my-reop-gar/java/target/classes
com.alibaba.fastffi.annotation.FFICodeGenerationException: Cannot find a TypeDef for 
        std::map<std::string, GraphArchive::VertexInfo> -> (@com.alibaba.fastffi.FFINameAlias("GetVertexInfos"),@com.alibaba.fastffi.CXXReference,@com.alibaba.fastffi.FFITypeAlias("std::map<std::string, GraphArchive::VertexInfo>") :: com.alibaba.graphar.stdcxx.StdMap<com.a
libaba.graphar.stdcxx.StdString,com.alibaba.graphar.graphinfo.VertexInfo>)
during generating
        (@com.alibaba.fastffi.FFINameAlias("GetVertexInfos"),@com.alibaba.fastffi.CXXReference,@com.alibaba.fastffi.FFITypeAlias("std::map<std::string, GraphArchive::VertexInfo>") :: com.alibaba.graphar.stdcxx.StdMap<com.alibaba.graphar.stdcxx.StdString,com.alibaba.graphar
.graphinfo.VertexInfo>) com.alibaba.graphar.graphinfo.GraphInfo::getVertexInfos()
        at com.alibaba.fastffi.annotation.TypeDefGenerator.reportError(TypeDefGenerator.java:212)
        at com.alibaba.fastffi.annotation.TypeDefGenerator.reportError(TypeDefGenerator.java:208)
        at com.alibaba.fastffi.annotation.TypeDefGenerator.reportError(TypeDefGenerator.java:204)
        at com.alibaba.fastffi.annotation.TypeDefGenerator.getInternalTypeName(TypeDefGenerator.java:1451)
        at com.alibaba.fastffi.annotation.TypeDefGenerator.generateFFIDelegateMethod(TypeDefGenerator.java:2302)
        at com.alibaba.fastffi.annotation.TypeDefGenerator.generateFFIPointer(TypeDefGenerator.java:2004)
        at com.alibaba.fastffi.annotation.TypeDefGenerator.generateFFIPointerOrFFILibrary(TypeDefGenerator.java:780)
        at com.alibaba.fastffi.annotation.TypeDefGenerator.generate(TypeDefGenerator.java:266)
        at com.alibaba.fastffi.annotation.TypeDefRegistry.lambda$generateFFIPointerFFILibrary$2(TypeDefRegistry.java:339)
        at com.alibaba.fastffi.annotation.TypeDefRegistry.processWorklist(TypeDefRegistry.java:325)
        at com.alibaba.fastffi.annotation.TypeDefRegistry.generateFFIPointerFFILibrary(TypeDefRegistry.java:337)
        at com.alibaba.fastffi.annotation.TypeDefRegistry.generate(TypeDefRegistry.java:302)
        at com.alibaba.fastffi.annotation.AnnotationProcessor.process(AnnotationProcessor.java:117)
        at com.sun.tools.javac.processing.JavacProcessingEnvironment.callProcessor(JavacProcessingEnvironment.java:802)
        at com.sun.tools.javac.processing.JavacProcessingEnvironment.discoverAndRunProcs(JavacProcessingEnvironment.java:713)
        at com.sun.tools.javac.processing.JavacProcessingEnvironment.access$1800(JavacProcessingEnvironment.java:91)
        at com.sun.tools.javac.processing.JavacProcessingEnvironment$Round.run(JavacProcessingEnvironment.java:1043)
        at com.sun.tools.javac.processing.JavacProcessingEnvironment.doProcessing(JavacProcessingEnvironment.java:1184)
        at com.sun.tools.javac.main.JavaCompiler.processAnnotations(JavaCompiler.java:1170)
        at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:856)
        at com.sun.tools.javac.main.Main.compile(Main.java:523)
        at com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:129)
        at com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:138)
        at org.codehaus.plexus.compiler.javac.JavaxToolsCompiler.compileInProcess(JavaxToolsCompiler.java:126)
        at org.codehaus.plexus.compiler.javac.JavacCompiler.performCompile(JavacCompiler.java:174)
        at org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:1129)
        at org.apache.maven.plugin.compiler.CompilerMojo.execute(CompilerMojo.java:188)
        at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:210)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:156)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:148)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
        at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56)
        at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:305)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192)
        at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105)
        at org.apache.maven.cli.MavenCli.execute(MavenCli.java:957)
        at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:289)
        at org.apache.maven.cli.MavenCli.main(MavenCli.java:193)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:282)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:225)
        at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:406)
        at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:347)
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /home/john/my-reop-gar/java/src/main/java/com/alibaba/graphar/graphinfo/GraphInfo.java:[197,33] Cannot find a TypeDef for
        std::map<std::string, GraphArchive::VertexInfo> -> (@com.alibaba.fastffi.FFINameAlias("GetVertexInfos"),@com.alibaba.fastffi.CXXReference,@com.alibaba.fastffi.FFITypeAlias("std::map<std::string, GraphArchive::VertexInfo>") :: com.alibaba.graphar.stdcxx.StdMap<com.a
libaba.graphar.stdcxx.StdString,com.alibaba.graphar.graphinfo.VertexInfo>)
  during generating
        (@com.alibaba.fastffi.FFINameAlias("GetVertexInfos"),@com.alibaba.fastffi.CXXReference,@com.alibaba.fastffi.FFITypeAlias("std::map<std::string, GraphArchive::VertexInfo>") :: com.alibaba.graphar.stdcxx.StdMap<com.alibaba.graphar.stdcxx.StdString,com.alibaba.graphar
.graphinfo.VertexInfo>) com.alibaba.graphar.graphinfo.GraphInfo::getVertexInfos()
  Type: com.alibaba.graphar.graphinfo.GraphInfo
  Executable: (@com.alibaba.fastffi.FFINameAlias("GetVertexInfos"),@com.alibaba.fastffi.CXXReference,@com.alibaba.fastffi.FFITypeAlias("std::map<std::string, GraphArchive::VertexInfo>") :: com.alibaba.graphar.stdcxx.StdMap<com.alibaba.graphar.stdcxx.StdString,com.alibaba.g
raphar.graphinfo.VertexInfo>) com.alibaba.graphar.graphinfo.GraphInfo::getVertexInfos()
[INFO] 1 error
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.125 s
[ERROR]   during generating
[ERROR]         (@com.alibaba.fastffi.FFINameAlias("GetVertexInfos"),@com.alibaba.fastffi.CXXReference,@com.alibaba.fastffi.FFITypeAlias("std::map<std::string, GraphArchive::VertexInfo>") :: com.alibaba.graphar.stdcxx.StdMap<com.alibaba.graphar.stdcxx.StdString,com.alibaba
.graphar.graphinfo.VertexInfo>) com.alibaba.graphar.graphinfo.GraphInfo::getVertexInfos()
[ERROR]   Type: com.alibaba.graphar.graphinfo.GraphInfo
[ERROR]   Executable: (@com.alibaba.fastffi.FFINameAlias("GetVertexInfos"),@com.alibaba.fastffi.CXXReference,@com.alibaba.fastffi.FFITypeAlias("std::map<std::string, GraphArchive::VertexInfo>") :: com.alibaba.graphar.stdcxx.StdMap<com.alibaba.graphar.stdcxx.StdString,com.a
libaba.graphar.graphinfo.VertexInfo>) com.alibaba.graphar.graphinfo.GraphInfo::getVertexInfos()
[ERROR]
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

// @FFIConst
// @CXXReference
// @FFITypeAlias("std::map<std::string, GraphArchive::VertexInfo>")
// StdMap<StdString, VertexInfo> getVertexInfos();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When compile with this function, met error : Cannot find a TypeDef for ....

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may remove @FFITypeAlias(...), FastFFI will automatically map the return type. Or try std::map<std::string,GraphArchive::VertexInfo> (no spaces).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!
Both ways can solve this problem.

@Thespica
Copy link
Contributor Author

PTAL :) @acezen @lixueclaire

I think this PR is ready, but there are some problems when generating docs. In fact, I haven't changed existing docs neither add new docs.

It's maybe related to incompatibility between Python3 and jinja2. We need to discuss it in our weekly meeting.

@Thespica Thespica changed the title [WIP][Java] Add Java grpahinfo [Java] Add Java grpahinfo Jul 31, 2023
@Thespica Thespica marked this pull request as ready for review July 31, 2023 15:40
@acezen
Copy link
Contributor

acezen commented Aug 2, 2023

hi, @Thespica, thanks for the job. Since you have added the unit tests for java SDK, I think you should include the test to CI, so we can test the java code in every PR automatically.
The CI you can refer the C++ or Spark in here: https://github.com/alibaba/GraphAr/tree/main/.github/workflows

@lixueclaire
Copy link
Contributor

@Thespica, thank you for your valuable contribution! In addition to the tests, I believe it would enhance the user experience if a simple example is provided to showcase the usage of the Java SDK. Perhaps, this example could be included when you add the documentation.

@Thespica
Copy link
Contributor Author

Thespica commented Aug 2, 2023

hi, @Thespica, thanks for the job. Since you has added the unit test for java SDK, I think you should include the test to CI, so we can test the java code in every PR automatically. The CI you can refer the C++ or Spark in here: https://github.com/alibaba/GraphAr/tree/main/.github/workflows

Thanks for your advice! I well do it as soon as possible.

@Thespica
Copy link
Contributor Author

Thespica commented Aug 2, 2023

@Thespica, thank you for your valuable contribution! In addition to the tests, I believe it would enhance the user experience if a simple example is provided to showcase the usage of the Java SDK. Perhaps, this example could be included when you add the documentation.

Thanks for your advice too!

I think adding examples when adding documentation is better for me.

@acezen
Copy link
Contributor

acezen commented Aug 9, 2023

BTW,the known problem can be recorded in an issue that we can follow in the future.

WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here should have a empty line.

WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto, empty line

echo "########################################### checking maven ..."
mvn --version
- name: Build, test and install FastFFI
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to build FastFFI from source? Maybe include the fastFFI in maven is enough

branches:
- main
paths:
- 'java/**'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

include cpp/src/** since the cpp may affect java

Copy link
Contributor

@acezen acezen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • The Java CI is a liitle complicated,simplify the CI with the builtin llvm and fastFFI. I have create a java test CI, You can refer to it。
  • It's better to write a simple README to tell others that how to BUILD, RUN and TEST the java sdk, you can refer to the README of cpp

branches:
- main
paths:
- 'java/**'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Diito

@acezen acezen changed the title [Java] Add Java grpahinfo [Feat][Java] Initialize JAVA SDK: add INFO implementation Aug 15, 2023
@@ -0,0 +1,27 @@
#!/bin/bash
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this file since we don't need to build llvm from the source

<dependency>
<groupId>com.alibaba.fastffi</groupId>
<artifactId>llvm4jni-runtime</artifactId>
<version>${fastffi.revision}</version>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add <classifier>${os.detected.classifier}</classifier> to install correct jar with os platform.

<dependency>
<groupId>com.alibaba.fastffi</groupId>
<artifactId>llvm</artifactId>
<version>${fastffi.revision}</version>
Copy link
Contributor

@acezen acezen Aug 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add <classifier>${os.detected.classifier}</classifier> to install correct jar with os platform.

@@ -0,0 +1,38 @@
# GraphAr Java

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a Working In Progress tag here to tell others that The JAVA SDK is working in progress and unfinished

Copy link
Contributor

@acezen acezen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM~
You can merge after the WIP tag add to README

@acezen acezen merged commit 76276f1 into apache:main Aug 16, 2023
@acezen acezen mentioned this pull request Aug 16, 2023
9 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants