py5generator build process breaks with Processing v 4.4.0

[hx2A]

The py5 build process is dependent on the Processing. Previously I would download one of the release archives (e.g. processing-4.3.4-linux-arm64.tgz) and unzip that, but as of 4.4.0, files like that are no longer being created. I need to adapt py5's build process to either use the result of Processing's new Gradle build process or use one of the portable files (e.g. processing-4.4.1-linux-x64-portable.zip).

To complete the build, I need all of the files and resources that are currently in these subdirectories:

https://github.com/py5coding/py5/tree/main/py5/jars
https://github.com/py5coding/py5/tree/main/py5/natives

Right now I think my best bet is to use the portable release file. I think I can get all of the jar files I need from these portable subdirectories:

lib/app/resources/core/library/
lib/app/resources/modes/java/libraries/svg/library/
lib/app/resources/modes/java/libraries/pdf/library/
lib/app/resources/modes/java/libraries/dxf/library/

Q: Are there jars in lib/app/resources/core/library/ that I don't need? I don't think the annotations, antlr, or kotlin-stdlib jars are needed. Is there anything missing that I am overlooking?

Q: And what about the native libraries currently in https://github.com/py5coding/py5/tree/main/py5/natives? These seem to be packaged up in the *native* jar files in the portable release's lib/app/resources/core/library/ directory. How do I use them? Currently py5 uses this function to add the native libraries to the java.library.path System property. How does this work with the native library files in jars? Do I just add the native jar file to java.library.path?

[Stefterv]

Which jars do you need? We can release those on maven.

EDIT: Just had a look myself, :core you can already grab from maven here: https://central.sonatype.com/artifact/org.processing/core
The gluegen and jogs-all you can grab from jogl's repo here: https://jogamp.org/deployment/maven/
and the dxf, pdf and svg libs I will make sure they get pushed to maven as soon as they are also moved to gradle.

Happy to have a look if you could use a hand automating this stuff

[hx2A]

Which jars do you need? We can release those on maven

I need core.jar, but it might be better for me if I am not going through maven, because sometimes when I am doing bug fixing I need make the core.jar file myself. But maybe I should consider adapting the build process to use maven? I will look into this later.

I also need the jogl and glugen jars, and the jars for svg, pdf, and dxf.

I now have part of the py5 build process working. I think the last sticking point is getting OpenGL to work. It needs the native libraries, which are now packaged in jar files. Adding them to the classpath is not enough. How does Processing add them? This is mentioned in the code but I don't see how you are handling this.

[Stefterv]

Okay would you be fine with cloning the processing repo in your build step and compiling the jar there? Then you could also apply your specific fixes there (Or you could even use a fork of the processing repo to provide those bug fixes back to us)

As for the natives, I don't have at the top of my head how I fixed it with the PDE itself but here is how I fixed it for the Gradle Plugin that I am working on that has the same issue: https://github.com/Stefterv/processing4/blob/runner/java/gradle/src/main/kotlin/ProcessingPlugin.kt

[hx2A]

Okay would you be fine with cloning the processing repo in your build step and compiling the jar there? Then you could also apply your specific fixes there (Or you could even use a fork of the processing repo to provide those bug fixes back to us)

Yes, that works for me. And of course I'd make PRs for any bug fixes. :)

As for the natives, I don't have at the top of my head how I fixed it with the PDE itself but here is how I fixed it for the Gradle Plugin that I am working on that has the same issue: https://github.com/Stefterv/processing4/blob/runner/java/gradle/src/main/kotlin/ProcessingPlugin.kt

This code looks like the Gradle stuff that the PDE uses to run someone's Sketch? How does this work when someone wants to use the core.jar code outside of the PDE?

Is the Grade code extracting the contents of these jar files? I did some more poking around and it seems that extracting the contents of these native jars is the best option. There doesn't seem to be a way for Java to access the native libraries in a jar file without extracting them first. Or am I wrong about that?

[hx2A]

@Stefterv , I updated py5's build process to extract the native libraries from the jars and to put them in the same location as before. Everything looks OK, but I still can't run OpenGL sketches. The error message I get is:

MESA: error: ZINK: vkCreateInstance failed (VK_ERROR_INCOMPATIBLE_DRIVER)

I never saw that before. No window opens. Any idea what I could be doing wrong?

It doesn't seem that these are the same native library files as before. Did you change the JOGL version or something? When I run a git diff command it indicates that the native libraries have changed. What is different about them?

[hx2A]

Everything looks OK, but I still can't run OpenGL sketches. The error message I get is:

MESA: error: ZINK: vkCreateInstance failed (VK_ERROR_INCOMPATIBLE_DRIVER)

I never saw that before. No window opens. Any idea what I could be doing wrong?

Looks like there is something weird happening with my computer. Even if I remove the version of py5 I just created and install the last release from pypi, I still get that error message. Since I know the version on pypi is fine, there must be some issue with my machine. :(

[hx2A]

Looks like there is something weird happening with my computer. Even if I remove the version of py5 I just created and install the last release from pypi, I still get that error message. Since I know the version on pypi is fine, there must be some issue with my machine.

It seems I have it working now? Will do more testing after dinner. @Stefterv , thanks for your assistance here.

[hx2A]

EDIT: Just had a look myself, :core you can already grab from maven here: https://central.sonatype.com/artifact/org.processing/core
The gluegen and jogs-all you can grab from jogl's repo here: https://jogamp.org/deployment/maven/

What is the right way to use the core.jar code on maven? I have the below in a pom.xml file for a project and it compiles and works correctly, but it seems sloppy to be excluding the jogl stuff like that.

  <dependencies>
    <dependency>
      <groupId>org.processing</groupId>
      <artifactId>core</artifactId>
      <version>4.4.1</version>
      <exclusions>
        <exclusion>
          <groupId>org.jogamp.gluegen</groupId>
          <artifactId>gluegen-rt-main</artifactId>
        </exclusion>
        <exclusion>
          <groupId>org.jogamp.jogl</groupId>
          <artifactId>jogl-all-main</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
  </dependencies>

[hx2A]

@Stefterv , I now have the py5 build working with the 4.4.1 release of Processing. Thanks again for your help here.

I do have one question, in addition the above question about Maven.

When I compile py5's Java code, I see these warning messages:

    [javac] warning: [path] bad path element "/var/home/jim/INSTALL/processing-4.4.1-portable/lib/app/resources/modes/java/application/launch4j/lib/commons-beanutils.jar": no such file or directory                    
    [javac] warning: [path] bad path element "/var/home/jim/INSTALL/processing-4.4.1-portable/lib/app/resources/modes/java/application/launch4j/lib/commons-logging.jar": no such file or directory                      
    [javac] warning: [path] bad path element "/var/home/jim/INSTALL/processing-4.4.1-portable/lib/app/resources/modes/java/application/launch4j/lib/flatlaf.jar": no such file or directory                              
    [javac] warning: [path] bad path element "/var/home/jim/INSTALL/processing-4.4.1-portable/lib/app/resources/modes/java/application/launch4j/lib/formsrt.jar": no such file or directory                              
    [javac] warning: [path] bad path element "/var/home/jim/INSTALL/processing-4.4.1-portable/lib/app/resources/modes/java/application/launch4j/lib/foxtrot.jar": no such file or directory                              
    [javac] warning: [path] bad path element "/var/home/jim/INSTALL/processing-4.4.1-portable/lib/app/resources/modes/java/application/launch4j/lib/jgoodies-common.jar": no such file or directory                      
    [javac] warning: [path] bad path element "/var/home/jim/INSTALL/processing-4.4.1-portable/lib/app/resources/modes/java/application/launch4j/lib/jgoodies-forms.jar": no such file or directory                       
    [javac] Note: Annotation processing is enabled because one or more processors were found                                                                                                                             
    [javac]   on the class path. A future release of javac may disable annotation processing                                                                                                                             
    [javac]   unless at least one processor is specified by name (-processor), or a search                                                                                                                               
    [javac]   path is specified (--processor-path, --processor-module-path), or annotation                                                                                                                               
    [javac]   processing is enabled explicitly (-proc:only, -proc:full).                                                                                                                                                 
    [javac]   Use -Xlint:-options to suppress this message.                                                                                                                                                              
    [javac]   Use -proc:none to disable annotation processing.                                                                                                                                                           
    [javac] 7 warnings  

They don't seem to cause any harm, but what is it saying here? I don't need or ask for any of those jar files. Why is it mentioning them?

[Stefterv]

Great! Those jar files exist in the binary to be able to package processing sketches into standalone apps. Not sure why it pops up here.

As for the maven question, which build system are you using? (Ant?) In Gradle or Maven it is pretty straightforward and you can add the JOGL maven repo to avoid these exclusions.

[hx2A]

As for the maven question, which build system are you using? (Ant?) In Gradle or Maven it is pretty straightforward and you can add the JOGL maven repo to avoid these exclusions.

I've been using ant but your work with Gradle has inspired me to give that a try! Also, to learn more about Kotlin. I saw in the build documentation that you want new classes to be written in Kotlin?

[Stefterv]

Yeah I think it make sense to start writing most things in Kotlin, as we are still somewhat stuck on an older JDK for now, we at least get the nice language features through Kotlin