On the Way to the Java Module System
This is an attempt to prepare a migration to the Java Module System (JMS). It is also an effort to get familiar with JMS and its challenges.
We will start with an analysis listing problems related to modularizing the JavaPOS source code for JMS, followed by proposal how the problems may be solved and how to the design modules. After that we will discuss backward compatibility issues, if any, and how to deal with them.
Based on that the first implementation attempt will be made checking whether the design approach fits the reality. New problems arising during implementation will be discussed here.
All the knowledge about the JMS and how to figure out problems and solution schemas has been taken from the Nicolai Parlog's book The Java Module System, Manning Publications, 2019. It is highly recommended to read this book if you are going to modularize your Java code!
The analysis is based on an AdoptOpenJDK distribution for Java 11.
denis@modula:~/javapos-jms$ apt show adoptopenjdk-11-hotspot
Package: adoptopenjdk-11-hotspot
Version: 11.0.5+10-2
Priority: extra
Section: java
Maintainer: AdoptOpenJDK
Installed-Size: 327 MB
Provides: java-compiler, java-sdk, java-sdk-headless, java10-sdk, java11-sdk, java2-sdk, java5-sdk, java6-sdk, [...]
Depends: ca-certificates, java-common, libasound2, libc6, libx11-6, libxext6, libxi6, libxrender1, libxtst6, zlib1g
Homepage: https://adoptopenjdk.net/
License: GPL-2.0+CE
Vendor: AdoptOpenJDK
Download-Size: 198 MB
APT-Manual-Installed: yes
APT-Sources: https://adoptopenjdk.jfrog.io/adoptopenjdk/deb xenial/main amd64 Packages
Description: OpenJDK Development Kit 11 (JDK) with Hotspot by AdoptOpenJDK
The analysis mainly utilizes the jdeps tool from the JDK: alias jdeps='/usr/lib/jvm/adoptopenjdk-11-hotspot-amd64/bin/jdeps'.
At first, we are going to check what we have. All JavaPOS JARs and their dependencies has been copied to a directory for analysis:
denis@modula:~/javapos-jms$ ls
javapos-config-loader-2.3.1.jar javapos-contracts-1.14.3.jar javapos-controls-1.14.1.jar xerces-1.2.3.jar
At a first call to jdeps on javapos-controls JAR we are getting a first overview of the problems we may have:
denis@modula:~/javapos-jms$ jdeps -s --class-path '*' javapos-controls-1.14.1.jar
Warning: split package: javax.xml.parsers jrt:/java.xml xerces-1.2.3.jar
Warning: split package: org.w3c.dom jrt:/java.xml xerces-1.2.3.jar
Warning: split package: org.w3c.dom.events jrt:/java.xml xerces-1.2.3.jar
Warning: split package: org.w3c.dom.html jrt:/jdk.xml.dom xerces-1.2.3.jar
Warning: split package: org.w3c.dom.ranges jrt:/java.xml xerces-1.2.3.jar
Warning: split package: org.w3c.dom.traversal jrt:/java.xml xerces-1.2.3.jar
Warning: split package: org.xml.sax jrt:/java.xml xerces-1.2.3.jar
Warning: split package: org.xml.sax.ext jrt:/java.xml xerces-1.2.3.jar
Warning: split package: org.xml.sax.helpers jrt:/java.xml xerces-1.2.3.jar
javapos-controls-1.14.1.jar -> java.base
javapos-controls-1.14.1.jar -> java.desktop
javapos-controls-1.14.1.jar -> javapos-config-loader-2.3.1.jar
javapos-controls-1.14.1.jar -> javapos-contracts-1.14.3.jar
This shows us, we need to get rid of the Xerces JAR of that version as it has massive split package problems with JDK modules. This was assumed as the Xerces JAR is coming as a very old quite out dated version.
Now, it would be good to get this overview information as a dependency graph representation. jdeps together with dot from the graphviz package can do that for us. We start the analysis from the top of the dependency tree, we know the javapos-controls JAR is, and let jdeps traverse the dependency hierarchy recursively which result in the a dots subdirectory with a image of the dependency graph created:
denis@modula:~/javapos-jms$ jdeps -summary -recursive --class-path '*' --dot-output dots javapos-controls-1.14.1.jar && dot -Tpng -O dots/summary.dot && tree
[warnings removed here]
.
├── dots
│ ├── summary.dot
│ └── summary.dot.png
├── javapos-config-loader-2.3.1.jar
├── javapos-contracts-1.14.3.jar
├── javapos-controls-1.14.1.jar
└── xerces-1.2.3.jar
1 directory, 6 files
Resulting in the following dependency graph:
Let's now drill deeper in our dependency hierarchy starting from the base, which we know is the javapos-contracts JAR.
denis@modula:~/javapos-jms$ jdeps --class-path '*' javapos-contracts-1.14.3.jar
javapos-contracts-1.14.3.jar -> java.base
javapos-contracts-1.14.3.jar -> java.desktop
jpos -> java.awt java.desktop
jpos -> java.lang java.base
jpos -> jpos.events javapos-contracts-1.14.3.jar
jpos.events -> java.lang java.base
jpos.events -> java.util java.base
jpos.loader -> java.lang java.base
jpos.loader -> jpos javapos-contracts-1.14.3.jar
jpos.services -> java.awt java.desktop
jpos.services -> java.lang java.base
jpos.services -> jpos javapos-contracts-1.14.3.jar
jpos.services -> jpos.events javapos-contracts-1.14.3.jar
jpos.services -> jpos.loader javapos-contracts-1.14.3.jar
OK, that looks fine. We do not have split package problems with the JDK modules or any "unknown packages" problems.
Continuing with the javapos-config-loader JAR, the next in the dependency hierarchy:
denis@modula:~/javapos-jms$ jdeps --class-path '*' javapos-config-loader-2.3.1.jar
Warning: split package: javax.xml.parsers jrt:/java.xml xerces-1.2.3.jar
Warning: split package: org.w3c.dom jrt:/java.xml xerces-1.2.3.jar
Warning: split package: org.w3c.dom.events jrt:/java.xml xerces-1.2.3.jar
Warning: split package: org.w3c.dom.html jrt:/jdk.xml.dom xerces-1.2.3.jar
Warning: split package: org.w3c.dom.ranges jrt:/java.xml xerces-1.2.3.jar
Warning: split package: org.w3c.dom.traversal jrt:/java.xml xerces-1.2.3.jar
Warning: split package: org.xml.sax jrt:/java.xml xerces-1.2.3.jar
Warning: split package: org.xml.sax.ext jrt:/java.xml xerces-1.2.3.jar
Warning: split package: org.xml.sax.helpers jrt:/java.xml xerces-1.2.3.jar
javapos-config-loader-2.3.1.jar -> java.base
javapos-config-loader-2.3.1.jar -> java.desktop
javapos-config-loader-2.3.1.jar -> java.xml
javapos-config-loader-2.3.1.jar -> javapos-contracts-1.14.3.jar
javapos-config-loader-2.3.1.jar -> xerces-1.2.3.jar
jpos.config -> java.io java.base
jpos.config -> java.lang java.base
jpos.config -> java.lang.reflect java.base
jpos.config -> java.net java.base
jpos.config -> java.util java.base
jpos.config -> jpos javapos-contracts-1.14.3.jar
jpos.config -> jpos.loader javapos-config-loader-2.3.1.jar
jpos.config -> jpos.util javapos-config-loader-2.3.1.jar
jpos.config -> jpos.util.tracing javapos-config-loader-2.3.1.jar
jpos.config.simple -> java.io java.base
jpos.config.simple -> java.lang java.base
jpos.config.simple -> java.net java.base
jpos.config.simple -> java.util java.base
jpos.config.simple -> java.util.zip java.base
jpos.config.simple -> jpos.config javapos-config-loader-2.3.1.jar
jpos.config.simple -> jpos.loader javapos-config-loader-2.3.1.jar
jpos.config.simple -> jpos.util javapos-config-loader-2.3.1.jar
jpos.config.simple -> jpos.util.tracing javapos-config-loader-2.3.1.jar
jpos.config.simple.xml -> java.io java.base
jpos.config.simple.xml -> java.lang java.base
jpos.config.simple.xml -> java.net java.base
jpos.config.simple.xml -> java.text java.base
jpos.config.simple.xml -> java.util java.base
jpos.config.simple.xml -> javax.xml.parsers java.xml
jpos.config.simple.xml -> jpos.config javapos-config-loader-2.3.1.jar
jpos.config.simple.xml -> jpos.config.simple javapos-config-loader-2.3.1.jar
jpos.config.simple.xml -> jpos.util javapos-config-loader-2.3.1.jar
jpos.config.simple.xml -> jpos.util.tracing javapos-config-loader-2.3.1.jar
jpos.config.simple.xml -> org.apache.xerces.dom xerces-1.2.3.jar
jpos.config.simple.xml -> org.apache.xerces.jaxp xerces-1.2.3.jar
jpos.config.simple.xml -> org.apache.xerces.parsers xerces-1.2.3.jar
jpos.config.simple.xml -> org.apache.xml.serialize xerces-1.2.3.jar
jpos.config.simple.xml -> org.w3c.dom java.xml
jpos.config.simple.xml -> org.xml.sax java.xml
jpos.config.simple.xml -> org.xml.sax.helpers java.xml
jpos.loader -> java.io java.base
jpos.loader -> java.lang java.base
jpos.loader -> java.lang.reflect java.base
jpos.loader -> java.net java.base
jpos.loader -> java.security java.base
jpos.loader -> java.util.jar java.base
jpos.loader -> jpos javapos-contracts-1.14.3.jar
jpos.loader -> jpos.config javapos-config-loader-2.3.1.jar
jpos.loader -> jpos.loader.simple javapos-config-loader-2.3.1.jar
jpos.loader -> jpos.profile javapos-config-loader-2.3.1.jar
jpos.loader -> jpos.util javapos-config-loader-2.3.1.jar
jpos.loader -> jpos.util.tracing javapos-config-loader-2.3.1.jar
jpos.loader.simple -> java.lang java.base
jpos.loader.simple -> java.lang.reflect java.base
jpos.loader.simple -> java.util java.base
jpos.loader.simple -> jpos javapos-contracts-1.14.3.jar
jpos.loader.simple -> jpos.config javapos-config-loader-2.3.1.jar
jpos.loader.simple -> jpos.config.simple javapos-config-loader-2.3.1.jar
jpos.loader.simple -> jpos.loader javapos-config-loader-2.3.1.jar
jpos.loader.simple -> jpos.loader javapos-contracts-1.14.3.jar
jpos.loader.simple -> jpos.profile javapos-config-loader-2.3.1.jar
jpos.loader.simple -> jpos.util javapos-config-loader-2.3.1.jar
jpos.loader.simple -> jpos.util.tracing javapos-config-loader-2.3.1.jar
jpos.profile -> java.io java.base
jpos.profile -> java.lang java.base
jpos.profile -> java.net java.base
jpos.profile -> java.util java.base
jpos.profile -> javax.xml.parsers java.xml
jpos.profile -> jpos.util javapos-config-loader-2.3.1.jar
jpos.profile -> jpos.util.tracing javapos-config-loader-2.3.1.jar
jpos.profile -> org.apache.xerces.parsers xerces-1.2.3.jar
jpos.profile -> org.w3c.dom java.xml
jpos.profile -> org.xml.sax java.xml
jpos.util -> java.awt java.desktop
jpos.util -> java.awt.event java.desktop
jpos.util -> java.io java.base
jpos.util -> java.lang java.base
jpos.util -> java.util java.base
jpos.util -> java.util.jar java.base
jpos.util -> java.util.zip java.base
jpos.util -> javax.swing java.desktop
jpos.util -> jpos.config javapos-config-loader-2.3.1.jar
jpos.util -> jpos.config.simple javapos-config-loader-2.3.1.jar
jpos.util -> jpos.loader javapos-config-loader-2.3.1.jar
jpos.util -> jpos.util.tracing javapos-config-loader-2.3.1.jar
jpos.util.tracing -> java.io java.base
jpos.util.tracing -> java.lang java.base
jpos.util.tracing -> java.util java.base
jpos.util.tracing -> jpos.util javapos-config-loader-2.3.1.jar
That's a quite long list. But no additional problem show up we didn't know already -- the split package problem in the Xerces JAR.
Let's get rid of Xerces for the further analysis to see the JavaPOS code internal problems more clearer.
denis@modula:~/javapos-jms$ rm xerces-1.2.3.jar && ls
dots javapos-config-loader-2.3.1.jar javapos-contracts-1.14.3.jar javapos-controls-1.14.1.jar
Finally, the last JAR on the top of the dependency graph is analyzed:
denis@modula:~/javapos-jms$ jdeps --class-path '*' javapos-controls-1.14.1.jar
javapos-controls-1.14.1.jar -> java.base
javapos-controls-1.14.1.jar -> java.desktop
javapos-controls-1.14.1.jar -> javapos-config-loader-2.3.1.jar
javapos-controls-1.14.1.jar -> javapos-contracts-1.14.3.jar
jpos -> java.awt java.desktop
jpos -> java.beans java.desktop
jpos -> java.lang java.base
jpos -> java.util java.base
jpos -> jpos.events javapos-contracts-1.14.3.jar
jpos -> jpos.loader javapos-config-loader-2.3.1.jar
jpos -> jpos.loader javapos-contracts-1.14.3.jar
jpos -> jpos.services javapos-contracts-1.14.3.jar
No new problems detected here.
Now, let's see whether we have a split package problem in the JavaPOS JARs itself. For that, we create a module path directory and move the base dependency JAR javapos-contracts to it. This puts javapos-contracts into the set of automatic modules letting it participate in the module graph construction made by jdeps and is similar to as if we had already modularized javapos-contracts.
denis@modula:~/javapos-jms$ mkdir mods && mv javapos-contracts-1.14.3.jar mods && tree
.
├── dots
│ ├── summary.dot
│ └── summary.dot.png
├── javapos-config-loader-2.3.1.jar
├── javapos-controls-1.14.1.jar
└── mods
└── javapos-contracts-1.14.3.jar
2 directories, 5 files
This results in the following jdeps analysis result for the javapos-controls JAR, the top of our dependency graph:
denis@modula:~/javapos-jms$ jdeps -summary -recursive --class-path '*' --module-path mods javapos-controls-1.14.1.jar
Warning: split package: jpos file:///home/denis/javapos-jms/mods/javapos-contracts-1.14.3.jar javapos-controls-1.14.1.jar
Warning: split package: jpos.loader file:///home/denis/javapos-jms/mods/javapos-contracts-1.14.3.jar javapos-config-loader-2.3.1.jar
javapos-controls-1.14.1.jar -> java.base
javapos-controls-1.14.1.jar -> java.desktop
javapos-controls-1.14.1.jar -> javapos.contracts
javapos-controls-1.14.1.jar -> not found
javapos.contracts -> java.base
javapos.contracts -> java.desktop
Here a new real problem in JavaPOS' package structure shows up. There are split packages when we will have made all the JavaPOS JARs to modules.
- The package jpos is split between javapos-contracts and javapos-controls,
- The package jpos.loader is split between javapos-contracts and javapos-config-loader.
These 2 problems have to be solved by redesign the JavaPOS package structure.
But wait. There is a new "not found" on the table. Where this comes from? Let's make a deeper class level analysis:
denis@modula:~/javapos-jms$ jdeps -verbose --class-path '*' --module-path mods javapos-controls-1.14.1.jar | grep "not found"
javapos-controls-1.14.1.jar -> not found
jpos.BaseJposControl -> jpos.loader.JposServiceConnection not found
jpos.BaseJposControl -> jpos.loader.JposServiceLoader not found
This comes from the fact that javapos-contracts is an automatic module now. jdeps computes the module graph with javapos-contract's package jpos.loader inside. And therefore, the Java module system does not allow to load the classes JposServiceConnection and JposServiceLoader of that package from another JAR (from the 'unnamed' module, to be precise), as those both classes are contained in the javapos-config-loader JAR. This JAR is on the class-path and therefore in the 'unnamed' module.
- Xerces JAR in version 1.2.3 is not JMS compliant and has split packages to several JDK modules.
- The package jpos is split between javapos-contracts and javapos-controls.
- The package jpos.loader is split between javapos-contracts and javapos-config-loader.
For reading JavaPOS configurations into memory from and writing to XML files, so called, populators are used (see interface jpos.config.JposRegPopulator. The JposRegPoplutator for XML writing and reading are implemented using the Xerces library. To get rid of the Xerces library, the XML file reading and writing functionality required by javapos-config-loader must be adopted.
The Xerces library aggregates 3 different namespaces:
- org.w3c.dom - implements the W3C DOM specification, a dependency for the Xerces project emanated implementation
- org.xml.sax - implements the Simple API for XML, for fast XML parser implementations, a dependency for the Xerces project emanated SAX parser implementation
- org.apache - the actual Xerces namespace containing the emanated Xerces implementation classes
The org.w3c.dom and the org.xml.sax namespaces provided through the Xerces library are also part of Java standard library. That's where the jdeps split package warnings for org.w3c.dom and org.xml.sax are coming from (see above). In fact, the packages of that namespace in the Xerces library are not in use as they are shadowed by the same ones from the Java standard library. Both namespaces have quite long been integral part of the Java standard library; at least since Java 8 (has been checked), maybe longer. They are part of Java 11 the same way(checked).
So, when we get rid of the Xerces library, the javapos-config-loader's functionality utilizing the classes from the org.w3c.dom and org.xml.sax namespaces is still in place and in fact the same as before (due to the shadowing).
That means, for getting rid of the Xerces library we only need to get rid of the classes imported from the org.apache namespace. The search for "import org.apache." produce 6 matches:
The class jpos.config.simple.xml.AbstractXercesRegPopulator uses classes from the org.apache namespace only for implementing the XML file writing. Today, this would be implemented differently, without referencing XML DOM implementation classes from the org.apache namespace directly. For an implementation example see this tutorial, e.g.
Using the XML DOM implementation just for writing the XML file from an in-memory JavaPOS configuration representation is quite cumbersome as the representation is first re-created as XML DOM tree which is then written to file using the XML DOM writing facilities. Instead it would be more easier to write the in-memory representation directly to a file. So, for getting rid of org.apache dependencies in this class, the method jpos.config.simple.xml.AbstractXercesRegPopulator.convertJposEntriesToXml(Enumeration, OutputStream) has to be reimplemented by iterating over the in-memory representation of the configuration and issuing it directly into a file, XML formatted (Consider to use the Xtend template engine for that).
The class jpos.config.simple.xml.Xerces2RegPopulator is using the org.apache.xerces.jaxp.SAXParserFactoryImpl only in the method jpos.config.simple.xml.Xerces2RegPopulator.getSAXParser() for instantiating a javax.xml.parsers.SAXParserFactory. This would be done today using a method javax.xml.parsers.SAXParserFactory.newInstance() which gets the right SAXParser from a JAXP properties configuration or trough the ServiceLoader mechanism. If no SAX parser implementation is found the default implementation com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl from the Java standard library will be used which is available at least since Java 8 (maybe earlier) and also in Java 11.
The last import of org.apache.xerces.parsers.DOMParser at jpos.profile.XercesProfileFactory is easy to fix. The compiler notes already that the declared field domParser of that type is never used.
The JavaPOS Device Control interfaces and the JavaPOS Device Control reference implementations for all device categories are both in the same package jpos. Because they are split into different libraries, javapos-contracts and javapos-controls respectively, the split package issues comes up.
To solve this, either one of both must be moved into another package or, both libraries has to be merged again building one module. Because the split package problem would disappear automatically.
The javapos-controls library was separated from the javapos-contracts to allow replacing the Device Control implementation, which is a reference implementation, by a vendor specific one. So merging both together again, would break the initial intention to allow vendor specific implementations with the same qualified names substituting the reference implementation by the vendor specific one on the class-path. Holding the Device Control implementation separate from the device-contract library this approach will continue to work when moving to a Java Module System design. That way, the Device Control reference implementation's module may be substituted on the module path by any vendor specific module with the same implementation classes).
When moving into another package, the JavaPOS Device Control interfaces should be moved to package jpos.controls parallel to the JavaPOS Device Service interfaces which reside at package jpos.services. The Device Control implementation should not be moved as its classes are used inside application code by direct reference. Moving them away could cause a big backward compatibility issue which should be avoided. However, moving the Device Control interfaces would only require the Device Control reference implementation and all vendor specific Device Control implementations to be changed, which is a smaller backward compatibility problem than the former one.
As an additional effort for a future design approach, there could be implemented a JMS compatible ServiceLoader based Device Control loading and instantiating mechanism for Device Control implementations based on the Device Control interfaces as service types. All applications utilizing this new approach for Device Control instantiation could make the device category's JavaPOS version the application is using more explicit and gaining a more secure way than patching the class path to deal with the vendor specific Device Control implementation it wants to use.
This is, maybe, the simplest of the problems.
It is causes by the interface jpos.loader.JposServiceInstance which is the only artifact in javapos-contracts package jpos.loader. Originally, it was created as part of the javapos-config-loader stand-alone project, before all the JavaPOS sources had been migrated to and joined at GitHub (for details see "Migrating to And Joining All JavaPOS Sources at GitHub).
In fact, it is an interface all Device Service interfaces are extending from and is used at the interface jpos.loader.JposServiceConnection and its reference implementation jpos.loader.simple.SimpleServiceConnection for creating and disposing Device Service instances and bounding them to Device Control instances. It is also the type returned by device provider specific jpos.loader.JposServiceInstanceFactory implementations.
So the solution is to move this interface to the jpos.services package as it is part of the interfaces defining a JavaPOS device service. This will not cause any backward compatibility issues the interface should only be referenced internally by classes at the javapos-contracts and javapos-config-loader modules.
As the solutions for the detected problems from above propose to stay with the library structure it is today, each library will be converted into a Java module. For those, modules names are required which should be unique inside the Java universe and describe its content in recognizable manner.
A good practice for JMS solutions is to use the reverse domain notation as it is known from Java package names and let the module name be the prefix of all packages it contains.
To fit the first best practice point, the modules could be named according to the domain javapos.org owned now by OMG, the standardization organization the UnifiedPOS standard JavaPOS is an implementation of is maintained at. This is also the group ID the old libraries are publishe under Maven Central and JCenter.
However, the second best practice is hard to met because all JavaPOS packages start with the prefix jpos, not org.javapos. And, in all 3 libraries the contained packages have only the prefix jpos_ in common. So, there is no library specific prefix available.
Therefore, the following module names will be chosen for the particular libraries:
- javapos-contracts: org.javapos.contracts
- javapos-controls: org.javapos.controls
- javapos-config-loader: org.javapos.config
The JMS encourages system designers to use the ServiceLoader based service provider mechanism extensively for module decoupling. It works seamlessly together with the new module system and is preferred over the reflection based instantiation mechanism (the combined calls to java.lang.Class.forName(String, boolean, ClassLoader), java.lang.Class.getConstructor(Class>...)_, and _java.lang.Class.getConstructor(Class>...)). It seems, this is the future at decoupled module design replacing reflection based approaches. The JMS based JavaPOS implementation should support this service provider mechanism as well as the reflection based instantiation mechanism for backward compatibility.
At this approach the java.util.ServiceLoader from the Java standard library provides a list of implementations for a particular service type it has been queried for. An example for how to implement such a combined approach can be found at this class implmentation: javax.xml.parsers.FactoryFinder in the Java 11 standard library.
There are several points for such a decoupling in JavaPOS:
- Device Service class instantiations through the jpos.loader.JposServiceInstanceFactory every Device Service provider must implement. The affected method is jpos.loader.simple.SimpleServiceConnection.connect().
- Device Control instantiation at application side (see Solving Problem 2) - this would be a new implementation
- Instantiation of a jpos.loader.JposServiceManager at the static initializer of jpos.loader.JposServiceLoader.
- Instantiation of a jpos.config.JposRegPopulator. The affected method is jpos.loader.simple.SimpleServiceManager.initRegPopulator().
There are several smaller issues to tackle regarding code cleanliness:
- Adding Junit tests for device control logic.
- Get rid of all compiler warnings.
- Introduce generic types for internal implementation parts.
- Extend public JavaPOS APIs regarding generic types and enabling for streaming APIs while preserving the old API for backward compatibility.
TODO