Datimprint fingerprint and verification software released using picocli

[garretwilson]

I want to share a new CLI program named Datimprint™ which I just released via my data processing company Jordial Corporation. The full source code is available in the Datimprint GitHub project.

The Datimprint program helps you verify that all of your data is intact by generating “data imprint” snapshots and later comparing them with current data. These data imprints contain fingerprint hashes of each file, along with other information such as file modification timestamps. The fingerprints are recursive and distinguish between a content tree (i.e. the bits) and other metadata (e.g. filename case and timestamp). Datimprint is written in Java, is multithreaded for quick checksum generation, and uses picocli for the actual command-line interface.

Datimprint creates a "data imprint" of a directory tree and stores it in datim file.

datimprint generate /var/data --output /var/imprints/data-2022-11-12.datim

You can later check the data tree against the imprint file, to verify a backup or to check for data degradation, for example.

datimprint check /var/backup/data --imprint /var/imprints/data-2022-11-12.datim

More instructions and explanations are available in the Datimprint overview.

I also want to discuss a little about how exactly Datimprint uses picocli. It leverages the globalmentor-application project, which provides a BaseCliApplication class (which I've discussed before in the picocli tickets) which does most of the work setting up picocli, auto-detecting terminal width, turning on ANSI, setting up default conversions, etc. Basing a CLI on BaseCliApplication allows an application to use the picocli annotations to define the application itself and not worry about all the setup/configuration boilerplate that is general for all/most applications.

Another interesting (and extremely useful) class is CliStatus, which provides a single, updating, color-coded status line that even supports multiple threads. You have to try it out to appreciate it. (Just run the generate command above on a large directory tree, for example.) Let's say some functionality is using a thread pool of eight threads (e.g. Datimprint traversing a file system to generate checksums). The status line will indicate that eight threads are working, but rather than trying to list all eight files being hashed, it will just display one of them. The way it is written, probability will cause the operations that take the longest (i.e. usually the largest files) to be be the ones displayed; the quick files won't be shown. This allows you to get an idea of the overall progress and gain insight into some of the operations taking the longest time without being lost in the weeds—which is the point of a status bar.

Finally to top this off, Datimprint uses the globalmentor-root POM as its Maven parent POM. The Datimprint CLI subproject then defines a single Maven property, exe.main.class, which will turn on generation of executable files. (The exe.base.filename is optional; it would default to the Maven artifact ID, here datimprint-cli, if unspecified.)

properties>
  <exe.main.class>com.jordial.datimprint.cli.DatimprintCli</exe.main.class>
  <exe.base.filename>datimprint</exe.base.filename>
</properties>

Simply setting these properties results in the automatic generation of the following:

• An executable JAR file datimprint-cli-x.x.x-exe.jar.
• A self-contained executable Linux file datimprint.
• A Windows executable datimprint.exe, with all the metadata (company, version, etc.) set. (Note that projects using the globalmentor-root POM will need to change the company information and such, which they should have defined in their Maven project anyway.)
• Both datimprint-cli-x.x.x-bin.zip and datimprint-cli-x.x.x-bin.tar.xz archives containing all the executable files listed above.

All the tricks I had to play to pull all that off in Maven (using the presence of a Maven property as a Boolean flag; automatic generation of a Windows executable; generation of an assembly descriptor during the build and then using it, etc.) is a separate discussion altogether.

globalmentor-application and globalmentor-root are open source and available on GitHub, should anyone want to use them to enhance and simplify their use of picocli.