Use newer gentyref dependency

[ghisvail]

The code for gentyref was moved to GitHub and the organisation was changed accordingly, from com.googlecode.gentyref to com.coekie.gentyref. Please consider updating scijava-common to use the newer version, which will improve its packageability by Linux distributions.

Unfortunately, the eventbus dependency is a trickier one to deal with. Does anyone know where the source code repository is?

Cheers,

[imagejan]

@ghisvail wrote:

Unfortunately, the eventbus dependency is a trickier one to deal with. Does anyone know where the source code repository is?

It seems like at least a fork of this project is here:

https://github.com/wumpz/eventbus

[ghisvail]

@imagejan Thanks for the pointers. Just found the official version:

https://github.com/michaelbushe/EventBus

[ctrueden]

Regarding GenTyRef, SciJava Common will stop depending on it soon; see the generic-types branch (6be1858 specifically). I'll close this issue when that branch merges to master.

If it would help with packageability, we could do something similar with EventBus, since it is also a small library, and does not seem to be actively developed anymore. Please let us know if so, and I can investigate; otherwise, I'll assume you are able to package eventbus successfully.

[ghisvail]

Regarding GenTyRef, SciJava Common will stop depending on it soon

Ok, so I'd probably wait then. Right now, I have parsington and imglib2 ready for submission and was walking down the dependency tree for ImageJ2, which lead me to scijava-common.

If it would help with packageability, we could do something similar with EventBus

If both the gentyref and eventbus dependencies can be dropped, then that would make scijava-common easier to package once parsington is accepted. So, yes please :-)

[ctrueden]

So, yes please :-)

Great, I filed issue #282 to remind me to do that. It will likely be a few weeks, due to other priorities, but I think merging the eventbus code into SJC is a good move regardless.

Regarding parsington, we are considering separating that dependency as well into a scijava-script component, so that SJC proper does not depend on it. Have not settled on doing that for sure, though. It wouldn't help you a lot, probably, because scijava-script would certainly then be a dependency of downstream things like imagej-scripting. I don't want to pull the rug out from under you while you're working through the components, so we can certainly wait on such changes for the time being.

I'm excited that you are working on this. Did you make a forum post yet about what you are trying to achieve? If so, could you point me to it? If not, could you start a quick thread on http://forum.imagej.net/ describing your goals? It would be awesome for the community to be aware of which distros you are targeting, etc., and possibly even help, since there are quite a few people who are anxiously awaiting (presumably some flavor(s) of Linux) packages for the ImageJ2 components!

[ctrueden]

One other thing: there are enough Java components that it would be really good to have an automated way of wrapping them as packages as new versions are released. Otherwise, the packages will quickly fall out of date, and you will be exhausted as a maintainer.

Relatedly, have you seen imagej/ImageJ#162? That discussion is with respect to Gentoo, but there have also been major efforts at packaging Fiji for Debian in the past by @mhl, which I can point you at if desired. He put some effort into automation, but at the time, the licensing of individual Fiji components was too hairy in some cases. We have since made great strides in purging incompatibly licensed components from Fiji, so it should in theory be possible to fully package Fiji now. Just a lot of work to do manually, since there are hundreds of JAR files.

[ghisvail]

Regarding parsington, we are considering separating that dependency as well into a scijava-script component

But parsington would still exist and be needed for the ImageJ2 stack, right?

Did you make a forum post yet about what you are trying to achieve?

I have not, since my effort is still in its infancy. Right now, I am trying to get to know the stack, whilst engaging with leaf dependencies like parsington and imglib2.

Relatedly, have you seen imagej/ImageJ#162? That discussion is with respect to Gentoo...

What applies to Gentoo will generally apply to other distros, so their advice is worth taking into account.

there have also been major efforts at packaging Fiji for Debian

The focus is on ImageJ2 for now, which will already be quite a task.

[ctrueden]

But parsington would still exist and be needed for the ImageJ2 stack, right?

Yes, that library is very stable and will remain a dependency. So efforts to package it are worthwhile. I just meant that it may not remain a dependency of scijava-common specifically. In general, we want scijava-common to have minimal dependencies.

Right now, I am trying to get to know the stack, whilst engaging with leaf dependencies like parsington and imglib2.

Sounds good.

What applies to Gentoo will generally apply to other distros, so their advice is worth taking into account.

Which distros are you personally looking at / interested in?

The focus is on ImageJ2 for now, which will already be quite a task.

Indeed. As part of that, we are willing/happy to discuss paring down the "core ImageJ2" dependencies to make this easier. For example, the imagej-scripting component exists to bring together all of the SciJava script language components under one roof for consumption by ImageJ2. But we do not necessarily need for it to be a required dependency of net.imagej:imagej. It would vastly reduce the dependency tree of ImageJ2 if we made that optional. My dream long-term is for ImageJ to detect when a feature is being used which is not currently on the classpath, and prompt the user to install the needed JAR file(s) in order to continue. This would let the actual net.imagej:imagej component have a much smaller footprint, although I realize it brings certain problems with package managers—in particular, such add-ons would always need to be installed in user space (e.g., under ~/.imagej/plugins) rather than system wide. Otherwise you end up with a bad situation like Eclipse, which plays horribly with Linux package managers.

[ghisvail]

Which distros are you personally looking at / interested in?

Debian

It would vastly reduce the dependency tree of ImageJ2 if we made that optional.

Which is a good thing as far as modularity is concerned too.

in particular, such add-ons would always need to be installed in user space (e.g., under ~/.imagej/plugins) rather than system wide

I have seen such pattern used in the Python stack too. This way, you can have support for both system-wide / packaged plugins and user specific ones.

Otherwise you end up with a bad situation like Eclipse, which plays horribly with Linux package managers.

Hence why Debian gave up on it.

[ghisvail]

@ctrueden Could you let me know what other leaf packages could be dealt with now, besides parsington and imglib2? Cheers,

[ctrueden]

what other leaf packages could be dealt with now, besides parsington
and imglib2?

Including third party packages? Or only SciJava ones?

Some third party ones which will probably not disappear any time soon:

• com.jcraft:jsch
• com.miglayout:miglayout
• org.jfree:jfreechart (though not a leaf; it depends on org.jfree:jcommon)
• com.fifesoft:rsyntaxtextarea

And it is likely we'll have a dependency on com.google.guava:guava in the future too.

Other third party leaf components which are not likely to disappear, but could be made to disappear if absolutely necessary:

• edu.mines:mines-jtk
• gov.nist.math:jama
• net.sf.trove4j:trove4j
• edu.ucar:udunits

At a quick glance, I didn't see any other leaf dependencies in the ImageJ software stack specifically.

And apologies if any of the above are not actually leaves; I just did a quick glance at mvn dependency:tree which is known to do some pruning of redundant dependencies.

[ghisvail]

And it is likely we'll have a dependency on com.google.guava:guava in the future too.

That's quite a big one :S

[ctrueden]

Yes Guava is big, but... no dependencies! 😛

[ghisvail]

Already packaged in Debian:

com.jcraft:jsch
com.miglayout:miglayout
org.jfree:jfreechart
com.fifesoft:rsyntaxtextarea
gov.nist.math:jama

Packaged but not at its latest major version:

net.sf.trove4j:trove4j

Version 2.x, latest is 3.x.

Not packaged yet:

edu.mines:mines-jtk
edu.ucar:udunits
com.google.guava:guava

[ghisvail]

what about the scifio stack?

[ctrueden]

Core SCIFIO:

[INFO] io.scif:scifio:jar:0.32.1-SNAPSHOT
[INFO] +- io.scif:scifio-jai-imageio:jar:1.1.0:compile
[INFO] +- net.imagej:imagej-common:jar:0.24.2:compile
[INFO] |  +- net.imglib2:imglib2-roi:jar:0.4.4:compile
[INFO] |  |  \- net.sf.trove4j:trove4j:jar:3.0.3:compile
[INFO] |  \- edu.ucar:udunits:jar:4.3.18:compile
[INFO] +- net.imglib2:imglib2:jar:4.2.1:compile
[INFO] +- org.scijava:scijava-common:jar:2.62.0:compile
[INFO] |  +- org.scijava:scijava-expression-parser:jar:3.1.0:compile
[INFO] |  +- com.googlecode.gentyref:gentyref:jar:1.1.0:compile
[INFO] |  \- org.bushe:eventbus:jar:1.4:compile
[INFO] +- org.mapdb:mapdb:jar:1.0.3:compile

The new stuff here is:

• io.scif:scifio-jai-imageio - project fork of JAI ImageIO with no dependencies. Only used for JPEG2000 support. We should probably make scifio-jpeg2000 its own component, to avoid inflicting this dependency on all SCIFIO consumers.
• org.mapdb:mapdb - MapDB; used for SCIFIO's caching, which will migrate to imglib2-cache instead. So I expect this dependency will become obsolete soon.

Unfortunately, it gets more complicated with the SCIFIO-Bio-Formats integration layer. Do you want to package Bio-Formats? Doing so would be more effort. Bio-Formats ships with Fiji, but is not considered part of "core ImageJ2."

[ghisvail]

io.scif:scifio-jai-imageio

How is that different than com.github.jai-imageio:jai-imageio-core?

org.mapdb:mapdb [...] which will migrate to imglib2-cache instead

So, neither gentyref nor mapdb will be part of the dependency chain in the future, right?

Do you want to package Bio-Formats?

Babysteps. First get solid packaging for the core, then additional features.

[ctrueden]

Sorry for the delay in reply.

How is that different than com.github.jai-imageio:jai-imageio-core?

The SCIFIO version is a fork, with various bug-fixes, and a renamed package prefix to avoid version skew with the original.

So, neither gentyref nor mapdb will be part of the dependency chain in the future, right?

Correct, at least at this level. The mapdb library may rear its head downstream as a dependency for specific plugins, but is unlikely to be a core dependency.

[ghisvail]

@ctrueden Do you know where the source for edu.ucar:udunits are?

[ghisvail]

The SCIFIO version is a fork, with various bug-fixes, and a renamed package prefix to avoid version skew with the original.

Any possibility to ever see you guys converge to com.github.jai-imageio:jai-imageio-core?

The licensing of io.scif:scifio-jai-imageio is unclear / contradictory. The main terms listed in file LICENSE.txt are permissive, but the terms listed in file LICENSE-codecLibJIIO.txt are proprietary (paragraph 2). Without clarification, this makes scifio-jai-imageio unfit for packaging.

On the other hand com.github.jai-imageio:jai-imageio-core appears to be free of such limitations.

[ctrueden]

Any possibility to ever see you guys converge to com.github.jai-imageio:jai-imageio-core?

I believe that @melissalinkert tried to get the changes merged upstream many years ago, and got no response from the upstream developers. However, looking at https://github.com/jai-imageio/jai-imageio-core now, I see that @rleigh-codelibre, another one of the OME developers, has now gotten some stuff merged upstream. Could the OME devs clarify the situation there? Will we continue to need a fork of ImageIO, or is the goal to merge everything upstream now?

On the other hand com.github.jai-imageio:jai-imageio-core appears to be free of such limitations.

Are you sure? I have always found the licensing of these old Sun libraries confusing. Check out the COPYRIGHT.txt of jai-imageio-core and tell me that doesn't concern you... 😦

It may indeed be the case that even if everything merged upstream, packaging it is legally dubious. Like I said above, we can work around this in part by creating a scifio-jpeg2000 component which isolates the dependency, so that you can at least package ImageJ2 without JPEG2000 support easily.

[rleigh-codelibre]

@ctrueden @ghisvail We haven't made a concrete decision yet, but we would like to merge the changes in our fork to the new jai-imageio upstream and then adopt it if that works out. Currently the upstream have been friendly but infrequently responsive. I'm still hopeful we can get everything merged there, but it might take a little while. If anyone wanted to help out with review of the open PRs that would be very helpful; not all our fork's changes are necessarily the optimal way of doing things upstream, so some might well need some rework. One of the changes (switching the default lossy tables to lossless) can't be merged and needs acomplishing in a less invasive way.

https://github.com/rleigh-codelibre/jai-tidy/commits/master is the set of patches we are submitting, for reference, after untangling them from the bioformats history and also https://trello.com/c/yyz0NwKo/459-jai-imageio-upstream-fork if you have access which is tracking which of those have been submitted and/or merged upstream.

[ghisvail]

I have always found the licensing of these old Sun libraries confusing.

@ctrueden you are right, these terms would likely not pass the DFSG. Not to mention the confusion arising from the mismatch between the restrictive copyright and permissive licensing terms.

so that you can at least package ImageJ2 without JPEG2000 support easily.

Looks to me that this is the only way forward. As things stand now, scifio-jai-imageio is not suitable for inclusion to Debian. Please keep me updated if you happen to make further progress in splitting the offending dependency on JPEG2000.

[ctrueden]

As of 6b71b7c, scijava-common no longer depends on the gentyref artifact. So this issue can be closed.

@ghisvail Please feel welcome to open additional issues along the lines of anything discussed above, as needed, to resolve other/related packaging issues.