Replace home-grown uberJar task with shadowJar plugin

[kwvanderlinde]

Identify the Bug or Feature request

Addresses #4354

Description of the Change

This isn't a huge change, but is still kind of involved in its details.

The main change is switching away from uberJar in favour of shadowJar in order to gain support for service file merging. This fixes SPI registration for our dependencies, including Graal languages (JS, regex), and jai-imageio-jpeg2000 (which we had a workaround for that is now removed). There was also a bunch of other stuff that was missing but is now present in the service files. Here's a complete diff of the service files for those interested:

diff '--color=auto' -rbBw MapTool-SNAPSHOT-7d89f48-uberJar/META-INF/services/com.oracle.truffle.api.TruffleLanguage$Provider MapTool-SNAPSHOT-e421d6e-shadowJar/META-INF/services/com.oracle.truffle.api.TruffleLanguage$Provider
0a1
> com.oracle.truffle.js.lang.JavaScriptLanguageProvider
diff '--color=auto' -rbBw MapTool-SNAPSHOT-7d89f48-uberJar/META-INF/services/javax.imageio.spi.ImageInputStreamSpi MapTool-SNAPSHOT-e421d6e-shadowJar/META-INF/services/javax.imageio.spi.ImageInputStreamSpi
0a1,5
> com.twelvemonkeys.imageio.stream.BufferedFileImageInputStreamSpi
> com.twelvemonkeys.imageio.stream.BufferedRAFImageInputStreamSpi
> com.twelvemonkeys.imageio.stream.BufferedInputStreamImageInputStreamSpi
> # Use SPI loading as a hook for early profile activation
> com.twelvemonkeys.imageio.color.ProfileDeferralActivator$Spi
diff '--color=auto' -rbBw MapTool-SNAPSHOT-7d89f48-uberJar/META-INF/services/javax.imageio.spi.ImageReaderSpi MapTool-SNAPSHOT-e421d6e-shadowJar/META-INF/services/javax.imageio.spi.ImageReaderSpi
0a1,141
> com.twelvemonkeys.imageio.plugins.jpeg.JPEGImageReaderSpi
> com.twelvemonkeys.imageio.plugins.psd.PSDImageReaderSpi
> com.twelvemonkeys.imageio.plugins.tiff.TIFFImageReaderSpi
> com.twelvemonkeys.imageio.plugins.tiff.BigTIFFImageReaderSpi
> com.twelvemonkeys.imageio.plugins.svg.SVGImageReaderSpi
> com.twelvemonkeys.imageio.plugins.wmf.WMFImageReaderSpi
> com.twelvemonkeys.imageio.plugins.tga.TGAImageReaderSpi
> #
> # Copyright (c) 2017, Harald Kuhr
> # All rights reserved.
> #
> # Redistribution and use in source and binary forms, with or without
> # modification, are permitted provided that the following conditions are met:
> #
> # * Redistributions of source code must retain the above copyright notice, this
> #   list of conditions and the following disclaimer.
> #
> # * Redistributions in binary form must reproduce the above copyright notice,
> #   this list of conditions and the following disclaimer in the documentation
> #   and/or other materials provided with the distribution.
> #
> # * Neither the name of the copyright holder nor the names of its
> #   contributors may be used to endorse or promote products derived from
> #   this software without specific prior written permission.
> #
> # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
> # AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
> # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
> # DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
> # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
> # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
> # SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
> # CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
> # OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> com.twelvemonkeys.imageio.plugins.bmp.BMPImageReaderSpi
> com.twelvemonkeys.imageio.plugins.bmp.CURImageReaderSpi
> com.twelvemonkeys.imageio.plugins.bmp.ICOImageReaderSpi
> #
> # $RCSfile: javax.imageio.spi.ImageReaderSpi,v $
> #
> # 
> # Copyright (c) 2005 Sun Microsystems, Inc. All  Rights Reserved.
> # 
> # Redistribution and use in source and binary forms, with or without
> # modification, are permitted provided that the following conditions
> # are met: 
> # 
> # - Redistribution of source code must retain the above copyright 
> #   notice, this  list of conditions and the following disclaimer.
> # 
> # - Redistribution in binary form must reproduce the above copyright
> #   notice, this list of conditions and the following disclaimer in 
> #   the documentation and/or other materials provided with the
> #   distribution.
> # 
> # Neither the name of Sun Microsystems, Inc. or the names of 
> # contributors may be used to endorse or promote products derived 
> # from this software without specific prior written permission.
> # 
> # This software is provided "AS IS," without a warranty of any 
> # kind. ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND 
> # WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, 
> # FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT, ARE HEREBY
> # EXCLUDED. SUN MIDROSYSTEMS, INC. ("SUN") AND ITS LICENSORS SHALL 
> # NOT BE LIABLE FOR ANY DAMAGES SUFFERED BY LICENSEE AS A RESULT OF 
> # USING, MODIFYING OR DISTRIBUTING THIS SOFTWARE OR ITS
> # DERIVATIVES. IN NO EVENT WILL SUN OR ITS LICENSORS BE LIABLE FOR 
> # ANY LOST REVENUE, PROFIT OR DATA, OR FOR DIRECT, INDIRECT, SPECIAL,
> # CONSEQUENTIAL, INCIDENTAL OR PUNITIVE DAMAGES, HOWEVER CAUSED AND
> # REGARDLESS OF THE THEORY OF LIABILITY, ARISING OUT OF THE USE OF OR
> # INABILITY TO USE THIS SOFTWARE, EVEN IF SUN HAS BEEN ADVISED OF THE
> # POSSIBILITY OF SUCH DAMAGES. 
> # 
> # You acknowledge that this software is not designed or intended for 
> # use in the design, construction, operation or maintenance of any 
> # nuclear facility. 
> #
> # $Revision: 1.2 $
> # $Date: 2007/09/05 00:21:08 $
> # $State: Exp $
> #
> # --- JAI-Image I/O ImageReader plug-ins ---
> #
> com.github.jaiimageio.jpeg2000.impl.J2KImageReaderSpi
> #
> # $RCSfile: javax.imageio.spi.ImageReaderSpi,v $
> #
> # 
> # Copyright (c) 2005 Sun Microsystems, Inc. All  Rights Reserved.
> # 
> # Redistribution and use in source and binary forms, with or without
> # modification, are permitted provided that the following conditions
> # are met: 
> # 
> # - Redistribution of source code must retain the above copyright 
> #   notice, this  list of conditions and the following disclaimer.
> # 
> # - Redistribution in binary form must reproduce the above copyright
> #   notice, this list of conditions and the following disclaimer in 
> #   the documentation and/or other materials provided with the
> #   distribution.
> # 
> # Neither the name of Sun Microsystems, Inc. or the names of 
> # contributors may be used to endorse or promote products derived 
> # from this software without specific prior written permission.
> # 
> # This software is provided "AS IS," without a warranty of any 
> # kind. ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND 
> # WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, 
> # FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT, ARE HEREBY
> # EXCLUDED. SUN MIDROSYSTEMS, INC. ("SUN") AND ITS LICENSORS SHALL 
> # NOT BE LIABLE FOR ANY DAMAGES SUFFERED BY LICENSEE AS A RESULT OF 
> # USING, MODIFYING OR DISTRIBUTING THIS SOFTWARE OR ITS
> # DERIVATIVES. IN NO EVENT WILL SUN OR ITS LICENSORS BE LIABLE FOR 
> # ANY LOST REVENUE, PROFIT OR DATA, OR FOR DIRECT, INDIRECT, SPECIAL,
> # CONSEQUENTIAL, INCIDENTAL OR PUNITIVE DAMAGES, HOWEVER CAUSED AND
> # REGARDLESS OF THE THEORY OF LIABILITY, ARISING OUT OF THE USE OF OR
> # INABILITY TO USE THIS SOFTWARE, EVEN IF SUN HAS BEEN ADVISED OF THE
> # POSSIBILITY OF SUCH DAMAGES. 
> # 
> # You acknowledge that this software is not designed or intended for 
> # use in the design, construction, operation or maintenance of any 
> # nuclear facility. 
> #
> # $Revision: 1.2 $
> # $Date: 2007/09/05 00:21:08 $
> # $State: Exp $
> #
> # --- JAI-Image I/O ImageReader plug-ins ---
> #
> #com.github.jaiimageio.impl.plugins.jpeg.CLibJPEGImageReaderSpi
> #com.github.jaiimageio.impl.plugins.png.CLibPNGImageReaderSpi
> #com.github.jaiimageio.impl.plugins.jpeg2000.J2KImageReaderSpi
> #com.github.jaiimageio.impl.plugins.jpeg2000.J2KImageReaderCodecLibSpi
> com.github.jaiimageio.impl.plugins.wbmp.WBMPImageReaderSpi
> com.github.jaiimageio.impl.plugins.bmp.BMPImageReaderSpi
> com.github.jaiimageio.impl.plugins.pcx.PCXImageReaderSpi
> com.github.jaiimageio.impl.plugins.pnm.PNMImageReaderSpi
> com.github.jaiimageio.impl.plugins.raw.RawImageReaderSpi
> com.github.jaiimageio.impl.plugins.tiff.TIFFImageReaderSpi
diff '--color=auto' -rbBw MapTool-SNAPSHOT-7d89f48-uberJar/META-INF/services/javax.imageio.spi.ImageWriterSpi MapTool-SNAPSHOT-e421d6e-shadowJar/META-INF/services/javax.imageio.spi.ImageWriterSpi
0a1,139
> com.twelvemonkeys.imageio.plugins.jpeg.JPEGImageWriterSpi
> com.twelvemonkeys.imageio.plugins.psd.PSDImageWriterSpi
> com.twelvemonkeys.imageio.plugins.tiff.TIFFImageWriterSpi
> com.twelvemonkeys.imageio.plugins.tiff.BigTIFFImageWriterSpi
> com.twelvemonkeys.imageio.plugins.tga.TGAImageWriterSpi
> #
> # Copyright (c) 2017, Harald Kuhr
> # All rights reserved.
> #
> # Redistribution and use in source and binary forms, with or without
> # modification, are permitted provided that the following conditions are met:
> #
> # * Redistributions of source code must retain the above copyright notice, this
> #   list of conditions and the following disclaimer.
> #
> # * Redistributions in binary form must reproduce the above copyright notice,
> #   this list of conditions and the following disclaimer in the documentation
> #   and/or other materials provided with the distribution.
> #
> # * Neither the name of the copyright holder nor the names of its
> #   contributors may be used to endorse or promote products derived from
> #   this software without specific prior written permission.
> #
> # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
> # AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
> # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
> # DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
> # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
> # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
> # SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
> # CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
> # OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> com.twelvemonkeys.imageio.plugins.bmp.BMPImageWriterSpi
> com.twelvemonkeys.imageio.plugins.bmp.ICOImageWriterSpi
> #
> # $RCSfile: javax.imageio.spi.ImageWriterSpi,v $
> #
> # 
> # Copyright (c) 2005 Sun Microsystems, Inc. All  Rights Reserved.
> # 
> # Redistribution and use in source and binary forms, with or without
> # modification, are permitted provided that the following conditions
> # are met: 
> # 
> # - Redistribution of source code must retain the above copyright 
> #   notice, this  list of conditions and the following disclaimer.
> # 
> # - Redistribution in binary form must reproduce the above copyright
> #   notice, this list of conditions and the following disclaimer in 
> #   the documentation and/or other materials provided with the
> #   distribution.
> # 
> # Neither the name of Sun Microsystems, Inc. or the names of 
> # contributors may be used to endorse or promote products derived 
> # from this software without specific prior written permission.
> # 
> # This software is provided "AS IS," without a warranty of any 
> # kind. ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND 
> # WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, 
> # FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT, ARE HEREBY
> # EXCLUDED. SUN MIDROSYSTEMS, INC. ("SUN") AND ITS LICENSORS SHALL 
> # NOT BE LIABLE FOR ANY DAMAGES SUFFERED BY LICENSEE AS A RESULT OF 
> # USING, MODIFYING OR DISTRIBUTING THIS SOFTWARE OR ITS
> # DERIVATIVES. IN NO EVENT WILL SUN OR ITS LICENSORS BE LIABLE FOR 
> # ANY LOST REVENUE, PROFIT OR DATA, OR FOR DIRECT, INDIRECT, SPECIAL,
> # CONSEQUENTIAL, INCIDENTAL OR PUNITIVE DAMAGES, HOWEVER CAUSED AND
> # REGARDLESS OF THE THEORY OF LIABILITY, ARISING OUT OF THE USE OF OR
> # INABILITY TO USE THIS SOFTWARE, EVEN IF SUN HAS BEEN ADVISED OF THE
> # POSSIBILITY OF SUCH DAMAGES. 
> # 
> # You acknowledge that this software is not designed or intended for 
> # use in the design, construction, operation or maintenance of any 
> # nuclear facility. 
> #
> # $Revision: 1.2 $
> # $Date: 2007/09/05 00:21:08 $
> # $State: Exp $
> #
> # --- JAI-Image I/O ImageWriter plug-ins ---
> #
> com.github.jaiimageio.jpeg2000.impl.J2KImageWriterSpi
> #
> # $RCSfile: javax.imageio.spi.ImageWriterSpi,v $
> #
> # 
> # Copyright (c) 2005 Sun Microsystems, Inc. All  Rights Reserved.
> # 
> # Redistribution and use in source and binary forms, with or without
> # modification, are permitted provided that the following conditions
> # are met: 
> # 
> # - Redistribution of source code must retain the above copyright 
> #   notice, this  list of conditions and the following disclaimer.
> # 
> # - Redistribution in binary form must reproduce the above copyright
> #   notice, this list of conditions and the following disclaimer in 
> #   the documentation and/or other materials provided with the
> #   distribution.
> # 
> # Neither the name of Sun Microsystems, Inc. or the names of 
> # contributors may be used to endorse or promote products derived 
> # from this software without specific prior written permission.
> # 
> # This software is provided "AS IS," without a warranty of any 
> # kind. ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND 
> # WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, 
> # FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT, ARE HEREBY
> # EXCLUDED. SUN MIDROSYSTEMS, INC. ("SUN") AND ITS LICENSORS SHALL 
> # NOT BE LIABLE FOR ANY DAMAGES SUFFERED BY LICENSEE AS A RESULT OF 
> # USING, MODIFYING OR DISTRIBUTING THIS SOFTWARE OR ITS
> # DERIVATIVES. IN NO EVENT WILL SUN OR ITS LICENSORS BE LIABLE FOR 
> # ANY LOST REVENUE, PROFIT OR DATA, OR FOR DIRECT, INDIRECT, SPECIAL,
> # CONSEQUENTIAL, INCIDENTAL OR PUNITIVE DAMAGES, HOWEVER CAUSED AND
> # REGARDLESS OF THE THEORY OF LIABILITY, ARISING OUT OF THE USE OF OR
> # INABILITY TO USE THIS SOFTWARE, EVEN IF SUN HAS BEEN ADVISED OF THE
> # POSSIBILITY OF SUCH DAMAGES. 
> # 
> # You acknowledge that this software is not designed or intended for 
> # use in the design, construction, operation or maintenance of any 
> # nuclear facility. 
> #
> # $Revision: 1.2 $
> # $Date: 2007/09/05 00:21:08 $
> # $State: Exp $
> #
> # --- JAI-Image I/O ImageWriter plug-ins ---
> #
> #com.github.jaiimageio.impl.plugins.jpeg.CLibJPEGImageWriterSpi
> #com.github.jaiimageio.impl.plugins.png.CLibPNGImageWriterSpi
> #com.github.jaiimageio.impl.plugins.jpeg2000.J2KImageWriterSpi
> #com.github.jaiimageio.impl.plugins.jpeg2000.J2KImageWriterCodecLibSpi
> com.github.jaiimageio.impl.plugins.wbmp.WBMPImageWriterSpi
> com.github.jaiimageio.impl.plugins.bmp.BMPImageWriterSpi
> com.github.jaiimageio.impl.plugins.gif.GIFImageWriterSpi
> com.github.jaiimageio.impl.plugins.pcx.PCXImageWriterSpi
> com.github.jaiimageio.impl.plugins.pnm.PNMImageWriterSpi
> com.github.jaiimageio.impl.plugins.raw.RawImageWriterSpi
> com.github.jaiimageio.impl.plugins.tiff.TIFFImageWriterSpi
diff '--color=auto' -rbBw MapTool-SNAPSHOT-7d89f48-uberJar/META-INF/services/javax.servlet.ServletContainerInitializer MapTool-SNAPSHOT-e421d6e-shadowJar/META-INF/services/javax.servlet.ServletContainerInitializer
0a1
> io.sentry.servlet.SentryServletContainerInitializer
diff '--color=auto' -rbBw MapTool-SNAPSHOT-7d89f48-uberJar/META-INF/services/org.apache.sis.internal.jaxb.TypeRegistration MapTool-SNAPSHOT-e421d6e-shadowJar/META-INF/services/org.apache.sis.internal.jaxb.TypeRegistration
0a1,2
> # Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements.
> org.apache.sis.internal.referencing.ReferencingTypes
diff '--color=auto' -rbBw MapTool-SNAPSHOT-7d89f48-uberJar/META-INF/services/org.apache.sis.storage.DataStoreProvider MapTool-SNAPSHOT-e421d6e-shadowJar/META-INF/services/org.apache.sis.storage.DataStoreProvider
0a1
> org.apache.sis.storage.netcdf.NetcdfStoreProvider

Outside of META-INF/services/, the shadowJar task only keeps one copy of any duplicated file. This is the same as uberJar, but the two don't necessarily pick the same copy. Mainly this happens with many unimportant text files, but I also noticed that there were differences in class files as well. Looking into this, I found that there was a conflict between our dependencies com.github.jai-imageio:jai-imageio-jpeg2000 and edu.ucar:jj2000. The latter comes from from tika-parser, a redistribution of a very old version of a library which we don't actually need (we only use stuff from tika-core). So I switched to using an updated tika-core from org.apache.tika instead, which avoids the conflicts while also reducing our dependency graph by a fair amount (315 -> 242).

Possible Drawbacks

As mentioned, a different set of files are produce by shadowJar vs uberJar. The differences should now be benign, but it is possible I overlooked something important. This can only impact the JAR release, not the other packaged releases.

As far as I can tell, tika-parsers does not accomplish anything for us beyond what tika-core provides. But if I'm wrong, this could negatively impact media type detection, and would affect the JAR release as well as the package releases.

Documentation Notes

JS UDFs can now be used with the standalone JAR.

Release Notes

• Fixed JS UDFs so they are able to be created when running on the standalone JAR release.

---

This change is