ANTLR library update

[bpintea]

*QL modules share the ANTLR library that the Painless module uses. The used library is currently at version 4.5.3 (released 5+ years ago).
Some improvement work in the parsing error reporting area in QL is considered, which however requires some improvements brought in 4.6 (besides apparently other "major performance improvement[s]") and 4.7.2 releases.
The issue proposes to upgrade the library either to the latter version above or potentially latest stable.

[elasticmachine]

Pinging @elastic/es-core-infra (Team:Core/Infra)

[bpintea]

To exemplify the issue with loading own ANTLR library in the SQL: the module lists the library as a dependency. Changing its version to a newer one (and listing org.antlr in the exclusion list for :modules:lang-painless dependency) works, it leads to a successful build.
However, starting ES fails due to a jar hell exception[0].

The dependency is used to implement org.elasticsearch.painless.spi.PainlessExtension, to export SQL's context whitelists, the module extending painlessț SPI.

[0]:

[2021-08-05T21:40:38,643][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [maclastic.local] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: failed to load plugin x-pack-sql due to jar hell
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:171) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:158) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:75) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:116) ~[elasticsearch-cli-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
        at org.elasticsearch.cli.Command.main(Command.java:79) ~[elasticsearch-cli-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:123) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:81) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
Caused by: java.lang.IllegalStateException: failed to load plugin x-pack-sql due to jar hell
        at org.elasticsearch.plugins.PluginsService.checkBundleJarHell(PluginsService.java:571) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
        at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:432) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
        at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:153) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
        at org.elasticsearch.node.Node.<init>(Node.java:333) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
        at org.elasticsearch.node.Node.<init>(Node.java:266) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
        at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:225) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:225) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:365) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:167) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
        ... 6 more
Caused by: java.lang.IllegalStateException: jar hell!
class: org.antlr.v4.runtime.ANTLRErrorListener
jar1: /Volumes/RAMDisk/distribution/archives/darwin-tar/build/install/elasticsearch-8.0.0-SNAPSHOT/modules/x-pack-sql/antlr-4.6-complete.jar
jar2: /Volumes/RAMDisk/distribution/archives/darwin-tar/build/install/elasticsearch-8.0.0-SNAPSHOT/modules/lang-painless/antlr4-runtime-4.5.3.jar
        at org.elasticsearch.jdk.JarHell.checkClass(JarHell.java:276) ~[elasticsearch-core-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
        at org.elasticsearch.jdk.JarHell.checkJarHell(JarHell.java:184) ~[elasticsearch-core-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
        at org.elasticsearch.plugins.PluginsService.checkBundleJarHell(PluginsService.java:557) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
        at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:432) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
        at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:153) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
        at org.elasticsearch.node.Node.<init>(Node.java:333) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
        at org.elasticsearch.node.Node.<init>(Node.java:266) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
        at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:225) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:225) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:365) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:167) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
        ... 6 more
uncaught exception in thread [main]
java.lang.IllegalStateException: failed to load plugin x-pack-sql due to jar hell
Likely root cause: java.lang.IllegalStateException: jar hell!
class: org.antlr.v4.runtime.ANTLRErrorListener
jar1: /Volumes/RAMDisk/distribution/archives/darwin-tar/build/install/elasticsearch-8.0.0-SNAPSHOT/modules/x-pack-sql/antlr-4.6-complete.jar
jar2: /Volumes/RAMDisk/distribution/archives/darwin-tar/build/install/elasticsearch-8.0.0-SNAPSHOT/modules/lang-painless/antlr4-runtime-4.5.3.jar

[rjernst]

Depending on all of painless should not be necessary. Painless provides a separate jar that contains all of the whitelist code. Can you please try changing the dependency to that (no exclusion necessary):

compileOnly(project(':modules:lang-painless:spi'))