-
Notifications
You must be signed in to change notification settings - Fork 594
Update Bookkeeper to 4.13.0 and Zookeeper to 3.6.3 #3692
Conversation
Converting to draft because I want to do some additional testing in Kubernetes. Will report back soon. |
…ubator-heron into nicknezis/bookkeeper-update-2
This was the error and resulting fix I found that explains why I had to update Guava library. |
I was able to successfully submit a topology to a Helm installed Heron cluster running in Kubernetes. So the DLog upload/download should be fine. |
|
||
git_repository( | ||
name = "com_github_johnynek_bazel_jar_jar", | ||
commit = "171f268569384c57c19474b04aebe574d85fde0d", # Latest commit SHA as at 2019/02/13 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like the comment with the date. +1 to you
This is great. Thanks! |
Adding this for historical context. This merge request was created to resolve this Bookkeeper issue: apache/bookkeeper#2302 |
Update Bookkeeper to 4.13.0 and Zookeeper to 3.6.3.
In testing, the Bookkeeper shaded jar was missing necessary libraries, because of this, I moved from using the shaded jar to the standard dependencies. Our Bazel logic should resolve the transitive dependencies, but we still need to make sure the transitives are listed or exported on any java_library or java_binary tasks that use code from the transitive.
Also in updating to newer Zookeeper, their build system has changed to leverage Maven. The Jute IDL generates code. This used to be run by Ant scripts, but now is Maven. Because some of our images are older, the provided version of Maven was too old to run properly. I was able to use the older Maven to build a newer Maven binary which did then work. But I was not able to successfully integrate with our Bazel build system. Instead I generated the code from the IDL which should be platform agnostic. The tradeoff being that any new updates to Zookeeper will need an update to the generated source. If the Zookeeper team provides a package that provides the C library in an easily compiled process, we can transition to that. But for now this seemed like a good compromise.
The update to Zookeeper needed a bump to Apache Curator from 2.9.0 to 5.1.0. Curator is a library that assists with some of the more mundane and complex Zookeeper logic.
Netty was also updated from 4.1.22 to 4.1.50.
Also in the course of doing this update, I had trouble with the
jarjar
sonatype plugin. I switched to the Bazel rules provided by johnynek/bazel_jar_jar.