Upgrade to Bazel 2.0.0

[nicknezis]

Update the Bazel build scripts to use this new capability: https://github.com/bazelbuild/rules_jvm_external

The goal would be to better manage the project's dependencies and not have to manually update the various transitive dependencies. This tool will use a dependency's pom.xml to dynamically generate the Bazel targets for each of the dependencies.

This approach started with the buildozer migration which did a lot of formatting cleanup, but ultimately didn't work. After the buildozer cleanup, I had to manually update the WORKSPACE and BUILD files to use the new maven_install feature.

[nicknezis]

I made a new pull request to better describe some of the issues I was running into when trying to use this feature. I'm able to get things working at this point as shown by the green CI build.

When I try to Pin the list of dependencies as described here, I run into my first issue. It complains about needing an attribute netrc which doesn't exist in Bazel 0.26.0. I can solve this issue by upgrading to 0.29.1, but it introduces other issues. I'm working on fixing those issues and trying to update things to the latest Bazel. As I dig into the issues to find fixes, many of the issues are actually due to the old Bazel and old OS versions.

[nicknezis]

An example of an odd issue I've run into when trying to use newer Bazel versions... I get errors about bdist_wheel not being a valid command. I couldn't find where in the Bazel files the wheel dependency was being installed. I thought it would be in the pex_rules BUILD file. It wasn't until later that I stumbled across the Travis CI config file which is actually doing a pip install wheel. If this is a dependency we should properly call it out. I'm not sure if the issue is only now impacting me because of the combination of Bazel upgrade and Python 3 upgrade. Either way I will try to resolve these issues, but wanted to give an example of the types of issues I'm running into.

[nwangtw]

Thanks!

Yeah. Wheel was installed and then used as a python tool instead of a library. I remember it is used to build platform independent CLI binary although I am not 100% sure. It could be nice if we can avoid the pip install but it might not be really necessary. Your call and whichever is easier should be good. :)

[nicknezis]

@nwangtw Yes once I saw the pip install wheel in the Travis CI config, that helped a lot. I was not able to build in the Docker containers and thought it was something I had broken with the newer Bazel. I'm thinking the yum or apt packages might be more consistent way to install the pip and wheel dependencies because it will be easier to keep the containers and travis setup in sync.

[Code0x58]

One thing that springs to mind looking at this PR, is that adding a Bazel linting/formatting stage to travis could be a good idea. That whay buildozer hopefully won't be responsible for such large diffs when it is used

[nicknezis]

@Code0x58 That's a great idea! I'm currently stumped with Python tests failing. It seems they are not able to find the protobuf generated python code. I assume I broke something in my edits.

[nicknezis]

Closing this. The work has moved to #3510