Polymer support

[hochhaus]

With the pending release of polymer 2.0 (built upon Shadow DOM v1 & Custom Elements v1) the polymer project is looking interesting. The closure tools blog has a series of posts (1, 2, 3, 4) about using closure and Polymer together. Also, the gerrit code review tool, as part of the PolyGerrit UI rewrite, has started adding bazel support for polymer build primitives (cripser and vulcanize). The polymer ecosystem currently integrates with several other build systems, however ideally first class bazel support would provided in a central location. If I had to guess, polymer, closure and bazel all play very well together inside Google. Possibly that code can't be easily open sourced. I thought I would mention it though before multiple different implementations of partially baked bazel support rules for polymer start popping up.

@jart What are your thoughts about having rules_closure natively support polymer? Is it out of scope?

[jart]

Making Closure Rules support Polymer is on the agenda. A webfiles rule is available at HEAD that was introduced by 5a8d6df twelve days ago. Check out jart/tensorflow@46574c9 for a sneak preview of how it's going to be used. Please be advised that I do not recommend webfiles for production quite yet (or Closure Rules at HEAD for that matter.) But it will be ready in the next few months. My primary assignment at Google right now is to bring order to TensorBoard builds, and TensorBoard is written in Polymer.

Polymer v2 is a big improvement over v1. I like that they're using real ES6 classes now, instead of that weird Polymer({...}) DSL that caused us to need --polymer_pass. I think there are nice things about Polymer. Once understood, it allows the developer to create contemporary looking UIs with very little effort. It also abstracts many patterns which are common in business applications.

But Polymer makes a lot of tradeoffs. It radically changes everything about how web development has been done for decades. It turns many Google best practices on its head, e.g. not writing code in HTML files, not copying vendor components into your codebase, not launching an autonomous open source project for a three line HTML file, etc. It requires you to use tools like vulcanize which have thousands of transitive NPM dependencies written by random people. It's difficult and non-obvious how to augment behaviors Polymer abstracts. It's very heavyweight and only natively supported by Chrome. It makes tooling such as Chrome debugger less useful. If one has greater dev resources, and the developers in question are highly experienced, then there is simply no comparison between Polymer and the Closure stack in terms of performance, safety, and maintainability.

That said, the build work we're doing is going to make Polymer much better and help it deliver on its promises of componentization in a remarkable way. We're planning to rewrite tools like vulcanize from scratch in Java so we don't need node and can leverage the Closure JavaScript and Stylesheets compilers for compile-time safety and optimization. It's going to be great. The build graph is going to be intelligent so JS can be written in its own files or in HTML. But it's not going to be easy.

[hochhaus]

Thanks for sharing all the context and your experience! Your plans sound amazing. The java rewrite of vulcanize is much better than what I was thinking. Avoiding the npm dependency mess would be outstanding. :)

Thanks for all your work on this repository! This project keeps becoming more and more impressive. Please let me know if I can do anything to help.

One other question. As I've started playing with web components, I'm a big fan of shadow dom and custom elements. However, I have a strong distaste for html templates. They seem like such a big step backwards from idom soy templates. Therefore, for components that I write myself, I was planning on skipping html templates all together and using shadow dom + custom element + idom soy. Based on your experience does this seem like a good or bad idea?

[jart]

I think you have good judgement and nothing about what you said sounds like a bad idea. Soy is pretty good and is the one place where Google has really put the most thought into preventing things like XSS attacks. It's not worth abandoning if you understand how to use it.

[davido]

Also, the gerrit code review tool, as part of the PolyGerrit UI rewrite, has started adding bazel support for polymer build primitives (cripser and vulcanize)

Closure compiler was integrated in gerrit code review bazel build tool chain: [1]. However, only white space mode works currently, because of google/closure-compiler#2042.

• [1] https://gerrit-review.googlesource.com/104504

[davido]

It turns out, that with whitespace optimization level es5_runtime isn't injected per default. Addressed here: [1].

• [1] https://bugs.chromium.org/p/gerrit/issues/detail?id=6110

[hochhaus]

@jart Could you provide a status update on polymer and rules_closure? The previous points you mentioned that are most interesting from my perspective are:

• Making Closure Rules support Polymer is on the agenda.
• We're planning to rewrite tools like vulcanize from scratch in Java so we don't need node and can leverage the Closure JavaScript and Stylesheets compilers for compile-time safety and optimization.
• The build graph is going to be intelligent so JS can be written in its own files or in HTML.

Any other points you are willing to share are appreciated as well. Lastly, do you have pointers to where portions of this code lives in open source repos (eg: tensorboard)?

[jart]

I wasn't able to attract the support within Google to generalize as much as I was hoping.

However, you can get pretty reasonable Polymer + TypeScript + Closure + Protobuf + Bazel support out of the box, by depending on TensorBoard's build system, and using tf_web_library.

This is an example of how to use TensorBoard's build as a dependency: https://github.com/tensorflow/tensorboard-plugin-example Please note that repo is pinned a few months behind. I'd greatly appreciate a PR that bumps it up to TensorBoard HEAD.

[jart]

Here are the main pieces of code to focus on, which explain what I mean:

Your WORKSPACE file would look like that.

Your BUILD files would look like that.

You use tensorboard_html_binary to vulcanize.

You use tensorboard_zip_file to package / distribute / deploy your web server files.

We already arduously defined, within TensorBoard's build system, probably at least half of the WORKSPACE definitions you're going to need, e.g. Polymer, Node, common libraries, etc. They're all defined using highly-available geographically redundant mirrors, out-of-the-box.

The Skylark code that makes these things possible is available under tensorboard/defs/.

[jart]

To give another example, Facets is another Google Research project that uses Polymer. It was able to open source rather quickly by depending on TensorBoard's build system. (Note: TensorBoard's build depends on rules_closure. There's no competition going on here.) They were even kind enough to give a shout out in their blog announcement.

[jart]

I'll also note that everything that's been baked into these builds, has received some vetting from myself and possibly also security folks. Please note that this can't be interpreted as a promise. Please see the Apache 2.0 license to learn more. Also, motivational reading: https://hackernoon.com/im-harvesting-credit-card-numbers-and-passwords-from-your-site-here-s-how-9a8cb347c5b5

[davido]

@jart Do you think gerrit code review could use rules_closure with tensorboard rules to use advanced optimization level on Polymer 1? We are using still WHITESPACE_ONLY mode, unfortunately.

https://github.com/GerritCodeReview/gerrit/blob/master/polygerrit-ui/app/rules.bzl#L17

[jart]

Right now TensorBoard skips Closure Compiler entirely and expresses its JS dependency relationships as <script> tags in HTML imports, specifically because Polymer 1.o is the way it is. A friend of mine on the Polymer team informed me a year ago that they do consider Closure Compiler support in Polymer 2 a priority. I haven't yet had time to delve into Polymer 2 so I'm unable to comment further.

One thing I will note is that, due to the flexibility of Bazel, you can write your Closure heavy code using closure_js_library and closure_js_binary. Then put the final resulting .min.js file as a srcs to your tf_web_library. But if you want to run Polymer({...}) class definitions through Closure, be prepared for pain.

[jart]

However, please note, the TensorBoard toolchain does support using Closure Compiler in "simple" mode with Polymer and it's baked into the Vulcanization process, and it does indeed work on Polymer 1. It's just that "simple" mode doesn't give us the full benefits of the Closure Compiler, and it takes a while to compile. TensorBoard currently doesn't require that level of minimization, which is the only reason why we skip it right now.

Source code: https://github.com/tensorflow/tensorboard/blob/master/tensorboard/java/org/tensorflow/tensorboard/vulcanize/Vulcanize.java

[jart]

Further note: You say tensorboard_html_binary(compiler = 1) to get it to run Closure. https://github.com/tensorflow/tensorboard/blob/master/tensorboard/defs/vulcanize.bzl#L110 Please note there's a lot of hard-coding done in that Java file. It's pretty generalized, but not 100%.