TFDV and TFMA visualizations in KFP UI?

[schmidt-jake]

Are there yaml component specifications that can do this already? Is AI Hub the only repo for pipelines/components? I don't know how to write the visual outputs/widgets from TFDV/TFMA to HTML for display in the KFP UI via writing to mlpipeline-ui-metadata.json as per the tutorial, and the links on that page to the example code are broken.

[andrewsmartin]

@JakeTheWise here's the commit that removed those files causing the broken links, hopefully this can set you in the right direction. 86163ec#diff-ef180dd8899f69cbf7b44c78a23e042eL196-L221

[ajchili]

Hey @JakeTheWise, I recently implemented support for Python Based Visualizations within KFP. This method of visualizing data allows for TFDV/TFMA and other Python visualization libraries to be easily used within the KFP UI without extensive knowledge of HTML or how to export visualizations. This might be the best option for your use case, you can checkout my post about them on Medium or the documentation within KFP here for details on how to get started. As a heads up, KFP needs to be at version 0.1.29 or later. Please let me know if you have any questions about the feature!

[Ark-kun]

Are there yaml component specifications that can do this already?

Yes, that should be possible. It's just that the outputs need to be explicit now.

pipelines/components/local/confusion_matrix/component.yaml

Line 20 in 9a9bd90

MLPipeline UI metadata: /mlpipeline-ui-metadata.json

The component.yaml just describes the component inputs and outputs and how to call the program. That spec does not care about the contents of those output files. Only the frontend does.

Is AI Hub the only repo for pipelines/components?

This repo is more up to date usually.

as per the tutorial, and the links on that page to the example code are broken.

We're sorry about this. I'll create an issue to fix those.

here's the commit that removed those files

BTW, Just because those files have been removed does not mean that those components should stop working. If they no longer work I'd consider that a bug that needs to be fixed.

Do you see any visualizations working at all? Including the 1st-party components from this repo.

[schmidt-jake]

Thanks for the help, everyone! Using the old commit provided by @andrewsmartin, I was able to make TFDV display properly in the KFP UI. However, I've only had partial success with TFMA. It displays incorrectly (the slice histograms don't display properly), and upon inspection it seems there is a CORS issue with a javascript file that TFMA uses. Here's the seemingly troublesome piece of HTML I'm rendering for the TFMA viz:

<link rel="import" href="https://cdn.rawgit.com/tensorflow/model-analysis/v0.6.0/tensorflow_model_analysis/static/vulcanized_template.html">

And here's the error I get in the browser console when inspecting the KFP UI:

Access to imported resource at '....cloud.goog/pipeline/nullnbextensions/tfma_widget_js//vulcanized_template.html' from origin 'null' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.

I tried adding crossorigin="anonymous" to the link tag but that didn't help. I'm quite unfamiliar with CORS (and frontend dev in general), so I plan to research this issue further, but if anyone has already encountered or solved this issue, it would be sweet to hear from you.

[schmidt-jake]

The source of the issue seems to be documented here tensorflow/model-analysis#10. Has anyone figured out how to resolve this issue?

[ajchili]

Hey @JakeTheWise, I investigated something similar in tensorflow/data-validation#80, where visualizations could not be rendered outside of a Jupyter Notebook generated environment. After a very quick look at the issue you provided it does not seem like the same approach can be provided. I will continue to investigate as I have time and keep you informed.

[Bobgy]

/assign @jingzhang36
Is this something you already tried or fixed?

[Bobgy]

Two changes needed to support this better:

1. integrate tfx tfma visualization TFMA auto-visualization for TFX components in KFP #3111
2. Improve loading speed of the hack by fixing KFP UI infinite redirection [UI] TFMA visualization loads slowly #3114

Users can use custom visualization as mentioned in tensorflow/model-analysis#10 (comment) now (but there's only one tfma version support).

[Bobgy]

p0 issues solved

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically closed because it has not had recent activity. Please comment "/reopen" to reopen it.