Skip to content

Conversation

@bmd3k
Copy link
Contributor

@bmd3k bmd3k commented Mar 16, 2020

  • Motivation for features / changes

Allow uploader users to specify the plugins for which data should be uploaded. It has two-fold purpose:
(1) Allow users to specify "experimental" plugins that would otherwise not be uploaded.
(2) Allow users to list a subset of launched plugins, if they do not want to upload all launched plugin data.

  • Technical description of changes

Add "--plugins" command line option for upload subcommand. The information is sent in the ServerInfoRequest (aka the handshake) and may impact the list of plugins returned in ServerInfoResponse.plugin_control.allowed_plugins.

Sample usage:
tensorboard dev upload --logdir --plugins scalars graphs histograms

Outside of these specific changes:

It's expected that supported servers will evaluate the list of plugins and decide whether it is valid. If valid, the server will respond with the entire list of plugins or perhaps a sublist. If the list is invalid then it will respond with a CompatibilityVerdict of VERDICT_ERROR and a useful detail message to print to console.

If --plugins is not specified then the server is expected to respond with a default list of plugins to upload.

@bmd3k bmd3k marked this pull request as ready for review March 16, 2020 19:36
@bmd3k bmd3k requested review from caisq and wchargin March 16, 2020 19:36
Copy link
Contributor

@wchargin wchargin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, given that we’re deferring --exclude_plugins or equivalent
to a future change.

nit: OSS commit style is imperative with no trailing punctuation with
optional section prefix, so: “Add --plugins option to uploader”, or:
“uploader: add --plugins option”. Could you please update the PR title
accordingly?

* Why? For consistency; this is the style of commits generated by Git
itself, as in git revert, git merge, etc.

@bmd3k bmd3k changed the title Uploader support for "--plugins" option. Add --plugins option to uploader Mar 17, 2020
bmd3k added 5 commits March 17, 2020 07:39
Fix other commands broken by --plugins implementation.
* Clarify type of upload_plugins arguments.
* Log request.
* --plugins flag now has default value of []. Can never be none.
* create_server_info implementation reflects this, too.
@bmd3k bmd3k requested a review from wchargin March 17, 2020 13:19
Copy link
Contributor

@caisq caisq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM modulo my comments.

Copy link
Contributor

@wchargin wchargin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great; thank you!

@bmd3k bmd3k merged commit 343456b into tensorflow:master Mar 17, 2020
@wchargin
Copy link
Contributor

This breaks tensorboard dev list. Let’s revert for the 2.2.0 release.
I’ll send a PR. (cc @bileschi as release owner)

$ bazel run //tensorboard -- dev list 2>&1 | san
Loading: 
Loading: 0 packages loaded
Analyzing: target //tensorboard:tensorboard (0 packages loaded, 0 targets configured)
INFO: Analyzed target //tensorboard:tensorboard (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
[0 / 2] [Prepa] BazelWorkspaceStatusAction stable-status.txt
Target //tensorboard:tensorboard up-to-date:
  bazel-bin/tensorboard/tensorboard
INFO: Elapsed time: 0.295s, Critical Path: 0.00s
INFO: 0 processes.
INFO: Build completed successfully, 1 total action
INFO: Running command line: bazel-bin/tensorboard/tensorboard dev list
INFO: Build completed successfully, 1 total action
WARNING:root:Limited tf.compat.v2.summary API due to missing TensorBoard installation.
WARNING:root:Limited tf.compat.v2.summary API due to missing TensorBoard installation.
WARNING:root:Limited tf.compat.v2.summary API due to missing TensorBoard installation.
WARNING:root:Limited tf.summary API due to missing TensorBoard installation.
Traceback (most recent call last):
  File "/HOMEDIR/.cache/bazel/_bazel_wchargin/52a95bbdd50941251730eb33b7476a66/execroot/org_tensorflow_tensorboard/bazel-out/k8-fastbuild/bin/tensorboard/tensorboard.runfiles/org_tensorflow_tensorboard/tensorboard/main.py", line 88, in <module>
    run_main()
  File "/HOMEDIR/.cache/bazel/_bazel_wchargin/52a95bbdd50941251730eb33b7476a66/execroot/org_tensorflow_tensorboard/bazel-out/k8-fastbuild/bin/tensorboard/tensorboard.runfiles/org_tensorflow_tensorboard/tensorboard/main.py", line 75, in run_main
    app.run(tensorboard.main, flags_parser=tensorboard.configure)
  File "/VIRTUAL_ENV/lib/python3.7/site-packages/absl/app.py", line 299, in run
    _run_main(main, args)
  File "/VIRTUAL_ENV/lib/python3.7/site-packages/absl/app.py", line 250, in _run_main
    sys.exit(main(argv))
  File "/HOMEDIR/.cache/bazel/_bazel_wchargin/52a95bbdd50941251730eb33b7476a66/execroot/org_tensorflow_tensorboard/bazel-out/k8-fastbuild/bin/tensorboard/tensorboard.runfiles/org_tensorflow_tensorboard/tensorboard/program.py", line 289, in main
    return runner(self.flags) or 0
  File "/HOMEDIR/.cache/bazel/_bazel_wchargin/52a95bbdd50941251730eb33b7476a66/execroot/org_tensorflow_tensorboard/bazel-out/k8-fastbuild/bin/tensorboard/tensorboard.runfiles/org_tensorflow_tensorboard/tensorboard/uploader/uploader_main.py", line 797, in run
    return _run(flags)
  File "/HOMEDIR/.cache/bazel/_bazel_wchargin/52a95bbdd50941251730eb33b7476a66/execroot/org_tensorflow_tensorboard/bazel-out/k8-fastbuild/bin/tensorboard/tensorboard.runfiles/org_tensorflow_tensorboard/tensorboard/uploader/uploader_main.py", line 312, in _run
    server_info = _get_server_info(flags)
  File "/HOMEDIR/.cache/bazel/_bazel_wchargin/52a95bbdd50941251730eb33b7476a66/execroot/org_tensorflow_tensorboard/bazel-out/k8-fastbuild/bin/tensorboard/tensorboard.runfiles/org_tensorflow_tensorboard/tensorboard/uploader/uploader_main.py", line 750, in _get_server_info
    server_info = server_info_lib.fetch_server_info(origin, flags.plugins)
AttributeError: 'Namespace' object has no attribute 'plugins'

wchargin added a commit that referenced this pull request Mar 19, 2020
This reverts commit 343456b.

Test Plan:
Running `bazel run //tensorboard -- dev list` now works. Previously, it
failed with:

```
  File ".../tensorboard/uploader/uploader_main.py", line 750, in _get_server_info
    server_info = server_info_lib.fetch_server_info(origin, flags.plugins)
AttributeError: 'Namespace' object has no attribute 'plugins'
```

wchargin-branch: revert-uploader-plugins-flag
wchargin added a commit that referenced this pull request Mar 20, 2020
This reverts commit 343456b.

Test Plan:
Running `bazel run //tensorboard -- dev list` now works. Previously, it
failed with:

```
  File ".../tensorboard/uploader/uploader_main.py", line 750, in _get_server_info
    server_info = server_info_lib.fetch_server_info(origin, flags.plugins)
AttributeError: 'Namespace' object has no attribute 'plugins'
```

wchargin-branch: revert-uploader-plugins-flag
@bmd3k bmd3k deleted the upload-plugins branch March 20, 2020 11:26
bmd3k added a commit to bmd3k/tensorboard that referenced this pull request Mar 20, 2020
…ensorflow#3400)"

This reverts commit 2b2a976.
Subsequent commits will fix the issue that caused the original
revert.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants