HParam table view sort not working

[patzm]

Environment information

Diagnostics

Diagnostics output

--- check: autoidentify
INFO: diagnose_tensorboard.py version 66d35fe98ca66dc3a5ae600631a8aa6bce785bc5

--- check: general                                                                                     
INFO: sys.version_info: sys.version_info(major=3, minor=6, micro=9, releaselevel='final', serial=0)    
INFO: os.name: posix                                                                                   
INFO: os.uname(): posix.uname_result(sysname='Linux', nodename='tensorboard', release='5.0.0-1026-gcp', version='#27~18.04.1-Ubuntu SMP Fri Nov 15 07:40:39 UTC 2019', machine='x86_64')
INFO: sys.getwindowsversion(): N/A                                                                     
                                                                                                       
--- check: package_management                                                                          
INFO: has conda-meta: False                                                                            
INFO: $VIRTUAL_ENV: None                                                                               
                                                                                                       
--- check: installed_packages
INFO: installed: tensorboard==2.0.0
INFO: installed: tensorflow==1.13.1
INFO: installed: tensorflow-estimator==1.13.0

--- check: tensorboard_python_version
INFO: tensorboard.version.VERSION: '2.0.0'

--- check: tensorflow_python_version
/home/patzm/.local/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/patzm/.local/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:527: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/patzm/.local/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:528: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/patzm/.local/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:529: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/patzm/.local/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:530: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/patzm/.local/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:535: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
INFO: tensorflow.__version__: '1.13.1'
INFO: tensorflow.__git_version__: "b'v1.13.1-0-g6612da8951'"

--- check: tensorboard_binary_path
INFO: which tensorboard: b'/home/patzm/.local/bin/tensorboard\n'

--- check: addrinfos
socket.has_ipv6 = True
socket.AF_UNSPEC = <AddressFamily.AF_UNSPEC: 0>
socket.SOCK_STREAM = <SocketKind.SOCK_STREAM: 1>
socket.AI_ADDRCONFIG = <AddressInfo.AI_ADDRCONFIG: 32>
socket.AI_PASSIVE = <AddressInfo.AI_PASSIVE: 1>
Loopback flags: <AddressInfo.AI_ADDRCONFIG: 32>
Loopback infos: [(<AddressFamily.AF_INET6: 10>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('::1', 0, 0, 0)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 0))]
Wildcard flags: <AddressInfo.AI_PASSIVE: 1>                                                            
Wildcard infos: [(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('0.0.0.0', 0)), (<AddressFamily.AF_INET6: 10>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('::', 0, 0, 0))]

--- check: readable_fqdn
INFO: socket.getfqdn(): 'tensorboard.c.tensorflow-training-176320.internal'

--- check: stat_tensorboardinfo
INFO: directory: /tmp/.tensorboard-info                                                                
INFO: os.stat(...): os.stat_result(st_mode=16895, st_ino=145168, st_dev=2049, st_nlink=2, st_uid=1005, st_gid=1006, st_size=4096, st_atime=1576537836, st_mtime=1576576519, st_ctime=1576576519)
INFO: mode: 0o40777

--- check: source_trees_without_genfiles
INFO: tensorboard_roots (1): ['/home/patzm/.local/lib/python3.6/site-packages']; bad_roots (0): []

--- check: full_pip_freeze

netifaces==0.10.4
numpy==1.17.2
oauthlib==2.0.6
PAM==0.4.2
parso==0.5.1
pbr==5.1.1
pip==9.0.1
pluggy==0.12.0
protobuf==3.10.0
pyasn1==0.4.2
pyasn1-modules==0.2.1
pycrypto==2.6.1
pygobject==3.26.1
PyJWT==1.5.3
pynvim==0.3.2
pyOpenSSL==17.5.0
pyserial==3.4
python-apt==1.6.4
python-debian==0.1.32
python-jsonrpc-server==0.2.0
python-language-server==0.28.1
pyxdg==0.25
PyYAML==3.12
requests==2.18.4
requests-unixsocket==0.1.5
SecretStorage==2.3.1
service-identity==16.0.0
setuptools==41.4.0
six==1.12.0
ssh-import-id==5.7
stevedore==1.30.0
systemd-python==234
tensorboard==2.0.0
tensorflow==1.13.1
tensorflow-estimator==1.13.0
termcolor==1.1.0
Twisted==17.9.0
ufw==0.36
unattended-upgrades==0.1
urllib3==1.22
virtualenv==16.1.0
virtualenv-clone==0.4.0
virtualenvwrapper==4.8.2
Werkzeug==0.16.0
wheel==0.33.6
zipp==0.5.2
zope.interface==4.3.2

For browser-related issues, please additionally specify:

• Firefox 71.0 on Manjaro Linux 64 bit
• Google Chrome Version 79.0.3945.79 (Official Build) (64-bit)

Issue description

In the Table View

sorting only works sporadically. Most of the time, it doesn't work at all, other times, sorting is applied to the previously selected column. This seems to be independent of whether you first select the sorting direction and then the sort by drop down, or otherwise round. This seems to be independent of the number of trials. Currently, I observe that for around ~50 runs.
On the server side that hosts the TensorBoard, I don't observe CPU core utilization or RAM saturation.

[rmothukuru]

@patzm,
Can you please let us know if you have set the Direction (Ascending or Descending) as well. If the issue still persists, can you please share the Screenshot, which demonstrates the issue. Thanks!

[patzm]

@rmothukuru, yes I am using the Direction drop down menu to select one of the two sorting direction choices.

Selection of sorting column and direction:

The very same column in the table view:

As you can see, these values are clearly not sorted. Sadly, I can't share the whole screenshot. If this is sufficient, I would like to avoid the effort of writing a reproducible script. Hopefully, this already helps you to narrow down the issue.

I refreshed the site as well with the browser refresh button, and the TensorBoard refresh trigger (upper right). Neither of which helped. Also, I tried various versions of TensorBoard: 1.15.0, 2.0.0, 2.0.2, 2.1.0. It is also noteworthy, that the model_dirs (of each trial) are quite huge. TensorBoard uses ~20GB of RAM for those ~50 trials.

[wchargin]

Hi @patzm! Thanks for the report and detailed background information.
I’m trying to reproduce this on a dataset with 50 runs, and having
trouble; the sorting seems to be working fine for me in both Chrome and
Firefox, on 64-bit Debian-based Linux.

I’m assuming from your screenshot that for each trial you have runs
named eval and train (or something), each with accuracy/top_1
summaries. Here’s the script that I’m using to generate test data:

# Context: <https://github.com/tensorflow/tensorboard/issues/3041>

import os
import random

import tensorflow as tf
from tensorboard.plugins.hparams import api as hp

HP_LR = hp.HParam("learning_rate", hp.RealInterval(0.01, 0.1))
HP_OPTIMIZER = hp.HParam("optimizer", hp.Discrete(["adam", "sgd"]))

SESSIONS = ("train", "eval")

METRIC_LOSS = "loss"
METRIC_ACC1 = "accuracy/top_1"
METRIC_ACC5 = "accuracy/top_5"

NUM_RUNS = 50
NUM_STEPS = 10
BASE_LOGDIR = "logs"


def main():
    rng = random.Random(0)

    for i in range(NUM_RUNS):
        session_dir = os.path.join(BASE_LOGDIR, "%03d" % i)
        with tf.summary.create_file_writer(session_dir).as_default():
            hp.hparams(
                {h: h.domain.sample_uniform(rng) for h in (HP_LR, HP_OPTIMIZER)}
            )
        for session in SESSIONS:
            logdir = os.path.join(session_dir, session)
            with tf.summary.create_file_writer(logdir).as_default():
                for step in range(NUM_STEPS):
                    tf.summary.scalar(METRIC_LOSS, rng.random(), step=step)
                    tf.summary.scalar(METRIC_ACC1, rng.random(), step=step)
                    tf.summary.scalar(METRIC_ACC5, rng.random(), step=step)


if __name__ == "__main__":
    main()

Changing the sort key and direction always seems to work fine for me.

Can you check and see if the data generated by that script works in your
environment? If not, we can try to track down the environmental
difference. If so, we’ll probably want to get your example down to a
minimal repro that you can share.

FWIW, I notice from your diagnostics that you’re running TensorBoard 2.0
with TensorFlow 1.13. This isn’t an officially supported configuration
(TensorBoard and TensorFlow need to have the same minor version), so you
could try upgrading TensorFlow (1.15 should be compatible with 1.13, and
then you can test in TensorBoard 1.15), but I would be a bit surprised
if that were the root cause here.

[wchargin]

(NB: I just edited the script in the above comment.)