Too many warnings printed with TF 2.X

[vandanavk]

While executing tests that involve reductions, too many warnings are being printed.

Example

tests/tensorflow2/test_keras_mirrored.py::test_base_reductions INFO:absl:Load dataset info from /root/tensorflow_datasets/mnist/3.0.1
--
587 | INFO:absl:Field info.citation from disk and from code do not match. Keeping the one from code.
588 | INFO:absl:Reusing dataset mnist (/root/tensorflow_datasets/mnist/3.0.1)
589 | INFO:absl:Constructing tf.data.Dataset for split None, from /root/tensorflow_datasets/mnist/3.0.1
590 | INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:GPU:0', '/job:localhost/replica:0/task:0/device:GPU:1', '/job:localhost/replica:0/task:0/device:GPU:2', '/job:localhost/replica:0/task:0/device:GPU:3')
591 | INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:GPU:0', '/job:localhost/replica:0/task:0/device:GPU:1', '/job:localhost/replica:0/task:0/device:GPU:2', '/job:localhost/replica:0/task:0/device:GPU:3')
592 | [2020-05-27 07:15:21.755 ae4bb918ce14:10191 INFO hook.py:191] tensorboard_dir has not been set for the hook. SMDebug will not be exporting tensorboard summaries.
593 | [2020-05-27 07:15:21.755 ae4bb918ce14:10191 INFO hook.py:236] Saving to /tmp/test
594 | [2020-05-27 07:15:21.755 ae4bb918ce14:10191 INFO state_store.py:67] The checkpoint config file /opt/ml/input/config/checkpointconfig.json does not exist.
595 | [2020-05-27 07:15:22.897 ae4bb918ce14:10191 INFO keras.py:68] Executing in TF2.x eager mode.SageMaker Debugger will not be saving gradients
596 | [2020-05-27 07:15:23.037 ae4bb918ce14:10191 INFO hook.py:372] Monitoring the collections: metrics, sm_metrics, biases, losses, weights
597 | INFO:tensorflow:batch_all_reduce: 6 all-reduces with algorithm = nccl, num_packs = 1, agg_small_grads_max_bytes = 0 and agg_small_grads_max_group = 10
598 | INFO:tensorflow:batch_all_reduce: 6 all-reduces with algorithm = nccl, num_packs = 1, agg_small_grads_max_bytes = 0 and agg_small_grads_max_group = 10
599 | INFO:tensorflow:batch_all_reduce: 6 all-reduces with algorithm = nccl, num_packs = 1, agg_small_grads_max_bytes = 0 and agg_small_grads_max_group = 10
600 | INFO:tensorflow:batch_all_reduce: 6 all-reduces with algorithm = nccl, num_packs = 1, agg_small_grads_max_bytes = 0 and agg_small_grads_max_group = 10
601 | [2020-05-27 07:15:28.373 ae4bb918ce14:10191 WARNING hook.py:562] Could not compute reduction l1 of conv2d/weights/conv2d/kernel:0 due to Improper number of dimensions to norm.
602 | [2020-05-27 07:15:28.373 ae4bb918ce14:10191 WARNING hook.py:562] Could not compute reduction l2 of conv2d/weights/conv2d/kernel:0 due to Improper number of dimensions to norm.
603 | [2020-05-27 07:15:28.382 ae4bb918ce14:10191 WARNING hook.py:562] Could not compute reduction l1 of conv2d/weights/conv2d/kernel:0 due to Improper number of dimensions to norm.
604 | [2020-05-27 07:15:28.382 ae4bb918ce14:10191 WARNING hook.py:562] Could not compute reduction l2 of conv2d/weights/conv2d/kernel:0 due to Improper number of dimensions to norm.
605 | [2020-05-27 07:15:28.391 ae4bb918ce14:10191 WARNING hook.py:562] Could not compute reduction l1 of conv2d/weights/conv2d/kernel:0 due to Improper number of dimensions to norm.
606 | [2020-05-27 07:15:28.391 ae4bb918ce14:10191 WARNING hook.py:562] Could not compute reduction l2 of conv2d/weights/conv2d/kernel:0 due to Improper number of dimensions to norm.
607 | [2020-05-27 07:15:28.399 ae4bb918ce14:10191 WARNING hook.py:562] Could not compute reduction l1 of conv2d/weights/conv2d/kernel:0 due to Improper number of dimensions to norm.
608 | [2020-05-27 07:15:28.399 ae4bb918ce14:10191 WARNING hook.py:562] Could not compute reduction l2 of conv2d/weights/conv2d/kernel:0 due to Improper number of dimensions to norm.
609 | WARNING:tensorflow:Method (on_train_batch_end) is slow compared to the batch update (0.102305). Check your callbacks.
610 | WARNING:tensorflow:Method (on_train_batch_end) is slow compared to the batch update (0.102305). Check your callbacks.
611 | [2020-05-27 07:15:28.636 ae4bb918ce14:10191 WARNING hook.py:562] Could not compute reduction l1 of conv2d/weights/conv2d/kernel:0 due to Improper number of dimensions to norm.
612 | [2020-05-27 07:15:28.636 ae4bb918ce14:10191 WARNING hook.py:562] Could not compute reduction l2 of conv2d/weights/conv2d/kernel:0 due to Improper number of dimensions to norm.
613 | [2020-05-27 07:15:28.644 ae4bb918ce14:10191 WARNING hook.py:562] Could not compute reduction l1 of conv2d/weights/conv2d/kernel:0 due to Improper number of dimensions to norm.
614 | [2020-05-27 07:15:28.644 ae4bb918ce14:10191 WARNING hook.py:562] Could not compute reduction l2 of conv2d/weights/conv2d/kernel:0 due to Improper number of dimensions to norm.
615 | [2020-05-27 07:15:28.652 ae4bb918ce14:10191 WARNING hook.py:562] Could not compute reduction l1 of conv2d/weights/conv2d/kernel:0 due to Improper number of dimensions to norm.
616 | [2020-05-27 07:15:28.653 ae4bb918ce14:10191 WARNING hook.py:562] Could not compute reduction l2 of conv2d/weights/conv2d/kernel:0 due to Improper number of dimensions to norm.
617 | [2020-05-27 07:15:28.661 ae4bb918ce14:10191 WARNING hook.py:562] Could not compute reduction l1 of conv2d/weights/conv2d/kernel:0 due to Improper number of dimensions to norm.
618 | [2020-05-27 07:15:28.661 ae4bb918ce14:10191 WARNING hook.py:562] Could not compute reduction l2 of conv2d/weights/conv2d/kernel:0 due to Improper number of dimensions to norm.
619 | [2020-05-27 07:15:28.862 ae4bb918ce14:10191 WARNING hook.py:562] Could not compute reduction l1 of conv2d/weights/conv2d/kernel:0 due to Improper number of dimensions to norm.
620 | [2020-05-27 07:15:28.862 ae4bb918ce14:10191 WARNING hook.py:562] Could not compute reduction l2 of conv2d/weights/conv2d/kernel:0 due to Improper number of dimensions to norm.
621 | [2020-05-27 07:15:28.872 ae4bb918ce14:10191 WARNING hook.py:562] Could not compute reduction l1 of conv2d/weights/conv2d/kernel:0 due to Improper number of dimensions to norm.
622 | [2020-05-27 07:15:28.873 ae4bb918ce14:10191 WARNING hook.py:562] Could not compute reduction l2 of conv2d/weights/conv2d/kernel:0 due to Improper number of dimensions to norm.
623 | [2020-05-27 07:15:28.883 ae4bb918ce14:10191 WARNING hook.py:562] Could not compute reduction l1 of conv2d/weights/conv2d/kernel:0 due to Improper number of dimensions to norm.
624 | [2020-05-27 07:15:28.883 ae4bb918ce14:10191 WARNING hook.py:562] Could not compute reduction l2 of conv2d/weights/conv2d/kernel:0 due to Improper number of dimensions to norm.
625 | [2020-05-27 07:15:28.892 ae4bb918ce14:10191 WARNING hook.py:562] Could not compute reduction l1 of conv2d/weights/conv2d/kernel:0 due to Improper number of dimensions to norm.
626 | [2020-05-27 07:15:28.892 ae4bb918ce14:10191 WARNING hook.py:562] Could not compute reduction l2 of conv2d/weights/conv2d/kernel:0 due to Improper number of dimensions to norm.
627 | [2020-05-27 07:15:29.083 ae4bb918ce14:10191 WARNING hook.py:562] Could not compute reduction l1 of conv2d/weights/conv2d/kernel:0 due to Improper number of dimensions to norm.
628 | [2020-05-27 07:15:29.084 ae4bb918ce14:10191 WARNING hook.py:562] Could not compute reduction l2 of conv2d/weights/conv2d/kernel:0 due to Improper number of dimensions to norm.
629 | [2020-05-27 07:15:29.095 ae4bb918ce14:10191 WARNING hook.py:562] Could not compute reduction l1 of conv2d/weights/conv2d/kernel:0 due to Improper number of dimensions to norm.
630 | [2020-05-27 07:15:29.096 ae4bb918ce14:10191 WARNING hook.py:562] Could not compute reduction l2 of conv2d/weights/conv2d/kernel:0 due to Improper number of dimensions to norm.
631 | [2020-05-27 07:15:29.106 ae4bb918ce14:10191 WARNING hook.py:562] Could not compute reduction l1 of conv2d/weights/conv2d/kernel:0 due to Improper number of dimensions to norm.
632 | [2020-05-27 07:15:29.107 ae4bb918ce14:10191 WARNING hook.py:562] Could not compute reduction l2 of conv2d/weights/conv2d/kernel:0 due to Improper number of dimensions to norm.
633 | [2020-05-27 07:15:29.115 ae4bb918ce14:10191 WARNING hook.py:562] Could not compute reduction l1 of conv2d/weights/conv2d/kernel:0 due to Improper number of dimensions to norm.
634 | [2020-05-27 07:15:29.115 ae4bb918ce14:10191 WARNING hook.py:562] Could not compute reduction l2 of conv2d/weights/conv2d/kernel:0 due to Improper number of dimensions to norm.
635 | 2020-05-27 07:15:29.518013: W tensorflow/core/kernels/data/cache_dataset_ops.cc:822] The calling iterator did not fully read the dataset being cached. In order to avoid unexpected truncation of the dataset, the partially cached contents of the dataset will be discarded. This can happen if you have an input pipeline similar to `dataset.cache().take(k).repeat()`. You should use `dataset.take(k).cache().repeat()` instead.
636 | 2020-05-27 07:15:29.524775: W tensorflow/core/kernels/data/cache_dataset_ops.cc:822] The calling iterator did not fully read the dataset being cached. In order to avoid unexpected truncation of the dataset, the partially cached contents of the dataset will be discarded. This can happen if you have an input pipeline similar to `dataset.cache().take(k).repeat()`. You should use `dataset.take(k).cache().repeat()` instead.
637 | 2020-05-27 07:15:29.544889: W tensorflow/core/kernels/data/generator_dataset_op.cc:103] Error occurred when finalizing GeneratorDataset iterator: Cancelled: Operation was cancelled
638 | 2020-05-27 07:15:29.545632: W tensorflow/core/kernels/data/generator_dataset_op.cc:103] Error occurred when finalizing GeneratorDataset iterator: Cancelled: Operation was cancelled
639 | 2020-05-27 07:15:29.546292: W tensorflow/core/kernels/data/generator_dataset_op.cc:103] Error occurred when finalizing GeneratorDataset iterator: Cancelled: Operation was cancelled
640 | 2020-05-27 07:15:29.546884: W tensorflow/core/kernels/data/generator_dataset_op.cc:103] Error occurred when finalizing GeneratorDataset iterator: Cancelled: Operation was cancelled
641 | [2020-05-27 07:15:29.549 ae4bb918ce14:10191 INFO local_trial.py:35] Loading trial test at path /tmp/test
642 | [2020-05-27 07:15:29.562 ae4bb918ce14:10191 INFO trial.py:198] Training has ended, will refresh one final time in 1 sec.
643 | [2020-05-27 07:15:30.563 ae4bb918ce14:10191 INFO trial.py:210] Loaded all steps
644 | PASSED