HParams: Port runs_data_source fetchHparamsMetadata to oss #6318
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
JamesHollyer
approved these changes
Apr 14, 2023
rileyajones
added a commit
that referenced
this pull request
Apr 14, 2023
…msInTimeSeries flag (#6317) Nothing will be displayed until ~#6318~ is merged ## Motivation for features / changes We are working on surfacing hparam data in the time series dashboard. As part of this effort we would like to show the value of hparams in the runs table. The runs table already has partial support for this but it is currently disabled. I am enabling the columns based on the feature flag I added in #6308. Note that you will not see anything in OSS until we fix the call to session_groups ## Technical description of changes I added a new selector to get the feature flag, then modified the metrics_container to read the value of the flag ## Screenshots of UI changes Disabled  Enabled  ## Detailed steps to verify changes work correctly (as executed by you) ## Alternate designs / implementations considered
rileyajones
added a commit
that referenced
this pull request
Apr 18, 2023
…ta (#6334) ## Motivation for features / changes The dashboard will not load when `inColab` is true because the runs data cannot be fetched. Googlers see b/278706604 for why this is important ## Technical description of changes When `inColab` is true `POST` requests are automatically converted to `GET` requests see [tensorboard/webapp/webapp_data_source/tb_http_client.ts](https://github.com/tensorflow/tensorboard/blob/master/tensorboard/webapp/webapp_data_source/tb_http_client.ts). This conversion assumes the provided body is of type `FormData`, however, when I open sourced `runs_data_source` in #6318 I was unaware of this conversion taking place and didn't modify the format of the post request body. ## Screenshots of UI changes Before:  After:  ## Detailed steps to verify changes work correctly (as executed by you) 1) Start tensorboard 2) Navigate to http://localhost:6006?tensorboardColab#timeseries 3) Assert that the dashboard loads ## Alternate designs / implementations considered I could have updated the conversion happening in [tensorboard/webapp/webapp_data_source/tb_http_client.ts](https://github.com/tensorflow/tensorboard/blob/master/tensorboard/webapp/webapp_data_source/tb_http_client.ts) to accept JSON bodies
rileyajones
added a commit
that referenced
this pull request
Apr 18, 2023
rileyajones
added a commit
that referenced
this pull request
Apr 20, 2023
#6337) ## Motivation for features / changes In #6318 I ported over the runs_data_source from internal tensorboard in order to begin fetching hparams data in the timeseries dashboard. However, I later learned that it didn't play well with internal Colab which does not allow POST requests and thus had to revert the change in #6336. ## Technical description of changes The current implementation of this "hackaround" converts post bodies to query params and assumes that all post bodies are of type `FormData`. This does not work in this situation for two reasons: 1) HParams Plugin backend does not accept `FormData` 2) The [the hparams plugin](https://github.com/tensorflow/tensorboard/blob/master/tensorboard/plugins/hparams/hparams_plugin.py#L191) only expects GET requests to have a query parameter titled "request" which is a serialized JSON object. ## Screenshots of UI changes N/A ## Alternate designs / implementations considered I considered modifying the hparams plugin to accept many query parameters but decided that the logic required to convert them to a single JSON object (including nested objects/arrays and parsing numbers) was messier than the just doing it on the client.
rileyajones
added a commit
that referenced
this pull request
Apr 25, 2023
…take 2 (#6340) ## Motivation for features / changes As part of the effort to surface hparams data in the time series dashboard we need to start actually fetching it. The logic existed internally but for some reason was not previously available in OSS. I originally attempted this in #6318 but due to incompatibilities with internal Colab had to revert it in #6336. The issue was that Colab does not allow `POST` requests and our method of mapping POST requests to GET requests didn't meet the expectations of the hparams_plugin backend. I've improved the tb_http_client in #6337 to now work with the hparams_plugin backend so this should now be fine ## Technical description of changes I mostly copied the internal code to the oss implementation and made a few small adjustments to imports. The biggest change is that I now utilize the `serializeUnder` parameter of our custom POST function to ensure the requests meet the expectations of the hparams backend ## Screenshots of UI changes ### enableHparamsInTimeSeries=false&tensorboardColab=false  ### enableHparamsInTimeSeries=true&tensorboardColab=false  ### enableHparamsInTimeSeries=false&tensorboardColab=true  ### enableHparamsInTimeSeries=true&tensorboardColab=true  ## Detailed steps to verify changes work correctly (as executed by you) I ensured all of the above states result in the runs table being displayed properly and that no errors are thrown by the hparams plugin backend
dna2github
pushed a commit
to dna2fork/tensorboard
that referenced
this pull request
May 1, 2023
…w#6318) ## Motivation for features / changes As part of the effort to surface hparams data in the time series dashboard we need to start actually fetching it. The logic existed internally but for some reason was not previously available in OSS. ## Technical description of changes I just copied the internal code to the oss implementation and made a few small adjustments to imports ## Screenshots of UI changes None ## Detailed steps to verify changes work correctly (as executed by you) Tests should pass Patch tensorflow#6317, enable the feature flag, and ensure the hparams appear in the runs table ## Alternate designs / implementations considered Make a more detailed stub?
dna2github
pushed a commit
to dna2fork/tensorboard
that referenced
this pull request
May 1, 2023
…msInTimeSeries flag (tensorflow#6317) Nothing will be displayed until ~tensorflow#6318~ is merged ## Motivation for features / changes We are working on surfacing hparam data in the time series dashboard. As part of this effort we would like to show the value of hparams in the runs table. The runs table already has partial support for this but it is currently disabled. I am enabling the columns based on the feature flag I added in tensorflow#6308. Note that you will not see anything in OSS until we fix the call to session_groups ## Technical description of changes I added a new selector to get the feature flag, then modified the metrics_container to read the value of the flag ## Screenshots of UI changes Disabled  Enabled  ## Detailed steps to verify changes work correctly (as executed by you) ## Alternate designs / implementations considered
dna2github
pushed a commit
to dna2fork/tensorboard
that referenced
this pull request
May 1, 2023
…ta (tensorflow#6334) ## Motivation for features / changes The dashboard will not load when `inColab` is true because the runs data cannot be fetched. Googlers see b/278706604 for why this is important ## Technical description of changes When `inColab` is true `POST` requests are automatically converted to `GET` requests see [tensorboard/webapp/webapp_data_source/tb_http_client.ts](https://github.com/tensorflow/tensorboard/blob/master/tensorboard/webapp/webapp_data_source/tb_http_client.ts). This conversion assumes the provided body is of type `FormData`, however, when I open sourced `runs_data_source` in tensorflow#6318 I was unaware of this conversion taking place and didn't modify the format of the post request body. ## Screenshots of UI changes Before:  After:  ## Detailed steps to verify changes work correctly (as executed by you) 1) Start tensorboard 2) Navigate to http://localhost:6006?tensorboardColab#timeseries 3) Assert that the dashboard loads ## Alternate designs / implementations considered I could have updated the conversion happening in [tensorboard/webapp/webapp_data_source/tb_http_client.ts](https://github.com/tensorflow/tensorboard/blob/master/tensorboard/webapp/webapp_data_source/tb_http_client.ts) to accept JSON bodies
dna2github
pushed a commit
to dna2fork/tensorboard
that referenced
this pull request
May 1, 2023
dna2github
pushed a commit
to dna2fork/tensorboard
that referenced
this pull request
May 1, 2023
tensorflow#6337) ## Motivation for features / changes In tensorflow#6318 I ported over the runs_data_source from internal tensorboard in order to begin fetching hparams data in the timeseries dashboard. However, I later learned that it didn't play well with internal Colab which does not allow POST requests and thus had to revert the change in tensorflow#6336. ## Technical description of changes The current implementation of this "hackaround" converts post bodies to query params and assumes that all post bodies are of type `FormData`. This does not work in this situation for two reasons: 1) HParams Plugin backend does not accept `FormData` 2) The [the hparams plugin](https://github.com/tensorflow/tensorboard/blob/master/tensorboard/plugins/hparams/hparams_plugin.py#L191) only expects GET requests to have a query parameter titled "request" which is a serialized JSON object. ## Screenshots of UI changes N/A ## Alternate designs / implementations considered I considered modifying the hparams plugin to accept many query parameters but decided that the logic required to convert them to a single JSON object (including nested objects/arrays and parsing numbers) was messier than the just doing it on the client.
dna2github
pushed a commit
to dna2fork/tensorboard
that referenced
this pull request
May 1, 2023
…take 2 (tensorflow#6340) ## Motivation for features / changes As part of the effort to surface hparams data in the time series dashboard we need to start actually fetching it. The logic existed internally but for some reason was not previously available in OSS. I originally attempted this in tensorflow#6318 but due to incompatibilities with internal Colab had to revert it in tensorflow#6336. The issue was that Colab does not allow `POST` requests and our method of mapping POST requests to GET requests didn't meet the expectations of the hparams_plugin backend. I've improved the tb_http_client in tensorflow#6337 to now work with the hparams_plugin backend so this should now be fine ## Technical description of changes I mostly copied the internal code to the oss implementation and made a few small adjustments to imports. The biggest change is that I now utilize the `serializeUnder` parameter of our custom POST function to ensure the requests meet the expectations of the hparams backend ## Screenshots of UI changes ### enableHparamsInTimeSeries=false&tensorboardColab=false  ### enableHparamsInTimeSeries=true&tensorboardColab=false  ### enableHparamsInTimeSeries=false&tensorboardColab=true  ### enableHparamsInTimeSeries=true&tensorboardColab=true  ## Detailed steps to verify changes work correctly (as executed by you) I ensured all of the above states result in the runs table being displayed properly and that no errors are thrown by the hparams plugin backend
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation for features / changes
As part of the effort to surface hparams data in the time series dashboard we need to start actually fetching it. The logic existed internally but for some reason was not previously available in OSS.
Technical description of changes
I just copied the internal code to the oss implementation and made a few small adjustments to imports
Screenshots of UI changes
None
Detailed steps to verify changes work correctly (as executed by you)
Tests should pass
Patch #6317, enable the feature flag, and ensure the hparams appear in the runs table
Alternate designs / implementations considered
Make a more detailed stub?