windows: explicitly register correct MIME types #3128

wchargin · 2020-01-10T04:58:30Z

Summary:
The standard library’s mimetypes.guess_type can give incorrect results
on Windows, because arbitrary programs can influence its behavior by
writing to the registry. This can be worked around by reinstating the
correct MIME types with mimetypes.add_type, as in this patch. Fixes
#3120.

Test Plan:
To check that this patch works at all on Windows, edited the registry to
add a bad content type:

reg add HKCR\.js /v "Content Type" /d application/x-testing-123

…then verified that before this change, mimetypes.guess_type("foo.js")
gives the wrong value and TensorBoard does not render properly (strict
MIME type checking error in Chrome, as expected), whereas after this
change mimetypes.guess_type("foo.js") gives the correct value and
TensorBoard returns the correct content type in its JavaScript
responses. TensorBoard still does not render on Windows due to unrelated
errors in the debugger plugin.

To check that this set of patches is sufficient, git grep guess_type
shows call sites only in core_plugin and projector_plugin. The
former takes file names from webfiles.zip, which has only html,
js, and woff2 files (plus a LICENSE). The latter has a fixed set
of inputs with extensions html and js.

wchargin-branch: windows-explicit-mime

Summary: The standard library’s `mimetypes.guess_types` can give incorrect results on Windows, because arbitrary programs can influence its behavior by writing to the registry. This can be worked around by reinstating the correct MIME types with `mimetypes.add_type`, as in this patch. Fixes #3120. Test Plan: To check that this patch works at all on Windows, edited the registry to add a bad content type: ``` reg add HKCR\.js /v "Content Type" /d application/x-testing-123 ``` …then verified that before this change, `mimetypes.guess_type("foo.js")` gives the wrong value and TensorBoard does not render properly (strict MIME type checking error in Chrome, as expected), whereas after this change `mimetypes.guess_type("foo.js")` gives the correct value and TensorBoard returns the correct content type in its JavaScript responses. TensorBoard still does not render on Windows due to unrelated errors in the debugger plugin. To check that this set of patches is sufficient, `git grep guess_type` shows call sites only in `core_plugin` and `projector_plugin`. The former takes file names from `webfiles.zip`, which has only `html`, `js`, and `woff2` files (plus a `LICENSE`). The latter has a fixed set of inputs with extensions `html` and `js`. wchargin-branch: windows-explicit-mime

wchargin · 2020-01-10T05:17:22Z

Can confirm that after patching in #3129, TensorBoard actually works on
Windows even with a bad MIME type in the registry (as described in the
test plan).

wchargin-branch: windows-explicit-mime wchargin-source: e069737c538dba27f74f4283f8e3ded6c2ebc887

psybuzz

I haven't verified it on my Window machine at home, but lgtm to merge.

rohangpatil · 2020-04-13T19:55:26Z

This is happening in mac as well. This is with Tensorboard 2.2.0

aviasd · 2020-05-08T11:17:52Z

Happening on AWS Sagemaker as well..
Tensorboard 2.2.1

@nfelt

…5746) Recently the test core_plugin_test.py::test_js_cache began to fail in some environments with the following error: ``` ... core_plugin_test.py", line 196, in test_js_cache self.assertEqual( AssertionError: - private, max-age=86400 + no-cache, must-revalidate ``` The root cause is that these systems have now been configured to return 'text/javascript' as the mimetype for js files, where they were previously returning 'application/javascript'. This means the 'Cache-Control' header was not set by this line: https://cs.opensource.google/tensorflow/tensorboard/+/master:tensorboard/plugins/core/core_plugin.py;l=137;drc=c061b659f50cde9dd9da5aa906ecb682ecdc68d4 ``` expires = ( JS_CACHE_EXPIRATION_IN_SECS if request.args.get("_file_hash") and mimetype == "application/javascript" else 0 ) ``` After further investigation (thanks @nfelt!) we determined that the change in mimetype is likely intentional after the recent publishing of RFC 9239 (https://www.rfc-editor.org/rfc/rfc9239), where 'text/javascript' becomes the standard mimetype for js files. We decided, therefore, to change the Cache-Control-setting logic to also recognize 'text/javascript'. Note, in practice, the change in mimetype had no impact on running tensorboards thanks to the logic added in #3128, which always ensures ".js" is mapped to mimetype "application/javascript" but does it at a higher level (ie outside either core plugin or the http utils).

@nfelt

…ensorflow#5746) Recently the test core_plugin_test.py::test_js_cache began to fail in some environments with the following error: ``` ... core_plugin_test.py", line 196, in test_js_cache self.assertEqual( AssertionError: - private, max-age=86400 + no-cache, must-revalidate ``` The root cause is that these systems have now been configured to return 'text/javascript' as the mimetype for js files, where they were previously returning 'application/javascript'. This means the 'Cache-Control' header was not set by this line: https://cs.opensource.google/tensorflow/tensorboard/+/master:tensorboard/plugins/core/core_plugin.py;l=137;drc=c061b659f50cde9dd9da5aa906ecb682ecdc68d4 ``` expires = ( JS_CACHE_EXPIRATION_IN_SECS if request.args.get("_file_hash") and mimetype == "application/javascript" else 0 ) ``` After further investigation (thanks @nfelt!) we determined that the change in mimetype is likely intentional after the recent publishing of RFC 9239 (https://www.rfc-editor.org/rfc/rfc9239), where 'text/javascript' becomes the standard mimetype for js files. We decided, therefore, to change the Cache-Control-setting logic to also recognize 'text/javascript'. Note, in practice, the change in mimetype had no impact on running tensorboards thanks to the logic added in tensorflow#3128, which always ensures ".js" is mapped to mimetype "application/javascript" but does it at a higher level (ie outside either core plugin or the http utils).

@nfelt

…5746) Recently the test core_plugin_test.py::test_js_cache began to fail in some environments with the following error: ``` ... core_plugin_test.py", line 196, in test_js_cache self.assertEqual( AssertionError: - private, max-age=86400 + no-cache, must-revalidate ``` The root cause is that these systems have now been configured to return 'text/javascript' as the mimetype for js files, where they were previously returning 'application/javascript'. This means the 'Cache-Control' header was not set by this line: https://cs.opensource.google/tensorflow/tensorboard/+/master:tensorboard/plugins/core/core_plugin.py;l=137;drc=c061b659f50cde9dd9da5aa906ecb682ecdc68d4 ``` expires = ( JS_CACHE_EXPIRATION_IN_SECS if request.args.get("_file_hash") and mimetype == "application/javascript" else 0 ) ``` After further investigation (thanks @nfelt!) we determined that the change in mimetype is likely intentional after the recent publishing of RFC 9239 (https://www.rfc-editor.org/rfc/rfc9239), where 'text/javascript' becomes the standard mimetype for js files. We decided, therefore, to change the Cache-Control-setting logic to also recognize 'text/javascript'. Note, in practice, the change in mimetype had no impact on running tensorboards thanks to the logic added in #3128, which always ensures ".js" is mapped to mimetype "application/javascript" but does it at a higher level (ie outside either core plugin or the http utils).

While investigating the test failure that lead to #5746, we realized that RFC 9239 (https://www.rfc-editor.org/rfc/rfc9239) has changed the preferred js mimetype to 'text/javascript' from 'application/javascript'. To be a good citizen of the internet, this change modifies the logic introduced in #3128 to force the mapping of js files to mimetype 'text/javascript'. We must also modify http_util.py to ensure that the proper charset is included in the Content-Type header. And, finally, we modify the example plugin to use 'text/javascript' as mimetype. To Test: * I ran local tensorboard and used the chrome developer tools to verify that the Content-Type for js files fetched by the browser was 'text/javascript; charset=utf-8'. I did a basic sanity test ensuring both Angular and Polymer portions of the application are running. * I ran a tensorboard with the example plugin installed and verified that index.js is returned with Content-Type 'text/javascript'.

@nfelt

…ensorflow#5746) Recently the test core_plugin_test.py::test_js_cache began to fail in some environments with the following error: ``` ... core_plugin_test.py", line 196, in test_js_cache self.assertEqual( AssertionError: - private, max-age=86400 + no-cache, must-revalidate ``` The root cause is that these systems have now been configured to return 'text/javascript' as the mimetype for js files, where they were previously returning 'application/javascript'. This means the 'Cache-Control' header was not set by this line: https://cs.opensource.google/tensorflow/tensorboard/+/master:tensorboard/plugins/core/core_plugin.py;l=137;drc=c061b659f50cde9dd9da5aa906ecb682ecdc68d4 ``` expires = ( JS_CACHE_EXPIRATION_IN_SECS if request.args.get("_file_hash") and mimetype == "application/javascript" else 0 ) ``` After further investigation (thanks @nfelt!) we determined that the change in mimetype is likely intentional after the recent publishing of RFC 9239 (https://www.rfc-editor.org/rfc/rfc9239), where 'text/javascript' becomes the standard mimetype for js files. We decided, therefore, to change the Cache-Control-setting logic to also recognize 'text/javascript'. Note, in practice, the change in mimetype had no impact on running tensorboards thanks to the logic added in tensorflow#3128, which always ensures ".js" is mapped to mimetype "application/javascript" but does it at a higher level (ie outside either core plugin or the http utils).

…5747) While investigating the test failure that lead to tensorflow#5746, we realized that RFC 9239 (https://www.rfc-editor.org/rfc/rfc9239) has changed the preferred js mimetype to 'text/javascript' from 'application/javascript'. To be a good citizen of the internet, this change modifies the logic introduced in tensorflow#3128 to force the mapping of js files to mimetype 'text/javascript'. We must also modify http_util.py to ensure that the proper charset is included in the Content-Type header. And, finally, we modify the example plugin to use 'text/javascript' as mimetype. To Test: * I ran local tensorboard and used the chrome developer tools to verify that the Content-Type for js files fetched by the browser was 'text/javascript; charset=utf-8'. I did a basic sanity test ensuring both Angular and Polymer portions of the application are running. * I ran a tensorboard with the example plugin installed and verified that index.js is returned with Content-Type 'text/javascript'.

@nfelt

…ensorflow#5746) Recently the test core_plugin_test.py::test_js_cache began to fail in some environments with the following error: ``` ... core_plugin_test.py", line 196, in test_js_cache self.assertEqual( AssertionError: - private, max-age=86400 + no-cache, must-revalidate ``` The root cause is that these systems have now been configured to return 'text/javascript' as the mimetype for js files, where they were previously returning 'application/javascript'. This means the 'Cache-Control' header was not set by this line: https://cs.opensource.google/tensorflow/tensorboard/+/master:tensorboard/plugins/core/core_plugin.py;l=137;drc=c061b659f50cde9dd9da5aa906ecb682ecdc68d4 ``` expires = ( JS_CACHE_EXPIRATION_IN_SECS if request.args.get("_file_hash") and mimetype == "application/javascript" else 0 ) ``` After further investigation (thanks @nfelt!) we determined that the change in mimetype is likely intentional after the recent publishing of RFC 9239 (https://www.rfc-editor.org/rfc/rfc9239), where 'text/javascript' becomes the standard mimetype for js files. We decided, therefore, to change the Cache-Control-setting logic to also recognize 'text/javascript'. Note, in practice, the change in mimetype had no impact on running tensorboards thanks to the logic added in tensorflow#3128, which always ensures ".js" is mapped to mimetype "application/javascript" but does it at a higher level (ie outside either core plugin or the http utils).

…5747) While investigating the test failure that lead to tensorflow#5746, we realized that RFC 9239 (https://www.rfc-editor.org/rfc/rfc9239) has changed the preferred js mimetype to 'text/javascript' from 'application/javascript'. To be a good citizen of the internet, this change modifies the logic introduced in tensorflow#3128 to force the mapping of js files to mimetype 'text/javascript'. We must also modify http_util.py to ensure that the proper charset is included in the Content-Type header. And, finally, we modify the example plugin to use 'text/javascript' as mimetype. To Test: * I ran local tensorboard and used the chrome developer tools to verify that the Content-Type for js files fetched by the browser was 'text/javascript; charset=utf-8'. I did a basic sanity test ensuring both Angular and Polymer portions of the application are running. * I ran a tensorboard with the example plugin installed and verified that index.js is returned with Content-Type 'text/javascript'.

wchargin requested a review from psybuzz January 10, 2020 04:58

googlebot added the cla: yes label Jan 10, 2020

wchargin added the os:windows label Jan 10, 2020

wchargin mentioned this pull request Jan 10, 2020

Blank Page in Browser (and other error) #3117

Closed

[update patch]

d90fabc

wchargin-branch: windows-explicit-mime wchargin-source: e069737c538dba27f74f4283f8e3ded6c2ebc887

psybuzz approved these changes Jan 10, 2020

View reviewed changes

wchargin merged commit 48b2c66 into master Jan 10, 2020

wchargin deleted the wchargin-windows-explicit-mime branch January 10, 2020 18:05

wchargin mentioned this pull request Jan 11, 2020

Directory traversal vulnerability in example raw scalars plugin #3134

Closed

psybuzz mentioned this pull request Mar 10, 2020

localhost:6006 is blank white screen despite TB --inspect loading and detecting logs in --logdir #3170

Closed

psybuzz mentioned this pull request Apr 8, 2020

TensorBoard just opens a blank webpage (windows) #3496

Closed

mxschmitt mentioned this pull request Dec 20, 2020

fix: Windows/firefox CI build playwright-community/playwright-go#58

Merged

joyceerhl mentioned this pull request Apr 28, 2021

Fix blank TensorBoard pane due to existing MIME type misconfiguration on Windows microsoft/vscode-python#16072

Closed

This was referenced Jun 7, 2022

Recognize text/javascript as valid js mimetype for caching purposes #5746

Merged

Change mimetype mapping of js files to 'text/javascript'. #5747

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

windows: explicitly register correct MIME types #3128

windows: explicitly register correct MIME types #3128

wchargin commented Jan 10, 2020 •

edited

Loading

wchargin commented Jan 10, 2020

psybuzz left a comment

rohangpatil commented Apr 13, 2020

aviasd commented May 8, 2020

windows: explicitly register correct MIME types #3128

windows: explicitly register correct MIME types #3128

Conversation

wchargin commented Jan 10, 2020 • edited Loading

wchargin commented Jan 10, 2020

psybuzz left a comment

Choose a reason for hiding this comment

rohangpatil commented Apr 13, 2020

aviasd commented May 8, 2020

wchargin commented Jan 10, 2020 •

edited

Loading