-
Notifications
You must be signed in to change notification settings - Fork 7k
[core] Augment browser request validation using CORS header checks
#59040
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request enhances the detection of browser requests by checking for additional CORS-related headers. This improves security by making it harder for non-browser clients to be misidentified. The changes are accompanied by a comprehensive set of new test cases that cover various browsers and scenarios.
My review focuses on improving code maintainability. I've suggested refactoring the header checks into a constant for clarity and moving the large set of test data into a separate file to make the test code more readable.
| return req.headers.get("User-Agent", "").startswith("Mozilla") or any( | ||
| h in req.headers | ||
| for h in ( | ||
| "Origin", | ||
| # Sec-Fetch headers are sent with many but not all `fetch` | ||
| # requests, and will eventually be sent on all requests. | ||
| "Sec-Fetch-Mode", | ||
| "Sec-Fetch-Dest", | ||
| "Sec-Fetch-Site", | ||
| "Sec-Fetch-User", | ||
| # CORS headers specifying which other headers are modified | ||
| "Access-Control-Request-Method", | ||
| "Access-Control-Request-Headers", | ||
| ) | ||
| ) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For improved readability and a minor performance gain, consider defining these headers as a module-level constant set instead of a tuple literal inside the function. This avoids recreating the collection on every call and makes the list of headers easier to manage.
For example, at the module level:
_BROWSER_HEADERS = {
"Origin",
"Sec-Fetch-Mode",
"Sec-Fetch-Dest",
"Sec-Fetch-Site",
"Sec-Fetch-User",
"Access-Control-Request-Method",
"Access-Control-Request-Headers",
}Then the function body can be simplified to:
return req.headers.get("User-Agent", "").startswith("Mozilla") or any(
h in req.headers for h in _BROWSER_HEADERS
)| testcases = ( | ||
| # chrome-invalid-tls.json | ||
| { | ||
| "Host": "localtest.me", | ||
| "Connection": "keep-alive", | ||
| "Content-Length": "0", | ||
| "Sec-Ch-Ua-Platform": '"macOS"', | ||
| "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/142.0.0.0 Safari/537.36", | ||
| "Sec-Ch-Ua": '"Chromium";v="142", "Google Chrome";v="142", "Not_A Brand";v="99"', | ||
| "Sec-Ch-Ua-Mobile": "?0", | ||
| "Accept": "*/*", | ||
| "Origin": "https://localtest.me", | ||
| "Sec-Fetch-Site": "same-origin", | ||
| "Sec-Fetch-Mode": "cors", | ||
| "Sec-Fetch-Dest": "empty", | ||
| "Referer": "https://localtest.me/", | ||
| "Accept-Encoding": "gzip, deflate, br, zstd", | ||
| "Accept-Language": "en-US,en;q=0.9", | ||
| }, | ||
| # chrome-localhost-notls.json | ||
| { | ||
| "Host": "localhost:5000", | ||
| "Connection": "keep-alive", | ||
| "Content-Length": "0", | ||
| "Sec-Ch-Ua-Platform": '"macOS"', | ||
| "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/142.0.0.0 Safari/537.36", | ||
| "Sec-Ch-Ua": '"Chromium";v="142", "Google Chrome";v="142", "Not_A Brand";v="99"', | ||
| "Sec-Ch-Ua-Mobile": "?0", | ||
| "Accept": "*/*", | ||
| "Origin": "http://localhost:5000", | ||
| "Sec-Fetch-Site": "same-origin", | ||
| "Sec-Fetch-Mode": "cors", | ||
| "Sec-Fetch-Dest": "empty", | ||
| "Referer": "http://localhost:5000/", | ||
| "Accept-Encoding": "gzip, deflate, br, zstd", | ||
| "Accept-Language": "en-US,en;q=0.9", | ||
| }, | ||
| # chrome-notlocalhost-notls.json | ||
| { | ||
| "Host": "localtest.me:5000", | ||
| "Connection": "keep-alive", | ||
| "Content-Length": "0", | ||
| "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/142.0.0.0 Safari/537.36", | ||
| "Accept": "*/*", | ||
| "Origin": "http://localtest.me:5000", | ||
| "Referer": "http://localtest.me:5000/", | ||
| "Accept-Encoding": "gzip, deflate", | ||
| "Accept-Language": "en-US,en;q=0.9", | ||
| }, | ||
| # chrome-notlocalhost-port80-notls.json | ||
| { | ||
| "Host": "localtest.me", | ||
| "Connection": "keep-alive", | ||
| "Content-Length": "0", | ||
| "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/142.0.0.0 Safari/537.36", | ||
| "Accept": "*/*", | ||
| "Origin": "http://localtest.me", | ||
| "Referer": "http://localtest.me/", | ||
| "Accept-Encoding": "gzip, deflate", | ||
| "Accept-Language": "en-US,en;q=0.9", | ||
| }, | ||
| # firefox-invalid-tls.json | ||
| { | ||
| "Host": "localtest.me", | ||
| "User-Agent": "Fake", | ||
| "Accept": "*/*", | ||
| "Accept-Language": "en-US,en;q=0.5", | ||
| "Accept-Encoding": "gzip, deflate, br, zstd", | ||
| "Referer": "https://localtest.me/", | ||
| "Origin": "https://localtest.me", | ||
| "Connection": "keep-alive", | ||
| "Sec-Fetch-Dest": "empty", | ||
| "Sec-Fetch-Mode": "cors", | ||
| "Sec-Fetch-Site": "same-origin", | ||
| "Priority": "u=0", | ||
| "Content-Length": "0", | ||
| }, | ||
| # firefox-localhost-notls.json | ||
| { | ||
| "Host": "localhost:5000", | ||
| "User-Agent": "Fake", | ||
| "Accept": "*/*", | ||
| "Accept-Language": "en-US,en;q=0.5", | ||
| "Accept-Encoding": "gzip, deflate, br, zstd", | ||
| "Referer": "http://localhost:5000/", | ||
| "Origin": "http://localhost:5000", | ||
| "Connection": "keep-alive", | ||
| "Sec-Fetch-Dest": "empty", | ||
| "Sec-Fetch-Mode": "cors", | ||
| "Sec-Fetch-Site": "same-origin", | ||
| "Priority": "u=0", | ||
| "Content-Length": "0", | ||
| }, | ||
| # firefox-notlocalhost-notls.json | ||
| { | ||
| "Host": "localtest.me:5000", | ||
| "User-Agent": "Fake", | ||
| "Accept": "*/*", | ||
| "Accept-Language": "en-US,en;q=0.5", | ||
| "Accept-Encoding": "gzip, deflate", | ||
| "Referer": "http://localtest.me:5000/", | ||
| "Origin": "http://localtest.me:5000", | ||
| "Connection": "keep-alive", | ||
| "Priority": "u=0", | ||
| "Content-Length": "0", | ||
| }, | ||
| # firefox-notlocalhost-port80-notls.json | ||
| { | ||
| "Host": "localtest.me", | ||
| "User-Agent": "Fake", | ||
| "Accept": "*/*", | ||
| "Accept-Language": "en-US,en;q=0.5", | ||
| "Accept-Encoding": "gzip, deflate", | ||
| "Referer": "http://localtest.me/", | ||
| "Origin": "http://localtest.me", | ||
| "Connection": "keep-alive", | ||
| "Priority": "u=0", | ||
| "Content-Length": "0", | ||
| }, | ||
| # safari-invalid-tls.json | ||
| { | ||
| "Host": "localtest.me", | ||
| "Accept": "*/*", | ||
| "Origin": "https://localtest.me", | ||
| "Sec-Fetch-Site": "same-origin", | ||
| "Sec-Fetch-Mode": "cors", | ||
| "User-Agent": "Fake", | ||
| "Referer": "https://localtest.me/", | ||
| "Sec-Fetch-Dest": "empty", | ||
| "Content-Length": "0", | ||
| "Accept-Language": "en-US,en;q=0.9", | ||
| "Priority": "u=3, i", | ||
| "Accept-Encoding": "gzip, deflate, br", | ||
| "Connection": "keep-alive", | ||
| }, | ||
| # safari-localhost-notls.json | ||
| { | ||
| "Host": "localhost:5000", | ||
| "Accept": "*/*", | ||
| "Origin": "http://localhost:5000", | ||
| "Sec-Fetch-Site": "same-origin", | ||
| "Sec-Fetch-Mode": "cors", | ||
| "User-Agent": "Fake", | ||
| "Referer": "http://localhost:5000/", | ||
| "Sec-Fetch-Dest": "empty", | ||
| "Content-Length": "0", | ||
| "Accept-Language": "en-US,en;q=0.9", | ||
| "Priority": "u=3, i", | ||
| "Accept-Encoding": "gzip, deflate", | ||
| "Connection": "keep-alive", | ||
| }, | ||
| # safari-notlocalhost-notls.json | ||
| { | ||
| "Host": "localtest.me:5000", | ||
| "User-Agent": "Fake", | ||
| "Accept": "*/*", | ||
| "Content-Length": "0", | ||
| "Referer": "http://localtest.me:5000/", | ||
| "Origin": "http://localtest.me:5000", | ||
| "Accept-Language": "en-US,en;q=0.9", | ||
| "Priority": "u=3, i", | ||
| "Accept-Encoding": "gzip, deflate", | ||
| "Connection": "keep-alive", | ||
| }, | ||
| # safari-notlocalhost-port80-notls.json | ||
| { | ||
| "Host": "localtest.me", | ||
| "User-Agent": "Fake", | ||
| "Accept": "*/*", | ||
| "Content-Length": "0", | ||
| "Referer": "http://localtest.me/", | ||
| "Origin": "http://localtest.me", | ||
| "Accept-Language": "en-US,en;q=0.9", | ||
| "Priority": "u=3, i", | ||
| "Accept-Encoding": "gzip, deflate", | ||
| "Connection": "keep-alive", | ||
| }, | ||
| # edge-valid-tls.json | ||
| { | ||
| "Content-Length": "0", | ||
| "Sec-Ch-Ua-Platform": '"Windows"', | ||
| "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/142.0.0.0 Safari/537.36 Edg/142.0.0.0", | ||
| "Sec-Ch-Ua": '"Chromium";v="142", "Microsoft Edge";v="142", "Not_A Brand";v="99"', | ||
| "Sec-Ch-Ua-Mobile": "?0", | ||
| "Accept": "*/*", | ||
| "Origin": "https://testing-dark-field-5895.fly.dev", | ||
| "Sec-Fetch-Site": "same-origin", | ||
| "Sec-Fetch-Mode": "cors", | ||
| "Sec-Fetch-Dest": "empty", | ||
| "Referer": "https://testing-dark-field-5895.fly.dev/", | ||
| "Accept-Encoding": "gzip, deflate, br, zstd", | ||
| "Accept-Language": "en-US,en;q=0.9", | ||
| "Priority": "u=1, i", | ||
| "X-Request-Start": "t=1764259434792741", | ||
| "Host": "testing-dark-field-5895.fly.dev", | ||
| }, | ||
| # edge-notlocalhost-notls | ||
| { | ||
| "Host": "localtest.me:5000", | ||
| "Connection": "keep-alive", | ||
| "Content-Length": "0", | ||
| "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/142.0.0.0 Safari/537.36 Edg/142.0.0.0", | ||
| "Accept": "*/*", | ||
| "Origin": "http://localtest.me:5000", | ||
| "Referer": "http://localtest.me:5000/", | ||
| "Accept-Encoding": "gzip, deflate", | ||
| "Accept-Language": "en-US,en;q=0.9", | ||
| }, | ||
| # edge-localhost-notls | ||
| { | ||
| "Host": "localhost:5000", | ||
| "Connection": "keep-alive", | ||
| "Content-Length": "0", | ||
| "Sec-Ch-Ua-Platform": '"Windows"', | ||
| "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/142.0.0.0 Safari/537.36 Edg/142.0.0.0", | ||
| "Sec-Ch-Ua": '"Chromium";v="142", "Microsoft Edge";v="142", "Not_A Brand";v="99"', | ||
| "Sec-Ch-Ua-Mobile": "?0", | ||
| "Accept": "*/*", | ||
| "Origin": "http://localhost:5000", | ||
| "Sec-Fetch-Site": "same-origin", | ||
| "Sec-Fetch-Mode": "cors", | ||
| "Sec-Fetch-Dest": "empty", | ||
| "Referer": "http://localhost:5000/", | ||
| "Accept-Encoding": "gzip, deflate, br, zstd", | ||
| "Accept-Language": "en-US,en;q=0.9", | ||
| }, | ||
| ) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This large block of test case data makes the test function hard to read and maintain. It's good practice to separate test data from test logic.
Consider moving this testcases tuple into a separate data file (e.g., a JSON file) and loading it at the start of the test. The comments in the code (e.g., # chrome-invalid-tls.json) suggest this data may have originated from JSON files anyway.
For example, you could create a test_dashboard_browser_headers.json file:
[
{
"name": "chrome-invalid-tls.json",
"headers": {
"Host": "localtest.me",
"Connection": "keep-alive",
"Content-Length": "0",
"Origin": "https://localtest.me"
}
},
{
"name": "chrome-localhost-notls.json",
"headers": {
"Host": "localhost:5000",
"Connection": "keep-alive",
"Origin": "http://localhost:5000"
}
}
]And then in your test:
import json
import os
# ...
def test_browser_no_post_no_put(enable_test_module, ray_start_with_dashboard):
# ...
test_data_path = os.path.join(os.path.dirname(__file__), "test_dashboard_browser_headers.json")
with open(test_data_path) as f:
testcases_data = json.load(f)
testcases = [item['headers'] for item in testcases_data]
# ... rest of the test logicThis would make the test function much cleaner.
…jection logic (ray-project#59042) ## Description Adds more headers to the denylist for recognising browser requests and denying them ## Related issues Supercedes ray-project#59040 Signed-off-by: Richo Healey <richo@anyscale.com>
…jection logic (ray-project#59042) ## Description Adds more headers to the denylist for recognising browser requests and denying them ## Related issues Supercedes ray-project#59040 Signed-off-by: Richo Healey <richo@anyscale.com> Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>
Rejects requests containing
"Access-Control-Request-Method"or"Access-Control-Request-Headers"headers.