[BUG] - _xsrf Cookie conflict error in JupyterHub token creation during tests

[viniciusdc]

Describe the bug

In our CI environment, we are encountering multiple test failures and errors related to creating tokens via the JupyterHub API.

...
FAILED tests/tests_deployment/test_jupyterhub_api.py::test_keycloak_roles_attributes_parsed_as_jhub_scopes[invalid-component-read:users:shares,read:groups:shares,users:shares-expected_scopes_difference1] - requests.cookies.CookieConflictError: There are multiple cookies with name, '_xsrf'
FAILED tests/tests_deployment/test_jupyterhub_api.py::test_keycloak_roles_attributes_parsed_as_jhub_scopes[invalid-component-admin:invalid-scope-expected_scopes_difference2] - requests.cookies.CookieConflictError: There are multiple cookies with name, '_xsrf'
FAILED tests/tests_deployment/test_jupyterhub_api.py::test_jupyterhub_loads_groups_from_keycloak - requests.cookies.CookieConflictError: There are multiple cookies with name, '_xsrf'
ERROR tests/tests_deployment/test_dask_gateway.py::test_dask_gateway - requests.cookies.CookieConflictError: There are multiple cookies with name, '_xsrf'
ERROR tests/tests_deployment/test_dask_gateway.py::test_dask_gateway_cluster_options - requests.cookies.CookieConflictError: There are multiple cookies with name, '_xsrf'
ERROR tests/tests_deployment/test_jupyterhub_ssh.py::test_simple_jupyterhub_ssh - requests.cookies.CookieConflictError: There are multiple cookies with name, '_xsrf'
ERROR tests/tests_deployment/test_jupyterhub_ssh.py::test_print_jupyterhub_ssh - requests.cookies.CookieConflictError: There are multiple cookies with name, '_xsrf'
ERROR tests/tests_deployment/test_jupyterhub_ssh.py::test_exact_jupyterhub_ssh - requests.cookies.CookieConflictError: There are multiple cookies with name, '_xsrf'
ERROR tests/tests_deployment/test_jupyterhub_ssh.py::test_contains_jupyterhub_ssh - requests.cookies.CookieConflictError: There are multiple cookies with name, '_xsrf'

This issue seems likely to come from the flakiness of the _xsrf token value in the final session requests, where it sometimes fails to update correctly. An incorrect cookie value can also be set (leading to a 403 unauthorized request), and a duplicate cookie with the same name might be generated for different paths (e.g., / and /hub/), leading to a requests.cookies.CookieConflictError.

The primary function below is the central logic of this problem. A long-term solution would involve completely refactoring the authentication logic to eliminate reliance on cookies and instead depend entirely on the JupyterHub API endpoint. However, such a mechanism has yet to be supported. The solution implemented in #2554 is the best approach for managing this conflict at least for now:

nebari/tests/tests_deployment/utils.py

Lines 29 to 41 in 3caf45b

	def create_jupyterhub_token(note):
	session = get_jupyterhub_session()
	xsrf_token = session.cookies.get("_xsrf")
	headers = {"Referer": f"https://{constants.NEBARI_HOSTNAME}/hub/token"}
	if xsrf_token:
	headers["X-XSRFToken"] = xsrf_token
	data = {"note": note, "expires_in": None}
	return session.post(
	f"https://{constants.NEBARI_HOSTNAME}/hub/api/users/{constants.KEYCLOAK_USERNAME}/tokens",
	headers=headers,
	json=data,
	verify=False,
	)

Expected behavior

• All tests reliant on the jupyterhub user's token should pass

OS and architecture in which you are running Nebari

Linux

How to Reproduce the problem?

• Running the tests locally

Command output

No response

Versions and dependencies used.

No response

Compute environment

None

Integrations

No response

Anything else?

No response