403 Forbidden Responses as anonymous user should be 401 Unauthorized

[jgough]

What is the bug?
On a cluster with anonymous access enabled, when I try to access a resource for which I do not have permissions then I receive a 403 Forbidden response

e.g.

< HTTP/1.1 403 Forbidden
< Content-Type: application/json; charset=UTF-8
< Date: Thu, 05 May 2022 06:09:13 GMT
< Transfer-Encoding: chunked
<
* Connection #0 to host opensearch left intact
{"error":{"root_cause":[{"type":"security_exception","reason":"no permissions for [indices:admin/get] and User [name=opendistro_security_anonymous, backend_roles=[opendistro_security_anonymous_backendrole], requestedTenant=null]"}],"type":"security_exception","reason":"no permissions for [indices:admin/get] and User [name=opendistro_security_anonymous, backend_roles=[opendistro_security_anonymous_backendrole], requestedTenant=null]"},"status":403}

According to RFC 2616 a 403 Forbidden means

The server understood the request, but is refusing to fulfill it. Authorization will not help and the request SHOULD NOT be repeated.

A 401 Unauthorized would be correct here as authentication is required.

The request requires user authentication

This is causing interoperability issues with https://github.com/lmenezes/cerebro which rightly does not request credentials to authenticate after a 403 Forbidden response.

Wikipedia states

HTTP 403 provides a distinct error case from HTTP 401; while HTTP 401 is returned when the client has not authenticated, and implies that a successful response may be returned following valid authentication, HTTP 403 is returned when the client is not permitted access to the resource despite providing authentication such as insufficient permissions of the authenticated account.

How can one reproduce the bug?
Set up a cluster with anonymous access enabled
As an anonymous user, try to access a resource the unauthenticated user has no permissions for and observe that a 403 Forbidden error is returned.

What is the expected behavior?
A 401 Unauthorized response instead of 403 Forbidden when unauthenticated users try to access resources they do not have permissions for.

[DarshitChanpura]

[Triage] You're right. There is an ambiguity to this area. Thank you for reporting this. As this is a breaking change w.r.t backwards compatibility, we're not going to change the response code in 2.0 release, but we'll keep this proposal in mind if there is an opportunity to update the behavior.