[Feature/Identity] Identity Module and tokens for internal authentication

[cwperks]

Description

Opening a draft PR to solicit feedback for implementation of internal authentication.

This PR introduces a new sandbox module identity that will use some of the existing extension points that the security plug-in does to authenticate rest requests and pass a token around on the header of the threadcontext of a task that identifies the user and can subsequently be used for authorization.

This new identity module uses a few existing extension points from the ActionPlugin and the NetworkPlugin.

From the ActionPlugin this branch uses:

• getRestHandlerWrapper to provide a wrapper that handles authentication. As of now, there is only a Basic auth mechanism that uses the internal IdP in this feature branch to authenticate the user and return a 403 if the request cannot be authenticated
• getActionFilters - This branch introduces an AuthorizationFilter that is intended to be used to perform authorization. This is mostly pass-through at the moment and right now it verifies that a token is present and valid before the TransportRequest performs its doExecute

From the NetworkPlugin this uses:

• getTransportInterceptors - The transport interceptor intercepts outgoing TransportRequests and can modify the request before its sent to another node. When testing this branch, I ran into problems with how the TransportMessageListener intercepted outgoing requests as the ThreadContext was not available to inspect to ensure that the token that received the RestRequest created a token before sending the transport request to other nodes. When running the test its clear to see that other nodes received it the created token, but the message listener is unable to get it because of how its wrapped in an ActionListener in OutboundHandler:

void sendRequest(...) throws IOException, TransportException {
        Version version = Version.min(this.version, channelVersion);
        OutboundMessage.Request message = new OutboundMessage.Request(
            threadPool.getThreadContext(),
            features,
            request,
            version,
            action,
            requestId,
            isHandshake,
            compressRequest
        );
        ActionListener<Void> listener = ActionListener.wrap(() -> messageListener.onRequestSent(node, requestId, action, request, options));
        sendMessage(channel, message, listener);
    }

The transport interceptor has access to the ThreadContext and the tests will be updated to use the interceptor.

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Commits are signed per the DCO using --signoff
• Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/7700/
• CommitID: e136240
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/7701/
• CommitID: c3e3712
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/7703/
• CommitID: c3e3712
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/7735/
• CommitID: ca619bb
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/7736/
• CommitID: c7f0844
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/7737/
• CommitID: 8d321cd
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/7741/
• CommitID: ce1ee4b
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/8103/
• CommitID: 22695a6
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/8111/
• CommitID: 88232b7
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: ABORTED ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/8104/
• CommitID: a3b6216
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/8112/
• CommitID: 27db2a1
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[dreamer-89]

Gradle Check (Jenkins) Run Completed with:

• RESULT: ABORTED ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/8104/
• CommitID: a3b6216
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

Failing due to version bump after 2.4.1 release. #5560.

* What went wrong:
Execution failed for task ':distribution:bwc:bugfix:buildBwcLinuxTar'.
> Building 2.4.1 didn't generate expected file /var/jenkins/workspace/gradle-check/search/distribution/bwc/bugfix/build/bwc/checkout-2.4/distribution/archives/linux-tar/build/distributions/opensearch-min-2.4.1-SNAPSHOT-linux-x64.tar.gz

[dreamer-89]

@cwperks : Can you please rebase your changes against latest changes in main ?

[cwperks]

@dreamer-89 Thank you for checking on that. Was that build issue resolved with this PR (#5570)? I will merge the latest from main into the identity feature branch and rebase this branch.

[dreamer-89]

@dreamer-89 Thank you for checking on that. Was that build issue resolved with this PR (#5570)? I will merge the latest from main into the identity feature branch and rebase this branch.

Thanks @cwperks. Yes, issue is resolved on latest main. Please let know if you see any other issue on main.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/8119/
• CommitID: 27db2a1
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS ✅
• URL: https://build.ci.opensearch.org/job/gradle-check/8120/
• CommitID: 459c18e

[codecov-commenter]

Codecov Report

Merging #5471 (1d56fa3) into feature/identity (31bcda6) will decrease coverage by 0.00%.
The diff coverage is 60.78%.

@@                  Coverage Diff                   @@
##             feature/identity    #5471      +/-   ##
======================================================
- Coverage               71.01%   71.00%   -0.01%     
- Complexity              58149    58509     +360     
======================================================
  Files                    4711     4768      +57     
  Lines                  277573   278956    +1383     
  Branches                40180    40296     +116     
======================================================
+ Hits                   197122   198079     +957     
- Misses                  64293    64783     +490     
+ Partials                16158    16094      -64     

Impacted Files	Coverage Δ
...search/transport/Netty4NioServerSocketChannel.java	0.00% <0.00%> (ø)
.../java/org/opensearch/transport/NettyAllocator.java	45.45% <0.00%> (ø)
...rch/authn/internal/InternalAccessTokenManager.java	0.00% <0.00%> (ø)
.../opensearch/authn/jwt/BadCredentialsException.java	0.00% <0.00%> (ø)
.../opensearch/authn/noop/NoopAccessTokenManager.java	0.00% <0.00%> (ø)
.../java/org/opensearch/authn/tokens/AccessToken.java	0.00% <0.00%> (ø)
.../java/org/opensearch/identity/ConfigConstants.java	0.00% <0.00%> (ø)
...rg/opensearch/identity/ThreadContextConstants.java	0.00% <0.00%> (ø)
.../main/java/org/opensearch/action/ActionModule.java	99.11% <ø> (+1.47%)	⬆️
...tion/admin/cluster/state/ClusterStateResponse.java	80.00% <0.00%> (-2.36%)	⬇️
... and 590 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: null ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/8142/
• CommitID: d53c4b8
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS ✅
• URL: https://build.ci.opensearch.org/job/gradle-check/8144/
• CommitID: 1d56fa3

[cwperks]

I opened a separate PR after a recent merge with main into feature/identity to squash these commits and simplify. Closing this PR in favor of: #5583

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS ✅
• URL: https://build.ci.opensearch.org/job/gradle-check/8147/
• CommitID: 4ddbd03