Move external command execution from token source provider to token source

[andrei-trandafir]

Tracking issue

#6254

Why are the changes needed?

External command doesn't get called again if authentication starts failing due to the returned token.

What changes were proposed in this pull request?

Create a new externalCommandTokenSource that runs the external command instead of the token source provider.
This results in:

• The external command is called as part of the MaterialiseCredentials function instead of the getTokenSourceAndMetadata function that is synchronized (in the flyteidl/go/client/admin/auth_interceptor.go).
• If the token in the token cache results in an Unauthenticated response from the flyte admin, the external command is called again.

How was this patch tested?

• Deployed the change to a flytepropeller instance using ExternalCommand auth that returns an expiring token
• Waited for the expiry of the token and checked whether authentication still works

Check all the applicable boxes

• I updated the documentation accordingly.
• All new and existing tests passed.
• All commits are signed-off.

Related PRs

The issue resulted from the following PR: #5686

Summary by Bito

Refactored external command token source implementation with improved token caching and refresh mechanism. Added mutex-protected token caching to prevent unnecessary command executions. Enhanced authentication system by implementing proper token expiration checks. Modified token generation logic to ensure efficient external command execution.

Unit tests added: False

Estimated effort to review (1-5, lower is better): 1

[welcome]

Thank you for opening this pull request! 🙌

These tips will help get your PR across the finish line:

• Most of the repos have a PR template; if not, fill it out to the best of your knowledge.
• Sign off your commits (Reference: DCO Guide).

[flyte-bot]

Code Review Agent Run #cbee27

Actionable Suggestions - 0

Review Details

• Files reviewed - 1 · Commit Range: dc31a56..dc31a56

  • flyteidl/clients/go/admin/token_source_provider.go
• Files skipped - 0
• Tools

  • Golangci-lint (Linter) - ✖︎ Failed
  • Whispers (Secret Scanner) - ✔︎ Successful
  • Detect-secrets (Secret Scanner) - ✔︎ Successful

---

AI Code Review powered by

[flyte-bot]

Changelist by Bito

This pull request implements the following key changes.

Key Change	Files Impacted
Feature Improvement - Authentication Token Source Refactoring	- token_source_provider.go - Refactored token source provider to use a dedicated externalCommandTokenSource struct for better token refresh handling

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 36.87%. Comparing base (a8b9db2) to head (13278eb).
Report is 3 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #6255   +/-   ##
=======================================
  Coverage   36.86%   36.87%           
=======================================
  Files        1318     1318           
  Lines      134767   134771    +4     
=======================================
+ Hits        49682    49692   +10     
+ Misses      80755    80749    -6     
  Partials     4330     4330           

Flag	Coverage Δ
unittests-datacatalog	51.58% <ø> (ø)
unittests-flyteadmin	51.87% <ø> (+0.02%)	⬆️
unittests-flytecopilot	30.99% <ø> (ø)
unittests-flytectl	62.33% <ø> (ø)
unittests-flyteidl	7.23% <100.00%> (+<0.01%)	⬆️
unittests-flyteplugins	54.03% <ø> (ø)
unittests-flytepropeller	42.78% <ø> (ø)
unittests-flytestdlib	55.33% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[flyte-bot]

Code Review Agent Run Status

• Limitations and other issues: ❌ Failure - The AI Code Review Agent skipped reviewing this change because it is configured to exclude certain pull requests based on the source/target branch or the pull request status. You can change the settings here, or contact the agent instance creator at eduardo@union.ai.

[eapolinario]

Looks reasonable, I have one small comment. Also, can you add a test to auth_interceptor_test.go?

[eapolinario]

Do we really need to define externalCommandTokenSource? We could implement Token in ExternalTokenSourceProvider, right?

[andrei-trandafir]

We could implement Token() in ExternalTokenSourceProvider, but I feel it would potentially feel a bit confusing to follow.
I don't personally mind either way.