-
Notifications
You must be signed in to change notification settings - Fork 16.3k
Re-work JWT Validation and Generation to use public/private key and official claims #46981
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
This comment was marked as resolved.
This comment was marked as resolved.
pierrejeambrun
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice, that's a big one!
f376eb1 to
1c9548c
Compare
1c9548c to
9ea089e
Compare
9ea089e to
c8dfbdd
Compare
97a4e5d to
cf06c57
Compare
cf06c57 to
3170f6e
Compare
I didn't want to unresolve the conversation, but I wanted to clarify that it was not strictly needed and there isn't any practical difference on the token. It is just a layer of flexibility for CLI since CLI is mostly used in automation, which could require longer periods than regular API calls coming from users. In the end, both are the same token and used in both places, but we could prevent that in the next steps. |
f8901d5 to
c57b9ee
Compare
17ed712 to
97723fe
Compare
7c56213 to
30d4aec
Compare
…I to use Since requests to Task Execution API can originate out-of-"cluser" so to speak, this PR re-works the JWTSigner class so that it is possible to use public/private keys (vs just a simple pre-shared secret). This is useful in many cloud environments where many companies have security requirements that ingress gateways must validate the JWT tokens on the way in, and the only way of doing this is with public keys So that we don't have two different ways of generating JWT tokens I have totally replaced the old "JWTSigner" class (which it turns out didn't have any unit test of its own, it was only tested indirectly through test_serve_logs etc). As part of this change I have also changed the JWT that was generated by the SimpleAuthManager and the AwsAuthManager (the only two we have that use JWT) to use the offical `sub` (subject) clain to place the user identifer rather than a custom claim name. And although it might seem slightly strange at first, I have made the JWTValidator an async class internally. (Hence `avalidated_claims` -- the `a` prefix signifies async, much like `aclose` or `aread` on HTTPX async responses). This allows us to periodically refresh the JWK document if configured in the background, and using asgiref's async_to_sync means we only have one version. And also conviently, this is the same tech that FastAPI uses, which means when this is called from within a FastAPI app, rather than creating a new/nested event loop it will "bubble out" to the main async loop that FastAPI is using.
30d4aec to
6721349
Compare
6721349 to
1eaa7c1
Compare
|
I think this PR introduces a bug in token validation. To reproduce:
After a very short investigation (sorry I could not put more time, I'll continue on Wednesday if it is not solved by then), the secret generated in |
|
Never mind, it has been fixed |
…fficial claims (apache#46981) Since requests to Task Execution API can originate out-of-"cluser" so to speak, this PR re-works the JWTSigner class so that it is possible to use public/private keys (vs just a simple pre-shared secret). This is useful in many cloud environments where many companies have security requirements that ingress gateways must validate the JWT tokens on the way in, and the only way of doing this is with public keys So that we don't have two different ways of generating JWT tokens I have totally replaced the old "JWTSigner" class (which it turns out didn't have any unit test of its own, it was only tested indirectly through test_serve_logs etc). As part of this change I have also changed the JWT that was generated by the SimpleAuthManager and the AwsAuthManager (the only two we have that use JWT) to use the offical `sub` (subject) clain to place the user identifer rather than a custom claim name. And although it might seem slightly strange at first, I have made the JWTValidator an async class internally. (Hence `avalidated_claims` -- the `a` prefix signifies async, much like `aclose` or `aread` on HTTPX async responses). This allows us to periodically refresh the JWK document if configured in the background, and using asgiref's async_to_sync means we only have one version. Co-authored-by: Jens Scheffler <jscheffl@apache.org>
…fficial claims (apache#46981) Since requests to Task Execution API can originate out-of-"cluser" so to speak, this PR re-works the JWTSigner class so that it is possible to use public/private keys (vs just a simple pre-shared secret). This is useful in many cloud environments where many companies have security requirements that ingress gateways must validate the JWT tokens on the way in, and the only way of doing this is with public keys So that we don't have two different ways of generating JWT tokens I have totally replaced the old "JWTSigner" class (which it turns out didn't have any unit test of its own, it was only tested indirectly through test_serve_logs etc). As part of this change I have also changed the JWT that was generated by the SimpleAuthManager and the AwsAuthManager (the only two we have that use JWT) to use the offical `sub` (subject) clain to place the user identifer rather than a custom claim name. And although it might seem slightly strange at first, I have made the JWTValidator an async class internally. (Hence `avalidated_claims` -- the `a` prefix signifies async, much like `aclose` or `aread` on HTTPX async responses). This allows us to periodically refresh the JWK document if configured in the background, and using asgiref's async_to_sync means we only have one version. Co-authored-by: Jens Scheffler <jscheffl@apache.org>
Since requests to Task Execution API can originate out-of-"cluser" so to
speak, this PR re-works the JWTSigner class so that it is possible to use
public/private keys (vs just a simple pre-shared secret). This is useful in
many cloud environments where many companies have security requirements that
ingress gateways must validate the JWT tokens on the way in, and the only way
of doing this is with public keys.
So that we don't have two different ways of generating JWT tokens I have
totally replaced the old "JWTSigner" class (which it turns out didn't have any
unit test of its own, it was only tested indirectly through test_serve_logs
etc).
This PR itself lands teh change to the JWT generation/validation codein Airflow,
but to keep things small(ish!) it doesn't actually use it inside the Execution API yet.
That will be in a future Pr
As part of this change I have also changed the JWT that was generated by the
SimpleAuthManager and the AwsAuthManager (the only two we have that use JWT)
to use the offical
sub(subject) clain to place the user identifer ratherthan a custom claim name.
And although it might seem slightly strange at first, I have made the
JWTValidator an async class internally. (Hence
avalidated_claims-- theaprefix signifies async, much like
acloseorareadon HTTPX asyncresponses). This allows us to periodically refresh the JWK document if
configured in the background, and using asgiref's async_to_sync means we only
have one version.
And also conveniently, this is the same tech that FastAPI uses, which means when
this is called from within a FastAPI app, rather than creating a new/nested
event loop it will "bubble out" to the main async loop that FastAPI is using.
Precursor to #45107
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.