-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
spanner: Application Default Credentials fail to work if you cancel the context used for spanner.NewClient() #945
Comments
If you're trying to add a deadline to the connection dial, you'll need to do it another way. Use grpc.WithTimeout. Pass it in using option.WithGRPCDialOption. |
@jba But according to the docs you linked, this function is deprecated (and understandably so!). Should I still use it despite that? The recommended alternative |
Sorry, my mistake. You should probably dial your own gRPC connection (using a different context) and pass it in with Yes, it's unusual to hold on to a context, but it does happen sometimes. But we shouldn't be doing it here. I filed #946 to track. |
Thanks @jba. 🙂 |
As #946 says, you shouldn't try to set a timeout for the dial at all, because it may (will?) happen lazily. Set timeouts on the RPCs. |
I have a similar situation, which may or may not be worthy of a separate issue. I'm also using application default credentials. I'm not explicitly setting a timeout, but in development i have services which use spanner clients that run for a long time. If I close the laptop for a while, for example, over night, the first request in the morning gives:
Like the original poster, I expected the client to automatically recover from severed connections (for whatever reason). It looks like it's trying to do that, but failing to re-read or refresh the credentials. |
Actually, @majelbstoat, could you open a separate issue for this? |
Client
Spanner
Describe Your Environment
Local development using Windows or macOS and the
gcloud
SDK tool.Expected Behavior
I expect the spanner package to correctly pick up my local "Application Default Credentials", which have been set up using
gcloud auth application-default login
.Actual Behavior
The following code should be quite standard for creating spanner clients:
But if you call "cancel()" before you use the client (for instance due to the needed
defer cancel()
) all further queries and operations using the client will fail.Cause
(The cause for this issue as far as I understand it)
The context used in
spanner.NewClient(WithConfig)
is declared herehttps://github.com/GoogleCloudPlatform/google-cloud-go/blob/01c1d9b2a0ed098a4e1a7656984d3730e4b56224/spanner/client.go#L115
and used with
gtransport.Dial
herehttps://github.com/GoogleCloudPlatform/google-cloud-go/blob/01c1d9b2a0ed098a4e1a7656984d3730e4b56224/spanner/client.go#L152
This will result in the context being used for
internal.Creds
herehttps://github.com/google/google-api-go-client/blob/e4126357c891acdef6dcd7805daa4c6533be6544/transport/grpc/dial.go#L65
which will one way or the other finally call
credentialsFromJSON
https://github.com/golang/oauth2/blob/fdc9e635145ae97e6c2cb777c48305600cf515cb/google/default.go#L85-L99
This will cause a
TokenSource
to be created using this functionhttps://github.com/golang/oauth2/blob/fdc9e635145ae97e6c2cb777c48305600cf515cb/oauth2.go#L211-L223
The issue is - as far as I understand - that this function holds onto the context and when it gets canceled it will cause client side connections to fail with Unauthenticated errors since the oauth2 package's token refresher will refuse to work that way.
The text was updated successfully, but these errors were encountered: