Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(pool): returning closed resources to the pool #154

Merged
merged 5 commits into from
Sep 6, 2021

Conversation

stakach
Copy link
Contributor

@stakach stakach commented Sep 3, 2021

returning closed resources is undesirable

@bcardiff
Copy link
Member

bcardiff commented Sep 4, 2021

I agree, but in your context, how is the connection closed in the first place? I would like to have a spec to cover that scenario. It seems easy to reintroduce in future refactors otherwise.

@stakach
Copy link
Contributor Author

stakach commented Sep 4, 2021

We did this: place-labs/crystal-pg@5db17e3
as the error was being handled before it made its way back to crystal-db, before that commit the actual connection was closed but the DB::Connection was still open. We made that change and then found it surprising that we were being served a closed connection from the pool.

it might actually be worthwhile calling close in the DB::ConnectionLost initialiser too

@stakach
Copy link
Contributor Author

stakach commented Sep 4, 2021

Spec added :)

@@ -141,7 +141,9 @@ module DB
idle_pushed = false

sync do
if can_increase_idle_pool
if resource.responds_to?(:closed?) && resource.closed?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this condition necessary for any practical application? I think it would be better to require a #closed? method. The interface already expects #close, so it would be nearby.
This might be a breaking change, but I'd accept that in order to avoid hitting this problem when a resource class doesn't implement closed?.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to say that is not needed, but some people want to use the pool with http client. The HTTP::Client has a #close, but has no #closed?.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably add that to HTTP::Client 🤔

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could, but the HTTP::Client might be reopened. I'm not sure what a good choice for interacting with it through a pool would be. But for now the responds_to? needs to stay because of that.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was wondering if there's an option for more conservative behaviour when the resource does not respond to #closed?. But I don't think there is one except unconditionally deletion, which would defeat the purpose of a pool.

Maybe it makes sense to consider a resource type that does not implement #closed? as being able to reset/reopen itself on subsequent checkouts.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it makes sense to consider a resource type that does not implement #closed? as being able to reset/reopen itself on subsequent checkouts.

And that would be the current behavior. Or you think it's not?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's it.

Copy link
Member

@bcardiff bcardiff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is good to go.

Regarding

it might actually be worthwhile calling close in the DB::ConnectionLost initialiser too

Yes, I think that is a good place to put it. That will also solve a leak in the prepared statement cache where the connection are cleaned if they are closed.

I was thinking if ConnectionRefused should also be closed, but in that exception type that is used upon connection creation there is no connection object. Because it couldn't initialize. So I think is on each driver to clean up resources in this situation.


We can add/discuss the connection close upon DB::ConnectionLost creation on a following PR.

@bcardiff bcardiff merged commit 6dc3f2d into crystal-lang:master Sep 6, 2021
@bcardiff
Copy link
Member

bcardiff commented Sep 6, 2021

Thanks @stakach!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants