Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fail to get new connections #115

Closed
yaroot opened this issue Oct 14, 2020 · 8 comments
Closed

Fail to get new connections #115

yaroot opened this issue Oct 14, 2020 · 8 comments
Assignees
Labels
bug Something isn't working

Comments

@yaroot
Copy link

yaroot commented Oct 14, 2020

This is a follow up for #110

Client would throw this error and not be able to connect through pgagroal

pg8000.exceptions.ProgrammingError: {'S': 'FATAL', 'V': 'FATAL', 'C': '53300', 'M': 'connection pool is full'}

@yaroot yaroot added the bug Something isn't working label Oct 14, 2020
@yaroot
Copy link
Author

yaroot commented Oct 14, 2020

pgagroal.log.gz

Screenshot_2020-10-14T10:21:36+09:00

Metrics are in UTC, logs are in UTC+9:00

@jesperpedersen jesperpedersen self-assigned this Oct 14, 2020
@jesperpedersen
Copy link
Collaborator

Thanks for the trace !

However, it shows that it is maxes out on 70 connections so the error is correct if you used the configuration in #110. Could you do another run and keep an eye on /methics ?

Also, if there is a problem check the output from ps aux | grep pgagroal ? The pgagroal_active_connections should be in line with the actual connections...

Do you have a test case that I can use locally ?

@yaroot
Copy link
Author

yaroot commented Oct 30, 2020

Sorry the problem is that I've only opened 60 connections (30 from each program) and after I ^ced the clients it stayed at 70.

I put the code in this repo

@jesperpedersen
Copy link
Collaborator

This is very helpful, thank you !

Although you are missing a conn.rollback() and a conn.close() I can replicate the issue you are seeing when doing the ^C testing.

@jesperpedersen
Copy link
Collaborator

I have pushed a fix to master and 0.9.x - if would be great if you could do a rerun of all your test cases.

The main issue is that the clients doesn't know about the cached connections since there is no prefill in play. I'll think some more about that scenario...

@yaroot
Copy link
Author

yaroot commented Nov 3, 2020

I rerun the test on 5e01f52 for a few hours, nothing happened so far, still trying to trigger connection pool is full, usually the scala code would do the trick.

I've added .rollback() on exception (the code didn't even reach this branch since the last few commits), the connection will be closed by the with construction.

jesperpedersen added a commit that referenced this issue Nov 3, 2020
jesperpedersen added a commit that referenced this issue Nov 3, 2020
@jesperpedersen
Copy link
Collaborator

ok, sounds good. My tests with your clients are successful now.

@yaroot
Copy link
Author

yaroot commented Nov 5, 2020

Closing as there's nothing more I could find, it's a pleasure to work with you.

@yaroot yaroot closed this as completed Nov 5, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants