-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Memory Leak and Connection Termination Issue in cap-js/Postgres during Mass Upserts #759
Comments
@GianfilippoDerga Thank you for reporting your observation. I have setup a test case to reproduce this issue here: #762 When I run the test case with the default behavior I get the following memory consumption pattern on Postgres: So I did some searching through the Postgres documentation and found the There doesn't seem to be a leak in the There is a feature in Postgres that will automatically call Additionally I would like to point out that it is possible with |
I used pgAdmin to execute a I have changed the pool config for db in package.json adding "db": {
"impl": "@cap-js/postgres",
"pool": {
"max": 50,
"min": 20,
"acquireTimeoutMillis": 5000,
"idleTimeoutMillis": 7500,
"softIdleTimeoutMillis": 5000,
"evictionRunIntervalMillis": 5000,
"numTestsPerEvictionRun": 50
},
"kind": "postgres",
"credentials": {
"host": "localhost",
"port": 5432,
"database": "sfatest",
"user": "postgres",
"password": "postgres"
},
"schema_evolution": "auto"
}, In the library node-pool if not specified if As I'm understanding, |
@GianfilippoDerga good to know that the problem could be solved with the pool configurations you shared. Your explanation of the pool configurations are the same as my understanding of them. |
looks like this has been resolved, feel free to ask any further questions here |
Description of erroneous behaviour
After migrating to cap-js from cds-dbm as a dbs i have a problem related to the db postgres resource handling by the library.
I'm using a specific logic to procedurally generate sql queries to execute massive upsert operations (5000 rows at a time) in my postegres db and after some time the database memory gets saturated and i get multiple errors with
Connection terminated unexpectedly
This errors generate some unhandled exeptions that make my Node.js application crash
db definition in package.json:
We have an insert function for our db tables that cause the problem defined as follow:
An shortened example of the query generated by the generateUpsert function is the following:
This first screen show the usage of memory (right graph).
After come time from the memory saturation the app crashes and the memory usage drops
If we increase the allocated memory the app does not crash but we can notice that the ram usage only increases and the memory usage doesn't get freed after the end of the import logic process execution.
Detailed Steps to Reproduce
Details about your project
The text was updated successfully, but these errors were encountered: