refactor(Postgres Node): Backport connection pooling to postgres v1

[despairblue]

Summary

This uses the pool manager for postgres v1, but additionally it also fixes the credential test for postgres which before would shut down the shared pool and lead to Connection pool of the database object has been destroyed. on any subsequent execution.

Taken from the pg-promise docs:

Object db represents the Database protocol with lazy connection, i.e. only the actual query methods acquire and release the connection automatically. Therefore, you should create only one global/shared db object per connection details.

Thus it's not necessary to release connections manually. They are managed by pg-promise.

You can verify this using these queries and constructing a workflow that executes them once a second:

Lists all open connections per db, grouped by state.

SELECT 
    datname,
    usename,
    state,
    COUNT(*) 
FROM pg_stat_activity 
WHERE backend_type = 'client backend'
GROUP BY datname, usename, state;

Long running query to simulated load.

SELECT 
    g.i,
    pg_sleep(0.001)  -- each row sleeps for 1ms
FROM generate_series(1, 1000) g(i);

By default the pool has a size of 10. If we need more then the next execution hangs until the next connection is free.

That limit is way too low.

I set it to 10_000 to test what happens if I ran out of the 100 connections my local postgres allows:

Also quickly after deactivating the workflow the pool was cleaned up and all 100 connections have been available again.

I would like to set it to unlimited for now, but that's not possible:
brianc/node-postgres#1977

But for that reason I made it configurable in the credential.

Related Linear tickets, Github issues, and Community forum posts

Fixes #12517
NODE-2240

Review / Merge checklist

• PR title and summary are descriptive. (conventions)
• Docs updated or follow-up ticket created.
• Tests included.
• PR Labeled with release/backport (if the PR is an urgent fix that needs to be backported)

[despairblue]

This would have left the connection hanging. It needs to be ended:

https://vitaly-t.github.io/pg-promise/Database.html#connect

The connection must be released in the end of the chain by calling done() on the connection object.

[despairblue]

This would shut down all pools in the process:
https://vitaly-t.github.io/pg-promise/module-pg-promise.html#~end

Shuts down all connection pools created in the process, so it can terminate without delay. It is available as pgp.end, after initializing the library.

Doing this would render any pool for any credential unusable until restarting n8n or waiting for the pool to be destroyed by the connection pool manager and every execution would throw this:
Connection pool of the database object has been destroyed.

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 22 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...es/nodes-base/nodes/Postgres/v1/PostgresV1.node.ts	0.00%	9 Missing ⚠️
...s-base/nodes/Postgres/v2/methods/credentialTest.ts	0.00%	5 Missing ⚠️
...mory/MemoryPostgresChat/MemoryPostgresChat.node.ts	0.00%	1 Missing ⚠️
...re/VectorStorePGVector/VectorStorePGVector.node.ts	0.00%	1 Missing ⚠️
...s-base/nodes/Postgres/PostgresTrigger.functions.ts	0.00%	1 Missing ⚠️
.../nodes-base/nodes/Postgres/PostgresTrigger.node.ts	0.00%	1 Missing ⚠️
...ges/nodes-base/nodes/Postgres/v2/actions/router.ts	0.00%	1 Missing ⚠️
...nodes-base/nodes/Postgres/v2/methods/listSearch.ts	0.00%	1 Missing ⚠️
...odes-base/nodes/Postgres/v2/methods/loadOptions.ts	0.00%	1 Missing ⚠️
...-base/nodes/Postgres/v2/methods/resourceMapping.ts	0.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[github-actions]

⚠️ Some Cypress E2E specs are failing, please fix them before merging

[cypress]

n8n    Run #8772

Run Properties:   Passed #8772  •  9ce7027c46: 🌳 🖥️ browsers:node18.12.0-chrome107 🤖 despairblue 🗃️ e2e/*

Project	n8n
Branch Review	backport-connection-pooling-to-postgres-v1
Run status	Passed #8772
Run duration	04m 55s
Commit	9ce7027c46: 🌳 🖥️ browsers:node18.12.0-chrome107 🤖 despairblue 🗃️ e2e/*
Committer	Danny Martini
View all properties for this run ↗︎

---

Test results
Failures	0
Flaky	2
Pending	0
Skipped	0
Passing	489
View all changes introduced in this branch ↗︎

[dana-gill]

tested locally and it works 🥳 lgtm!

[github-actions]

⚠️ Some Cypress E2E specs are failing, please fix them before merging

[HermesMacedo]

Any news? release 1.75.0 has not been resolved.

[github-actions]

✅ All Cypress E2E specs passed

[despairblue]

Any news? release 1.75.0 has not been resolved.

@HermesMacedo This will be part of 1.75.1 which is scheduled to be released today.

[gonzalogcontacto]

Many people are waiting the fix to be deployed 🥲 Thanks for that fix @despairblue 🙌🏻

[janober]

Got released with n8n@1.74.2

[g-roliveira]

The problem still persists for me in version 1.74.2 and 1.75.1

[HermesMacedo]

@janober The problem still persists for me in version 1.75.1

It responds on the first interaction, but the error occurs on subsequent interactions.

{
"errorMessage": "Cannot use a pool after calling end on the pool",
"errorDescription": "Cannot use a pool after calling end on the pool",
"errorDetails": {},
"n8nDetails": {
"time": "17/01/2025, 10:09:11",
"n8nVersion": "1.75.1 (Self Hosted)",
"binaryDataMode": "default",
"cause": {}
}
}

[zberg007]

Same here on both versions. Cloud hosted. All subsequent interactions fail.

[netroy]

Please let's move the conversation to the issue #12517 . That way we don't have to keep posting the same message in multiple places.
#12517 (comment)