Skip to content

Commit

Permalink
[cypress] PostgreSQL Connection pool for System Tests (#43924)
Browse files Browse the repository at this point in the history
* Use Connection pool for PostgreSQL in System Tests

Sometimes single specs in the System Tests fail with:
    CypressError: `cy.task('queryDB')` failed with the following error:
    > duplicate key value violates unique constraint "cpostgresmax_users_pkey"

The error happens on different specs in System Tests especially running on drone.
It could be reproduced in running the System Tests tree times (first with installation, 2nd and 3rd run w/o):
    npx cypress run
    npx cypress run --spec 'tests/System/integration/administrator/**/*.cy.{js,jsx,ts,tsx},tests/System/integration/site/**/*.cy.{js,jsx,ts,tsx},tests/System/integration/api/**/*.cy.{js,jsx,ts,tsx},tests/System/integration/plugins/**/*.cy.{js,jsx,ts,tsx}'
    npx cypress run --spec 'tests/System/integration/administrator/**/*.cy.{js,jsx,ts,tsx},tests/System/integration/site/**/*.cy.{js,jsx,ts,tsx},tests/System/integration/api/**/*.cy.{js,jsx,ts,tsx},tests/System/integration/plugins/**/*.cy.{js,jsx,ts,tsx}'

Checking the session usage in pgAdmin shows the maximum number of 100 configured sessions appears to have been reached.
Therefore the usage of postgres connection pool is implemented.
The session usage is reduced to max 12 and the error could no more reproduced.

* Simplify implementation

- Delete function as only called once
- Renamed to postgresConnectionPool

Contributed by @laoneo

---------

Co-authored-by: Allon Moritz <allon.moritz@digital-peak.com>
  • Loading branch information
muhme and laoneo authored Aug 29, 2024
1 parent e491457 commit 53a3390
Showing 1 changed file with 21 additions and 14 deletions.
35 changes: 21 additions & 14 deletions tests/System/plugins/db.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ import postgres from 'postgres';
// Items cache which are added by an insert statement
let insertedItems = [];

// Use of the PostgreSQL connection pool to limit the number of sessions, see
// https://github.com/porsager/postgres?tab=readme-ov-file#connection-details
let postgresConnectionPool = null;

/**
* Does run the given query against the database from the configuration. It caches all inserted items.
*
Expand All @@ -29,17 +33,20 @@ function queryTestDB(joomlaQuery, config) {
insertedItems.push(insertItem);
}

// Check if the DB is from postgres
// Do we use PostgreSQL?
if (config.env.db_type === 'pgsql' || config.env.db_type === 'PostgreSQL (PDO)') {
const connection = postgres({
host: config.env.db_host,
port: config.env.db_port,
database: config.env.db_name,
username: config.env.db_user,
password: config.env.db_password,
idle_timeout: 1,
max_lifetime: 1,
});

if (postgresConnectionPool === null) {
// Initialisation on the first call
postgresConnectionPool = postgres({
host: config.env.db_host,
port: config.env.db_port,
database: config.env.db_name,
username: config.env.db_user,
password: config.env.db_password,
max: 10, // Use only this (unchanged default) maximum number of connections in the pool
});
}

// Postgres delivers the data direct as result of the insert query
if (insertItem) {
Expand All @@ -49,7 +56,7 @@ function queryTestDB(joomlaQuery, config) {
// Postgres needs double quotes
query = query.replaceAll('`', '"');

return connection.unsafe(query).then((result) => {
return postgresConnectionPool.unsafe(query).then((result) => {
// Select query should always return an array
if (query.indexOf('SELECT') === 0 && !Array.isArray(result)) {
return [result];
Expand All @@ -64,12 +71,12 @@ function queryTestDB(joomlaQuery, config) {
insertItem.rows.push(result[0].id);
}

// Normalize the object
// Normalize the object and return from PostgreSQL
return { insertId: result[0].id };
});
}

// Return a promise which runs the query
// Return a promise which runs the query for MariaDB / MySQL
return new Promise((resolve, reject) => {
// Create the connection and connect
const connection = mysql.createConnection({
Expand All @@ -94,7 +101,7 @@ function queryTestDB(joomlaQuery, config) {
insertItem.rows.push(results.insertId);
}

// Resolve the result
// Resolve the result from MariaDB / MySQL
return resolve(results);
});
});
Expand Down

0 comments on commit 53a3390

Please sign in to comment.