Skip to content

Update from upstream #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Apr 12, 2016
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 3 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ var conString = "postgres://username:password@localhost/database";

//this initializes a connection pool
//it will keep idle connections open for a (configurable) 30 seconds
//and set a limit of 20 (also configurable)
//and set a limit of 10 (also configurable)
pg.connect(conString, function(err, client, done) {
if(err) {
return console.error('error fetching client from pool', err);
Expand Down Expand Up @@ -130,31 +130,8 @@ Follow me [@briancarlson](https://twitter.com/briancarlson) to keep up to date.

## Extras

node-postgres is by design pretty light on abstractions. These are some handy modules we've been using over the years to complete the picture:

- [brianc/node-pg-native](https://github.com/brianc/node-pg-native) - Simple interface abstraction on top of [libpq](https://github.com/brianc/node-libpq)
- [brianc/node-pg-query-stream](https://github.com/brianc/node-pg-query-stream) - Query results from node-postgres as a readable (object) stream
- [brianc/node-pg-cursor](https://github.com/brianc/node-pg-cursor) - Query cursor extension for node-postgres
- [brianc/node-pg-copy-streams](https://github.com/brianc/node-pg-copy-streams) - COPY FROM / COPY TO for node-postgres. Stream from one database to another, and stuff.
- [brianc/node-postgres-pure](https://github.com/brianc/node-postgres-pure) - node-postgres without any of the C/C++ stuff
- [brianc/node-pg-types](https://github.com/brianc/node-pg-types) - Type parsing for node-postgres
- [Suor/pg-bricks](https://github.com/Suor/pg-bricks) - A higher level wrapper around node-postgres to handle connection settings, sql generation, transactions and ease data access.
- [grncdr/node-any-db](https://github.com/grncdr/node-any-db) - Thin and less-opinionated database abstraction layer for node.
- [brianc/node-sql](https://github.com/brianc/node-sql) - SQL generation for node.js
- [hiddentao/squel](https://hiddentao.github.io/squel/) - SQL query string builder for Javascript
- [CSNW/sql-bricks](https://github.com/CSNW/sql-bricks) - Transparent, Schemaless SQL Generation
- [datalanche/node-pg-format](https://github.com/datalanche/node-pg-format) - Safely and easily create dynamic SQL queries with this Node implementation of [PostgreSQL format()](http://www.postgresql.org/docs/9.3/static/functions-string.html#FUNCTIONS-STRING-FORMAT).
- [iceddev/pg-transact](https://github.com/iceddev/pg-transact) - A nicer API on node-postgres transactions
- [sehrope/node-pg-db](https://github.com/sehrope/node-pg-db) - Simpler interface, named parameter support, transaction management and event hooks.
- [vitaly-t/pg-promise](https://github.com/vitaly-t/pg-promise) - Use node-postgres via [Promises/A+](https://promisesaplus.com/).
- [kriasoft/node-pg-client](https://github.com/kriasoft/node-pg-client) - Promise-based wrapper for `node-postgres` designed for easy use with ES7 async/await.
- [pg-then](https://github.com/coderhaoxin/pg-then) A tiny wrapper of `pg` for promise api.
- [pg-rxjs](https://github.com/jadbox/pg-rxjs) Another tiny wrapper like `pg-then` but for [RxJS](https://github.com/Reactive-Extensions/RxJS)
- [acarl/pg-restify](https://github.com/acarl/pg-restify) - Creates a generic REST API for a postgres database using restify.
- [XeCycle/pg-template-tag](https://github.com/XeCycle/pg-template-tag) - Write queries with ES6 tagged template literals, a "poor man's query builder".
- [recursivefunk/pg-gen](https://github.com/recursivefunk/pg-gen) - Use ES6 Generators to paginate through large Postgres result sets
- [vitaly-t/pg-minify](https://github.com/vitaly-t/pg-minify) - Minifies PostgreSQL scripts.
- [MassiveJS](https://github.com/robconery/massive-js) - A simple relational data access tool that has full JSONB document support for Postgres.
node-postgres is by design pretty light on abstractions. These are some handy modules we've been using over the years to complete the picture.
Entire list can be found on [wiki](https://github.com/brianc/node-postgres/wiki/Extras)

## License

Expand Down
2 changes: 1 addition & 1 deletion lib/connection.js
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@ Connection.prototype.cancel = function(processID, secretKey) {
.addInt16(5678)
.addInt32(processID)
.addInt32(secretKey)
.addCString('').flush();
.flush();

var length = bodyBuffer.length + 4;

Expand Down
4 changes: 2 additions & 2 deletions lib/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ var Connection = require('./connection');
var PG = function(clientConstructor) {
EventEmitter.call(this);
this.defaults = defaults;
this.Client = pool.Client = clientConstructor;
this.Client = clientConstructor;
this.Query = this.Client.Query;
this.pools = pool;
this.pools = pool(clientConstructor);
this.Connection = Connection;
this.types = require('pg-types');
};
Expand Down
162 changes: 82 additions & 80 deletions lib/pool.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,93 +3,95 @@ var EventEmitter = require('events').EventEmitter;
var defaults = require('./defaults');
var genericPool = require('generic-pool');

var pools = {
//dictionary of all key:pool pairs
all: {},
//reference to the client constructor - can override in tests or for require('pg').native
Client: require('./client'),
getOrCreate: function(clientConfig) {
clientConfig = clientConfig || {};
var name = JSON.stringify(clientConfig);
var pool = pools.all[name];
if(pool) {
return pool;
}
pool = genericPool.Pool({
name: name,
max: clientConfig.poolSize || defaults.poolSize,
idleTimeoutMillis: clientConfig.poolIdleTimeout || defaults.poolIdleTimeout,
reapIntervalMillis: clientConfig.reapIntervalMillis || defaults.reapIntervalMillis,
log: clientConfig.poolLog || defaults.poolLog,
create: function(cb) {
var client = new pools.Client(clientConfig);
// Ignore errors on pooled clients until they are connected.
client.on('error', Function.prototype);
client.connect(function(err) {
if(err) return cb(err, null);

// Remove the noop error handler after a connection has been established.
client.removeListener('error', Function.prototype);
module.exports = function(Client) {
var pools = {
//dictionary of all key:pool pairs
all: {},
//reference to the client constructor - can override in tests or for require('pg').native
getOrCreate: function(clientConfig) {
clientConfig = clientConfig || {};
var name = JSON.stringify(clientConfig);
var pool = pools.all[name];
if(pool) {
return pool;
}
pool = genericPool.Pool({
name: name,
max: clientConfig.poolSize || defaults.poolSize,
idleTimeoutMillis: clientConfig.poolIdleTimeout || defaults.poolIdleTimeout,
reapIntervalMillis: clientConfig.reapIntervalMillis || defaults.reapIntervalMillis,
log: clientConfig.poolLog || defaults.poolLog,
create: function(cb) {
var client = new Client(clientConfig);
// Ignore errors on pooled clients until they are connected.
client.on('error', Function.prototype);
client.connect(function(err) {
if(err) return cb(err, null);

//handle connected client background errors by emitting event
//via the pg object and then removing errored client from the pool
client.on('error', function(e) {
pool.emit('error', e, client);
// Remove the noop error handler after a connection has been established.
client.removeListener('error', Function.prototype);

// If the client is already being destroyed, the error
// occurred during stream ending. Do not attempt to destroy
// the client again.
if (!client._destroying) {
pool.destroy(client);
}
});
//handle connected client background errors by emitting event
//via the pg object and then removing errored client from the pool
client.on('error', function(e) {
pool.emit('error', e, client);

// Remove connection from pool on disconnect
client.on('end', function(e) {
// Do not enter infinite loop between pool.destroy
// and client 'end' event...
if ( ! client._destroying ) {
// If the client is already being destroyed, the error
// occurred during stream ending. Do not attempt to destroy
// the client again.
if (!client._destroying) {
pool.destroy(client);
}
});

// Remove connection from pool on disconnect
client.on('end', function(e) {
// Do not enter infinite loop between pool.destroy
// and client 'end' event...
if ( ! client._destroying ) {
pool.destroy(client);
}
});
client.poolCount = 0;
return cb(null, client);
});
},
destroy: function(client) {
client._destroying = true;
client.poolCount = undefined;
client.end();
}
});
pools.all[name] = pool;
//mixin EventEmitter to pool
EventEmitter.call(pool);
for(var key in EventEmitter.prototype) {
if(EventEmitter.prototype.hasOwnProperty(key)) {
pool[key] = EventEmitter.prototype[key];
}
}
//monkey-patch with connect method
pool.connect = function(cb) {
var domain = process.domain;
pool.acquire(function(err, client) {
if(domain) {
cb = domain.bind(cb);
}
if(err) return cb(err, null, function() {/*NOOP*/});
client.poolCount++;
cb(null, client, function(err) {
if(err) {
pool.destroy(client);
} else {
pool.release(client);
}
});
client.poolCount = 0;
return cb(null, client);
});
},
destroy: function(client) {
client._destroying = true;
client.poolCount = undefined;
client.end();
}
});
pools.all[name] = pool;
//mixin EventEmitter to pool
EventEmitter.call(pool);
for(var key in EventEmitter.prototype) {
if(EventEmitter.prototype.hasOwnProperty(key)) {
pool[key] = EventEmitter.prototype[key];
}
};
return pool;
}
//monkey-patch with connect method
pool.connect = function(cb) {
var domain = process.domain;
pool.acquire(function(err, client) {
if(domain) {
cb = domain.bind(cb);
}
if(err) return cb(err, null, function() {/*NOOP*/});
client.poolCount++;
cb(null, client, function(err) {
if(err) {
pool.destroy(client);
} else {
pool.release(client);
}
});
});
};
return pool;
}
};
};

module.exports = pools;
return pools;
};
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "pg",
"version": "4.5.1",
"version": "4.5.3",
"description": "PostgreSQL client - pure javascript & libpq with the same API",
"keywords": [
"postgres",
Expand Down
27 changes: 27 additions & 0 deletions test/integration/gh-issues/981-tests.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
var helper = require(__dirname + '/../test-helper');

//native bindings are only installed for native tests
if(!helper.args.native) {
return;
}

var assert = require('assert')
var pg = require('../../../lib')
var native = require('../../../lib').native

var JsClient = require('../../../lib/client')
var NativeClient = require('../../../lib/native')

assert(pg.Client === JsClient);
assert(native.Client === NativeClient);

pg.connect(function(err, client, done) {
assert(client instanceof JsClient);
client.end();

native.connect(function(err, client, done) {
assert(client instanceof NativeClient);
client.end();
});
});

15 changes: 7 additions & 8 deletions test/unit/pool/basic-tests.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@ var util = require('util');
var EventEmitter = require('events').EventEmitter;

var libDir = __dirname + '/../../../lib';
var poolsFactory = require(libDir + '/pool')
var defaults = require(libDir + '/defaults');
var pools = require(libDir + '/pool');
var poolId = 0;

require(__dirname + '/../../test-helper');
Expand All @@ -21,6 +21,7 @@ FakeClient.prototype.connect = function(cb) {
FakeClient.prototype.end = function() {
this.endCalled = true;
}
var pools = poolsFactory(FakeClient);

//Hangs the event loop until 'end' is called on client
var HangingClient = function(config) {
Expand All @@ -41,8 +42,6 @@ HangingClient.prototype.end = function() {
clearInterval(this.intervalId);
}

pools.Client = FakeClient;

test('no pools exist', function() {
assert.empty(Object.keys(pools.all));
});
Expand Down Expand Up @@ -115,7 +114,7 @@ test('on client error, client is removed from pool', function() {
});

test('pool with connection error on connection', function() {
pools.Client = function() {
var errorPools = poolsFactory(function() {
return {
connect: function(cb) {
process.nextTick(function() {
Expand All @@ -124,9 +123,10 @@ test('pool with connection error on connection', function() {
},
on: Function.prototype
};
};
})

test('two parameters', function() {
var p = pools.getOrCreate(poolId++);
var p = errorPools.getOrCreate(poolId++);
p.connect(assert.calls(function(err, client) {
assert.ok(err);
assert.equal(client, null);
Expand All @@ -136,7 +136,7 @@ test('pool with connection error on connection', function() {
}));
});
test('three parameters', function() {
var p = pools.getOrCreate(poolId++);
var p = errorPools.getOrCreate(poolId++);
var tid = setTimeout(function() {
assert.fail('Did not call connect callback');
}, 100);
Expand All @@ -155,7 +155,6 @@ test('pool with connection error on connection', function() {

test('returnning an error to done()', function() {
var p = pools.getOrCreate(poolId++);
pools.Client = FakeClient;
p.connect(function(err, client, done) {
assert.equal(err, null);
assert(client);
Expand Down
5 changes: 3 additions & 2 deletions test/unit/pool/timeout-tests.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ var EventEmitter = require('events').EventEmitter;

var libDir = __dirname + '/../../../lib';
var defaults = require(libDir + '/defaults');
var pools = require(libDir + '/pool');
var poolsFactory = require(libDir + '/pool');
var poolId = 0;

require(__dirname + '/../../test-helper');
Expand All @@ -25,8 +25,9 @@ FakeClient.prototype.end = function() {
defaults.poolIdleTimeout = 10;
defaults.reapIntervalMillis = 10;

var pools = poolsFactory(FakeClient)

test('client times out from idle', function() {
pools.Client = FakeClient;
var p = pools.getOrCreate(poolId++);
p.connect(function(err, client, done) {
done();
Expand Down