Skip to content

Commit fe015f6

Browse files
jmd-mongojmd-mongo
authored
DOCSP-16501 adds connection pool overview (#288)
1 parent 70e968a commit fe015f6

File tree

7 files changed

+301
-1
lines changed

7 files changed

+301
-1
lines changed

‎snooty.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ toc_landing_pages = [
1717
"/administration/analyzing-mongodb-performance",
1818
"/administration/backup-sharded-clusters",
1919
"/administration/configuration-and-maintenance",
20+
"/administration/connection-pool-overview",
2021
"/administration/install-community",
2122
"/administration/install-enterprise-linux",
2223
"/administration/install-enterprise",

‎source/administration/analyzing-mongodb-performance.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -259,6 +259,7 @@ analyzing or debugging issues with support from MongoDB Inc. engineers.
259259
:titlesonly:
260260
:hidden:
261261

262+
/administration/connection-pool-overview
262263
/tutorial/manage-the-database-profiler
263264
/tutorial/transparent-huge-pages
264265
/reference/ulimit

‎source/administration/connection-pool-overview.txt

Lines changed: 183 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,183 @@
1+
.. _connection-pool-overview:
2+
3+
========================
4+
Connection Pool Overview
5+
========================
6+
7+
.. default-domain:: mongodb
8+
9+
.. contents:: On this page
10+
:local:
11+
:backlinks: none
12+
:depth: 1
13+
:class: singlecol
14+
15+
This document describes how to use a connection pool to manage
16+
connections between applications and MongoDB instances.
17+
18+
What is a Connection Pool?
19+
--------------------------
20+
21+
Definition
22+
~~~~~~~~~~
23+
24+
A :ref:`connection pool <connection-pool-overview>` is a cache of open,
25+
ready-to-use database connections maintained by the :driver:`driver </>`.
26+
Your application can seamlessly get connections from the pool, perform
27+
operations, and return connections back to the pool. Connection pools
28+
are thread-safe.
29+
30+
Benefits of a Connection Pool
31+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
32+
33+
A connection pool helps reduce application latency and the number of
34+
times new connections are created.
35+
36+
A connection pool creates connections at startup. Applications do not
37+
need to manually return connections to the pool. Instead, connections
38+
return to the pool automatically.
39+
40+
Some connections are active and some are inactive but available.
41+
If your application requests a connection and there’s an available
42+
connection in the pool, a new connection does not need to be created.
43+
44+
Create and Use a Connection Pool
45+
--------------------------------
46+
47+
Use an Instance of your Driver's ``MongoClient`` Object
48+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
49+
50+
Most :driver:`drivers </>` provide an object of type ``MongoClient``.
51+
52+
Use one ``MongoClient`` instance per application unless the
53+
application is connecting to many separate clusters. Each
54+
``MongoClient`` instance manages its own connection pool to the
55+
MongoDB cluster or node specified when the ``MongoClient`` is created.
56+
``MongoClient`` objects are thread-safe in most drivers.
57+
58+
.. note::
59+
60+
Store your ``MongoClient`` instance in a place that is globally
61+
accessible by your application.
62+
63+
Authentication
64+
~~~~~~~~~~~~~~
65+
66+
To use a connection pool with LDAP, see
67+
:ref:`LDAP Connection Pool Behavior <ldap-connection-pool-behavior>`.
68+
69+
Sharded Cluster Connection Pooling
70+
----------------------------------
71+
72+
:binary:`~bin.mongos` routers have connection pools for each node in the
73+
cluster. The availability of connections to individual nodes within a
74+
sharded cluster affects latency. Operations must wait for a connection
75+
to be established.
76+
77+
Connection Pool Configuration Settings
78+
--------------------------------------
79+
80+
To configure the connection pool, set the options:
81+
82+
- through the :ref:`MongoDB URI <mongodb-uri>`,
83+
84+
- programmatically when building the ``MongoClient`` instance, or
85+
86+
- in your application framework's configuration files.
87+
88+
Settings
89+
~~~~~~~~
90+
91+
.. list-table::
92+
:widths: 25,75
93+
:header-rows: 1
94+
95+
* - Setting
96+
- Description
97+
98+
* - :urioption:`maxPoolSize`
99+
100+
- .. _maxpoolsize-cp-setting:
101+
102+
Maximum number of connections opened in the pool. When the
103+
connection pool reaches the maximum number of connections, new
104+
connections wait up until to the value of
105+
:urioption:`waitQueueTimeoutMS`.
106+
107+
*Default:* ``100``
108+
109+
* - :urioption:`minPoolSize`
110+
111+
- .. _minpoolsize-cp-setting:
112+
113+
Minimum number of connections opened in the pool.
114+
The value of :urioption:`minPoolSize` must be less than
115+
the value of :urioption:`maxPoolSize`.
116+
117+
*Default*: ``0``
118+
119+
* - :urioption:`connectTimeoutMS`
120+
121+
- Most drivers default to never time out. Some versions of the
122+
Java drivers (for example, version 3.7) default to ``10``.
123+
124+
*Default:* ``0`` for most drivers. See your :driver:`driver </>`
125+
documentation.
126+
127+
* - :urioption:`socketTimeoutMS`
128+
129+
- Number of milliseconds to wait before timeout on a TCP
130+
connection.
131+
132+
Do *not* use :urioption:`socketTimeoutMS` as a mechanism for
133+
preventing long-running server operations.
134+
135+
Setting low socket timeouts may result in operations that error
136+
before the server responds.
137+
138+
*Default*: ``0``, which means no timeout. See your
139+
:driver:`driver </>` documentation.
140+
141+
* - :urioption:`maxIdleTimeMS`
142+
143+
- Amount of time that a connection can be idle in the pool before
144+
closing. Idle connections close until the number of
145+
open connections equals :urioption:`minPoolSize`.
146+
147+
*Default:* See your :driver:`driver </>` documentation.
148+
149+
* - :urioption:`waitQueueTimeoutMS`
150+
151+
- Maximum wait time in milliseconds that a can thread wait for
152+
a connection to become available. A value of ``0`` means there
153+
is no limit.
154+
155+
*Default*: ``0``. See your :driver:`driver </>` documentation.
156+
157+
* - :parameter:`ShardingTaskExecutorPoolMinSize`
158+
159+
- Minimum number of outbound connections each TaskExecutor
160+
connection pool can open to any given :binary:`~bin.mongod`
161+
instance.
162+
163+
*Default*: ``1``. See
164+
:parameter:`ShardingTaskExecutorPoolMinSize`.
165+
166+
This setting only applies to sharded deployments.
167+
168+
* - :parameter:`ShardingTaskExecutorPoolMaxSize`
169+
170+
- Maximum number of outbound connections each TaskExecutor
171+
connection pool can open to any given :binary:`~bin.mongod`
172+
instance.
173+
174+
*Default*: 2\ :sup:`64` - 1. See
175+
:parameter:`ShardingTaskExecutorPoolMaxSize`.
176+
177+
This setting only applies to sharded deployments.
178+
179+
.. toctree::
180+
:titlesonly:
181+
:hidden:
182+
183+
/tutorial/connection-pool-performance-tuning

‎source/core/security-ldap-external.txt

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -92,6 +92,8 @@ authorization:
9292
- :ref:`security-kerberos`
9393
- :ref:`security-auth-x509`
9494

95+
.. _ldap-connection-pool-behavior:
96+
9597
Connection Pool
9698
~~~~~~~~~~~~~~~
9799

‎source/reference/glossary.txt

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -188,6 +188,11 @@ Glossary
188188
associated with a :term:`sharded cluster`.
189189
See :ref:`sharding-config-server`.
190190

191+
connection pool
192+
A cache of database connections maintained by the driver. These
193+
connections are re-used when connections to the database are
194+
required, instead of opening new connections.
195+
191196
container
192197
A collected set of software and its dependent libraries that are
193198
packaged together to make transferring between computing

‎source/tutorial.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -116,7 +116,7 @@ Development Patterns
116116
- :doc:`/tutorial/troubleshoot-map-function`
117117
- :doc:`/tutorial/troubleshoot-reduce-function`
118118
- :doc:`/tutorial/store-javascript-function-on-server`
119-
119+
- :doc:`/tutorial/connection-pool-performance-tuning`
120120

121121

122122
.. _tutorials-text-search:

‎source/tutorial/connection-pool-performance-tuning.txt

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,108 @@
1+
.. _connection-pool-performance-tuning:
2+
3+
====================================
4+
Tuning Your Connection Pool Settings
5+
====================================
6+
7+
.. default-domain:: mongodb
8+
9+
.. contents:: On this page
10+
:local:
11+
:backlinks: none
12+
:depth: 1
13+
:class: singlecol
14+
15+
16+
.. list-table::
17+
:widths: 25,75
18+
:header-rows: 1
19+
20+
* - Problem
21+
22+
- Solution
23+
24+
* - Slow application-side operation times that are not reflected in
25+
the database :doc:`server logs </reference/log-messages>` or the
26+
real time panel.
27+
28+
- Use :urioption:`connectTimeoutMS` to ensure the driver does not
29+
wait indefinitely during the connection phase.
30+
31+
Set :urioption:`connectTimeoutMS` to a value greater than the
32+
longest network latency you have to a member of the set.
33+
34+
For example: if a member has a latency of 10000 milliseconds,
35+
setting :urioption:`connectTimeoutMS` to ``5000``
36+
(milliseconds) prevents the driver from connecting to that
37+
member.
38+
39+
* - A misconfigured firewall closes a socket connection incorrectly
40+
and the driver cannot detect that the connection closed
41+
improperly.
42+
43+
- Use :urioption:`socketTimeoutMS` to ensure that sockets are
44+
always closed.
45+
46+
Set :urioption:`socketTimeoutMS` to two or three times the
47+
length of the slowest operation that the driver runs.
48+
49+
* - The :doc:`server logs </reference/log-messages>` or real time panel
50+
show that the application spends too much time creating new
51+
connections.
52+
53+
- Not enough connections are available at startup.
54+
Allocate connections in the pool by setting
55+
:urioption:`minPoolSize`.
56+
57+
Set :ref:`minPoolSize <minpoolsize-cp-setting>` to the number
58+
of connections you want to be available at startup.
59+
60+
The ``MongoClient`` instance ensures that number of
61+
connections exists at all times.
62+
63+
* - The load on the database is low and there's a small number of
64+
active connections at any time. Application performs fewer
65+
operations at any one time than expected.
66+
67+
- Increase :ref:`maxPoolSize <maxpoolsize-cp-setting>`, or increase
68+
the number of active threads in your application or the framework
69+
you are using.
70+
71+
* - Database CPU usage is higher than expected. The
72+
:doc:`server logs </reference/log-messages>` or real time panel
73+
show more connection attempts than expected.
74+
75+
- Decrease the :ref:`maxPoolSize <maxpoolsize-cp-setting>` or
76+
reduce the number of threads in your application. This can reduce
77+
load and response times.
78+
79+
.. warning::
80+
81+
Do not use :urioption:`socketTimeoutMS` to prevent long-running
82+
server operations. Instead, use :method:`~cursor.maxTimeMS` with
83+
queries so that the server can cancel long-running operations.
84+
85+
86+
Calculate Maximum Number of Connections
87+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
88+
89+
Calculate usage to find the number of operations running for each
90+
connection.
91+
92+
Consider four application servers connecting to a replica set with three
93+
members. In this scenario, each of the four application servers
94+
creates a connection pool for each replica set member.
95+
96+
Calculate the maximum number of connections that are opened by
97+
each application server by multiplying :urioption:`maxPoolSize` by the
98+
number of members.
99+
100+
Calculate outgoing connections from an application to a three-member
101+
replica set:
102+
103+
**100** (:urioption:`maxPoolSize` default ``100``) x **3** (replica set members) = **300** (outgoing connections from the application).
104+
105+
Calculate incoming connections from four application servers to a
106+
replica set:
107+
108+
**100** (:urioption:`maxPoolSize` default ``100``) x **4** (application servers) = **400** (incoming connections to each mongod).

0 commit comments

Comments
 (0)