diff --git a/source/core/sharded-cluster-internals.txt b/source/core/sharded-cluster-internals.txt index 47de4739b10..b900174ae37 100644 --- a/source/core/sharded-cluster-internals.txt +++ b/source/core/sharded-cluster-internals.txt @@ -448,6 +448,8 @@ when modifying chunk size: chunks must grow through insertion or updates until they reach the new size. +.. _sharding-shard-size: + Shard Size ~~~~~~~~~~ @@ -462,7 +464,11 @@ command. This will prevent the :term:`balancer` from migrating chunks to the shard when the value of :data:`mem.mapped ` exceeds the ``maxSize`` setting. -.. seealso:: :doc:`/administration/monitoring`. +.. seealso:: + + :ref:`sharded-cluster-config-max-shard-size` + + :doc:`/administration/monitoring`. .. _sharding-chunk-migration: @@ -506,9 +512,9 @@ All chunk migrations use the following procedure: no open cursors on the chunk, the source shard starts deleting its copy of documents from the migrated chunk. -When the ``_secondaryThrottle`` is ``true`` for :dbcommand:`moveChunk` -or the :term:`balancer`, MongoDB ensure that *one* :term:`secondary` -member has replicated changes before allowing new chunk migrations. +If enabled, the ``_secondaryThrottle`` setting causes the balancer to +wait for replication to secondaries. For more information, see +:ref:`sharded-cluster-config-secondary-throttle`. Detect Connections to :program:`mongos` Instances ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/source/reference/command/moveChunk.txt b/source/reference/command/moveChunk.txt index de253c6c7b4..1e5cb8b669c 100644 --- a/source/reference/command/moveChunk.txt +++ b/source/reference/command/moveChunk.txt @@ -29,17 +29,12 @@ moveChunk :param to: The identifier of the shard, that you want to migrate the chunk to. - :param _secondaryThrottle: Optional. Set to ``false`` by - default. Provides :ref:`write concern ` - support for chunk - migrations. - - If you set ``_secondaryThrottle`` to ``true``, during chunk - migrations when a :term:`shard` hosted by a :term:`replica set`, - the :program:`mongod` will wait until the :term:`secondary` members - replicate the migration operations continuing to migrate chunk - data. You may also configure ``_secondaryThrottle`` in the balancer - configuration. + :param _secondaryThrottle: Optional. Set to ``false`` by default. If + set to ``true``, the balancer waits for + replication to :term:`secondaries ` + while copying and deleting + data during migrations. For details, see + :ref:`sharded-cluster-config-secondary-throttle`. Use the :method:`sh.moveChunk()` helper in the :program:`mongo` shell to migrate chunks manually. diff --git a/source/sharding.txt b/source/sharding.txt index eea981b4677..cac149adde2 100644 --- a/source/sharding.txt +++ b/source/sharding.txt @@ -54,6 +54,7 @@ Sharded Cluster Maintenance and Administration tutorial/manage-sharded-cluster-config-server tutorial/manage-chunks-in-sharded-cluster + tutorial/configure-sharded-cluster-balancer tutorial/manage-sharded-cluster-balancer tutorial/remove-shards-from-cluster diff --git a/source/tutorial/configure-sharded-cluster-balancer.txt b/source/tutorial/configure-sharded-cluster-balancer.txt new file mode 100644 index 00000000000..cd2e9495b01 --- /dev/null +++ b/source/tutorial/configure-sharded-cluster-balancer.txt @@ -0,0 +1,151 @@ +.. index:: balancing; configure + +========================================================== +Configure Behavior of Balancer Process in Sharded Clusters +========================================================== + +.. default-domain:: mongodb + +The balancer runs on a single :program:`mongos` instance and distributes +chunks evenly throughout a sharded cluster. In most deployments, you do +not need to configure the balancer. The balancer automatically +distributes chunks in an optimal manner. However, administrators might +need to modify balancer behavior depending on application or operational +requirements. Should a situation arise where modifying balancer behavior +is necessary, this page describes settings that can be changed. + +For conceptual information about the balancer, see +:ref:`sharding-balancing` and :ref:`sharding-balancing-internals`. + +.. _sharded-cluster-config-balancing-window: + +Schedule a Window of Time for Balancing to Occur +------------------------------------------------ + +You can schedule a window of time during which the balancer is allowed +to migrate chunks, as described in the following procedures: + +- :ref:`sharding-schedule-balancing-window` + +- :ref:`sharding-balancing-remove-window`. + +The configured time is evaluated relative to the time zone of each +individual :program:`mongos` instance in the sharded cluster. + +.. _sharded-cluster-config-default-chunk-size: + +Configure Default Chunk Size +---------------------------- + +The default chunk size for a sharded cluster is 64 megabytes. In most +situations, the default size is appropriate for splitting and migrating +chunks. For information on how chunk size affects deployments, see +details, see :ref:`sharding-chunk-size`. + +Changing the default chunk size affects chunks that are processes during +migrations and auto-splits but does not retroactively affect all chunks. + +To configure default chunk size, see :ref:`sharding-balancing-modify-chunk-size`. + +.. _sharded-cluster-config-max-shard-size: + +Change the Maximum Storage Size for a Given Shard +------------------------------------------------- + +The ``maxSize`` field in the :data:`~config.shards` collection in the +:ref:`config database ` sets the maximum size for a +shard, allowing you to control whether the balancer will migrate chunks +to a shard. If :data:`dataSize <~dbStats.dataSize>` is above a shard's +``maxSize``, the balancer will not move chunks to the shard. Also, the +balancer will not move chunks off an overloaded shard. This must happen +manually. The ``maxSize`` value only affects the balancer's selection of +destination shards. + +By default, ``maxSize`` is not specified, allowing shards to consume the +total amount of available space on their machines if necessary. + +You can set ``maxSize`` both when adding a shard and once a shard is +running. + +To set ``maxSize`` when adding a shard, set the :dbcommand:`addShard` +command's ``maxSize`` parameter to the maximum size in megabytes. For +example, the following command run in the :program:`mongo` shell adds a +shard with a maximum size of 125 megabytes: + +.. code-block:: javascript + + db.runCommand( { addshard : "example.net:34008", maxSize : 125 } ) + +To set ``maxSize`` on an existing shard, insert or update the +``maxSize`` field in the :data:`~config.shards` collection in the +:ref:`config database `. Set the ``maxSize`` in +megabytes. + +.. example:: + + Assume you have the following shard without a ``maxSize`` field: + + .. code-block:: javascript + + { "_id" : "shard0000", "host" : "example.net:34001" } + + Run the following sequence of commands in the :program:`mongo` shell + to insert a ``maxSize`` of 125 megabytes: + + .. code-block:: javascript + + use config + db.shards.update( { _id : "shard0000" }, { $set : { maxSize : 125 } } ) + + To later increase the ``maxSize`` setting to 250 megabytes, run the + following: + + .. code-block:: javascript + + use config + db.shards.update( { _id : "shard0000" }, { $set : { maxSize : 250 } } ) + +.. index:: balancing; secondary throttle +.. index:: secondary throttle +.. _sharded-cluster-config-secondary-throttle: + +Require Replication before Chunk Migration (Secondary Throttle) +--------------------------------------------------------------- + +.. versionadded:: 2.2.1 + +You can configure the balancer to wait for replication to secondaries +during migrations. You do so by enabling the balancer's +``_secondaryThrottle`` parameter, which reduces throughput (i.e., +"throttles") in order to decrease the load on secondaries. You might do +this, for example, if you have migration-caused I/O peaks that impact +other workloads + +When enabled, secondary throttle puts a ``{ w : 2 }`` write concern on +deletes and on copies, which means the balancer waits for those +operations to replicate to at least one secondary before migrating +chunks. + +.. BACKGROUND NOTES + Specifically, secondary throttle affects the first and fourth + phases (informal phases) of chunk migration. Migration can happen during + the second and third phases (the "steady state"): + 1) copies the documents in the chunk from shardA to shardB + 2) continues to copy over ongoing changes that occurred during the initial copy step, + as well as current changes to that chunk range + 3) Stop writes, allow shardB to get final changes, commit migration to config server + 4) cleanup now-inactive data on shardA in chunk range (once all cursors are done) + +You enable ``_secondaryThrottle`` directly in the +:data:`settings <~config.settings>` collection in the :ref:`config database +` by running the following commands from the +:program:`mongo` shell: + +.. code-block:: javascript + + use config + db.settings.update( { "_id" : "balancer" } , { $set : { "_secondaryThrottle" : true } , { upsert : true } } ) + +You also can enable secondary throttle when issuing the +:dbcommand:`moveChunk` command by setting ``_secondaryThrottle`` to +``true``. For more information, see :dbcommand:`moveChunk`. diff --git a/source/tutorial/manage-chunks-in-sharded-cluster.txt b/source/tutorial/manage-chunks-in-sharded-cluster.txt index 9f606877224..39fd7473712 100644 --- a/source/tutorial/manage-chunks-in-sharded-cluster.txt +++ b/source/tutorial/manage-chunks-in-sharded-cluster.txt @@ -293,9 +293,10 @@ to pre-splitting. .. versionadded:: 2.2 :dbcommand:`moveChunk` command has the: ``_secondaryThrottle`` - parameter. When set to ``true``, MongoDB ensures that - :term:`secondary` members have replicated operations before allowing - new chunk migrations. + parameter. When set to ``true``, MongoDB ensures replication to + :term:`secondaries ` before allowing + new chunk migrations. For more information, see + :ref:`sharded-cluster-config-secondary-throttle`. .. warning:: diff --git a/source/tutorial/manage-sharded-cluster-balancer.txt b/source/tutorial/manage-sharded-cluster-balancer.txt index 2ce2c6e9a1c..454e340805b 100644 --- a/source/tutorial/manage-sharded-cluster-balancer.txt +++ b/source/tutorial/manage-sharded-cluster-balancer.txt @@ -1,5 +1,4 @@ .. index:: balancing; operations -.. _sharding-balancing-operations: =============================== Manage Sharded Cluster Balancer @@ -22,6 +21,10 @@ This page includes the following: - :ref:`sharding-balancing-disable-temporally` +.. seealso:: + + :doc:`/tutorial/configure-sharded-cluster-balancer` + .. _sharding-balancing-check-lock: Check the Balancer Lock