diff --git a/source/applications/indexes.txt b/source/applications/indexes.txt index 22a4256a806..cba3b90c2f9 100644 --- a/source/applications/indexes.txt +++ b/source/applications/indexes.txt @@ -29,7 +29,7 @@ understanding of: The best overall strategy for designing indexes is to profile a variety of index configurations with data sets similar to the ones you'll be -running in production and to see which configurations perform best. +running in production to see which configurations perform best. MongoDB can only use *one* index to support any given operation. However, each clause of an :operator:`$or` query can use @@ -41,14 +41,14 @@ Create Indexes to Support Your Queries -------------------------------------- If you only ever query on a single key in a given collection, then you need -create just one single-key index for that collection. For example, you +to create just one single-key index for that collection. For example, you might create an index on ``category`` in the ``product`` collection: .. code-block:: javascript db.products.ensureIndex( { "category": 1 } ) -However, if you sometimes query on only one key but and at other times +However, if you sometimes query on only one key and at other times query on that key combined with a second key, then creating a :ref:`compound index ` is more efficient. MongoDB will use the compound index for both queries. For example, you might @@ -65,8 +65,8 @@ you also can query on ``category`` combined with ``item``. With the exception of queries that use the :operator:`$or` operator, a query cannot use multiple indexes. A query must use only one index. -.. _covered-queries: -.. _indexes-covered-queries: +.. _compound-key-indexes: +.. _indexes-compound-key-indexes: Use Compound Indexes to Support Several Different Queries --------------------------------------------------------- @@ -111,6 +111,9 @@ can support all the queries that search a "prefix" subset of those fields. the query. For more information on sorting, see :ref:`sorting-with-indexes`. +.. _covered-queries: +.. _indexes-covered-queries: + Create Indexes that Support Covered Queries ------------------------------------------- @@ -119,9 +122,9 @@ part of an index. They are "covered queries" because an index "covers" the query MongoDB can fulfill the query by using *only* the index. MongoDB need not scan documents from the database. -Querying *only* the index is much faster than querying documents. -Indexes keys are typically smaller than the documents they catalog, and indexes are -typically stored in RAM or located sequentially on disk. +Querying *only* the index is much faster than querying documents. Index +keys are typically smaller than the documents they catalog, and indexes +are typically stored in RAM or located sequentially on disk. Mongod automatically uses a covered query when possible. To ensure use of a covered query, create an index that includes all the fields listed @@ -132,7 +135,7 @@ explicitly exclude the ``_id`` field from the result set, unless the index includes ``_id``. MongoDB cannot use a covered query if any of the indexed fields in any -of the documents in the collection include an array. If an indexed field +of the documents in the collection includes an array. If an indexed field is an array, the index becomes a :ref:`multi-key index ` index and cannot support a covered query. @@ -194,10 +197,10 @@ are equality matches. .. note:: - When the :method:`sort() ` method performs an - in-memory sort operation without the use of an index, the operation - is significantly slower than for operations that use an index, and - the operation aborts when it consumes 32 megabytes of memory. + For in-memory sorts that do not use an index, the :method:`sort() + ` operation is significantly slower. The + :method:`~cursor.sort()` operation will abort when it uses 32 + megabytes of memory. .. _indexes-ensure-indexes-fit-ram: @@ -223,7 +226,7 @@ available but also must have RAM available for the rest of the If you have and use multiple collections, you must consider the size of all indexes on all collections. The indexes and the working set must be able to -fit RAM at the same time. +fit in RAM at the same time. There are some limited cases where indexes do not need to fit in RAM. See :ref:`indexing-right-handed`. @@ -334,8 +337,8 @@ Consider Performance when Creating Indexes for Write-heavy Applications If your application is write-heavy, then be careful when creating new indexes, since each additional index with impose a write-performance penalty. In general, don't be careless about adding -indexes. Add indexes complement your queries. Always have -a good reason for adding a new index, and make sure you've benchmarked +indexes. Add indexes to complement your queries. Always have +a good reason for adding a new index, and be sure to benchmark alternative strategies. Consider Insert Throughput @@ -344,17 +347,16 @@ Consider Insert Throughput .. todo:: insert link to /source/core/write-operations when that page is complete. Do we want to link to write concern? -bg -MongoDB must update *all* indexes associated with a collection after every -insert, update, or delete operation. For update operations, if the updated document -does not move to a new location, then only the modified, MongoDB only needs -to update the updated fields in the index. -Therefore, every index on a -collection adds some amount of overhead to these write operations. In almost -every case, the performance gains that indexes realize for read +MongoDB must update *all* indexes associated with a collection after +every insert, update, or delete operation. For update operations, if +the updated document does not move to a new location, then MongoDB only +modifies the updated fields in the index. Therefore, every index on a +collection adds some amount of overhead to these write operations. In +almost every case, the performance gains that indexes realize for read operations are worth the insertion penalty. However, in some cases: - An index to support an infrequent query might incur more - insert-related costs than saved read-time. + insert-related costs than savings in read-time. .. todo:: How do you determine if the above is the case? Empirically. @@ -368,5 +370,5 @@ operations are worth the insertion penalty. However, in some cases: - If your indexes and queries are not sufficiently :ref:`selective `, the speed improvements for query operations - might not offset the costs of maintaining an index. For more + may not offset the costs of maintaining an index. For more information see :ref:`index-selectivity`. diff --git a/source/reference.txt b/source/reference.txt index 5ca0e513910..c09b462cc7d 100644 --- a/source/reference.txt +++ b/source/reference.txt @@ -74,6 +74,7 @@ Status and Reporting reference/replication-info reference/current-op reference/exit-codes + reference/explain-output Internal Metadata ----------------- diff --git a/source/reference/explain-output.txt b/source/reference/explain-output.txt new file mode 100644 index 00000000000..51c3a7ced0a --- /dev/null +++ b/source/reference/explain-output.txt @@ -0,0 +1,297 @@ +============== +Explain Output +============== + +.. default-domain:: mongodb + +This document explains the output of the :operator:`$explain` operator +and the :program:`mongo` shell method :method:`explain() +`. + +Core Explain Output +------------------- + +The :ref:`explain-output-fields-core` fields display information for +queries on non-sharded collections. For queries on sharded collections, +this information is displayed for each accessed shard. + +.. code-block:: javascript + + { + "cursor" : "", + "isMultiKey" : , + "n" : , + "nscannedObjects" : , + "nscanned" : , + "nscannedObjectsAllPlans" : , + "nscannedAllPlans" : , + "scanAndOrder" : , + "indexOnly" : , + "nYields" : , + "nChunkSkips" : , + "millis" : , + "indexBounds" : { }, + "allPlans" : [ + { "cursor" : "", + "n" : , + "nscannedObjects" : , + "nscanned" : , + "indexBounds" : { } + }, + ... + ], + "oldPlan" : { + "cursor" : "", + "indexBounds" : { } + } + "server" : "", + } + +$OR Queries +----------- + +Queries with :operator:`$or` operator execute each clause of the +:operator:`$or` expression in parallel and can use separate indexes on +the individual clauses. If the indexes on the clauses are used, the +explain output contains the :ref:`explain-output-fields-core` for each +clause as well as the cumulative information for the query: + +.. code-block:: javascript + + { + "clauses" : [ + { + + }, + { + + }, + ... + ], + "n" : , + "nscannedObjects" : , + "nscanned" : , + "nscannedObjectsAllPlans" : , + "nscannedAllPlans" : , + "millis" : , + "server" : "" + } + +Sharded Collections +------------------- + +For queries on a sharded collection, the output contains the +:ref:`explain-output-fields-core` for each accessed shard and +:ref:`cumulative shard +information `: + +.. code-block:: javascript + + { + "clusteredType" : "", + "shards" : { + + "" : [ + { + + } + ], + "" : [ + { + + } + ], + ... + }, + "millisShardTotal" : , + "millisShardAvg" : , + "numQueries" : , + "numShards" : , + "cursor" : "", + "n" : , + "nChunkSkips" : , + "nYields" : , + "nscanned" : , + "nscannedAllPlans" : , + "nscannedObjects" : , + "nscannedObjectsAllPlans" : , + "millis" : + } + +Fields +------ + +.. _explain-output-fields-core: + +Core Explain Output +~~~~~~~~~~~~~~~~~~~ + +.. data:: cursor + + Specifies the type of cursor used in the query operation: + + - ``BasicCursor`` indicates use of full collection scan. + + - ``BtreeCursor`` indicates use of an B-tree index. The cursor + information includes the index name. With the use of an index, the + :method:`explain() ` output will include the + :data:`indexBounds` details. + + - ``GeoSearchCursor`` indicates use of a geospatial index. + +.. data:: isMultiKey + + Specifies whether a :ref:`multikey index on an array field + ` was used. + +.. data:: n + + Specifies the number of documents that match the query selection + criteria. + +.. data:: nscannedObjects + + Specifies the total number of documents scanned during the query. + The :data:`nscannedObjects` may be lower than :data:`nscanned`, such + as if the index is a covered index. + +.. data:: nscanned + + Specifies the total number of documents or index entries scanned + during the database operation. You want :data:`n` and + :data:`nscanned` to be close in value as possible. The + :data:`nscanned` value may be higher than the + :data:`nscannedObjects` value, such as if the index is a covered + index. + +.. data:: nscannedObjectsAllPlans + + .. versionadded:: 2.2 + + Specifies the total number of documents scanned for all query plans + during the database operation. + +.. data:: nscannedAllPlans + + .. versionadded:: 2.2 + + Specifies the total number of documents or index entries scanned for + all query plans during the database operation. + +.. data:: scanAndOrder + + .. versionadded:: 2.2 + + Specifies whether the index could not be used for sorting. + + If ``true``, the index could *not* be used for sorting. Instead, the + documents are sorted after being retrieved either from the index if + an index is used to return the documents, or from the full + collection if a collection scan is performed. + +.. data:: indexOnly + + Specifies whether the query is a :ref:`covered query + ` that can be fulfilled by using just the + index. + +.. data:: nYields + + Specifies the number of times this query yielded the read lock to + allow waiting writes execute. + +.. data:: nChunkSkips + + Specifies the number of documents skipped because of active chunk + migrations in a sharded system. Typically this will be zero. A + number greater than zero is ok, but indicates a little bit of + inefficiency. + +.. data:: millis + + Specifies the number of milliseconds to complete the query. + +.. data:: indexBounds + + Specifies the lower and upper index key bounds. + + .. code-block:: javascript + + "indexBounds" : { + "start" : { : , ... }, + "end" : { : , ... } + }, + or + + "indexBounds" : { "" : [ [ , ] ], + ... + } + +.. data:: allPlans + + Specifies the list of plans the query optimizer runs in order to + select the index for the query. Displays only if the ```` + parameter to the :method:`explain() ` is set to + ``true`` or ``1``. + +.. data:: oldPlan + + .. versionadded:: 2.2 + + Specifies the previous plan selected by the query optimizer for the + query. Displays only if the ```` parameter to the + :method:`explain() ` is set to ``true`` or ``1``. + +.. data:: server + + .. versionadded:: 2.2 + + Specifies the MongoDB server. + +.. _explain-output-field-or-clauses: + +$OR Query Output +~~~~~~~~~~~~~~~~ + +.. data:: clauses + + Contains the :ref:`explain-output-fields-core` information for each + clause of the :operator:`$or` expression. Displays only if indexes + are used for the clauses in the :operator:`$or` expression. + + +.. _explain-output-fields-sharded-collection: + +Sharded Collections Output +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. data:: clusteredType + + Specifies how the shards are accessed: + + - ``ParallelSort`` means all shards are queried in parallel + - ``SerialServer`` means the shards are queried sequentially + +.. data:: shards + + Specifies the shards accessed during the query and individual + :ref:`explain-output-fields-core` for each shard. + +.. data:: millisShardTotal + + Specifies the total time in milliseconds for the query to run on the + shards. + +.. data:: millisShardAvg + + Specifies the average time in millisecond for the query to run on + each shard. + +.. data:: numQueries + + Specifies the total number of queries executed. + +.. data:: numShards + + Specifies the total number of shards queried. + diff --git a/source/reference/method/cursor.explain.txt b/source/reference/method/cursor.explain.txt index 95281a72c45..cc9eb27c934 100644 --- a/source/reference/method/cursor.explain.txt +++ b/source/reference/method/cursor.explain.txt @@ -4,36 +4,82 @@ cursor.explain() .. default-domain:: mongodb +.. EDITS to cursor.explain.txt must be carried over to the operator + explain.txt and vice versa + .. method:: cursor.explain() - :returns: A document that describes the process used to return the - query. + The :method:`cursor.explain()` method provides information on the + query plan. The query plan is the plan the server uses to find the + matches for a query. This information may be useful when optimizing + a query. + + :param boolean verbose: + + Specifies the level of detail to include in the output. If + ``true`` or ``1``, include the ``allPlans`` and ``oldPlan`` + fields in the :doc:`output document + `. + + :returns: A :doc:`document ` that + describes the process used to return the query results. + + Retrieve the query plan by appending :method:`explain() + ` to a :method:`find()` query, as in the following + example: + + .. code-block:: javascript + + db.products.find().explain() + + For details on the output, see :doc:`/reference/explain-output`. - This method may provide useful insight when attempting to optimize - a query. When you call the :method:`explain ` on - a query, the query system reevaluates the query plan. As a result, - these operations only provide a realistic account of *how* MongoDB - would perform the query, and *not* how long the query would take. + :method:`explain ` runs the actual query to + determine the result. Although there are some differences between + running the query with :method:`explain ` and + running without, generally, the performance will be similar between + the two. So, if the query is slow, the :method:`explain + ` will also be slow. + + Additionally, when you call the :method:`explain ` + method on a query, the query system reevaluates a set of candidate + query plans, which may cause some queries to run slower. As a + result, these operations generally provide an accurate account of + *how* MongoDB would perform the query [#explain-cache-plan]_, but + not necessarily *how long* the query would take. - .. _explain-output: + To determine the performance of a particular index, you can use + :method:`hint() ` and in conjunction with + :method:`explain() `, as in the following example: - The method's output includes these fields: + .. code-block:: javascript - - ``cursor``: The value for cursor can be either ``BasicCursor`` or - ``BtreeCursor``. The second of these indicates that the given query - is using an index. + db.products.find().hint( { type: 1 } ).explain() - - ``nscanned``: The number of index entries scanned. + When you run :method:`explain ` with + :method:`hint() `, the query optimizer does not + reevaluate the query plans. - - ``n``: the number of documents returned by the query. You want the - value of ``n`` to be close to the value of ``nscanned``. You want to - avoid a "collection scan," which is a scan where every document in - the collection is accessed. This is the case when ``nscanned`` is - equal to the number of documents in the collection. + .. [#explain-cache-plan] In some situations, the :method:`explain + ` operation may differ from the actual query + plan used to perform the query. The :method:`explain + ` operation evaluates the set of query plans + and reports on the winning plan for the query. This plan is + cached to be used for the execution of the query and similar + queries at a later time. When you run similar queries, this + cached plan is selected even though the cached plan may not be + the best plan for similar queries. So, if you run + :method:`explain ` operation on similar + queries, the winning plan may be different. - - ``millis``: the number of milliseconds require to complete the - query. This value is useful for comparing indexing strategies. + .. seealso:: + + - :operator:`$explain` + + - :wiki:`Optimization` wiki page for information regarding + optimization strategies. - .. seealso:: :operator:`$explain` for related functionality and - the ":wiki:`Optimization`" wiki page for information regarding - optimization strategies. + - :wiki:`Database Profiler` wiki page for information regarding + optimization strategies. + + - :doc:`Current Operation Reporting ` diff --git a/source/reference/operator/explain.txt b/source/reference/operator/explain.txt index 8b6b57208f8..b35b3cba90d 100644 --- a/source/reference/operator/explain.txt +++ b/source/reference/operator/explain.txt @@ -4,21 +4,79 @@ $explain .. default-domain:: mongodb +.. EDITS to explain.txt must be carried over to the method + cursor.explain.txt and vice versa + .. operator:: $explain - Use the :operator:`$explain` operator to return a :term:`document` - that describes the process and indexes used to return the - query. This may provide useful insight when attempting to optimize - a query. Consider the following example: + The :operator:`$explain` operator provides information on the query + plan. The query plan is the plan the server uses to find the matches + for a query. This information may be useful when optimizing a query. + + It returns a :doc:`document ` that + describes the process and indexes used to return the query result. + + :program:`mongo` shell also provides the :method:`explain() + ` method: + + .. code-block:: javascript + + db.collection.find().explain() + + You can also specify the option in either of the following forms: .. code-block:: javascript db.collection.find()._addSpecial( "$explain", 1 ) + db.collection.find( { $query: {}, $explain: 1 } ) + + For details on the output, see :doc:`/reference/explain-output`. + + :operator:`$explain` runs the actual query to determine the result. + Although there are some differences between running the query with + :operator:`$explain` and running without, generally, the performance + will be similar between the two. So, if the query is slow, the + :operator:`$explain` will likely be slow. + + Additionally, when you call explain on a query, the query system + reevaluates a set of candidate query plans, which may cause some + queries to run slower. As a result, these operations generally + provide an accurate account of *how* MongoDB would perform the query + [#explain-cache-plan]_, but not necessarily *how long* the query + would take. - The JavaScript function :method:`cursor.explain()` provides equivalent - functionality in the :program:`mongo` shell. See the following - example, which is equivalent to the above: + To determine the performance of a particular index, you can use + :method:`hint() ` and in conjunction with + :method:`explain() `, as in the following example: .. code-block:: javascript - db.collection.find().explain() + db.products.find().hint( { type: 1 } ).explain() + + When you run :method:`explain ` with + :method:`hint() `, the query optimizer does not + reevaluate the query plans. + + .. [#explain-cache-plan] In some situations, the :method:`explain + ` operation may differ from the actual query + plan used to perform the query. The :method:`explain + ` operation evaluates the set of query plans + and reports on the winning plan for the query. This plan is + cached to be used for the execution of the query and similar + queries at a later time. When you run similar queries, this + cached plan is selected even though the cached plan may not be + the best plan for similar queries. So, if you run + :method:`explain ` operation on similar + queries, the winning plan may be different. + + .. seealso:: + + - :method:`cursor.explain()` + + - :wiki:`Optimization` wiki page for information regarding + optimization strategies. + + - :wiki:`Database Profiler` wiki page for information regarding + optimization strategies. + + - :doc:`Current Operation Reporting `