Skip to content

Commit 373e976

Browse files
jwilliams-mongojwilliams-mongo
authored
(DOCSP-10559): mdb-shell Aggregation Pipeline Tutorial (#14)
* (DOCSP-10559): Aggregation Pipeline Tutorial * (DOCSP-10559): add tip * (DOCSP-10559): copy review feedback * (DOCSP-10559): minor tweaks * (DOCSP-10559): add in pipeline explanation * (DOCSP-10559): copy review rd 2 * (DOCSP-10559): removing note about multi-line paste * (DOCSP-10559): editor mode cleanup
1 parent 43a4d88 commit 373e976

File tree

2 files changed

+321
-8
lines changed

2 files changed

+321
-8
lines changed

‎source/index.txt

Lines changed: 7 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -63,8 +63,10 @@ Once you have installed the |mdb-shell| and added it to you system
6363
``PATH``, you can connect to a MongoDB deployment. To learn more, see
6464
:ref:`mdb-shell-connect`.
6565

66-
Multi-Line Operations in the |mdb-shell|
67-
----------------------------------------
66+
.. _mdb-shell-multi-line:
67+
68+
|mdb-shell| Editor Mode
69+
-----------------------
6870

6971
Run the ``.editor`` command within the |mdb-shell| to open the editor
7072
and manage comprehensive multiline functions:
@@ -102,12 +104,6 @@ and manage comprehensive multiline functions:
102104
When you close the editor, the |mdb-shell| does not save any functions
103105
you had entered into the editor.
104106

105-
.. note::
106-
107-
Currently, ``editor()`` is the only way to paste multiline commands
108-
into the |mdb-shell|.
109-
110-
111107
The |mdb-shell| versus the Legacy ``mongo`` Shell
112108
-------------------------------------------------
113109

@@ -134,6 +130,8 @@ Learn More
134130

135131
- :ref:`Perform CRUD Operations <mdb-shell-crud>`
136132

133+
- :ref:`Run Aggregation Pipelines <mdb-shell-aggregation>`
134+
137135
- :ref:`View Available Methods in the MongoDB Shell <mdb-shell-methods>`
138136

139137
.. class:: hidden
@@ -143,4 +141,5 @@ Learn More
143141

144142
/connect
145143
/crud
144+
/run-agg-pipelines
146145
/reference

‎source/run-agg-pipelines.txt

Lines changed: 314 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,314 @@
1+
.. _mdb-shell-aggregation:
2+
3+
=========================
4+
Run Aggregation Pipelines
5+
=========================
6+
7+
.. default-domain:: mongodb
8+
9+
.. contents:: On this page
10+
:local:
11+
:backlinks: none
12+
:depth: 1
13+
:class: singlecol
14+
15+
.. include:: /includes/admonitions/fact-mdb-shell-beta.rst
16+
17+
You can run :manual:`aggregation pipelines </aggregation>` on your
18+
collections using the |mdb-shell|. Aggregation pipelines transform your
19+
documents into aggregated results based on selected
20+
:manual:`pipeline stages </reference/operator/aggregation-pipeline/>`.
21+
22+
Common uses for aggregation include:
23+
24+
- Grouping data by a given expression.
25+
26+
- Calculating results based on multiple fields and storing those results
27+
in a new field.
28+
29+
- Filtering data to return a subset that matches a given criteria.
30+
31+
- Sorting data.
32+
33+
When you run an aggregation, |mdb-shell| outputs the results directly to
34+
the terminal.
35+
36+
Understand the Aggregation Syntax
37+
---------------------------------
38+
39+
The MongoDB aggregation pipeline consists of :manual:`stages
40+
</reference/operator/aggregation-pipeline/#aggregation-pipeline-operator-reference>`.
41+
Each stage transforms the documents as they pass through the pipeline.
42+
Pipeline stages do not need to produce one output document for every
43+
input document; e.g., some stages may generate new documents or filter
44+
out documents.
45+
46+
To create an aggregation pipeline, use the following syntax in the
47+
|mdb-shell|:
48+
49+
.. code-block:: javascript
50+
:linenos:
51+
52+
db.<collection>.aggregate([
53+
{
54+
<$stage1>
55+
},
56+
{
57+
<$stage2>
58+
}
59+
...
60+
])
61+
62+
Example
63+
-------
64+
65+
.. include:: /includes/fact-sample-data-examples.rst
66+
67+
The example below uses the ``movies`` collection in the |service|
68+
:atlas:`sample_mflix </sample-data/sample-mflix/>` sample dataset.
69+
70+
Example Document
71+
~~~~~~~~~~~~~~~~
72+
73+
Each document in the ``movies`` collection describes a movie:
74+
75+
.. code-block:: javascript
76+
:linenos:
77+
:copyable: false
78+
79+
{
80+
_id: 573a1397f29313caabce8347,
81+
fullplot: 'In a cyberpunk vision of the future, man has developed the technology to create replicants, human clones used to serve in the colonies outside Earth but with fixed lifespans. In Los Angeles, 2019, Deckard is a Blade Runner, a cop who specializes in terminating replicants. Originally in retirement, he is forced to re-enter the force when four replicants escape from an off-world colony to Earth.',
82+
imdb: { rating: 8.2, votes: 419589, id: 83658 },
83+
year: 1982,
84+
plot: 'A blade runner must pursue and try to terminate four replicants who stole a ship in space and have returned to Earth to find their creator.',
85+
genres: [ 'Sci-Fi', 'Thriller' ],
86+
rated: 'R',
87+
metacritic: 88,
88+
title: 'Blade Runner',
89+
lastupdated: '2015-09-04 00:05:51.990000000',
90+
languages: [ 'English', 'German', 'Cantonese', 'Japanese', 'Hungarian' ],
91+
writers: [
92+
'Hampton Fancher (screenplay)',
93+
'David Webb Peoples (screenplay)',
94+
'Philip K. Dick (novel)'
95+
],
96+
type: 'movie',
97+
tomatoes: {
98+
viewer: { rating: 4, numReviews: 331213, meter: 91 },
99+
dvd: 1997-08-27T00:00:00.000Z,
100+
critic: { rating: 8.5, numReviews: 102, meter: 90 },
101+
lastUpdated: 2015-09-12T17:48:21.000Z,
102+
consensus: "Misunderstood when it first hit theaters, the influence of Ridley Scott's mysterious, neo-noir Blade Runner has deepened with time. A visually remarkable, achingly human sci-fi masterpiece.",
103+
rotten: 10,
104+
production: 'Warner Bros. Pictures',
105+
fresh: 92
106+
},
107+
poster: 'https://m.media-amazon.com/images/M/MV5BNzQzMzJhZTEtOWM4NS00MTdhLTg0YjgtMjM4MDRkZjUwZDBlXkEyXkFqcGdeQXVyNjU0OTQ0OTY@._V1_SY1000_SX677_AL_.jpg',
108+
num_mflix_comments: 1,
109+
released: 1982-06-25T00:00:00.000Z,
110+
awards: {
111+
wins: 13,
112+
nominations: 15,
113+
text: 'Nominated for 2 Oscars. Another 11 wins & 15 nominations.'
114+
},
115+
countries: [ 'USA', 'Hong Kong', 'UK' ],
116+
cast: [
117+
'Harrison Ford',
118+
'Rutger Hauer',
119+
'Sean Young',
120+
'Edward James Olmos'
121+
],
122+
directors: [ 'Ridley Scott' ],
123+
runtime: 117
124+
}
125+
126+
The documents aggregated in this tutorial reside in the
127+
``sample_mflix.movies`` collection. Use the following command to switch
128+
to the database that contains this collection:
129+
130+
.. code-block:: javascript
131+
132+
use sample_mflix
133+
134+
.. _mdb-shell-example-agg-pipeline:
135+
136+
Example Aggregation Pipeline
137+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
138+
139+
Consider the following pipeline:
140+
141+
.. code-block:: javascript
142+
:linenos:
143+
144+
db.movies.aggregate([
145+
146+
// First Stage
147+
148+
{ $project: { _id: 0, genres: 1, imdb: 1, title: 1 } },
149+
150+
// Second Stage
151+
152+
{ $unwind: "$genres" },
153+
154+
// Third Stage
155+
156+
{ $group:
157+
{ _id: "$genres",
158+
averageGenreRating: { $avg: "$imdb.rating" }
159+
}
160+
},
161+
162+
// Fourth Stage
163+
164+
{ $sort: { averageGenreRating: -1 } }
165+
] )
166+
167+
This pipeline performs an aggregation in four stages:
168+
169+
First Stage
170+
The :pipeline:`$project <$project>` stage passes documents that
171+
contain the following fields to the next pipeline stage:
172+
173+
- ``genres``
174+
- ``imdb``
175+
- ``title``
176+
177+
Documents from the collection that don't include all of these fields
178+
are not passed to the next pipeline stage.
179+
180+
.. note::
181+
182+
The ``$project`` stage specifies ``_id: 0`` to suppress the ``_id``
183+
field from the documents it passes to the next stage.
184+
185+
For more information, see the :manual:`MongoDB Manual
186+
</reference/operator/aggregation/project/#suppress-id-field-in-the-output-documents>`.
187+
188+
The ``$project`` stage transforms the example document and passes
189+
the following output to the next pipeline stage:
190+
191+
.. code-block:: javascript
192+
:linenos:
193+
:copyable: false
194+
195+
{
196+
imdb: { rating: 8.2, votes: 419589, id: 83658 },
197+
genres: [ 'Sci-Fi', 'Thriller' ],
198+
title: 'Blade Runner'
199+
}
200+
201+
Second Stage
202+
The :pipeline:`$unwind <$unwind>` stage passes a document for each
203+
element in the ``genres`` array to the next pipeline stage.
204+
205+
The ``$unwind`` stage generates the following two documents from the
206+
original example document, then passes them to the next pipeline
207+
stage:
208+
209+
.. code-block:: javascript
210+
:linenos:
211+
:copyable: false
212+
213+
{
214+
imdb: { rating: 8.2, votes: 419589, id: 83658 },
215+
genres: 'Sci-Fi',
216+
title: 'Blade Runner'
217+
}
218+
219+
.. code-block:: javascript
220+
:linenos:
221+
:copyable: false
222+
223+
{
224+
imdb: { rating: 8.2, votes: 419589, id: 83658 },
225+
genres: 'Thriller',
226+
title: 'Blade Runner'
227+
}
228+
229+
Third Stage
230+
The :pipeline:`$group <$group>` stage:
231+
232+
- Retrieves the distinct genre values from the documents it receives
233+
from the previous pipeline stage,
234+
- Creates a document for each distinct genre where the ``_id`` is the
235+
genre name,
236+
- Adds a field, ``averageGenreRating``, to each new document that
237+
contains the average ``imdb.rating`` of all documents that match the
238+
genre, and
239+
- Passes the new documents to the next pipeline stage.
240+
241+
This stage sends documents that look similar to the following to
242+
the next pipeline stage:
243+
244+
.. code-block:: javascript
245+
:linenos:
246+
:copyable: false
247+
248+
{ _id: 'Sport', averageGenreRating: 6.781233933161954 },
249+
{ _id: 'History', averageGenreRating: 7.202306920762287 },
250+
{ _id: 'Biography', averageGenreRating: 7.097142857142857 },
251+
{ _id: 'Adventure', averageGenreRating: 6.527788649706458 },
252+
{ _id: 'Family', averageGenreRating: 6.36096256684492 },
253+
{ _id: 'Crime', averageGenreRating: 6.730478683620045 },
254+
{ _id: 'Western', averageGenreRating: 6.879197080291971 },
255+
{ _id: 'Fantasy', averageGenreRating: 6.42495652173913 },
256+
{ _id: 'Talk-Show', averageGenreRating: 7 },
257+
{ _id: 'Documentary', averageGenreRating: 7.365266635205286 },
258+
{ _id: 'War', averageGenreRating: 7.183944374209861 },
259+
{ _id: 'Short', averageGenreRating: 7.355813953488372 },
260+
{ _id: 'Horror', averageGenreRating: 5.84110718492344 },
261+
{ _id: 'Film-Noir', averageGenreRating: 7.503809523809523 },
262+
{ _id: 'News', averageGenreRating: 7.254901960784314 },
263+
{ _id: 'Thriller', averageGenreRating: 6.322121555303888 },
264+
{ _id: 'Action', averageGenreRating: 6.3774842271293375 },
265+
{ _id: 'Music', averageGenreRating: 6.923452380952381 },
266+
{ _id: 'Animation', averageGenreRating: 6.917993795243019 },
267+
{ _id: 'Drama', averageGenreRating: 6.830528688822631 }
268+
269+
Fourth Stage
270+
The :pipeline:`$sort <$sort>` stage sorts the documents it receives
271+
from the previous stage in descending order based on the value of
272+
the ``averageGenreRating`` field.
273+
274+
When you run the :ref:`example pipeline
275+
<mdb-shell-example-agg-pipeline>`, the |mdb-shell| prints documents
276+
similar to the following to the terminal:
277+
278+
.. code-block:: javascript
279+
:linenos:
280+
:copyable: false
281+
282+
[
283+
{ _id: 'Film-Noir', averageGenreRating: 7.503809523809523 },
284+
{ _id: 'Documentary', averageGenreRating: 7.365266635205286 },
285+
{ _id: 'Short', averageGenreRating: 7.355813953488372 },
286+
{ _id: 'News', averageGenreRating: 7.254901960784314 },
287+
{ _id: 'History', averageGenreRating: 7.202306920762287 },
288+
{ _id: 'War', averageGenreRating: 7.183944374209861 },
289+
{ _id: 'Biography', averageGenreRating: 7.097142857142857 },
290+
{ _id: 'Talk-Show', averageGenreRating: 7 },
291+
{ _id: 'Music', averageGenreRating: 6.923452380952381 },
292+
{ _id: 'Animation', averageGenreRating: 6.917993795243019 },
293+
{ _id: 'Western', averageGenreRating: 6.879197080291971 },
294+
{ _id: 'Drama', averageGenreRating: 6.830528688822631 },
295+
{ _id: 'Sport', averageGenreRating: 6.781233933161954 },
296+
{ _id: 'Crime', averageGenreRating: 6.730478683620045 },
297+
{ _id: 'Musical', averageGenreRating: 6.696913580246913 },
298+
{ _id: 'Romance', averageGenreRating: 6.695711554220159 },
299+
{ _id: 'Mystery', averageGenreRating: 6.563317384370015 },
300+
{ _id: 'Adventure', averageGenreRating: 6.527788649706458 },
301+
{ _id: 'Comedy', averageGenreRating: 6.479626461362988 },
302+
{ _id: 'Fantasy', averageGenreRating: 6.42495652173913 }
303+
]
304+
305+
.. seealso::
306+
307+
- To learn more about the available aggregation stages, see
308+
:manual:`Aggregation Pipeline Stages
309+
</reference/operator/aggregation-pipeline/>`.
310+
311+
- To learn more about the available aggregation operators you
312+
can use within stages, see
313+
:manual:`Aggregation Pipeline Operators
314+
</reference/operator/aggregation/>`.

0 commit comments

Comments
 (0)