Skip to content

Commit 344ce8c

Browse files
committed
Add docs about multiple streams and groupings. Fixes #252.
1 parent 24a6ba6 commit 344ce8c

File tree

1 file changed

+40
-0
lines changed

1 file changed

+40
-0
lines changed

doc/source/topologies.rst

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -91,6 +91,46 @@ To do that you just need to use the :meth:`streamparse.ShellBolt.spec` and
9191
``script`` arguments to specify a binary to run and its string-separated
9292
arguments.
9393

94+
Multiple Streams
95+
^^^^^^^^^^^^^^^^
96+
97+
To specify that a component has multiple output streams, instead of using a
98+
list of strings for :attr:`~streamparse.dsl.component.ComponentSpec.outputs`,
99+
you must specify a list of :class:`~streamparse.Stream` objects, as shown below.
100+
101+
.. code-block:: python
102+
103+
class FancySpout(Spout):
104+
outputs = [Stream(fields=['good_data'], name='default'),
105+
Stream(fields=['bad_data'], name='errors')]
106+
107+
To select one of those streams as the input for a downstream
108+
:class:`~streamparse.Bolt`, you simply use ``[]`` to specify the stream you
109+
want. Without any stream specified, the ``default`` stream will be used.
110+
111+
.. code-block:: python
112+
113+
class ExampleTopology(Topology):
114+
fancy_spout = FancySpout.spec()
115+
error_bolt = ErrorBolt.spec(inputs=[fancy_spout['errors']])
116+
process_bolt = ProcessBolt.spec(inputs=[fancy_spout])
117+
118+
119+
Groupings
120+
^^^^^^^^^
121+
122+
By default, Storm uses a :attr:`~streamparse.Grouping.SHUFFLE` grouping to route
123+
tuples to particular executors for a given component, but you can also specify
124+
other groupings by using the appropriate :class:`~streamparse.Grouping`
125+
attribute. The most common grouping is probably the
126+
:meth:`~streamparse.Grouping.fields` grouping, which will send all the tuples
127+
with the same value for the specified fields to the same executor. This can be
128+
seen in the prototypical word count topology:
129+
130+
.. literalinclude:: ../../examples/redis/topologies/wordcount_mem.py
131+
:language: python
132+
133+
94134
Topology-Level Configuration
95135
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
96136

0 commit comments

Comments
 (0)