Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Redis Example: Fix AttributeError #336

Merged
merged 2 commits into from
Nov 16, 2016
Merged

Update Redis Example: Fix AttributeError #336

merged 2 commits into from
Nov 16, 2016

Conversation

cyhsutw
Copy link
Contributor

@cyhsutw cyhsutw commented Nov 16, 2016

Hi,

First, thanks for building streamparse, it's an awesome project!


What Happened?

I encountered the following while running the provided redis example:

28417 [Thread-23] ERROR o.a.s.t.ShellBolt - ShellLog pid:91105, name:count_bolt 2016-11-16 14:35:32,581 - pystorm.component.count_bolt - Exception in RedisWordCountBolt.run()
Traceback (most recent call last):
  File "/Users/cyhsutw/.virtualenvs/streamparse-dev/lib/python2.7/site-packages/pystorm/component.py", line 481, in run
    self._run()
  File "/Users/cyhsutw/.virtualenvs/streamparse-dev/lib/python2.7/site-packages/pystorm/bolt.py", line 197, in _run
    self.process(tup)
  File "/private/var/folders/fl/s44qnfxs6lldrpy8w7k6rd400000gn/T/b34e7e46-7298-4068-89dc-f3f470c61998/supervisor/stormdist/wordcount_redis-1-1479278116/resources/bolts.py", line 24, in process
    self.emit([word, self.counter[word]])
AttributeError: 'RedisWordCountBolt' object has no attribute 'counter'

Full log: https://gist.github.com/cyhsutw/999a37329e79b61943540e588289c622

System Information

  • Python: 2.7.12, 3.5.2
  • storm: 1.0.2
  • streamparse: HEAD@bc49df
  • Redis: 3.2.5

Proposed Solution

Looking into the source files, I found that RedisWordCountBolt is a subclass of WordCountBolt where the counter attribute is defined. However, the initialize method in RedisWordCountBolt does not call initialize on its superclass.

While adding super(RedisWordCountBolt, self).initialize(conf, ctx) will fix the issue, I am thinking if we could just separate the implementations of these two classes since they are quite different in data storage.

I've drafted a possible implementation and would love to hear your feedback on it.

How to Test

  1. Spin up a Redis server locally
  2. Install dependencies of the example
  3. Run parse run --name wordcount_redis
  4. Watch the activities using top.sh and watch.sh

Thank you! 😄

@cyhsutw cyhsutw changed the title Update Redis Example: Fix Attribute Error Update Redis Example: Fix AttributeError Nov 16, 2016
@coveralls
Copy link

Coverage Status

Coverage remained the same at 44.534% when pulling bd9a52b on cyhsutw:cyhsutw/update-redis-example into bc49dfd on Parsely:master.

@dan-blanchard
Copy link
Member

Thanks for fixing this. I must not have tried the Redis example in a long time.

@dan-blanchard dan-blanchard merged commit 705a402 into pystorm:master Nov 16, 2016
@cyhsutw cyhsutw deleted the cyhsutw/update-redis-example branch November 17, 2016 10:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants