Skip to content

RuntimeError: deque mutated during iteration #2645

Closed
@Havefun404

Description

@Havefun404

kafka-python version==2.2.11
After running days, at some point it terminated with this:

  File "T_kafkas_4.py", line 318, in <module>
    for msg in consumer:
  File "/data/home/.local/lib/python3.6/site-packages/kafka/consumer/group.py", line 1188, in __next__
    return next(self._iterator)
  File "/data/home/.local/lib/python3.6/site-packages/kafka/consumer/group.py", line 1160, in _message_generator_v2
    record_map = self.poll(timeout_ms=timeout_ms, update_offsets=False)
  File "/data/home/.local/lib/python3.6/site-packages/kafka/consumer/group.py", line 684, in poll
    records = self._poll_once(timer, max_records, update_offsets=update_offsets)
  File "/data/home/.local/lib/python3.6/site-packages/kafka/consumer/group.py", line 716, in _poll_once
    futures = self._fetcher.send_fetches()
  File "/data/home/.local/lib/python3.6/site-packages/kafka/consumer/fetcher.py", line 150, in send_fetches
    for node_id, (request, fetch_offsets) in six.iteritems(self._create_fetch_requests()):
  File "/data/home/.local/lib/python3.6/site-packages/kafka/consumer/fetcher.py", line 629, in _create_fetch_requests
    for partition in self._fetchable_partitions():
  File "/data/home/.local/lib/python3.6/site-packages/kafka/consumer/fetcher.py", line 610, in _fetchable_partitions
    discard = {fetch.topic_partition for fetch in self._completed_fetches}
  File "/data/home/.local/lib/python3.6/site-packages/kafka/consumer/fetcher.py", line 610, in <setcomp>
    discard = {fetch.topic_partition for fetch in self._completed_fetches}
RuntimeError: deque mutated during iteration

Another issue(#1399) mentioned this before. It has been fixed but why still have this?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions