Skip to content

Commit

Permalink
handle a subtle iteration issue in the cache with extra locking
Browse files Browse the repository at this point in the history
  • Loading branch information
Jacob Beck committed Oct 4, 2018
1 parent 198883a commit 96e7fdb
Showing 1 changed file with 7 additions and 6 deletions.
13 changes: 7 additions & 6 deletions dbt/adapters/cache.py
Original file line number Diff line number Diff line change
Expand Up @@ -184,13 +184,14 @@ def dump_graph(self):
known relation is a key with a value of a list of keys it is referenced
by.
"""
# we have to hold the lock while iterating, if other threads modify
# self.relations during iteration it's a runtime error
# we have to hold the lock for the entire dump, if other threads modify
# self.relations or any cache entry's referenced_by during iteration
# it's a runtime error!
with self.lock:
# consume the full iterator inside the lock
items = list(self.relations.items())

return {dot_separated(k): v.dump_graph_entry() for k, v in items}
return {
dot_separated(k): v.dump_graph_entry()
for k, v in self.relations.items()
}

def _setdefault(self, relation):
"""Add a relation to the cache, or return it if it already exists.
Expand Down

0 comments on commit 96e7fdb

Please sign in to comment.