Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

concurrent schema test writes can break #383

Closed
drewbanin opened this issue Apr 18, 2017 · 1 comment
Closed

concurrent schema test writes can break #383

drewbanin opened this issue Apr 18, 2017 · 1 comment
Assignees

Comments

@drewbanin
Copy link
Contributor

ompiled 2 models, 2 tests, 0 archives, 0 analyses, 12 macros
Concurrency: 2 threads (target='dev')

09:14:25 | Running 2 tests
09:14:25 |
09:14:25 | 1 of 2 START test not_null_integers_i................................ [RUN]
09:14:25 | 2 of 2 START test unique_integers_i.................................. [RUN]
09:14:26 | 2 of 2 PASS unique_integers_i........................................ [PASS in 0.01s]
Traceback (most recent call last):
  File "/usr/local/bin/dbt", line 11, in <module>
    load_entry_point('dbt==0.8.0', 'console_scripts', 'dbt')()
  File "/usr/local/Cellar/dbt/0.8.0/libexec/lib/python3.6/site-packages/dbt/main.py", line 28, in main
    handle(args)
  File "/usr/local/Cellar/dbt/0.8.0/libexec/lib/python3.6/site-packages/dbt/main.py", line 46, in handle
    res = run_from_args(parsed)
  File "/usr/local/Cellar/dbt/0.8.0/libexec/lib/python3.6/site-packages/dbt/main.py", line 100, in run_from_args
    result = task.run()
  File "/usr/local/Cellar/dbt/0.8.0/libexec/lib/python3.6/site-packages/dbt/task/test.py", line 31, in run
    results = runner.run_tests(include, exclude, set())
  File "/usr/local/Cellar/dbt/0.8.0/libexec/lib/python3.6/site-packages/dbt/runner.py", line 852, in run_tests
    flatten_graph=True)
  File "/usr/local/Cellar/dbt/0.8.0/libexec/lib/python3.6/site-packages/dbt/runner.py", line 831, in run_types_from_graph
    on_failure, should_run_hooks)
  File "/usr/local/Cellar/dbt/0.8.0/libexec/lib/python3.6/site-packages/dbt/runner.py", line 696, in execute_nodes
    for node in nodes_to_execute]):
  File "/usr/local/opt/python3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py", line 699, in next
    raise value
  File "/usr/local/opt/python3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/usr/local/Cellar/dbt/0.8.0/libexec/lib/python3.6/site-packages/dbt/runner.py", line 577, in safe_execute_node
    raise e
  File "/usr/local/Cellar/dbt/0.8.0/libexec/lib/python3.6/site-packages/dbt/runner.py", line 535, in safe_execute_node
    node = compiler.compile_node(node, flat_graph)
  File "/usr/local/Cellar/dbt/0.8.0/libexec/lib/python3.6/site-packages/dbt/compilation.py", line 323, in compile_node
    injected_node['wrapped_sql'])
  File "/usr/local/Cellar/dbt/0.8.0/libexec/lib/python3.6/site-packages/dbt/compilation.py", line 166, in __write
    os.makedirs(os.path.dirname(target_path))
  File "/usr/local/Cellar/dbt/0.8.0/libexec/bin/../lib/python3.6/os.py", line 220, in makedirs
    mkdir(name, mode)
FileExistsError: [Errno 17] File exists: 'target/build/test_project/schema_test'

Because of this code:

        if not os.path.exists(os.path.dirname(target_path)):
            os.makedirs(os.path.dirname(target_path))

We can just catch the FileExistsError. Does anything like this happen elsewhere?

@drewbanin drewbanin self-assigned this Apr 21, 2017
@drewbanin
Copy link
Contributor Author

fixed: #398

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant