On mostly-duplicate git urls, pick whichever came first (#1084) #1428
Conversation
|
I checked out your comment here and I think I follow your stated approaches. I'm happy with the approach presented here, but I also think there could be merit to resolving this problem only when a conflict occurs. In the issue, you noted:
This actually sounds like the best possible option because we:
What's involved in a pick-a-winner approach? |
|
The pick-a-winner approach is a bit more complicated - I'll modify ProjectList to change this bit to check if the incorporated package is a GItPackage, and if so do some additional checks. |
First write wins, on the assumption that all matching URLs are valid
beckjake
force-pushed
the
fix/strip-url-trailing-dotgit
branch
from
590c310
to
3af88b0
Compare
beckjake
changed the title
core/dbt/task/deps.py
Outdated
| @@ -216,9 +216,17 @@ class GitPackage(Package): | |||
|
|
|||
| def __init__(self, *args, **kwargs): | |||
| super(GitPackage, self).__init__(*args, **kwargs) | |||
| # strip trailing '.git's from urls | |||
There was a problem hiding this comment.
| # strip trailing '.git's from urls |
|
|
||
| def __getitem__(self, key): | ||
| if isinstance(key, basestring): | ||
| return super(PackageListing, self).__getitem__(key) |
There was a problem hiding this comment.
When i use the following package spec, I see a backtrace on this line:
packages:
- git: "https://github.com/fishtown-analytics/dbt-utils"
revision: 0.1.12
- git: "https://github.com/fishtown-analytics/dbt-event-logging.git"
revision: 0.1.5
2019-05-07 21:48:28,778 (MainThread): Traceback (most recent call last):
File "/Users/drew/fishtown/dbt/core/dbt/main.py", line 80, in main
results, succeeded = handle_and_check(args)
File "/Users/drew/fishtown/dbt/core/dbt/main.py", line 153, in handle_and_check
task, res = run_from_args(parsed)
File "/Users/drew/fishtown/dbt/core/dbt/main.py", line 201, in run_from_args
results = task.run()
File "/Users/drew/fishtown/dbt/core/dbt/task/deps.py", line 525, in run
final_deps[name].resolve_version()
File "/Users/drew/fishtown/dbt/core/dbt/task/deps.py", line 397, in __getitem__
return super(PackageListing, self).__getitem__(key)
KeyError: 'https://github.com/fishtown-analytics/dbt-utils.git'
I don't totally understand what's happening here, or why this case is different than the example provided in the integration test in this PR. Can you check it out?
There was a problem hiding this comment.
It looks like while evaluating subpackages, dbt was using the chosen canonical key of the package instead of the package itself for the collision check, which bypassed the extra other_name handling. That's no longer valid since the two dicts might have distinct keys that should match!
remove incorrect comment fix an issue where we looked deps up by the wrong key add a test
beckjake
force-pushed
the
fix/strip-url-trailing-dotgit
branch
from
905bdbc
to
12f0887
Compare
|
I've seen this a couple times now. It's a transient CI-only thing (I can never reproduce it locally) - the thread that is supposed to send the kill seems to not manage to do so in the 15s it takes the Maybe we could take the lazy way out and decorate it with this pytest plugin? |
|
Flaky looks cool! not something to solve in this PR though.... I just kicked off the tests again - let's merge when they're all 🍏 📗 💚 🥗 🇬🇱 |
Fixes #1084
The first encountered of duplicate URLs (the same URL with
.gitor without.gitare considered duplicates) is used as the official URL.