[Core] Default behavior of dbt deps changed in v1.7

[dbeatty10]

Link to the page(s) on docs.getdbt.com requiring updates

For v1.7+, the default behavior of dbt deps has changed.

This can be surprising to long-time users of dbt deps, so we should document this in both the migration guide for v1.7 and anywhere else it is applicable (like here or here).

Tell us more about this update

See the GitHub links below.

Reviewers/Stakeholders/SMEs

{{ this }}

Related GitHub issues

See dbt-labs/dbt-core#9796 for user request for updated documentation.

Also see the original issue + PRs in GitHub:

• [CT-1845] [Feature] Write a file of installed versions during dbt deps dbt-core#6643
• ✨ restructure deps command with package-lock.yml dbt-core#6735
• Deps lock by justbldwn dbt-core#8408

Additional information

No response

[matthewshaver]

@dbeatty10 I'll work on adding the deps info to the linked pages. We have outlined it in the v1.7 migration guide, is there anything specific you would like to see added to that page?

[dbeatty10]

@dbeatty10 I'll work on adding the deps info to the linked pages. We have outlined it in the v1.7 migration guide, is there anything specific you would like to see added to that page?

Thanks for reaching out @matthewshaver !

Here is the instigating issue: dbt-labs/dbt-core#9796 (comment)

This is the behavior change that was surprising to the user:

When further commits are added to this branch, despite calling dbt clean and dbt deps again, this remains hard pinned to the earlier commit and I have to delete the package lock file to get it to fetch the latest commit on the branch instead.

Although we do have some information in the migration guide, we don't clearly explain the behavior that is different in v1.7 nor how to achieve the same behavior as v1.6.

I mean, we kinda do, but there's too much to read in between the lines -- we should just be clear and concise about:

1. What is different
2. What the user can do to keep the same behavior

Equivalent to v1.6 and earlier

I didn't confirm or deny, but the get the same behavior as v1.6 and earlier, it sounds like you'd need to do one of the following options:

1. dbt deps --upgrade

   It's possible to force package resolution to rerun, even if the packages config hasn't changed, by running dbt deps --upgrade. This enables you to get the latest commits from the main branch of an internally maintained git package while accepting the risk of unpredictable builds.

   As a replacement every time was dbt deps was called previously:

   dbt deps --upgrade

2. Git ignore package-lock.yml

   An alternative to running dbt deps --upgrade in production is to "ignore" the lock file by adding package-lock.yml to your project's .gitignore file. If you pursue either approach, dbt Labs strongly recommends adding version pins for third-party packages within your packages config.

   One-time action:

   echo "\npackage-lock.yml" >> .gitignore
   git add .gitignore
   git commit -m "Add package-lock.yml to .gitignore"
   # open up a PR, etc.

3. Delete package-lock.yml

   As a replacement every time was dbt deps was called previously:

   rm package-lock.yml
   dbt deps

Not equivalent to v1.6 and earlier

This one is not equivalent to v1.6 and earlier because it only recreates a new package-lock.yml if it detects a (meaningful) change to the packages.yml file.

4. dbt deps --lock && dbt deps

   To only check for changes to the packages config and update the lock file accordingly without installing those packages, provide the --lock flag (that is, dbt deps --lock).

   As a replacement every time was dbt deps was called previously:

   dbt deps --lock
   dbt deps

[rlh1994]

Yeah I think for context I was probably looking for the --upgrade flag to be detailed in this section where it says you can provide a branch, tag, or commit hash. At least to me I think of a branch as the head commit, and the other two are pinned to specific commits, so maybe just something there to highlight if you are using a branch the package lock will pin to the head at time of the first dbt deps unless you run it with the upgrade flag?

https://docs.getdbt.com/docs/build/packages#git-packages

[matthewshaver]

Thank you @dbeatty10 and @rlh1994 I will work on updating the impacted areas. I appreciate the feedback and details