Support grouping for "must-be-updated-together" dependencies

[just-jeb]

To be clear, this feature request is different from #1190.
It is not about convenience (creating one PR instead of creating multiple PRs to keep the history clean), but rather about something that must-have when you're working with scoped packages or packages that are released together.

The issue:
There are some packages that are released and work together and if you update just one of them the build will essentially fail.
Examples for such packages are @angular/core & @angular/compiler, @angular/cli & @angular-devkit/build-angular, react and react-dom and so on.

The request:
Allow (either implicitly like in greenkeeper) or explicitly (like in renovate) updating these group dependencies in a single PR.
The lack of this functionality is a blocker for anyone who uses multiple packages that are tightly coupled and have the same release schedule (and even a few packages I specified above are, well, quite popular).
All the beautiful work Dependabot does is worth nothing if one still have to spend time on manually updating group dependencies.

[feelepxyz]

@just-jeb we have the ability to group tightly coupled dependencies but have only done so for vue and vue-template-compiler (because it breaks the build if using different versions).

From my understanding react and react-dom minor/patch versions can be out of sync without breaking. Whenever there's a major version bump we'll update them together (due to the peer dependency requirement).

Trying to figure out if we should be grouping angular and seeing that @angular/core, @angular/compiler, @angular/cli have a slightly different versions scheme and release schedule to @angular-devkit/build-angular and @angular/pwa.
You know if these should be grouped differently? Is there anything in angular that breaks if
for example, core and compiler versions don't match up?

Looking at the greenkeeper list it looks like we could maybe just use their definitions list to group these dependencies.

[just-jeb]

Using greenkeeper definitions is a good start, although making it configurable is a preferable option.
Regarding Angular I know for sure that @angular/cli and @angular-devkit are tightly coupled and can break even with minor version changes. I'd say for Angular everything (the whole @angular scope and @Angular-devkit scope) should be grouped together as they are always released together.
There is an open PR (for almost a year now) in Greenkeeper monorepo-definitions repository where you can get more details.

[feelepxyz]

@just-jeb yeah definitely keen to support custom grouping in future but pretty stacked atm integrating with Github so won't get to anytime soon unfortunately. Will see what makes sense on grouping cli/devkit. Keen to keen the grouping to stuff that's breaking atm until we have a way of configuring it.

[olson-sean-k]

Just wanted to bump this and report that I've encountered the same issue with graphics and windowing crates in the Rust ecosystem. gfx, glutin, and gfx_window_glutin often require lockstep upgrades and dependabot has opened separate PRs against my repositories to upgrade each of these independently. Those PRs fail CI with no (obvious) way to merge/combine them.

[SebastienGllmt]

I would love this. For example I would like all NPM packages that start with @babel/ to be updated together and all packages that start with @storybook/ to be updated together.

My npm outdated often looks like this

which would be nice if it was just 2 PRs (although to be fair it's happened before that some specific babel plugin dependency breaks my build but the others don't but I can debug this manually when it happens)

[hisuwh]

I would like to point out this is also the case for .NET core sdks, there was a recent bump from 2.2 to 3 and we had about 30 prs created that I had to manually consolidate into one branch to get it to build

[connorshea]

Worth mentioning that this is also relevant to RubyGems with Sorbet, which should update sorbet, sorbet-static, and sorbet-runtime all at once.

[snebjorn]

This seems to be related to #317

[aminya]

Isn't there a solution for this yet?

Dependant bot spams my email every day to update every single package individually! I prefer to bump all of them at the same time in a single pull request if my tests pass!

Unfortunately, the dependant bot follows a rigid architecture of Ruby classes which makes it hard for adding new features. These features could be a simple shell command rather than a Ruby class. I can do this very easily in using npm-check-updates and GitHub actions.

name: Bump

on:
  schedule:
    - cron: "0 0 * * *"
      
jobs:
  bump:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-node@v2-beta
        with:
          node-version: '14'
      - run: |
          npm install -g npm-check-updates
          ncu "/^@babel\/.*$/"     # or `ncu -u`
          npm install
      - uses: peter-evans/create-pull-request@v2
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          commit-message:  Update dependencies
          title: "[AUTO] Update dependencies"
          labels: Dependencies
          branch: "Bump"

[ashughes]

I haven't been able to find whether this is documented anywhere, but I discovered a workaround (at least for Gradle dependencies): #1618 (comment)

[aminya]

@ashughes Thanks for the answer. That seems to be specific to Gradle. There is no such thing in other languages. That is why a shell coding interface is necessary for the Dependant bot to allow others to execute custom dependency management tasks.

[ashughes]

@aminya To be clear, I wasn't suggesting that this issue is solved. Just that for Gradle there appears to be a workaround (and I'm not sure if there are similar workarounds for other build systems). Still would like to see official support for this in Dependabot!

[mercuriete]

The answer is simple: talk with the guys in charge of package.json syntax and add a variable interpolation as is in maven or gradle.
I miss that feature on node that is on java for decades.
Upgrading angular dependencies is a hell without having such a feature.
I think is not a dependabot problem but a package.json problem.

[aminya]

@mercuriete That is already solved by ncu. The problem is that there is no shell interface for dependant bot.

See my comment: #1296 (comment)

[joaorosado]

Just my 2 cents on this.
I'm really struggling with this missing capability on my repositories :(

I think I can workaround some like it was said above with a variable for some external libraries where the numbers match, but that is not always the case.
The main issue is when there are many upstream repositories that contribute to mine, and the version numbers are all independent and cause most builds to fail with "downgrade detected" errors.

Note: in my scenario it is for C#/nuget projects

[sebaslogen]

This is also required for updating Android Gradle dependencies because Google couples the Jetpack Compose dependency with a specific Kotlin language version and also a specific KSP version.
All three dependencies have different numbers, so they can't just use the same version number stored in a variable, but all three of them need to be updated in the same PR

[abdulapopoola]

Update: We've started doing some grouped updates work! This particular issue might not be part of the first ship but if you want to track our updates, do follow #1190.

[jtbandes]

If anyone's looking for a DIY solution, we've had some success in foxglove/studio by using the yarn constraints feature.

First we define version groups in constraints.pro:
(note: the Yarn team is planning to migrate the constraints feature from Prolog to JavaScript in Yarn v4: yarnpkg/berry#1276)

% Define version groups for dependencies. All dependencies in the same group will be required to have the same version.

% All `@storybook/*` packages, including `storybook`, but excluding `@storybook/testing-library`, must have the same version.
version_group(Dep, storybook) :-
  Dep = 'storybook';
  has_prefix('@storybook/', Dep), Dep \= '@storybook/testing-library'.
% All `@typescript-eslint/*` packages must have the same version.
version_group(Dep, typescript_eslint) :-
  has_prefix('@typescript-eslint/', Dep).

Then we have a GitHub actions workflow that runs on Dependabot PRs:

    if: |
      github.actor == 'dependabot[bot]' &&
      contains(github.event.pull_request.head.ref, 'dependabot/npm_and_yarn/')

This workflow runs yarn constraints --fix, yarn install, yarn dedupe, and pushes a commit with the resulting package.json and lockfile changes. This way, we get PRs that actually bump multiple packages at once. For example, https://github.com/foxglove/studio/pull/5891 bumps @typescript-eslint/eslint-plugin and @typescript-eslint/parser.

We also run yarn constraints during our normal CI actions on every PR to check that the constraints are satisfied.

This approach only works for packages that are bumped together and have the exact same version number. It doesn't just bump multiple packages each to their latest versions, but it enforces that they have the same version.

[IvanPizhenko]

This is also required for updating Android Gradle dependencies because Google couples the Jetpack Compose dependency with a specific Kotlin language version and also a specific KSP version. All three dependencies have different numbers, so they can't just use the same version number stored in a variable, but all three of them need to be updated in the same PR

@sebaslogen In this case you can use Gradle's lib.versions.toml file, dependabot knows how to work with it.
For working real-life example see my SO post: https://stackoverflow.com/a/76067491/1540501
Realted Gradle documentation: https://docs.gradle.org/current/userguide/platforms.html#sub:conventional-dependencies-toml

[sebaslogen]

This is also required for updating Android Gradle dependencies because Google couples the Jetpack Compose dependency with a specific Kotlin language version and also a specific KSP version. All three dependencies have different numbers, so they can't just use the same version number stored in a variable, but all three of them need to be updated in the same PR

@sebaslogen In this case you can use Gradle's lib.versions.toml file, dependabot knows how to work with it.
For working real-life example see my SO post: https://stackoverflow.com/a/76067491/1540501
Realted Gradle documentation: https://docs.gradle.org/current/userguide/platforms.html#sub:conventional-dependencies-toml

That doesn't solve the problem because the libraries that need to be updated in the same group do not share the same version number (like suggested in the SO answer).
Example: Compose version 1.4.1 works only with kotlin 1.8.0

[Marvin-Brouwer]

Just as a reference, I have a similar problem in .net.
These packages are always updated together, just updating one of them results in compiler/build errors.

• Microsoft.CodeAnalysis
• Microsoft.CodeAnalysis.CSharp
• Microsoft.CodeAnalysis.Compilers

I've implemented the workaround from: #1618 (comment).
Dependabot seems to understand this properly, so I guess this is good enough for now.

[j0k3r]

Looks like something is coming dependabot/fetch-metadata#389 & dependabot/fetch-metadata#374

[mtripg6666tdr]

I appreciate the dependabot team worked for the useful feature!

[j0k3r]

If you didn't notice the blog post, it's available in beta https://github.blog/changelog/2023-06-30-grouped-version-updates-for-dependabot-public-beta/

[jtbandes]

Awesome, thanks to the team for working on this! It seems to be working well for us so far: https://github.com/foxglove/studio/pull/6367

[edmorley]

Thank you for adding this! It's working pretty well so far, though we did run into #7305.

[carogalvin]

@j0k3r already mentioned this, but just wanted to pop in from the Dependabot team to repeat that we have configurable grouping rules in public beta, which should solve this problem! You can see the docs on how to set these up here: https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#groups

I'm aiming to leave this issue open until we go to "official" release. In the meantime, if you have any feedback, please feel free to email me at carogalvin@github.com or set up a meeting with me by picking a convenient time in my calendar here.

[yurishkuro]

Thank you for this long awaited feature. Worked very well for Go modules, with one minor, non-blocking issue #7521.

[abdulapopoola]

Closing this out as we've officially released grouped version updates; please feel free to reopen or reach out if there are more questions or feedback.

[jeppester]

@abdulapopoola
I was hoping to use this for packages in our rails projects.
It's common for rails packages (e.g. active-storage, action-cable, shakapacker) to have both a ruby gem and an npm package.

Unfortunately the new functionality is limited to packages within a single package ecosystem. Is there any technical reason for this limitation?

[deivid-rodriguez]

@jeppester I think you may want to track #8126?