Create 1 PR for the same update across update configs

[hisuwh]

We have a monorepo for some microservices and have config something like this:

version: 1
update_configs:
- package_manager: "javascript"
  directory: "/path/to/project1"
  update_schedule: "daily"

- package_manager: "javascript"
  directory: "/path/to/project2"
  update_schedule: "daily"

- package_manager: "dotnet:nuget"
  directory: "/path/to/project3"
  update_schedule: "daily"

- package_manager: "dotnet:nuget"
  directory: "/path/to/project4"
  update_schedule: "daily"

So if for example Typescript gets updated we end up with two PRs, 1 each for project1 and project2.

Its worse with dotnet because we have common assemblies between the two projects, so if a dependency in one of them changes, we end up with two PRs for the same project. This obviously resolves and dependabot closes the second one when we merge the first one but it is still extra noise.

Perhaps you could support multiple folders per configuration so they can be easily grouped?

version: 1
update_configs:
- package_manager: "javascript"
  directory:
  - "/path/to/project1"
  - "/path/to/project2"
  update_schedule: "daily"

- package_manager: "dotnet:nuget"
  directory: 
  - "/path/to/project3"
  - "/path/to/project4"
  update_schedule: "daily"

[infin8x]

Similar to #1190

[scottrobertson]

This would be awesome for us.

[abdulapopoola]

Update: We've started doing some grouped updates work! This particular issue might not be part of the first ship but if you want to track our updates, do follow #1190.

[abdulapopoola]

Following up; we've started doing some exploration to allow multiple directories to be grouped together.

[abdulapopoola]

tagging @Nishnha and @honeyankit

[danielknell]

I have been finding this increasingly problematic with local interdependencies within a python monorepo.

if package1 depends on package2 and a dep is bumped in package2 then the poetry lock file in package1 still depends on the older version of the transitive dependency.

this will fail the build, as the lock file for package1 contains the old version of dep1, which does not meet the new transitive requirement from package2, so we end up having to manually untangle that.

updating across the entire repo would prevent this.

[yeikel]

I have been finding this increasingly problematic with local interdependencies within a python monorepo.

if package1 depends on package2 and a dep is bumped in package2 then the poetry lock file in package1 still depends on the older version of the transitive dependency.

this will fail the build, as the lock file for package1 contains the old version of dep1, which does not meet the new transitive requirement from package2, so we end up having to manually untangle that.

updating across the entire repo would prevent this.

I ignore how other ecosystems work, but can your Python package manager store and share version definitions across projects?

In the Java world, this is easily solved defining the version in the parent project and letting the children projects inherit the version.

When this happens, not only dependabot but any tool or developer can update the version in a single place

[danielknell]

I have been finding this increasingly problematic with local interdependencies within a python monorepo.
if package1 depends on package2 and a dep is bumped in package2 then the poetry lock file in package1 still depends on the older version of the transitive dependency.
this will fail the build, as the lock file for package1 contains the old version of dep1, which does not meet the new transitive requirement from package2, so we end up having to manually untangle that.
updating across the entire repo would prevent this.

I ignore how other ecosystems work, but can your Python package manager store and share version definitions across projects?

In the Java world, this is easily solved defining the version in the parent project and letting the children projects inherit the version.

When this happens, not only dependabot but any tool or developer can update the version in a single place

no, this is not supported by the tooling at the moment.

[yeikel]

I have been finding this increasingly problematic with local interdependencies within a python monorepo.
if package1 depends on package2 and a dep is bumped in package2 then the poetry lock file in package1 still depends on the older version of the transitive dependency.
this will fail the build, as the lock file for package1 contains the old version of dep1, which does not meet the new transitive requirement from package2, so we end up having to manually untangle that.
updating across the entire repo would prevent this.

I ignore how other ecosystems work, but can your Python package manager store and share version definitions across projects?

In the Java world, this is easily solved defining the version in the parent project and letting the children projects inherit the version.

When this happens, not only dependabot but any tool or developer can update the version in a single place

no, this is not supported by the tooling at the moment.

Relevant python-poetry/poetry#2270

[danielknell]

Relevant python-poetry/poetry#2270

I don't see that ticket being realistically touched in the foreseeable future, it's just an unnecessary distraction from the issue here.

[yeikel]

Relevant python-poetry/poetry#2270

I don't see that ticket being realistically touched in the foreseeable future, it's just an unnecessary distraction from the issue here.

Apologies, I did not mean to suggest that the issue would solve the problem here.

It just answered my question and could help others with a similar curiosity. Dependabot can definitely support this independently even if poetry never does

[magnusjtvisma]

Is this now supported given this recent PR: #8541 ?

[jzabroski]

@magnusjtvisma Not sure but I opened #8808 because I can't read Ruby well enough to know how to configure whatever it is 8541 is doing.

[abdulapopoola]

Multi-dir configuration is now in public beta; open to hearing your thoughts as you try out the beta!

Tagging @carlincherry

[bjorhn]

I've attemped to modify our dependabot.yml according to the blog post but it doesn't appear to be working correctly on my end. The below code snippet results in 4 pull requests being created, only for the project file in the root ("/") folder, despite several packages in the other project files being out of date.

When I check the dependabot logs it says "Errored with the message "Dependabot cannot open any more pull requests"". I don't recall 4 being the limit but perhaps it is nowadays. Should I expect to see a single pull request if the same package is referenced in multiple project files, or one pull request for each project where the package is referenced?

All projects are Poetry projects in pyproject.toml format.

  # Enable version updates for pip
  - package-ecosystem: "pip"
    directories:
      - "/"
      - "/cli"
      - "/dbt"
      - "/agent"
      - "/report"
      - "/pipeline"
      - "/realtime"
      - "/deployment"
      - "/flow_audit"
      - "/integration"
      - "/maintenance"
      - "/orchestration"
      - "/scripts"
    # Check for updates once a month
    schedule:
      interval: "monthly"
      day: "monday"
      time: "04:00"
      timezone: "Europe/Stockholm"

Edit: My mistake, of course I had to create groups as well. Now things work as expected!

[jzabroski]

You should be able to configure the PR limit.

open-pull-requests-limit: 10

The use case to limit it is that sometimes it can be overbearing to review a bunch of dependencies.

[jzabroski]

For me, the annoying part so far testing out the beta is that I can't seem to get dependabot to update existing incorrect PRs. I tried @dependabot recreate and it told me that it was previously created with a different config. It then suggested I close the PR: "The dependabot.yml entry that created this PR has been deleted so this PR can't be recreated. Please close the PR so Dependabot can create a new one with the current dependabot.yml." I did so. It then responded with "OK, I won't notify you again about this release, but will get in touch when a new version is available. [...]" - is this message incorrect?

Edit: If I go to Insights -> Dependency Graph -> Dependabot, I can see the dependabot logs that one of the directories was a top-level directory, and so dependabot failed to unify the directories to find csproj files to consider. So, the message may be incorrect.

[stevehipwell]

This worked great in the repo I tested it in. I had 3 outstanding PRs and after changing the config these were closed and replaced with a single PR.

[carlincherry]

For me, the annoying part so far testing out the beta is that I can't seem to get dependabot to update existing incorrect PRs. I tried @dependabot recreate and it told me that it was previously created with a different config. It then suggested I close the PR: "The dependabot.yml entry that created this PR has been deleted so this PR can't be recreated. Please close the PR so Dependabot can create a new one with the current dependabot.yml." I did so. It then responded with "OK, I won't notify you again about this release, but will get in touch when a new version is available. [...]" - is this message incorrect?

Edit: If I go to Insights -> Dependency Graph -> Dependabot, I can see the dependabot logs that one of the directories was a top-level directory, and so dependabot failed to unify the directories to find csproj files to consider. So, the message may be incorrect.

Hi @jzabroski I also replied in another issue but wanted to add here too in case folks are reading this thread. I'm sorry you experienced this behavior; this is a known issue; our team is actively investigating and we'll update here as we investigate.

[legoktm]

Thanks for working on this!

Multi-dir configuration is now in public beta; open to hearing your thoughts as you try out the beta!

We tried this in the freedomofpress/securedrop-client repository (dependabot.yml) but it is timing out:

Is our repository/dependency tree too big to combine like this? Or something else we can tweak? Let me know if more information is useful or if I should leave this comment elsewhere.

[carlincherry]

Is our repository/dependency tree too big to combine like this? Or something else we can tweak? Let me know if more information is useful or if I should leave this comment elsewhere.

@legoktm Possibly -- a couple thoughts:

• Maybe try a more specific grouping pattern(s) than *?
• If you're experiencing timeouts, one option to try could be larger GitHub-hosted (unfortunately, larger GH-hosted runners cost $, but you might get rid of the timeouts) or self-hosted runners (configure yourself to give more compute power to avoid the timeouts 😄). Check out these docs to see if either larger GitHub-hosted or self-hosted runners might be of interest to you!

[Nishnha]

Hi folks, this feature is now live! 🎉
https://github.blog/changelog/2024-06-25-simplified-dependabot-yml-configuration-with-multi-directory-key-directories-and-wildcard-glob-support/

If you encounter any issues with this please reach out to GitHub support.