Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parse gemspec files #10616

Open
HonkingGoose opened this issue Jun 26, 2021 · 21 comments
Open

Parse gemspec files #10616

HonkingGoose opened this issue Jun 26, 2021 · 21 comments
Labels
help wanted Help is needed or welcomed on this issue manager:bundler Bundler (Ruby) package manager priority-3-medium Default priority, "should be done" but isn't prioritised ahead of others type:feature Feature (new functionality)

Comments

@HonkingGoose
Copy link
Collaborator

What would you like Renovate to be able to do?

Copy/paste of comment from @rarkins in discussion #10585.

I would definitely like to parse gemspec files, but only using JS-based parsing and therefore they may need to confirm to "conventions" rather than being completely free of convention. Let's have a feature request issue to discuss it if we don't already

Did you already have any implementation ideas?

Nope. 😄

@HonkingGoose HonkingGoose added type:feature Feature (new functionality) status:requirements Full requirements are not yet known, so implementation should not be started priority-5-triage manager:bundler Bundler (Ruby) package manager labels Jun 26, 2021
@viceice
Copy link
Member

viceice commented Jun 26, 2021

As far as i understand the docs, we can use a simple regex to parse the deps.

@HonkingGoose HonkingGoose added priority-3-medium Default priority, "should be done" but isn't prioritised ahead of others and removed priority-5-triage labels Jun 29, 2021
@marvin-bitterlich
Copy link

I encountered this same need just now and took a stab at a regexManager, its not working but maybe someone with more knowledge about these might know what is missing to get this over the line:

  "regexManagers": [
    {
      "fileMatch": ["^*.gemspec"],
      "matchStrings": [
        ".*add.*dependency '(?<depName>.*?)','(?<currentValue>.*?)'"
      ],
      "datasourceTemplate": "rubygems"
    }
  ]

@rarkins rarkins added the auto:reproduction A minimal reproduction is necessary to proceed label Aug 20, 2021
@github-actions
Copy link
Contributor

Hi there,

Help us by making a minimal reproduction repository.

Before we can start work on your issue we first need to know exactly what's causing the current behavior. A minimal reproduction helps us with this.

To get started, please read our guide on creating a minimal reproduction to understand what is needed.

We may close the issue if you (or someone else) have not provided a minimal reproduction within two weeks. If you need more time, or are stuck, please ask for help or more time in a comment.

Good luck,

The Renovate team

@marvin-bitterlich
Copy link

The discussion already references examples like https://github.com/mdub/config_hound/blob/master/config_hound.gemspec
If that helps :)

@marvin-bitterlich
Copy link

Gem::Specification.new do |spec|
  spec.name          = "renovate-gemspec-example"
  spec.version       = '1.0.0'
  spec.authors       = ["reggaemuffin"]
  spec.email         = ["reggaemuffin@xxx.com"]

  spec.summary       = %q{A minimal example for a gemspec file to parse}
  spec.description   = %q{A minimal example for a gemspec file to parse}
  spec.homepage      = "https://github.com/renovatebot/renovate/issues/10616"
  spec.license       = "MIT"

  spec.files         = `git ls-files -z`.split("\x0").reject do |f|
    f.match(%r{^(spec|images)/})
  end
  spec.bindir        = "exe"
  spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
  spec.require_paths = ["lib"]

  spec.required_ruby_version = '>= 2.1.0' # keyword args

  spec.add_runtime_dependency 'apollo-federation', '~> 1.0'
  spec.add_runtime_dependency 'apollo-tracing', '~> 1.6'
  spec.add_runtime_dependency 'activesupport', '~> 6.0', '>= 5.0'
  spec.add_dependency 'batch-loader', '~> 1.5'
  spec.add_dependency 'graphql', '~> 1.9.19'
  spec.add_dependency 'vault', '0.12.0'
  spec.add_development_dependency 'bundler', '~> 2.1', '>= 2.1.4'
  spec.add_development_dependency 'rspec', '~> 3.9', '>= 3.9.0'
  spec.add_development_dependency 'pry', '~> 0.13'
  spec.add_development_dependency "rake", "~> 13.0"
  spec.add_development_dependency "benchmark-ips", "~> 2.7"
  spec.add_development_dependency "ruby-prof", "~> 0.16"
end

This would be an example with some random diverse esample of dependencies from a few public repositories, I tried to get a few semi edge cases in there that can of course be removed too

@marvin-bitterlich
Copy link

image

It seems that ^.*add.*dependency.*['"](?<depName>.*?)['"],.*['"](?<currentValue>.*?)['"].* might work as a generic enough regex for this example

@marvin-bitterlich
Copy link

Tested this locally and it does not register in the logs of renovate except in the config

So far unclear:

  • Is datasource rubygems or bundler or something else, and should it be datasource or the template
  • Is semver supporting the ~> versions or should that be excluded
  • Dealing with multiple possible versions could easily be supported but right now it just takes the first one which should be good enough for now
  • Are there any other good practices that https://docs.renovatebot.com/modules/manager/regex/ is not telling me?

@rarkins
Copy link
Collaborator

rarkins commented Aug 20, 2021

So for a feature request like this, our goal would be for a gemspec which is both minimal as well as covering all the use cases. There's no point in having 20 entries all with the same syntax, for example - and it slows things down by making debugging harder to do. We can move this to reproduction:provided and status:ready once someone can put such a thing into a public repo which an be used as part of development

@rarkins
Copy link
Collaborator

rarkins commented Aug 20, 2021

  • Is datasource rubygems or bundler or something else, and should it be datasource or the template

You can see the full list here: https://docs.renovatebot.com/modules/datasource/

(the answer is rubygems)

  • Is semver supporting the ~> versions or should that be excluded

See https://docs.renovatebot.com/modules/versioning/#ruby-versioning

There is no one "semver" because semver itself defines no ranges at all, while most package managers have different syntax to each other for ranges.

@marvin-bitterlich
Copy link

marvin-bitterlich commented Aug 20, 2021

I created https://github.com/reggaemuffin/renovate-gemspec-minimal-example and will try to update it with the ruby version strategy to see if that works

@marvin-bitterlich
Copy link

You can see a run of it as https://app.renovatebot.com/dashboard#github/reggaemuffin/renovate-gemspec-minimal-example/432999060

image

Not sure how to extract more information out of this but let me know if you have any ideas

@rarkins rarkins added reproduction:provided and removed auto:reproduction A minimal reproduction is necessary to proceed status:requirements Full requirements are not yet known, so implementation should not be started labels Aug 20, 2021
@mattwynne
Copy link

We'd also like something like this. We're currently manually upgrading gemspecs using a bash script.

@narwold
Copy link
Contributor

narwold commented Aug 3, 2022

Any progress on this per chance?

@mattwynne
Copy link

FWIW @narwold we switched to DependaBot which supports this out of the box.

@mpkorstanje
Copy link
Contributor

mpkorstanje commented Dec 24, 2022

@marvin-bitterlich at a glance it seems your attempt at using the regex matcher won't work because you start the regex with ^.

The regex manager matches are done per-file and not per-line, you should be aware when using the ^ and/or $ regex assertions.
https://docs.renovatebot.com/modules/manager/regex/

@mpkorstanje
Copy link
Contributor

mpkorstanje commented Dec 24, 2022

I've created a working version of the regex manager here:

https://github.com/mpkorstanje/renovate-gemspec-minimal-example

{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "extends": [
    "config:base",
  ],
  "regexManagers": [
    {
      "fileMatch": [
        "^*.gemspec"
      ],
      "matchStrings": [
        ".*\\.add_(:?(?<depType>.*?)_)?dependency\\s*(['\"])(?<depName>[^'\"]+)(['\"])(\\s*,\\s*(?<currentValue>(['\"])[^'\"]+['\"](\\s*,\\s*['\"][^'\"]+['\"])?))?"
      ],
      "datasourceTemplate": "rubygems",
      "versioningTemplate": "ruby"
    }
  ]
}

@AlexWayfer
Copy link

regexManagers is deprecated, as I see, so I've tried to actualize it:

"customManagers": [
	{
		"customType": "regex",
		"fileMatch": ["^*.gemspec"],
		"matchStrings": [
			".*\\.add_(:?(?<depType>.*?)_)?dependency\\s*(['\"])(?<depName>[^'\"]+)(['\"])(\\s*,\\s*(?<currentValue>(['\"])[^'\"]+['\"](\\s*,\\s*['\"][^'\"]+['\"])?))?"
		],
		"datasourceTemplate": "rubygems",
		"versioningTemplate": "ruby"
	}
],

But now I have this error in dashboard:

image

rangeStrategy is disallowed in custom managers,replace value in root gives the same error, so now we're in a dead-end.

AlexWayfer added a commit to AlexWayfer/benchmark_toys that referenced this issue Nov 2, 2023
@mpkorstanje
Copy link
Contributor

mpkorstanje commented Mar 22, 2024

Yup. Fairly easy to reproduce. Updated https://github.com/mpkorstanje/renovate-gemspec-minimal-example with the customManagers.

It appears that this code here:

export function getRangeStrategy(config: RangeConfig): RangeStrategy | null {
const { manager, rangeStrategy } = config;
if (!manager || !managers.has(manager)) {
return null;
}

Is checking against the managerList rather than the allManagersList:

const managerList = Array.from(managers.keys()); // does not include custom managers
export const getManagerList = (): string[] => managerList;
export const getManagers = (): Map<string, ManagerApi> => managers;
export const allManagersList = [...managerList, ...customManagerList];

Started a new issue here #28090

@D1X7R4
Copy link

D1X7R4 commented Mar 28, 2024

Hi!

Is there any chance of gemspec support soon?

On our particular case we defined gemspec in our Gemfile. Where the gemspec file specifies the dependencies required by the gem and the Gemfile the required ones for testing and development.

In this case, the manager should update ranges in the gemspec but also the Gemfile.lock definition.

References:

@rarkins rarkins added the help wanted Help is needed or welcomed on this issue label Mar 28, 2024
@ThomasKoppensteiner
Copy link

How can we move on with this issue?
What is the blocker for parsing and updating dependencies in gemspec files?

@mpkorstanje
Copy link
Contributor

mpkorstanje commented Dec 19, 2024

Currently someone has to do the work. Either to get the custom manager working or to provide a fully featured manager for gemspec files.

I made an analysis of the problems with the current custom manager in #28090 and I think they should be relatively easy to solve. But I've not yet found the space for it.

Feel free to help out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Help is needed or welcomed on this issue manager:bundler Bundler (Ruby) package manager priority-3-medium Default priority, "should be done" but isn't prioritised ahead of others type:feature Feature (new functionality)
Projects
None yet
Development

No branches or pull requests

10 participants