forbidigo: better support for configuring complex rules #3612

pohly · 2023-02-18T13:20:29Z

forbidigo 1.4.0 introduced a new configuration mechanism with multiple fields per rule. It still supports a single string, but allowing structs as alternative to such strings makes the configuration easier to read and write.

A new global flag is needed to enable the more advanced analysis.

.golangci.reference.yml

ashanbrown · 2023-02-18T21:51:48Z

pkg/golinters/forbidigo.go

@@ -40,14 +41,20 @@ func NewForbidigo(settings *config.ForbidigoSettings) *goanalysis.Linter {
 		},
 	}

+	loadMode := goanalysis.LoadModeSyntax
+	if settings != nil && settings.ExpandExpressions {


I believe we also need to set the analyzer flag called analyze_types when we see this set.

Which analyzer flag?

goanalysis uses different "levels", not individual flags: https://pkg.go.dev/github.com/golangci/golangci-lint/pkg/golinters/goanalysis#LoadMode

I was thinking we were using forbidigo as an analyzer, per the new guidelines for linters. For an analyzer, we'd set the analyze_types flag at https://github.com/ashanbrown/forbidigo/blob/master/pkg/analyzer/analyzer.go#L55. That said, it looks like we've set up forbidigo to automatically expand types when TypesInfo is provided. I think I hadn't quite realized we were using that as a flag, rather than just information. In retrospect, my preference would have been to explicitly add a new analyzeTypes option at https://github.com/ashanbrown/forbidigo/blob/master/forbidigo/forbidigo.go#L65 (similar for what we did for the analyzer mode).

By using types, your linter is now considered a slow linter.
#3639

@ldez That makes sense, and we had anticipated this issue, although I didn't realize that it was formally checked in golangci-lint. The linter can work without types, but there is an opt-in behavior that allows us to use types to match more things. Is there any precedent for a linter being both fast and slow? I don't suppose we could have both fast and slow versions of the same linter? I'd imagine they would have to have separate names and configurations.

Is type analysis triggered by the line at https://github.com/golangci/golangci-lint/pull/3612/files#diff-b647b23fb0e6073b7f161d88a8cd16af6dd3e90de1d6a81006d2d9a4ee416707R424? In this PR, it appears that setting is conditionally executed based on the configuration.

The goal was to not change behavior unless the user explicitly asks for it in the config. It looks like there are two places where that is relevant:

GetAllSupportedLinterConfigs: covered by this PR

NewForbidigo: covered by this PR

The already merged commit unconditionally reconfigures forbidigo to enable type analysis. There's a certain risk that this breaks users because existing patterns get interpreted differently. I missed earlier that the PR wasn't just the automatic bump of the forbidigo package.

I think we should merge this PR here soon because it is required to finish the forbidigo update to 1.5.x properly.

Wait, I'm not keeping up with you guys 😢

So forbidigo 1.5.x is new, and the upgrade of it in #3639 is something that I really only saw today because it's also new.

That PR makes forbidigo a "slow" linter, without actually enabling the feature which uses the additional information (no call of OptionAnalyzeTypes(true).

So my statement above still stands: this PR is needed to finish the forbidigo update. I just need to add OptionAnalyzeTypes(true).

I've added it.

ashanbrown · 2023-02-25T14:08:58Z

FYI, I've bumped forbidigo to v1.5.0 which now requires a new option called OptionAnalyzeTypes to be set to enable analyzing types. I'm also looking at integrating one more change to remove pkg/errors, so it might not hurt to upgrade all that together. That will be version v1.5.1.

ashanbrown · 2023-02-27T05:24:14Z

.golangci.reference.yml

-      # Optionally put comments at the end of the regex, surrounded by `(# )?`
-      # Escape any special characters.
+      # Optional message that gets included in error reports.
+      - pattern: ^fmt\.Print.*$


I think we ended up calling this "p" at https://github.com/golangci/golangci-lint/pull/3612/files#diff-cb1fbdbfccd3fd0d8fe9d9006138fbcd1afcc3a701b37180c6fe50ed0c5c390fR332.

True, and I did the same for ForbidigoPattern in linter_settings.go. I've fixed the reference file to match.

It's worth calling out that we can use pattern here if p is considered too brief: we can map that in linter_settings.go:

// ForbidigoPattern corresponds to forbidigo.pattern and adds // mapstructure support. The YAML field names must match what // forbidigo expects. type ForbidigoPattern struct { // patternString gets populated when the config contains a string as // entry in ForbidigoSettings.Forbid[] because ForbidigoPattern // implements encoding.TextUnmarshaler and the reader uses the // mapstructure.TextUnmarshallerHookFunc as decoder hook. // // If the entry is a map, then the other fields are set as usual // by mapstructure. patternString string Pattern string `yaml:"p" mapstructure:"pattern"` // <===== Package string `yaml:"pkg,omitempty" mapstructure:"pkg,omitempty"` Msg string `yaml:"msg,omitempty" mapstructure:"msg,omitempty"` }

I myself think golangci-lint should follow the choice made for upstream forbidigo and use p, too.

ashanbrown · 2023-02-27T05:26:05Z

pkg/lint/lintersdb/manager.go

+				WithURL("https://github.com/ashanbrown/forbidigo")
+			if forbidigoCfg != nil && forbidigoCfg.AnalyzeTypes {
+				// Need more information for the analysis.
+				l = l.WithLoadForGoAnalysis()


@ldez As @pohly mentioned, we are making this call (which sets IsSlow) conditional on the configuration. Are you ok with having this otherwise "fast" linter become slow if this flag is set or would that be considered too unexpected?

GetAllSupportedLinterConfigs is called in different situations, the linter config will not be available in all those situations, so please remove this and don't try to create an asymetrical configuration between WithLoadForGoAnalysis and LoadModeTypesInfo.

I see how not setting IsSlow in GetAllSupportedLinterConfigs when called without config is wrong (or at least potentially misleading).

But is it also useful in NewForbidigo to use LoadModeTypesInfo whether it's needed or not? Can something break when combining WithLoadForGoAnalysis with LoadModeSyntax?

because it's not related to isSlow but to loadMode.
if the 2 load modes are not synced, some unexpected behavior will appear.

Okay. I've pushed an update which always uses the more expensive mode. In the long run I'd expect users to switch to it anyway. It's just not the default because of some risk of breaking certain patterns.

Back to the main point of this discussion: @ldez, you had asked that we not be asymmetric in our use of WithLoadForGoAnalysis in one place and not the other. Could we simply report that we need the requirements of a fast linter except when this "experimental" setting is set in the config file? When we see the experimental config, we'd add the extra load requirements. I realize this is still asymmetric, although at least it would no longer call WithLoadForGoAnalysis. Is the concern that by reporting a linter as fast, say in a help message, and later making it slow, we would be confusing the user, or is there something that would actually break?

go test -v -run TestSourcesFromTestdata/forbidigo ./test doesn't pass when GetAllSupportedLinterConfigs doesn't use WithLoadForGoAnalysis for forbidigo, so it has technical implications.

If we absolutely need a "fast forbidigo" (not convinced yet myself, but usage patterns are different and I don't want to make forbidigo less useful for its original author!) and a "throrough forbidigo" (a strong yes from me for Kubernetes, because import aliases make the current source-code based rules unreliable), then we could register forbidigo twice:

Once with the current configuration and mode of operation (no changes).

Again with a different name (working title "forbidigo-analyze-types", better suggestions welcome) and the new configuration without the AnalyzeTypes field.

I'm not sure it's LoadModeTypesInfo that is causing us to be considered a slow linter. For example, durationCheck uses this load mode and is still considered fast. I think it's the requirement to load dependencies that would make this a slow linter. WithLoadForGoAnalysis does this:

lc.LoadMode |= packages.NeedImports | packages.NeedDeps | packages.NeedExportFile | packages.NeedTypesSizes

This was added at #1844. Perhaps @ldez can fill us on as to why we can't just set IsSlow based on the presence of select LoadMode flags.

I'd personally be happy with the multiple-linter configs but that had sounded like a non-starter to me.

goanalysis.LoadModeTypesInfo required WithLoadForGoAnalysis().

A linter with two "speed states" doesn't seem right.

"slow" or "fast" don't really describe the speed of a linter, it's just a best-effort state.
The "slow" linters share a state for the slow part, so if you run only one linter you will pay the cost but if your run several linters the cost will be shared.

Duplicating the linter doesn't seem like the right idea too.

I think it's not really important if your linter is flagged as "slow" or "fast".
But if it's important to you, you have to make choice inside the linter: use types or not.

I don't want to create unneeded complexity inside the configuration, I also prefer maintainability over possibilities.

Would it work to register it with WithLoadForGoAnalysis unconditionally and then decide in NewForbidigo based on the actual setting? At least the tests are passing that way.

Would it make the linter run faster? I'm not so sure.

For Kubernetes, golangci-lint run --disable-all --enable forbidigo -v with that approach uses LoadModeSyntax:

INFO Execution took 8.982228038s

That's after cleaning the golangci-lint cache, but with a warm Go build cache.

Without this hack it's 9.556500684s - a bit slower, but not much.

With an empty Go cache (go clean --cache), the difference is 56.58711488s (with that approach) vs. 57.399810283s (without).

For comparison, with forbidigo master (985efcc) it's 9.639382845s and 57.23967985s.

These numbers are so close together, I don't see a relevant difference. It probably comes down to noise - otherwise master would have to be the fastest.

@ashanbrown: can you give a specific example where you want to use "fast" analysis? Does this PR really make a significant difference?

ldez · 2023-02-27T08:10:40Z

pkg/lint/lintersdb/manager.go

+				WithURL("https://github.com/ashanbrown/forbidigo")
+			if forbidigoCfg != nil && forbidigoCfg.AnalyzeTypes {
+				// Need more information for the analysis.
+				l = l.WithLoadForGoAnalysis()


GetAllSupportedLinterConfigs is called in different situations, the linter config will not be available in all those situations, so please remove this and don't try to create an asymetrical configuration between WithLoadForGoAnalysis and LoadModeTypesInfo.

pohly · 2023-03-01T07:59:49Z

My latest push triggered some linter warnings. I've fixed those:

diff --git a/pkg/golinters/forbidigo.go b/pkg/golinters/forbidigo.go
index cad20b4e..b46c8698 100644
--- a/pkg/golinters/forbidigo.go
+++ b/pkg/golinters/forbidigo.go
@@ -16,6 +16,7 @@ import (
 
 const forbidigoName = "forbidigo"
 
+//nolint:dupl
 func NewForbidigo(settings *config.ForbidigoSettings) *goanalysis.Linter {
        var mu sync.Mutex
        var resIssues []goanalysis.Issue
@@ -59,9 +60,7 @@ func runForbidigo(pass *analysis.Pass, settings *config.ForbidigoSettings) ([]go
                forbidigo.OptionExcludeGodocExamples(settings.ExcludeGodocExamples),
                // disable "//permit" directives so only "//nolint" directives matters within golangci-lint
                forbidigo.OptionIgnorePermitDirectives(true),
-       }
-       if settings != nil && settings.AnalyzeTypes {
-               options = append(options, forbidigo.OptionAnalyzeTypes(true))
+               forbidigo.OptionAnalyzeTypes(settings.AnalyzeTypes),
        }
 
        // Convert patterns back to strings because that is what NewLinter

It looks like runForbidigo will never be called with a nil pointer (existing code already relied on that), so I removed the if check.

pkg/lint/lintersdb/manager.go

sbrownjc · 2023-04-25T15:38:42Z

Looking forward to this getting merged! Once it is, please update the Schema stored here: https://github.com/SchemaStore/schemastore/blob/e321c0e165562a845e3d05b28f4ce2054bc11d7e/src/schemas/json/golangci-lint.json#L911-L913

ashanbrown · 2023-04-29T23:37:13Z

@pohly Could you let us know where this PR is at? I'm still not excited about this never being a fast linter (although I think that change went in inadvertently with the dependency bump), but I figure it's better that we push it out and get feedback, given that there seems to be interest in the new features. Thanks.

pohly · 2023-05-01T08:42:44Z

@ashanbrown: I think @ldez is just waiting for your okay to merge it as it is now. You were the one who had concerns and it wouldn't be fair to just ignore that.

The discussion in #3612 (comment) stopped with me asking about some specific example where I can try out myself what the actual difference is. If you feel that this is not necessary, then we can also just go ahead and merge.

ashanbrown · 2023-05-01T15:31:03Z

Got it @pohly. I can approve this. As I mentioned, I don't think this causes any performance regression vs the previous release.

I don't quite understand what your benchmarks you showed are trying to cover. Once we call WithLoadForGoAnalysis then we will load all the type information for all the dependencies. I'm pretty certain this will significantly degrade performance for pre-commit checks, where for speed we'd like to analysis only files that have changed and not the entire project.

pkg/config/linters_settings.go

forbidigo 1.4.0 introduced a new configuration mechanism with multiple fields per rule. It still supports a single string, but allowing structs as alternative to such strings makes the configuration easier to read and write. A new global flag is needed to enable the more advanced analysis.

ldez

LGTM

pohly mentioned this pull request Feb 18, 2023

.golangci.reference.yml validation #3613

Closed

ldez added linter: update Update the linter implementation inside golangci-lint enhancement New feature or improvement area: config Related to .golangci.yml and/or cli options labels Feb 18, 2023

ashanbrown reviewed Feb 18, 2023

View reviewed changes

.golangci.reference.yml Outdated Show resolved Hide resolved

ashanbrown reviewed Feb 18, 2023

View reviewed changes

pohly force-pushed the forbidigo-update branch from 208ecbc to bd4c77e Compare February 21, 2023 09:44

ldez self-requested a review February 21, 2023 10:54

pohly mentioned this pull request Feb 26, 2023

build(deps): bump github.com/ashanbrown/forbidigo from 1.4.0 to 1.5.1 #3639

Merged

pohly force-pushed the forbidigo-update branch from bd4c77e to aa112f6 Compare February 26, 2023 19:54

ashanbrown reviewed Feb 27, 2023

View reviewed changes

pohly force-pushed the forbidigo-update branch from aa112f6 to 8a2f7e7 Compare February 27, 2023 08:04

ldez requested changes Feb 27, 2023

View reviewed changes

pohly force-pushed the forbidigo-update branch 3 times, most recently from 176e87c to a5cb733 Compare March 1, 2023 07:58

pohly mentioned this pull request Mar 1, 2023

writing-good-e2e-tests.md: polling, contexts and failure messages kubernetes/community#7021

Merged

alexandear reviewed Mar 2, 2023

View reviewed changes

pkg/lint/lintersdb/manager.go Outdated Show resolved Hide resolved

pohly force-pushed the forbidigo-update branch from a5cb733 to f20d983 Compare March 3, 2023 07:46

ashanbrown mentioned this pull request Apr 9, 2023

How to write method checks under third-party packages ashanbrown/forbidigo#31

Closed

sbrownjc mentioned this pull request Apr 25, 2023

Add analyze_types config option for forbidigo #3802

Closed

4 tasks

ashanbrown approved these changes May 1, 2023

View reviewed changes

alexandear reviewed May 3, 2023

View reviewed changes

pkg/config/linters_settings.go Outdated Show resolved Hide resolved

pohly force-pushed the forbidigo-update branch from dd7aab8 to 7176f2d Compare May 3, 2023 12:24

pohly mentioned this pull request May 22, 2023

hack/verify-test-code.sh: replace with proper linter kubernetes/kubernetes#115234

Open

6 tasks

ldez force-pushed the forbidigo-update branch from 6843516 to f507714 Compare May 31, 2023 13:34

review: fix dependencies and cosmetic changes

ed7f938

ldez force-pushed the forbidigo-update branch from f507714 to ed7f938 Compare May 31, 2023 13:35

ldez approved these changes May 31, 2023

View reviewed changes

ldez merged commit afd0ba5 into golangci:master May 31, 2023

ldez added this to the v1.53 milestone Mar 6, 2024

alexandear mentioned this pull request Mar 15, 2024

docs fix broken links alexandear/golangci-lint#1

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

forbidigo: better support for configuring complex rules #3612

forbidigo: better support for configuring complex rules #3612

pohly commented Feb 18, 2023

ashanbrown Feb 18, 2023

pohly Feb 21, 2023

ashanbrown Feb 25, 2023

ldez Feb 26, 2023

ashanbrown Feb 26, 2023

ashanbrown Feb 26, 2023 •

edited

Loading

pohly Feb 26, 2023

pohly Feb 26, 2023

pohly Feb 26, 2023

pohly Feb 26, 2023

ashanbrown commented Feb 25, 2023

ashanbrown Feb 27, 2023

pohly Feb 27, 2023

ashanbrown Feb 27, 2023

ldez Feb 27, 2023

pohly Feb 28, 2023

ldez Feb 28, 2023 •

edited

Loading

pohly Feb 28, 2023

ashanbrown Mar 4, 2023

pohly Mar 5, 2023

ashanbrown Mar 5, 2023

ldez Mar 5, 2023

pohly Mar 6, 2023

ldez Feb 27, 2023

pohly commented Mar 1, 2023

sbrownjc commented Apr 25, 2023

ashanbrown commented Apr 29, 2023

pohly commented May 1, 2023

ashanbrown commented May 1, 2023

ldez left a comment

forbidigo: better support for configuring complex rules #3612

forbidigo: better support for configuring complex rules #3612

Conversation

pohly commented Feb 18, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ashanbrown Feb 26, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ashanbrown commented Feb 25, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ldez Feb 28, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pohly commented Mar 1, 2023

sbrownjc commented Apr 25, 2023

ashanbrown commented Apr 29, 2023

pohly commented May 1, 2023

ashanbrown commented May 1, 2023

ldez left a comment

Choose a reason for hiding this comment

ashanbrown Feb 26, 2023 •

edited

Loading

ldez Feb 28, 2023 •

edited

Loading