Code Analysis rule documentation should list all editorconfig names for a rule

[daiplusplus]

Old Issue description

• The documentation for CA1xxx rules do not state what their full .editorconfig rule names are.
  • This makes it impossible to suppress or configure rules using .editorconfig given that the Error List's Suppress and Configure menus are currently broken at-present if you don't already know what the name is.
  • Also, given that some rules (especially code style rules) have both IDE0000-style names and Editorconfig-standard names this makes things even more confusing.
  • Additionally so when rules' names change between legacy FxCop, Roslyn-based, and NuGet-package-based releases.
• Additionally, the generic example given in the documentation implies that every rule is under dotnet_code_quality which is not true, but it's used for every rule's documentation. For example, CA1054 is dotnet_diagnostic.CA1054, not dotnet_code_quality.CA1054. However I couldn't find any documentation for CA1054's full name - I found it out through trial-and-error.

(Revised) Issue description

Ask 1: Please list all .editorconfig names for each rule (at the top of each page):

The documentation pages for both Code Analysis and Code Style do not clearly document all of the .editorconfig names for each rule.

Specifically, the problem I experienced is that each rule has very different names for setting a rule's options compared to setting a rule's severity or for suppressing it - which also differs from the name used for suppressing entire categories.

For example, for CA1000 (comments inline):

# If you want to disable CA1000 then you need this:
dotnet_diagnostic.CA1000.severity = none

# If you want to configure CA1000 then you need this:
dotnet_code_quality.CA1000.api_surface = private, intern

# If you want to suppress all rules in the same category then you need this:
dotnet_analyzer_diagnostic.category-Design.severity = none

For Code style it's even more inconsistent:

# If you want to disable IDE0058 then you need this:
dotnet_diagnostic.IDE0058.severity = none

# If you want to configure IDE0058 then you need this:
csharp_style_unused_value_expression_statement_preference = discard_variable

# ...but you can do this too:
csharp_style_unused_value_expression_statement_preference = discard_variable:none

# If you want to suppress all rules in the same category then you need this:
dotnet_analyzer_diagnostic.category-Style.severity = none

To clarify: I'm not complaining that the names are inconsistent (well, I am...), but I'm complaining that the documentation does not serve as a quick and easy reference to discover the other names of a rule.

So I'm asking for the info-table in every rule's documentation to show all 3 .editorconfig names (the specific rule's name, the name for when suppressing it or configuring its severity, and the name for when configuring its category's severity), as well as all available options in a succinct list.

So this:

Should be something like this:

Ask 2: Include copypastable severity syntaxes in every rule page:

Given the most likely reason for visiting Code Analysis documentation is to figure out how to suppress a rule, I'd also like to see table rows containing copypastable suppression syntaxes, in all 3 formats:

• Preprocessor directive:
  • #pragma warning disable CA1000 // Do not declare static members on generic types
• Global Suppression File:
  • [assembly: SuppressMessage("Microsoft.Design","CA1000:DoNotDeclareStaticMembersOnGenericTypes", Scope = "Type")]
• .editorconfig (single rule):
  • dotnet_diagnostic.CA1000.severity = none (or silent)
• .editorconfig (entire rule category):
  • dotnet_analyzer_diagnostic.category-Design.severity = none (or silent)

Ask 3: Document the .editorconfig rule prefixes:

I would also like to see a page document why the rule name prefixes are different, and as they are different, what they represent. I'm asking because the names are not self-describing at all:

• dotnet_diagnostic
• dotnet_analyzer_diagnostic
• dotnet_code_quality

Ask 4: The Code quality rule configuration options page should also mention severity

The Code quality rule configuration options page should also have a section for configuring a rule's severity and/or how to suppress it, as I was under the mistaken belief that setting the severity uses the same syntax as setting an OptionValue.

[Youssef1313]

I thought that setting the severity is dotnet_diagnostic.ID.severity = value for all rules (this is a roslyn thing, unrelated to the analyzer implementation).

@mavasani Is the documentation incorrect here for dotnet_code_quality.CA1000.severity?

Edit: Just noticed the second screenshot is hand-edited suggestion, not the actual documentation.

[Youssef1313]

@Jehoel I believe that setting severity is done using dotnet_diagnostic.ID.severity for everything.

Can you be more specific about some page as an example that you think gives a wrong example or is missing some information?

[daiplusplus]

@Youssef1313

I believe that setting severity is done using dotnet_diagnostic.ID.severity for everything.

Some settings have more verbose names then just an ID. For example, IDE0063 is csharp_prefer_simple_using_statement.

I appreciate the above is not setting the severity for the rule , but it's just another case of things being inconsistent without an obvious reason. I wouldn't be complaining if I could do either of these instead:

csharp_prefer_simple_using_statement = true
csharp_prefer_simple_using_statement.severity = warning

or:

dotnet_diagnostic.IDE0063.value = true
dotnet_diagnostic.IDE0063.severity = warning

As it is, I need to put this instead: (Notice how they have very different forms, a casual reader isn't going to know that they're both referring to the same rule unless someone adds a clarifying comment).

csharp_prefer_simple_using_statement = true
dotnet_analyzer_diagnostic.IDE0063.severity = warning

---

Inconsistency with suppressing categories of rules:

While writing my reply above and checking the documentation, I found out that if I want to suppress an entire category then there's more inconsistency:

• This page says that to suppress an entire category, the syntax is this:
  • dotnet_code_quality.RuleCategory.OptionName = OptionValue
  • Therefore I need to use this:
    • dotnet_code_quality.category-Style.severity = warning
• If I let Visual Studio 2019 set the severity for the rule category via the Lightbulb menu then it writes this to my .editorconfig:
  • dotnet_analyzer_diagnostic.category-Style.severity = warning
• But both have different name prefixes compared to when I configured the individual rule above, so I would have expected it was this:
  • dotnet_diagnostic.category-Style.severity = warning

[daiplusplus]

@Youssef1313

Can you be more specific about some page as an example that you think gives a wrong example or is missing some information?

All of the CAxxxx pages I've seen give examples using dotnet_code_quality when for many (if not all? or many?) should be under dotnet_diagnostic instead (or is it dotnet_analyzer_diagnostic...?)

• https://docs.microsoft.com/en-us/dotnet/fundamentals/code-analysis/quality-rules/ca1000
  • dotnet_code_quality.CAXXXX.api_surface = private, internal
• https://docs.microsoft.com/en-us/dotnet/fundamentals/code-analysis/quality-rules/ca1001
  • dotnet_code_quality.CAXXXX.excluded_symbol_names = MyType
  • dotnet_code_quality.CAXXXX.excluded_type_names_with_derived_types = MyType
• https://docs.microsoft.com/en-us/dotnet/fundamentals/code-analysis/quality-rules/ca1002
  • dotnet_code_quality.CAXXXX.api_surface = private, internal
• https://docs.microsoft.com/en-us/dotnet/fundamentals/code-analysis/quality-rules/ca1003
  • dotnet_code_quality.CAXXXX.api_surface = private, internal
• https://docs.microsoft.com/en-us/dotnet/fundamentals/code-analysis/quality-rules/ca1005
  • dotnet_code_quality.CAXXXX.api_surface = private, internal
• https://docs.microsoft.com/en-us/dotnet/fundamentals/code-analysis/quality-rules/ca1008
  • dotnet_code_quality.CAXXXX.api_surface = private, internal
• Etc

[Youssef1313]

@Jehoel None of the pages suggests using dotnet_code_quality.ID.severity.

I see one disconnect here is that you think "severity" is applicable when a page says the word "option".

i.e, when the page says dotnet_code_quality.OptionName = OptionValue, it doesn't mean dotnet_code_quality.severity.

Regarding the code style names, for example the csharp_prefer_simple_using_statement you mentioned. It's documented in the IDE0063 page. So I don't quite understand why you think the following:

As it is, I need to put this instead: (Notice how they have very different forms, a casual reader isn't going to know that they're both referring to the same rule unless someone adds a clarifying comment).

[Youssef1313]

Regarding the examples for api_surface, I believe the form mentioned in the docs is correct.

See for example this test:

https://github.com/dotnet/roslyn-analyzers/blob/0d8961a69316b92c263d2eb367f0d5fbdada5c69/src/NetAnalyzers/UnitTests/Microsoft.CodeQuality.Analyzers/Maintainability/ReviewUnusedParametersTests.cs#L833-L848

If it's not working with you, then it may be a product issue/feedback rather than a documentation issue.

[daiplusplus]

None of the pages suggests using dotnet_code_quality.ID.severity.
I see one disconnect here is that you think "severity" is applicable when a page says the word "option".

Yes, I concede you're correct.

... but there is nothing on those CA1000-1008 rules' documentation pages to suggest that severity isn't an option - or that it requires a totally different .editorconfig entry name to set severity or to suppress it - for that you need to click on more links and read-through more pages.

So I don't quite understand why you think...

Because using two completely different names to configure a rule and then to set its severity is inconsistent and completely non-obvious.

Regarding IDE0063 and api_surface, the issue isn't that those specific rules (CA1000 through CA1008) are under dotnet_code_quality; the issue is that the documentation for those rules doesn't make it clear enough that they're under dotnet_code_quality.

It's because the only mention of dotnet_code_quality is in a single example line that's exactly the same across all CAxxxx rules, the examples all have "CAXXXX" which makes me think that the entire line is a fictional or arbitrary example - which sends me on a quest through documentation to find the "real" full-name of a given CA rule.

---

Perhaps my position might be easier to understand if I share what my thought-process tonight was:

1. I want to globally suppress these CA1000 warnings in the MSBuild output - and my team's rules require the .editorconfig to be used instead of [SuppressMessage].
   • I note that due to this bug, I can't just right-click the CA1000 warning to have VS add the Supression rule for me.
2. So I google Bing search for CA1000 and I get this page: https://docs.microsoft.com/en-us/dotnet/fundamentals/code-analysis/quality-rules/ca1000
3. The "Suppress a warning" header tells me to read this page: https://docs.microsoft.com/en-us/dotnet/fundamentals/code-analysis/suppress-warnings
4. The "How to suppress code analysis warnings" page tells me to disable the rule in .editorconfig by specifying dotnet_diagnostic.<rule-ID>.severity = none
5. I did that and it didn't suppress the rule. I assumed that it didn't work because the dotnet_diagnostic prefix was just a placeholder, just like <rule-ID> was.
6. So I went digging up more documentation to find out what the first part of the name should be to set the rule severity...
7. I went back to the CA1000 page and I scrolled down the page and I saw the example giving dotnet_code_quality.CAXXXX.api_surface = private, internal
8. So I tried that and put dotnet_code_quality.CA1000.severity= none in the .editorconfig and it didn't work.
9. After some headbanging I saw references to dotnet_analyzer_diagnostic in other .editorconfig files and tried that (as dotnet_analyzer_diagnostic.CA1000.severity= none) and that didn't work either.
10. Then I gave up and posted this thread.
11. Then (just now) after your assertions that it is dotnet_diagnostic.CA1000.severity = none, I tried it again (this time doing a manual nuke-and-build (because MSBuild clean really doesn't actually clean all that much) it did work - this isn't the first time that stale build artifacts have ruined my day :(

If it's not working with you, then it may be a product issue/feedback rather than a documentation issue.

Now that I see that I was wrong - I'd like to rephrase my ask as:

• Please add the .editorconfig suppression / set-severity syntax to every code-analysis rule's documentation page, as it is different and non-obvious when compared to the non-ID-based name of a rule or style rule.
• Please document why there are different rule prefixes and what they represent:
  • dotnet_diagnostic
  • dotnet_analyzer_diagnostic
  • dotnet_code_quality

[Youssef1313]

but there's nothing to suggest that severity isn't an option

There is, but probably should be made more obvious. The rest of the article lists what options can be used. But noting that above may reduce confusion here.

---

Because using two completely different names to configure a rule and then to set its severity is inconsistent and completely non-obvious.

The main issue here is that the compiler doesn't know anything about the code_style option names. All what the compiler knows is the rule id. This is the reason you have to use the rule id to set the severity. (You can use option_name = value:severity, but it works only in IDE since the compiler doesn't know about it, i.e, it doesn't work from command-line builds).

I believe @mavasani is working on enhancing things here in dotnet/roslyn#52991.

Back to the docs, each IDExxxx page mentions the option names (if any) for this rule, it however, don't mention how to change the severity. Tagging @gewarren to consider adding a link to https://docs.microsoft.com/en-us/dotnet/fundamentals/code-analysis/suppress-warnings in all IDExxxx pages, as well as updating the same page to have some examples for IDExxxx (it's the same as CAxxxx, but just to make the reader know that the page is applicable to both IDE and CA analyzers)

---

but on my computer dotnet_diagnostic did not suppress the message, I needed to use dotnet_analyzer_diagnostic instead.

dotnet_diagnostic is the correct form to suppress a particular rule of a given ID, while dotnet_analyzer_diagnostic is the correct form to bulk-suppress either all analyzers or a given category. So if you have used dotnet_analyzer_diagnostic to suppress a specific rule with specific id, then my guess it's a product issue rather than a doc issue.

[daiplusplus]

BTW, @Youssef1313 Thank you for putting up with my opinionated documentation rants and stresses tonight (it's 1am now...) and working with me to to educate me on the right answer! :)

[Youssef1313]

Welcome @Jehoel

So to summarize things that needs to be modified in docs:

• Update https://docs.microsoft.com/dotnet/fundamentals/code-analysis/code-quality-rule-options to early say that the option should be one of the listed below.
• Add a link to https://docs.microsoft.com/dotnet/fundamentals/code-analysis/suppress-warnings in every rule doc page
• Update https://docs.microsoft.com/dotnet/fundamentals/code-analysis/suppress-warnings to mention that it's applicable to IDExxxx, CAxxxx, and third-party analyzers as well.

Is there something else you would like to be addressed @Jehoel ?

[daiplusplus]

@Youssef1313 I have rewritten my initial post so it now accurately describes the problems I experienced and how things could be made better for others in future.

---

Update https://docs.microsoft.com/dotnet/fundamentals/code-analysis/code-quality-rule-options to early say that the option should be one of the listed below.

Yes, I've written that up as the 4th item in my edited post.

---

Add a link to https://docs.microsoft.com/dotnet/fundamentals/code-analysis/suppress-warnings in every rule doc page

That link already exists. What I'm asking for is for a copypastable snippet in every rule's page instead.

So instead of this:

Suppress a warning

There are various ways to suppress a code analysis warning, including disabling the rule for the project, using a preprocessor directive to disable it for a specific line of code, or by applying the SuppressMessageAttribute attribute. For more information, see How to suppress code analysis warnings.

It should be something like this (note how there are no placeholders, every identifier and reference to the current rule is specific and correct):

Suppress a warning

Using #pragma

#pragma warning disable CA1000 // Do not declare static members on generic types

Using .editorconfig

[*.cs]
# This suppresses this specific rule
dotnet_diagnostic.CA1000.severity = none
# This suppresses this rule's entire category
dotnet_analyzer_diagnostic.category-Design.severity = none

Using [SuppressMessage]

// This suppresses the rule assembly-wide:
[assembly: SuppressMessage("Usage", "CA1000:Do not declare static members on generic types", Justification = "TODO")]
// This suppresses the rule on a single type:
public class GenericType<T>
{
   [SuppressMessage("Usage", "CA1000:Do not declare static members on generic types", Justification = "TODO")]
   public static void Foo() {}
}

For more information, see How to suppress code analysis warnings.

---

Update https://docs.microsoft.com/dotnet/fundamentals/code-analysis/suppress-warnings to mention that it's applicable to IDExxxx, CAxxxx, and third-party analyzers as well.

Yes please.

[Youssef1313]

That link already exists.

It doesn't exist in all pages, for example IDE0063. I expect this to be an issue for IDExxxx in general.