Scoped nowarn #18049

Martin521 · 2024-11-22T08:58:41Z

Description

Implements Scoped Nowarn according to draft RFC FS-1146.

This PR has taken a while. I had to deal with much more complexity than I imagined when I naively volunteered to tackle the feature request. Anyway, here we are.

I have split the PR into 7 commits that can be reviewed in sequence.
All of them compile, 1 and 4 - 7 also pass all tests locally.

Add the feature flag, baseline tests, and the core WarnScopes module. See src/Compiler/SyntaxTree/WarnScopes.fsi and the RFC for the functionality of the module.
Add the necessary changes to lexing and parsing. Note that the warn directives can no longer be collected during parsing (since they can now appear not only in top-level modules, but anywhere). So we collect them during lexing, similar to the processing of #if/#else/#endif directives.
Remove legacy #nowarn processing (but hold off AST changes)
Integrate the WarnScopes functionality and test it
Add warn directive trivia (but hold off AST changes)
Enable warn directive trivia (which means AST changes)
Remove defunct types and parameters related to former #nowarn processing (more AST changes)

There is also a separate commit for the IlVerify baseline updates (change in line numbers only)

Checklist

Test cases added
Performance benchmarks added in case of performance changes
Release notes entry updated
Create documentation update PRs (see RFC)

github-actions · 2024-11-22T08:59:24Z

❗ Release notes required

✅ Found changes and release notes in following paths:

Change path Release notes path Description

src/Compiler docs/release-notes/.FSharp.Compiler.Service/9.0.200.md

LanguageFeatures.fsi docs/release-notes/.Language/preview.md

psfinaki · 2024-11-29T11:39:48Z

Hi @Martin521 - thanks for the contribution. It's a substantial effort and we appreciate it. The PR is on our radar - just keep in mind that it's big and specific, and it will take time to find capacity for it.

If anyone from the community gets to thoroughly review it, that would be valuable as well.

Thanks for your diligence and patience :)

KevinRansom · 2024-12-12T21:30:56Z

src/Compiler/Driver/CompilerConfig.fs

@@ -877,6 +878,14 @@ type TcConfigBuilder =
            compilationMode = TcGlobals.CompilationMode.Unset
        }

+    member tcConfigB.SetLangVersion v =


Adding a mutator for langVersion is fine but mostly unnecessary, but please don't let the langVersion mutator set other values in TcConfig that is not the paradigm we use. Yes SetPrimaryAssembly and SetUseSdkRefs also set tcConfigB.fxResolver, they probably shouldn't, actually I think I may fix them so they don't.

TcConfigBuilder is a mutable record and so modifying values is kind of what it's about, after building this is passed to the constructor of TcConfig which then creates the immutable TcConfig that is used to access these values. Perhaps the best thing is to mutate fsharpDiagnosticOptions with the language version when constructing the TcConfig object. Then you would plumb the langversion to the places where FSharpDiagnosticOptions.Default is used and add a langVersion argument to FSharpDiagnosticOptions.Default which would enable the error and warning defaults to change on a per language version basis which is probably usefull longer term. Unfortunately plumbing data about is never fun.

When tcConfigB.diagnosticsOptions is set to FSharpDiagnosticOptions.Default, the langversion is not yet known. The langversion comes only much later with ParseCompilerOpions.
There are two alternatives that I considered but didn't like.
The first was to add to tcConfigB an updated FSharpDiagnosticOptions (updated with the langversion) in all the places where ParseCompilerOptions is called (in fsc.fs, fsi.fs, BackgroundCompiler.fs, TransparentCompiler.fs and FSharpCheckerResults.fs).
The second was to thread the whole tcConfig through all the function calls to the places where I need only the diagnosticsOptions.
But let me check if I find still another way.

KevinRansom

I still have a long way to go, this is an interesting feature.

KevinRansom · 2024-12-13T00:39:29Z

src/Compiler/Facilities/DiagnosticOptions.fs

@@ -20,6 +20,8 @@ type FSharpDiagnosticOptions =
        WarnOn: int list
        WarnAsError: int list
        WarnAsWarn: int list
+        WarnScopesFeatureIsSupported: bool
+        mutable WarnScopeData: obj option


DiagnosticOptions should remain immutable. Perhaps collect this data separately and pass it to where it needs to go.

Yes, I did not really like to add a mutable field, but here are my "excuses".

I believe that the information about the warn scopes belongs into FSharpDiagnosticsOptions as much as all the other fields. All these fields are used only in one single place in the whole compiler, namely in PhasedDiagnostics.AddSeverity. Passing the information separately does not fit its purpose and would add an additional parameter to many functions.

This information needs to be updated during postparse for every file. This is in the nature of the data: the #nowarn/#warnon directives change the way warnings are emitted.

I modeled this in some way after the BufferLocalStore of lexbuf. Just a way to store data for a certain aspect for the current compiler phase.

WarnScopeData is accessed only via private functions WarnScopes.getWarnScopeData and WarnScopes.setWarnScopeData in the WarnScopes module.

I will look into alternatives and see if I find something acceptable.
(For fsc, we could probably go back to tcConfigB and update the diagnosticsOptions there. But this may not work well for the other use cases. And it would not be safer or nicer or more idiosyncratic than the mutable field.)

KevinRansom · 2024-12-13T01:32:56Z

src/Compiler/SyntaxTree/WarnScopes.fs

+    /// a list of mapped sections (surrogate and original start lines).
+    type private LineMaps = Map<FileIndex, FileIndex * (LineNumber * LineNumber) list>
+
+    type private TempData =


Name is a bit generic.

I agree. I had actually made a note about it already :-)
I will change it.

I renamed it.

KevinRansom · 2024-12-13T01:57:43Z

src/Compiler/SyntaxTree/WarnScopes.fs

+        ns |> removeQuotes |> Option.bind removePrefix |> Option.bind parseInt
+
+    let private regex =
+        Regex(""" *#(nowarn|warnon|\S+)(?: +([^ \r\n/;]+))*(?:;;)? *(\/\/.*)?$""", RegexOptions.CultureInvariant)


I believe this regex is slightly incorrect:

for better or worse the F# syntax for directives is

[optional spaces]#[optional spaces]directive[space][arguments]

so this code will turn off error 44

# nowarn 44

For example paste this code into dotnet fsi

open System [<Obsolete("This function is obsolete. Use newFunction instead.")>] let oldFunction x = printfn "This is the old function with value %d" x let newFunction x = printfn "This is the new function with value %d" x oldFunction 10;; # nowarn 44;; oldFunction 30;;

Observe:

Oh and perhaps a description of the syntax the regex implements.

I would consider the fact that the compiler accepts spaces after the hash sign a bug.
But it can certainly be replicated in the new version (it has to happen here and in the lex grammar then).
I guess this should become another RFC discussion.

KevinRansom · 2024-12-13T02:04:08Z

.../FSharp.Compiler.Service.Tests/FSharp.Compiler.Service.SurfaceArea.netstandard20.release.bsl

@@ -2236,9 +2236,6 @@ FSharp.Compiler.CodeAnalysis.FSharpParsingOptions: FSharp.Compiler.CodeAnalysis.
 FSharp.Compiler.CodeAnalysis.FSharpParsingOptions: FSharp.Compiler.CodeAnalysis.FSharpParsingOptions get_Default()
 FSharp.Compiler.CodeAnalysis.FSharpParsingOptions: FSharp.Compiler.Diagnostics.FSharpDiagnosticOptions DiagnosticOptions
 FSharp.Compiler.CodeAnalysis.FSharpParsingOptions: FSharp.Compiler.Diagnostics.FSharpDiagnosticOptions get_DiagnosticOptions()
-FSharp.Compiler.CodeAnalysis.FSharpParsingOptions: Int32 CompareTo(FSharp.Compiler.CodeAnalysis.FSharpParsingOptions)


Do we understand why these public Apis were removed? is it likely to impact existing projects?

These methods are no longer auto-generated because FSharpDiagnosticsOptions has now a obj member and thus is no longer comparable.

KevinRansom · 2024-12-13T02:04:53Z

.../FSharp.Compiler.Service.Tests/FSharp.Compiler.Service.SurfaceArea.netstandard20.release.bsl

 FSharp.Compiler.Diagnostics.FSharpDiagnosticOptions: System.String ToString()
-FSharp.Compiler.Diagnostics.FSharpDiagnosticOptions: Void .ctor(Int32, Boolean, Microsoft.FSharp.Collections.FSharpList`1[System.Int32], Microsoft.FSharp.Collections.FSharpList`1[System.Int32], Microsoft.FSharp.Collections.FSharpList`1[System.Int32], Microsoft.FSharp.Collections.FSharpList`1[System.Int32])


Do we understand why these public Apis were removed? is it likely to impact existing projects?

These methods are no longer auto-generated because FSharpDiagnosticsOptions has now a obj member and thus is no longer comparable.

KevinRansom · 2024-12-13T02:14:45Z

tests/fsharp/core/load-script/ProjectDriver.fsx

@@ -1,9 +1,10 @@
 // #Conformance #FSI 
 #load "ThisProject.fsx"
+#nowarn "44"


This test case change looks very specific.

It appears to verify that #nowarn in an imported source file named "ThisProject.fsx" will impact the currently compiling sourcefile

I have just tested with the released code and the #nowarn does indeed flow.

By all means add a new test case to ensure that #nowarn "44" works, although probably there are plenty. But this test case needs to pass unmodified. Or this feature introduces a breaking change, and if it does, we will have to discuss that.

This is implementing the RFC, so it should be discussed over there.
See also this comment.

KevinRansom · 2024-12-13T02:28:42Z

tests/service/data/SyntaxTree/Attribute/RangeOfAttribute.fs.bsl

@@ -30,4 +30,5 @@ ImplFile
           Expr (Do (Const (Unit, (3,3--3,5)), (3,0--3,5)), (3,0--3,5))],
          PreXmlDocEmpty, [], None, (2,0--4,0), { LeadingKeyword = None })],
      (true, true), { ConditionalDirectives = []
+                      WarnDirectives = []


It would be nice if we didn't have this right now, it is very hard to use github web interface with 900 baseline files that have just this WarnDirectives = []. Perhaps remove the WarnDirectives addition to the syntax tree right now so that what we have is reviewable? We can add it back when we are confident the core changes are acceptable?

That's why I spent effort to create the successive commits. Almost all of the baseline changes are introduced only in commit 6. (See the PR description above)

KevinRansom · 2024-12-13T02:39:48Z

tests/FSharp.Compiler.ComponentTests/CompilerDirectives/NonStringArgs.fs

-        |> shouldFail
-        |> withDiagnostics [
-            if languageVersion = "8.0" then
-                (Error 3350, Line 4, Col 5, Line 4, Col 7, "Feature '# directives with non-quoted string arguments' is not available in F# 8.0. Please use language version 9.0 or greater.")


I assume that testing # nowarn with each argument on a separate line is tested elsewhere can you point me to the test cases that make this test redundent

Thanks

This is implementing the RFC, which assumes that compiler directives are single-line.

KevinRansom · 2024-12-13T03:11:03Z

src/Compiler/lex.fsl

@@ -1080,6 +1082,13 @@ rule token (args: LexArgs) (skip: bool) = parse
     lexbuf.StartPos <- lexbuf.StartPos.ShiftColumnBy(n)
     HASH_IDENT(lexemeTrimLeft lexbuf (n+1)) }

+ | anywhite* ("#nowarn" | "#warnon") anystring


So you are not using the existing lexing for directives? will any existing programs break, or is this implementation compatible?

For example:

#nowarn #nowarn "1" 44 // This construct is deprecated

The directives were processed in the parser. This is no longer possible if we want to enable them everywhere (not only in top-level modules. Therefore I had to pull it forward into the lexer. (Commit 2 includes these changes).

src/Compiler/Utilities/range.fs

auduchinok · 2024-12-13T11:01:20Z

src/Compiler/SyntaxTree/WarnScopes.fsi

+open FSharp.Compiler.Text
+open FSharp.Compiler.UnicodeLexing
+
+module internal WarnScopes =


Any reason to make it internal?

We should somehow make the scopes information accessible to FCS users, given that the previous implementation is removed from SyntaxTree.

The directives are available as trivia (now in ParsedImplFileInputTrivia / ParsedSigFileInputTrivia), see commit 5 of this PR.
But if more information is useful for tooling, please let me know what I should make available.

Helpers like IsWarnon and IsNowarn are definitely useful.

I made IsWarnon and IsNowarn public.
I would recommend, though, to rather rely on the filtering mechanism of the diagnostics loggers of the compiler (which use these two functions internally).

changed a private name to make it more readable

Martin521 added 9 commits November 22, 2024 08:45

WarnScopes (1): feature flag, baseline tests, and WarnScopes module

7498b0b

WarnScopes (2): changes to lexing and parsing

9590899

WarnScopes (3): remove legacy #nowarn processing

7b2eeb2

WarnScopes (4): integrate the new functionality and add tests

4520e55

WarnScopes (5): add warn directive trivia

0fdaa43

WarnScopes (6): enable warn directive trivia

fb848d9

WarnScopes (7): remove defunct types and parameters

e623fde

WarnScopes (8): IlVerify baseline update

62eb32a

WarnScopes (9): ran fantomas, updated release notes

5c77ecd

Martin521 requested a review from a team as a code owner November 22, 2024 08:58

Martin521 added 10 commits November 22, 2024 09:02

updated PR number in release notes

bbdea9d

adapted scipt test to WarnScopes

1c97f60

updated PR link in release notes (now really)

ca6fd7a

Merge remote-tracking branch 'upstream/main' into scoped-nowarn-dev

0d0e7bf

skipped a test in vsintegration that might need a vs update

29212c5

Merge remote-tracking branch 'upstream/main' into scoped-nowarn-dev

6c3ab12

Merge remote-tracking branch 'upstream/main' into scoped-nowarn-dev

ceb7e38

merge upstream/main

23e366f

moved ilverify baselines

a727e67

Merge remote-tracking branch 'upstream/main' into scoped-nowarn

cdfef12

Martin521 mentioned this pull request Nov 29, 2024

Allow F# compiler directives like #nowarn to span less than an entire file. fsharp/fslang-suggestions#278

Open

merge upstream/main

e8a35f4

Martin521 mentioned this pull request Nov 29, 2024

A F# Compiler Tour: Reviewing "Scoped Nowarn" amplifying-fsharp/sessions#57

Open

Martin521 added 5 commits December 2, 2024 17:14

merge upstream/main

2214e83

update ilverify baselines

5f358b3

merge upstream/main

30d66e2

Merge remote-tracking branch 'upstream/main' into scoped-nowarn

387049e

Merge remote-tracking branch 'upstream/main' into scoped-nowarn

d617fbd

Martin521 added 3 commits December 4, 2024 21:11

update ilverify baseline (line number changes only)

e7eb32f

merge upstream/main

f05f993

merge upstream/main

eba854f

KevinRansom reviewed Dec 12, 2024

View reviewed changes

KevinRansom approved these changes Dec 13, 2024

View reviewed changes

auduchinok reviewed Dec 13, 2024

View reviewed changes

Martin521 added 7 commits December 15, 2024 16:47

merge upstream/main

7fa6352

fix merge issue

1301542

ilverify baselines

48d0875

Merge remote-tracking branch 'upstream/main' into scoped-nowarn

2d29b15

made WarnScopes.IsWarnon / IsNowarn public

2f44500

changed a private name to make it more readable

merge upstream/main

f526549

added surface area change (IsNowarn/IsWarnon) to baseline

7e96882

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Scoped nowarn #18049

Scoped nowarn #18049

Martin521 commented Nov 22, 2024 •

edited

Loading

github-actions bot commented Nov 22, 2024 •

edited

Loading

psfinaki commented Nov 29, 2024

KevinRansom Dec 12, 2024 •

edited

Loading

Martin521 Dec 15, 2024

KevinRansom left a comment

KevinRansom Dec 13, 2024

Martin521 Dec 15, 2024

KevinRansom Dec 13, 2024

Martin521 Dec 13, 2024

Martin521 Dec 16, 2024

KevinRansom Dec 13, 2024

KevinRansom Dec 13, 2024

Martin521 Dec 16, 2024

KevinRansom Dec 13, 2024

Martin521 Dec 13, 2024 •

edited

Loading

KevinRansom Dec 13, 2024

Martin521 Dec 13, 2024

KevinRansom Dec 13, 2024

Martin521 Dec 13, 2024

KevinRansom Dec 13, 2024

Martin521 Dec 13, 2024

KevinRansom Dec 13, 2024

Martin521 Dec 13, 2024

KevinRansom Dec 13, 2024

Martin521 Dec 13, 2024

auduchinok Dec 13, 2024

Martin521 Dec 13, 2024

auduchinok Dec 13, 2024

Martin521 Dec 16, 2024

		FSharp.Compiler.Diagnostics.FSharpDiagnosticOptions: System.String ToString()
		FSharp.Compiler.Diagnostics.FSharpDiagnosticOptions: Void .ctor(Int32, Boolean, Microsoft.FSharp.Collections.FSharpList`1[System.Int32], Microsoft.FSharp.Collections.FSharpList`1[System.Int32], Microsoft.FSharp.Collections.FSharpList`1[System.Int32], Microsoft.FSharp.Collections.FSharpList`1[System.Int32])

Scoped nowarn #18049

Are you sure you want to change the base?

Scoped nowarn #18049

Conversation

Martin521 commented Nov 22, 2024 • edited Loading

Description

Checklist

github-actions bot commented Nov 22, 2024 • edited Loading

❗ Release notes required

psfinaki commented Nov 29, 2024

KevinRansom Dec 12, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

KevinRansom left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Martin521 Dec 13, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Martin521 commented Nov 22, 2024 •

edited

Loading

github-actions bot commented Nov 22, 2024 •

edited

Loading

KevinRansom Dec 12, 2024 •

edited

Loading

Martin521 Dec 13, 2024 •

edited

Loading