Skip to content

Commit

Permalink
Merge pull request #17584 from dotnet/merges/main-to-release/dev17.12
Browse files Browse the repository at this point in the history
Merge main to release/dev17.12
  • Loading branch information
vzarytovskii committed Aug 22, 2024
2 parents e8d381b + bab72ca commit 60fc17d
Show file tree
Hide file tree
Showing 977 changed files with 3,241 additions and 3,238 deletions.
3 changes: 3 additions & 0 deletions .git-blame-ignore-revs
Original file line number Diff line number Diff line change
@@ -1,2 +1,5 @@
# Format src/Compiler/Checking/CheckComputationExpressions.fs, https://github.com/dotnet/fsharp/pull/16512
603a310cdfd9902ec1d29b399377dcc9ac56235b

# Spelling, https://github.com/dotnet/fsharp/pull/16212
823d5e99fdd13f696ea8fe572d502e5fa68f6fd1
2 changes: 1 addition & 1 deletion .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Fixes # (issue, if applicable)
>
> Release notes files:
> - If anything under `src/Compiler` has been changed, please make sure to make an entry in `docs/release-notes/.FSharp.Compiler.Service/<version>.md`, where `<version>` is usually "highest" one, e.g. `42.8.200`
> - If language feature was added (i.e. `LanguageFeatures.fsi` was changed), please add it to `docs/releae-notes/.Language/preview.md`
> - If language feature was added (i.e. `LanguageFeatures.fsi` was changed), please add it to `docs/release-notes/.Language/preview.md`
> - If a change to `FSharp.Core` was made, please make sure to edit `docs/release-notes/.FSharp.Core/<version>.md` where version is "highest" one, e.g. `8.0.200`.
> Information about the release notes entries format can be found in the [documentation](https://fsharp.github.io/fsharp-compiler-docs/release-notes/About.html).
Expand Down
2 changes: 1 addition & 1 deletion DEVGUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -342,4 +342,4 @@ See the "Debugging The Compiler" section of this [article](https://medium.com/@w

If you are behind a proxy server, NuGet client tool must be configured to use it:

See the Nuget config file documention for use with a proxy server [https://learn.microsoft.com/nuget/reference/nuget-config-file](https://learn.microsoft.com/nuget/reference/nuget-config-file)
See the Nuget config file documentation for use with a proxy server [https://learn.microsoft.com/nuget/reference/nuget-config-file](https://learn.microsoft.com/nuget/reference/nuget-config-file)
2 changes: 1 addition & 1 deletion Directory.Build.props
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
When developers load the FSharp.Compiler.Service solution we set FSHARPCORE_USE_PACKAGE to true if it hasn't already been set to a value.
This option ensures that building and testing uses the specified FSharp.Core nuget package instead of the local FSharp.Core project.
We also disable arcade and reset certain artifacts and compiler paths to use default ones.
All settings below can be overriden via CLI switches if needed. -->
All settings below can be overridden via CLI switches if needed. -->

<PropertyGroup Condition="'$(SolutionName)' == 'FSharp.Compiler.Service' and '$(BUILDING_USING_DOTNET)' != 'false'">
<BUILDING_USING_DOTNET>true</BUILDING_USING_DOTNET>
Expand Down
10 changes: 5 additions & 5 deletions TESTGUIDE.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# F# Compiler, Core Library and Visual F# Tools Tests

Where this guide mentions the command `build` it means either `build.cmd` in the root folder for Windows, or `build.sh` for Linux/MacOS.
Where this guide mentions the command `build` it means either `build.cmd` in the root folder for Windows, or `build.sh` for Linux/macOS.

## In this guide

Expand Down Expand Up @@ -49,9 +49,9 @@ Some test groups can only be run in `CI` configuration, for that, you need to pa

To run tests, from a command prompt, use variations such as the following, depending on which test suite and build configuration you want.

### Tests that can be run on Linux and MacOS
### Tests that can be run on Linux and macOS

If you're using Linux or MacOS to develop, the group of tests that are known to succeed are all in `-testCoreClr`. Any other `-testXXX` argument will currently fail. An effort is underway to make testing and running tests easier on all systems.
If you're using Linux or macOS to develop, the group of tests that are known to succeed are all in `-testCoreClr`. Any other `-testXXX` argument will currently fail. An effort is underway to make testing and running tests easier on all systems.

### Tests that can only be run in Release mode

Expand Down Expand Up @@ -123,7 +123,7 @@ This is compiled using [tests\fsharp\FSharp.Tests.FSharpSuite.fsproj](tests/fsha
Tests are grouped in folders per area. Each test compiles and executes a `test.fsx|fs` file in its folder using some combination of compiler or FSI flags specified in the FSharpSuite test project.
If the compilation and execution encounter no errors, the test is considered to have passed.

There are also negative tests checking code expected to fail compilation. See note about baseline under "Other Tips" bellow for tests checking expectations against "baseline" (.bsl) files.
There are also negative tests checking code expected to fail compilation. See note about baseline under "Other Tips" below for tests checking expectations against "baseline" (.bsl) files.

### FSharpQA Suite

Expand Down Expand Up @@ -174,7 +174,7 @@ All test execution logs and result files will be dropped into the `tests\TestRes
### Working with baseline tests

FSharp Test Suite works with a couple of `.bsl` (or `.bslpp`) files describing "expected test results" and are called the _Baseline Tests_. Those are matched against the actual output that resides under `.err` or `.vserr` files of the same name during test execution.
When doing so keep in mind to carefully review the diff before comitting updated baseline files.
When doing so keep in mind to carefully review the diff before committing updated baseline files.

The `.bslpp` (for: baseline pre-process) files are specially designed to enable substitution of certain tokens to generate the `.bsl` file. You can look further about the pre-processing logic under [tests/fsharp/TypeProviderTests.fs](tests/fsharp/TypeProviderTests.fs), this is used only for type provider tests for now.

Expand Down
10 changes: 5 additions & 5 deletions buildtools/README-fslexyacc.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ That means we have to build up FsLex and FsYacc from scratch, _including_ their
We can't pick up the source from "packages" because in a build-from-source scenario we can't even fetch those
packages - we really have to build from just our source tree and .NET SDK.

Please do _not_ modify the code in these directories except by copying over from an upgraded FsLexYacc pacakge.
Please do _not_ modify the code in these directories except by copying over from an upgraded FsLexYacc package.
Without the testing and documentation in the `FsLexYacc` repo, this copied code is just a bunch of untested, undocumented and
largely generated code checked into our source tree.

Expand All @@ -31,14 +31,14 @@ For (1), to improve the code/table generators, make a PR to the `FsLexYacc` repo

For (2), normally for FsLexYacc-based tools the runtime is either a source inclusion of `Lexing.fs`, Lexing.fsi, Parsing.fs, Parsing.fsi or a reference to the `FsLexYacc.Runtime` package. The runtime contains LexBuffer and the lexing/parsing table interpreters.

However long ago we decided to duplicate and ingest the _runtime_ files for FsLex and FsYacc into the F# compiler rather than taking them directly from the FsLexYacc project. This was mainly because we wanted to squeeze optimizations out of them based on profiling and simplify them a bit. The duplicated files are `prim-lexing.fs`, `prim-parsing.fs` and the corresponding `.fsi` files in `src/utils`. These files are sufficient to implement the contracts exepcted by the FsLex/FsYacc generated code, and require exactly the same table formats as generated by FsLex/FsYacc.
However long ago we decided to duplicate and ingest the _runtime_ files for FsLex and FsYacc into the F# compiler rather than taking them directly from the FsLexYacc project. This was mainly because we wanted to squeeze optimizations out of them based on profiling and simplify them a bit. The duplicated files are `prim-lexing.fs`, `prim-parsing.fs` and the corresponding `.fsi` files in `src/utils`. These files are sufficient to implement the contracts excepted by the FsLex/FsYacc generated code, and require exactly the same table formats as generated by FsLex/FsYacc.

This means you can improve some aspects of the _runtime_ for FsLex and FsYacc by making direct changes to `prim-lexing.fs` and `prim-parsing.fs`.

For example, the _actual_ `LexBuffer` type being used in the F# compiler (for all three lexers and grammars) is this one: https://github.com/dotnet/fsharp/blob/bdb64624f0ca220ca4433c83d02dd5822fe767a5/src/Compiler/Facilities/prim-lexing.fsi#L102 . (That version of the Lex/Yacc runtime has added some things: `BufferLocalStore` for example, which we use for the `XmlDoc` accumulator as we strip those out. It's also dropped any mention of async lexing, and any mention of `byte`. The use
of generics for `LexBuffer<'Char>` is also superfluous because `'Char` is always `char` but is needed because the FsLex/FsYacc generated code expects this type to be generic.)

## What if I want to eridicate our use of FsLex and FsYacc?
## What if I want to eradicate our use of FsLex and FsYacc?

The use of FsLex and FsYacc in this repo is somewhat controversial since the C# compiler implementation uses hand-written lexers and parsers.

Expand All @@ -48,7 +48,7 @@ overly difficult given the declarative nature of `FsLex` tokenization.
The use of a table-driven LALR(1) parser is more controversial: there is a general feeling that it would be great to
somehow move on from FsYacc and do parsing some other way. However, it is not at all easy to do that and remain
fully compatible. For this reason it is unlikely we will remove the use of FsYacc any time soon. However incremental
modifications to extract more information from the grammer may yield good results.
modifications to extract more information from the grammar may yield good results.

## Why aren't FsLex and FsYacc just ingested into this repo if we depend on them (and even have an exact copy of them for build-from-source)?

Expand All @@ -57,7 +57,7 @@ considered carefully. While we are open to adding features to these tools specif
independently. For these reasons it is generally best that these tools live in their own repository.

The copy of the `fslex` and `fsyacc` source code in `buildtools` is an exact copy and is not tested or documented
apart from what's been done before in FsLexYacc repo. Adjusting these copies is not allowed and would be wrong from an engineering persepctive,
apart from what's been done before in FsLexYacc repo. Adjusting these copies is not allowed and would be wrong from an engineering perspective,
because there's no place to put documentation or tests.

Occasionally we discuss ingesting FsLex and FsYacc into this repository. This often comes up in the hope that by doing so
Expand Down
2 changes: 1 addition & 1 deletion buildtools/checkpackages/check.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ if ($LASTEXITCODE -eq 0)
{
$package = Get-Content -Path .\Version.txt
Write-Error "
Package restore succeded for '${package}', expected to fail.
Package restore succeeded for '${package}', expected to fail.
This usually means that the package has been already published.
Please, bump the version to fix this failure." -ErrorAction Stop
} else {
Expand Down
2 changes: 1 addition & 1 deletion buildtools/fslex/Lexing.fs
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ and [<Sealed>]
let extendBufferAsync = (fun () -> match filler.fillAsync with Some refill -> refill this | None -> invalidOp "attempt to read asynchronously from a synchronous lex buffer")
let mutable buffer=[||]

/// number of valid charactes beyond bufferScanStart
/// number of valid characters beyond bufferScanStart
let mutable bufferMaxScanLength=0

/// count into the buffer when scanning
Expand Down
4 changes: 2 additions & 2 deletions buildtools/fslex/Lexing.fsi
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ type AsciiTables =
/// Interpret tables for an ascii lexer generated by fslex.
member Interpret: initialState:int * LexBuffer<byte> -> int

[<System.Obsolete("Async interretation of lexer tables is now obsolete and will be removed. Please use a background thread if you want async lexing")>]
[<System.Obsolete("Async interpretation of lexer tables is now obsolete and will be removed. Please use a background thread if you want async lexing")>]
/// Interpret tables for an ascii lexer generated by fslex, processing input asynchronously
member AsyncInterpret: initialState:int * LexBuffer<byte> -> Async<int>

Expand All @@ -150,7 +150,7 @@ type UnicodeTables =
/// Interpret tables for a unicode lexer generated by fslex.
member Interpret: initialState:int * LexBuffer<char> -> int

[<System.Obsolete("Async interretation of lexer tables is now obsolete and will be removed. Please use a background thread if you want async lexing")>]
[<System.Obsolete("Async interpretation of lexer tables is now obsolete and will be removed. Please use a background thread if you want async lexing")>]
/// Interpret tables for a unicode lexer generated by fslex, processing input asynchronously
member AsyncInterpret: initialState:int * LexBuffer<char> -> Async<int>

Expand Down
2 changes: 1 addition & 1 deletion buildtools/fslex/Parsing.fs
Original file line number Diff line number Diff line change
Expand Up @@ -431,7 +431,7 @@ module Implementation =
// and an EOF token.
if inEofCountDown && eofCountDown < 10 then
#if __DEBUG
if Flags.debug then printfn "poppin stack, lokking to shift both 'error' and that token, during end-of-file error recovery"
if Flags.debug then printfn "poppin stack, looking to shift both 'error' and that token, during end-of-file error recovery"
#endif
popStackUntilErrorShifted(if haveLookahead then Some(lookaheadToken) else None)

Expand Down
4 changes: 2 additions & 2 deletions buildtools/fslex/Parsing.fsi
Original file line number Diff line number Diff line change
Expand Up @@ -113,9 +113,9 @@ type Tables<'tok> =
/// Returns an object indicating the final synthesized value for the parse.
member Interpret : lexer:(LexBuffer<'char> -> 'tok) * lexbuf:LexBuffer<'char> * startState:int -> obj

/// Indicates an accept action has occured
/// Indicates an accept action has occurred
exception Accept of obj
/// Indicates a parse error has occured and parse recovery is in progress
/// Indicates a parse error has occurred and parse recovery is in progress
exception RecoverableParseError

#if __DEBUG
Expand Down
2 changes: 1 addition & 1 deletion buildtools/fslex/fslexast.fs
Original file line number Diff line number Diff line change
Expand Up @@ -258,7 +258,7 @@ let LexerStateToNfa ctx (macros: Map<string,_>) (clauses: Clause list) =
// These cases unwind the difficult cases in the syntax that rely on knowing the
// entire alphabet.
//
// Note we've delayed the expension of these until we've worked out all the 'special' Unicode characters
// Note we've delayed the expansion of these until we've worked out all the 'special' Unicode characters
// mentioned in the entire lexer spec, i.e. we wait until GetAlphabet returns a reliable and stable answer.
| Inp (UnicodeCategory uc) ->
let re = Alt(fun ctx ->
Expand Down
2 changes: 1 addition & 1 deletion buildtools/fsyacc/Lexing.fs
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ and [<Sealed>]
let extendBufferAsync = (fun () -> match filler.fillAsync with Some refill -> refill this | None -> invalidOp "attempt to read asynchronously from a synchronous lex buffer")
let mutable buffer=[||]

/// number of valid charactes beyond bufferScanStart
/// number of valid characters beyond bufferScanStart
let mutable bufferMaxScanLength=0

/// count into the buffer when scanning
Expand Down
4 changes: 2 additions & 2 deletions buildtools/fsyacc/Lexing.fsi
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ type AsciiTables =
/// Interpret tables for an ascii lexer generated by fslex.
member Interpret: initialState:int * LexBuffer<byte> -> int

[<System.Obsolete("Async interretation of lexer tables is now obsolete and will be removed. Please use a background thread if you want async lexing")>]
[<System.Obsolete("Async interpretation of lexer tables is now obsolete and will be removed. Please use a background thread if you want async lexing")>]
/// Interpret tables for an ascii lexer generated by fslex, processing input asynchronously
member AsyncInterpret: initialState:int * LexBuffer<byte> -> Async<int>

Expand All @@ -150,7 +150,7 @@ type UnicodeTables =
/// Interpret tables for a unicode lexer generated by fslex.
member Interpret: initialState:int * LexBuffer<char> -> int

[<System.Obsolete("Async interretation of lexer tables is now obsolete and will be removed. Please use a background thread if you want async lexing")>]
[<System.Obsolete("Async interpretation of lexer tables is now obsolete and will be removed. Please use a background thread if you want async lexing")>]
/// Interpret tables for a unicode lexer generated by fslex, processing input asynchronously
member AsyncInterpret: initialState:int * LexBuffer<char> -> Async<int>

Expand Down
2 changes: 1 addition & 1 deletion buildtools/fsyacc/Parsing.fs
Original file line number Diff line number Diff line change
Expand Up @@ -431,7 +431,7 @@ module Implementation =
// and an EOF token.
if inEofCountDown && eofCountDown < 10 then
#if __DEBUG
if Flags.debug then printfn "poppin stack, lokking to shift both 'error' and that token, during end-of-file error recovery"
if Flags.debug then printfn "poppin stack, looking to shift both 'error' and that token, during end-of-file error recovery"
#endif
popStackUntilErrorShifted(if haveLookahead then Some(lookaheadToken) else None)

Expand Down
4 changes: 2 additions & 2 deletions buildtools/fsyacc/Parsing.fsi
Original file line number Diff line number Diff line change
Expand Up @@ -113,9 +113,9 @@ type Tables<'tok> =
/// Returns an object indicating the final synthesized value for the parse.
member Interpret : lexer:(LexBuffer<'char> -> 'tok) * lexbuf:LexBuffer<'char> * startState:int -> obj

/// Indicates an accept action has occured
/// Indicates an accept action has occurred
exception Accept of obj
/// Indicates a parse error has occured and parse recovery is in progress
/// Indicates a parse error has occurred and parse recovery is in progress
exception RecoverableParseError

#if __DEBUG
Expand Down
14 changes: 7 additions & 7 deletions buildtools/fsyacc/fsyaccast.fs
Original file line number Diff line number Diff line change
Expand Up @@ -134,9 +134,9 @@ let ProcessParserSpecAst (spec: ParserSpec) =
// Process LALR(1) grammars to tables

type ProductionIndex = int
type ProdictionDotIndex = int
type ProductionDotIndex = int

/// Represent (ProductionIndex,ProdictionDotIndex) as one integer
/// Represent (ProductionIndex,ProductionDotIndex) as one integer
type Item0 = uint32

let mkItem0 (prodIdx,dotIdx) : Item0 = (uint32 prodIdx <<< 16) ||| uint32 dotIdx
Expand Down Expand Up @@ -287,7 +287,7 @@ type ProductionTable(ntTab:NonTerminalTable, termTab:TerminalTable, nonTerminals
if n >= syms.Length then None else Some syms.[n]
member prodTab.Productions = productions

/// A mutable table maping kernels to sets of lookahead tokens
/// A mutable table mapping kernels to sets of lookahead tokens
type LookaheadTable() =
let t = Dictionary<KernelItemIndex,Set<TerminalIndex>>()
member table.Add(x,y) =
Expand All @@ -311,7 +311,7 @@ type KernelTable(kernels) =
member t.Index(kernel) = toIdxMap.[kernel]
member t.Kernel(i) = ofIdxMap.[i]

/// Hold the results of cpmuting the LALR(1) closure of an LR(0) kernel
/// Hold the results of computing the LALR(1) closure of an LR(0) kernel
type Closure1Table() =
let t = Dictionary<Item0,HashSet<TerminalIndex>>()
member table.Add(a,b) =
Expand Down Expand Up @@ -648,7 +648,7 @@ let CompilerLalrParserSpec logf (spec : ProcessedParserSpec): CompiledSpec =

// Compute the "spontaneous" and "propagate" maps for each LR(0) kernelItem
//
// Input: The kernal K of a set of LR(0) items I and a grammar symbol X
// Input: The kernel K of a set of LR(0) items I and a grammar symbol X
//
// Output: The lookaheads generated spontaneously by items in I for kernel items
// in goto(I,X) and the items I from which lookaheads are propagated to kernel
Expand All @@ -661,7 +661,7 @@ let CompilerLalrParserSpec logf (spec : ProcessedParserSpec): CompiledSpec =
//
// PLAN TO OPTIMIZE THIS;
// - Clarify and comment what's going on here
// - verify if we really have to do these enormouos closure computations
// - verify if we really have to do these enormous closure computations
// - assess if it's possible to use the symbol we're looking for to help trim the jset

reportTime(); printf "computing lookahead relations..."; stdout.Flush();
Expand Down Expand Up @@ -741,7 +741,7 @@ let CompilerLalrParserSpec logf (spec : ProcessedParserSpec): CompiledSpec =
// into the table, taking into account precedences etc. and reporting errors.
let addResolvingPrecedence (arr: _[]) kernelIdx termIdx (precNew, actionNew) =
// printf "DEBUG: state %d: adding action for %s, precNew = %a, actionNew = %a\n" kernelIdx (termTab.OfIndex termIdx) outputPrec precNew OutputAction actionNew;
// We add in order of precedence - however the precedences may be the same, and we give warnings when rpecedence resolution is based on implicit file orderings
// We add in order of precedence - however the precedences may be the same, and we give warnings when precedence resolution is based on implicit file orderings

let _, actionSoFar as itemSoFar = arr.[termIdx]

Expand Down
Loading

0 comments on commit 60fc17d

Please sign in to comment.