release 3.3 (#6)

* fix:  Const initializations  problems dafny-lang#1111 (dafny-lang#1509)

Fixed dafny-lang#1111

* Only emit captureState when debugging with /mv (dafny-lang#1465)

### Changes

- Do not emit `assume {:capturestate ...} true` statements unless the `/mv:<file>` option, which is only used for debugging, is passed. These statements contained source locations which prevent proof isolation.

### Testing

- A side effect of the first change is that source locations in error messages may shift, since previously sometimes the location attached to an `assume {:capturestate ...} true` at the end of a block, which would be the location of any `ensures` clauses, was being picked as the return location of the block. I believe the new behavior is better so I've updated the tests, but please verify.

* Separate document verification from load (dafny-lang#1516)

* Extracted the text change migration into a new service

* Added cancellation token to the position resolution

* Added the possible exceptions to the XML doc.

* Extracted the symbol relocation into a dedicated service

* Removed the default token argument

* Moved document updates to database to make it fully responsible of cancellation

* Added integration tests for OnSave/Never verification status messages

* Made document opening more self-descriptive

* Moved the verification responsibility to the document database

* Removed the SerializedCounterExamples from the document constructor

* Restructured code

* Reduce the TPL stress by mutually exclude verification asynchronously

* Moved verification back to the document loader

* Fixed doc typos

* Now verifying documents on the large stack thread

Co-authored-by: Remy Willems <rgv.willems@gmail.com>

* Enhance `Import` printer (dafny-lang#1487)

This commit optimises `Import M = M` to `Import M`, and `Import
M\`{N}` to `Import M\`N`. It also fixes and adds to the existing tests
for module insertion.

Co-authored-by: Hira Syeda <syedahir@amazon.com>
Co-authored-by: Remy Willems <rgv.willems@gmail.com>

* Counterexamples from Command Line (dafny-lang#1511)

* Move position extraction to DafnyModelState

* Move DafnyModel parsing to DafnyModel class

* Add /extractCounterExample option

* Add a lit Test

* Update option description

* Change source location reporting

Co-authored-by: Robin Salkeld <salkeldr@amazon.com>

* chore(version): Bump version for pre-release (dafny-lang#1524)

* Bump version for pre-release
* Allow PR title to satisfy semantic PRs app

* chore: Fix version number in attributes (dafny-lang#1525)

* Add document symbols to support outline (dafny-lang#1483)

Enable support for document outlines and the breadcrumb navigation at the top of the editor.

Co-authored-by: Remy Willems <rgv.willems@gmail.com>

* chore: xUnit-based lit test runner (dafny-lang#680)

Adds an xUnit-based test runner for all of the lit tests under Test.

The new runner supports executing commands such as %dafny by directly invoking the main entry point of the corresponding C# package (i.e. DafnyDriver), which makes running the debugger against a particular test much more convenient. By default this is disabled and the runner creates a separate dafny process just as lit does, however. This is because the main CLI implementation currently has shared static state, which causes errors when invoking the CLI in the same process on multiple inputs in sequence, much less in parallel. Future changes will address this so that the in-process Main invocation can be used instead, however, which will likely improve performance but more importantly allow us to measure code coverage of the test suite.

Where this alternate runner really shines is in IDEs that support .NET test frameworks:

Getting a tree of test results in the UI
Being able to re-run/debug individual tests by right-clicking on them
Filtering by test name (e.g. dotnet test --filter DisplayName~allocated)
(Earlier versions of this PR went further and aimed to convert the lit tests to a new format. We may still want to do this in the future, but parsing and understanding the existing test suite is a necessary prerequisite anyway.)

This change adds two new .NET packages:

XUnitExtensions - Contains a few generic extensions to xUnit to support file-based parameterized tests (a.k.a. Theories), and the alternate Lit test interpreter.
IntegrationTests - Contains a single file-based parameterized xUnit test to run all lit tests in the Test directory.
Other .NET projects that use lit, such as Boogie, could also leverage this runner by depending on the generic XUnitExtensions package and defining a similar second package.

There is one fundamental difference in behavior between the two runners: our current lit configuration uses the immediate directory of each test case as the current directory, whereas .NET test runners always run in the single project output directory (e.g. IntegrationTests/bin/Debug/net5.0). It wouldn't be feasible to change the current directory to match lit's behavior, since multiple parallel tests cannot do this at the same time. Instead, I prepended the %S substitution macro to a handful of command arguments that were relative paths, and otherwise ensured the tests did not depend on the current directory.

Because the IntegrationTests package runs all tests out of the output directory rather than the Test directory, its csproj configuration and its dependencies ensure that a few additional files are also copied into that directory, such as z3 and the necessary runtime files. I also applied this technique to the other test projects to avoid the manual step of copying z3 as described in the wiki (and will remove that step once this PR is merged).

Co-authored-by: Alex Cioc <4691979+acioc@users.noreply.github.com>
Co-authored-by: Remy Willems <rgv.willems@gmail.com>

* Update Boogie to 2.9.4 (dafny-lang#1531)

Co-authored-by: Remy Willems <>

* No more `assume 2 < 2;` (dafny-lang#1354)

* Add test file

* Don't remove Lit brackets when assert/assume

By removing this optimization (?), a condition like `false` is no longer passed to Boogie that way, but instead of `Lit(false)`.

* Refactor Assert helper methods

This change also removes a redundant `this.assertionCount++;`. This increment is done by the `builder.Add` method anyway.

* Redo changes lost in the merge

Co-authored-by: Robin Salkeld <salkeldr@amazon.com>
Co-authored-by: Remy Willems <rgv.willems@gmail.com>

* fix: Document outline for functions and methods without body (dafny-lang#1536)

This PR resolves an issue where the document outline won't be rendered within VSCode if there are methods or functions without a body (i.e., only the signature).

The root cause is that VSCode does not render any document outline if there are symbols with invalid locations (e.g., a negative line number). The range of a declaration is computed using tok and BodyEndTok. However, BodyEndTok is set to Token.NoToken if there is no body, leading to a range ending at line -1.

Co-authored-by: Remy Willems <rgv.willems@gmail.com>
Co-authored-by: Christoph Amrein <camrein@users.noreply.github.com>
Co-authored-by: hirataqdees <syeda.hira.taqdees@gmail.com>
Co-authored-by: Hira Syeda <syedahir@amazon.com>
Co-authored-by: Aleksandr Fedchin <sasha.fedchin@gmail.com>
Co-authored-by: Robin Salkeld <salkeldr@amazon.com>
Co-authored-by: Alex Cioc <4691979+acioc@users.noreply.github.com>
Co-authored-by: Rustan Leino <leino@amazon.com>

Source/DafnyLanguageServer.Test/Various/CompilationStatusNotificationTest.cs

@@ -266,5 +266,97 @@ lemma Something(i: int)
  Assert.AreEqual(documentItem.Version, compilation.Version);
  Assert.AreEqual(CompilationStatus.ParsingFailed, compilation.Status);
  }
+
+ [TestMethod, Timeout(MaxTestExecutionTimeMs)]
+ public async Task DocumentLoadWithOnSaveVerificationDoesNotSendVerificationStatuses() {
+ var source = @"
+method Abs(x: int) returns (y: int)
+ ensures y >= 0
+{
+ return x;
+}
+".TrimStart();
+ await SetUp(new Dictionary<string, string>() {
+ { $"{DocumentOptions.Section}:{nameof(DocumentOptions.Verify)}", nameof(AutoVerification.OnSave) }
+ });
+
+ // We load two documents. If no verification is executed, we should receive each
+ // compilation status twice without any verification status inbetween.
+ var documentItem1 = CreateTestDocument(source, "test_1.dfy");
+ await client.OpenDocumentAndWaitAsync(documentItem1, CancellationToken);
+ var documentItem2 = CreateTestDocument(source, "test_2dfy");
+ await client.OpenDocumentAndWaitAsync(documentItem2, CancellationToken);
+
+ var compilation1 = await notificationReceiver.AwaitNextNotificationAsync(CancellationToken);
+ Assert.AreEqual(documentItem1.Uri, compilation1.Uri);
+ Assert.AreEqual(documentItem1.Version, compilation1.Version);
+ Assert.AreEqual(CompilationStatus.CompilationSucceeded, compilation1.Status);
+
+ var compilation2 = await notificationReceiver.AwaitNextNotificationAsync(CancellationToken);
+ Assert.AreEqual(documentItem2.Uri, compilation2.Uri);
+ Assert.AreEqual(documentItem2.Version, compilation2.Version);
+ Assert.AreEqual(CompilationStatus.CompilationSucceeded, compilation2.Status);
+ }
+
+ [TestMethod, Timeout(MaxTestExecutionTimeMs)]
+ public async Task DocumentSaveWithOnSaveVerificationSendsVerificationStatuses() {
+ var source = @"
+method Abs(x: int) returns (y: int)
+ ensures y >= 0
+{
+ return x;
+}
+".TrimStart();
+ await SetUp(new Dictionary<string, string>() {
+ { $"{DocumentOptions.Section}:{nameof(DocumentOptions.Verify)}", nameof(AutoVerification.OnSave) }
+ });
+
+ var documentItem = CreateTestDocument(source, "test_1.dfy");
+ await client.OpenDocumentAndWaitAsync(documentItem, CancellationToken);
+ await client.SaveDocumentAndWaitAsync(documentItem, CancellationToken);
+
+ bool verificationStartedReceived = false;
+ bool verificationFailedReceived = false;
+ while (!verificationStartedReceived || !verificationFailedReceived) {
+ var notification = await notificationReceiver.AwaitNextNotificationAsync(CancellationToken);
+ Assert.AreEqual(documentItem.Uri, notification.Uri);
+ Assert.AreEqual(documentItem.Version, notification.Version);
+ verificationStartedReceived = verificationStartedReceived || notification.Status == CompilationStatus.VerificationStarted;
+ verificationFailedReceived = verificationFailedReceived || notification.Status == CompilationStatus.VerificationFailed;
+ }
+ }
+
+ [TestMethod, Timeout(MaxTestExecutionTimeMs)]
+ public async Task DocumentLoadAndSaveWithNeverVerifySendsNoVerificationStatuses() {
+ var source = @"
+method Abs(x: int) returns (y: int)
+ ensures y >= 0
+{
+ return x;
+}
+".TrimStart();
+ await SetUp(new Dictionary<string, string>() {
+ { $"{DocumentOptions.Section}:{nameof(DocumentOptions.Verify)}", nameof(AutoVerification.Never) }
+ });
+
+ // We load two and save two documents. If no verification is executed, we should receive each
+ // compilation status twice without any verification status inbetween.
+ var documentItem1 = CreateTestDocument(source, "test_1.dfy");
+ await client.OpenDocumentAndWaitAsync(documentItem1, CancellationToken);
+ await client.SaveDocumentAndWaitAsync(documentItem1, CancellationToken);
+ var documentItem2 = CreateTestDocument(source, "test_2dfy");
+ await client.OpenDocumentAndWaitAsync(documentItem2, CancellationToken);
+ await client.SaveDocumentAndWaitAsync(documentItem2, CancellationToken);
+
+ var compilation1 = await notificationReceiver.AwaitNextNotificationAsync(CancellationToken);
+ Assert.AreEqual(documentItem1.Uri, compilation1.Uri);
+ Assert.AreEqual(documentItem1.Version, compilation1.Version);
+ Assert.AreEqual(CompilationStatus.CompilationSucceeded, compilation1.Status);
+
+ var compilation2 = await notificationReceiver.AwaitNextNotificationAsync(CancellationToken);
+ Assert.AreEqual(documentItem2.Uri, compilation2.Uri);
+ Assert.AreEqual(documentItem2.Version, compilation2.Version);
+ Assert.AreEqual(CompilationStatus.CompilationSucceeded, compilation2.Status);
+ }
  }
 }

Source/DafnyLanguageServer/Workspace/ChangeProcessors/ISymbolTableRelocator.cs

@@ -0,0 +1,22 @@
+using Microsoft.Dafny.LanguageServer.Language.Symbols;
+using OmniSharp.Extensions.LanguageServer.Protocol.Models;
+using System.Threading;
+
+namespace Microsoft.Dafny.LanguageServer.Workspace.ChangeProcessors {
+ /// <summary>
+ /// Implementations of this interface are responsible to relocate the symbols of a given
+ /// <see cref="SymbolTable"/> to the new positions according to the given <see cref="DidChangeTextDocumentParams"/>.
+ /// </summary>
+ public interface ISymbolTableRelocator {
+ /// <summary>
+ /// Relocates the symbols of the given table with the given text changes.
+ /// </summary>
+ /// <param name="originalSymbolTable">The symbol table whose symbols should be relocated.</param>
+ /// <param name="changes">The applied changes to the text document that should be used for the relocation.</param>
+ /// <param name="cancellationToken">A token to stop the relocation prior completion.</param>
+ /// <returns>A new symbol table whose symbols are placed according to the given changes.</returns>
+ /// <exception cref="System.OperationCanceledException">Thrown when the cancellation was requested before completion.</exception>
+ /// <exception cref="System.ObjectDisposedException">Thrown if the cancellation token was disposed before the completion.</exception>
+ SymbolTable Relocate(SymbolTable originalSymbolTable, DidChangeTextDocumentParams changes, CancellationToken cancellationToken);
+ }
+}