Skip to content

Commit

Permalink
Merge pull request #8 from Microsoft/master
Browse files Browse the repository at this point in the history
merge from dev
  • Loading branch information
AviAvni authored Nov 2, 2016
2 parents 7f13a25 + 0e60f38 commit 500c342
Show file tree
Hide file tree
Showing 221 changed files with 5,894 additions and 21,424 deletions.
10 changes: 7 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -49,13 +49,13 @@
/tests/*FSharp_Failures.lst
/tests/fsharpqa/Source/CodeGen/EmittedIL/StaticInit/StaticInit_Module01.dll
/tests/fsharpqa/Source/CodeGen/EmittedIL/StaticInit/StaticInit_Module01.pdb
/tests/XFSharpQA_Failures.log.*
/tests/Xnet40-fsharpqa-suite-failures.log.*
/vsintegration/src/vs/FsPkgs/FSharp.Project/FS/FSharp.ProjectSystem.FSharp.fsi
/vsintegration/src/vs/FsPkgs/FSharp.Project/FS/ctofiles/
/tests/fsharpqa/Source/CodeGen/EmittedIL/QueryExpressionStepping/Utils.dll
/tests/fsharpqa/Source/CodeGen/EmittedIL/ComputationExpressions/ComputationExprLibrary.dll
/tests/fsharpqa/Source/*FSharpQA_Failures.env
/tests/fsharpqa/Source/*FSharpQA_Failures.lst
/tests/fsharpqa/Source/*net40-fsharpqa-suite-failures.env
/tests/fsharpqa/Source/*net40-fsharpqa-suite-failures.lst
/tests/**/FSharp.Core.dll
lib/debug
lib/release
Expand Down Expand Up @@ -102,3 +102,7 @@ tests/fsharp/core/array/dont.run.peverify
tests/fsharp/core/innerpoly/dont.run.peverify
tests/fsharp/typecheck/sigs/neg94-pre.dll
times
/tests/fsharpqa/testenv/bin/System.ValueTuple.dll
source_link.json
/.vs
/VSRelease/net40/bin
2 changes: 1 addition & 1 deletion BuildToolsVersion.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
1.0.26-prerelease-00625-01
1.0.26-prerelease-00920-02
127 changes: 25 additions & 102 deletions DEVGUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ To learn what F# is and why it's interesting, go to [fsharp.org](http://fsharp.o

The primary technical documents for the F# compiler code are

* [The F# Language and Core Library RFC Process](http://fsharp.github.io/2016/09/26/fsharp-rfc-process.html)

* [The F# Language Specification](http://fsharp.org/specs/language-spec/)

* [The F# Compiler Technical Guide](http://fsharp.github.io/2015/09/29/fsharp-compiler-guide.html)
Expand All @@ -21,44 +23,41 @@ The primary technical documents for the F# compiler code are

## Quick Start: Build, Test, Develop

You can build the compiler+tools and run the subset the tests used for continuous integration as follows:
You can build the F# compiler for .NET Framework as follows:

build.cmd

This is the same as

build.cmd net40

There are various qualifiers:

build.cmd release -- build release (the default)
build.cmd debug -- build debug instead of release

build.cmd proto -- force the rebuild of the Proto bootstrap compiler in addition to other things
build.cmd protofx -- build using a .NET Framework proto (no .NET Core is used)
build.cmd net40 -- build .NET Framework compiler (the default)
build.cmd coreclr -- build .NET Core compiler
build.cmd vs -- build the Visual F# IDE Tools
build.cmd pcls -- build the PCL FSharp.Core libraries
build.cmd all -- build all

build.cmd net40 -- build/tests for .NET Framework version of the compiler (not the Visual F# IDE Tools or .NET Core)
build.cmd coreclr -- build/tests only the coreclr version compiler (not the Visual F# IDE Tools or .NET Framework)
build.cmd vs -- build/tests the Visual F# IDE Tools
build.cmd pcls -- build/tests the PCL FSharp.Core libraries
build.cmd proto -- force the rebuild of the Proto bootstrap compiler in addition to other things

build.cmd build -- build, do not test
build.cmd ci -- build, run the same tests as CI
build.cmd all -- build, run all tests
build.cmd notests -- turn off testing (used in conjunction with other options)
build.cmd test -- build default targets, run suitable tests
build.cmd net40 test -- build net40, run suitable tests
build.cmd coreclr test -- build coreclr, run suitable tests
build.cmd vs test -- build Visual F# IDE Tools, run all tests
build.cmd all test -- build all, run all tests

build.cmd test-smoke -- build, run smoke tests
build.cmd test-coreunit -- build, run FSharp.Core tests
build.cmd test-coreclr -- build, run .NET Core tests
build.cmd test-pcls -- build, run PCL tests
build.cmd test-fsharp -- build, run tests\fsharp suite
build.cmd test-fsharpqa -- build, run tests\fsharpqa suite
build.cmd test-vs -- build, run Visual F# IDE Tools unit tests
build.cmd test-net40-fsharp -- build, run tests\fsharp suite for .NET Framework
build.cmd test-net40-fsharpqa -- build, run tests\fsharpqa suite for .NET Framework

**Notes**
To build and test Visual F# IDE Tools, you must use [Visual Studio "vNext" (aka "Dev15")](https://www.visualstudio.com/en-us/downloads/visual-studio-next-downloads-vs.aspx). This is the one after Visual Studio 2015 (aka "Dev 14"). You must also install Visual Studio SDK (also called _Visual Studio Extensibility SDK_ on the Visual Studio installer) before building Visual F# IDE Tools.
Please ensure that the Visual Studio SDK version is matched with your current Visual Studio to ensure successful builds. For example: Visual Studio 2015 Update 1 requires Visual Studio 2015 SDK Update 1. Any installation of Visual Studio 2015 and later provides Visual Studio SDK as part of the installation of Visual Studio 2015 as feature installation.

Combinations are also allowed:

build.cmd debug,compiler,notests -- build the debug compiler and run smoke tests

After you build the first time you can open and use this solution:

.\VisualFSharp.sln
Expand All @@ -71,86 +70,7 @@ Building ``VisualFSharp.sln`` builds _nearly_ everything. However building porta
FSharp.Core.dll is not included. If you are just developing the core compiler, library
and Visual F# Tools then building the solution will be enough.

## Step by Step:

### 1. Building a Proto Compiler

The compiler is compiled as a set of .NET 4.0 components using a bootstrap process.
This uses a Last Known Good (LKG) compiler checked into this repository to build.

msbuild src\fsharp-proto-build.proj

### 2. Building an F# (Debug) library and compiler

This uses the proto compiler to build `FSharp.Core.dll`, `FSharp.Compiler.dll`, `fsc.exe`, and `fsi.exe`.

msbuild src/fsharp-library-build.proj
msbuild src/fsharp-compiler-build.proj

You can now use the updated F# compiler in `debug\net40\bin\fsc.exe` and F# Interactive in `debug\net40\bin\fsi.exe` to develop and test basic language and tool features.

**Note:** The updated library is not used until you run `update.cmd`, see below. The updated compiler is not run 'pre-compiled' until you run `update.cmd -ngen`, see below.

### 3. Full Steps Before Running Tests

See [TESTGUIDE.md](TESTGUIDE.md) for full details on how to run tests.

Prior to a full **Debug** test run, you need to complete **all** of the steps in build.cmd

build.cmd debug,build

Likewise prior to a **Release** test run:

build.cmd release,build

For **Debug** this corresponds to these steps, which you can run individually for more incremental builds:

msbuild src/fsharp-library-build.proj
msbuild src/fsharp-compiler-build.proj
msbuild src/fsharp-compiler-unittests-build.proj
msbuild src/fsharp-library-build.proj /p:TargetFramework=portable47
msbuild src/fsharp-library-build.proj /p:TargetFramework=portable7
msbuild src/fsharp-library-build.proj /p:TargetFramework=portable78
msbuild src/fsharp-library-build.proj /p:TargetFramework=portable259
msbuild src/fsharp-library-unittests-build.proj
msbuild src/fsharp-library-unittests-build.proj /p:TargetFramework=portable47
msbuild src/fsharp-library-unittests-build.proj /p:TargetFramework=portable7
msbuild src/fsharp-library-unittests-build.proj /p:TargetFramework=portable78
msbuild src/fsharp-library-unittests-build.proj /p:TargetFramework=portable259
msbuild vsintegration/fsharp-vsintegration-src-build.proj
msbuild vsintegration/fsharp-vsintegration-project-templates-build.proj
msbuild vsintegration/fsharp-vsintegration-item-templates-build.proj
msbuild vsintegration/fsharp-vsintegration-deployment-build.proj
msbuild vsintegration/fsharp-vsintegration-unittests-build.proj
msbuild tests/fsharp/FSharp.Tests.fsproj
src\update.cmd debug -ngen
tests\BuildTestTools.cmd debug


For **Release** this corresponds to these steps, which you can run individually for more incremental builds:

msbuild src/fsharp-library-build.proj /p:Configuration=Release
msbuild src/fsharp-compiler-build.proj /p:Configuration=Release
msbuild src/fsharp-compiler-unittests-build.proj /p:Configuration=Release
msbuild src/fsharp-library-build.proj /p:TargetFramework=portable47 /p:Configuration=Release
msbuild src/fsharp-library-build.proj /p:TargetFramework=portable7 /p:Configuration=Release
msbuild src/fsharp-library-build.proj /p:TargetFramework=portable78 /p:Configuration=Release
msbuild src/fsharp-library-build.proj /p:TargetFramework=portable259 /p:Configuration=Release
msbuild src/fsharp-library-unittests-build.proj /p:Configuration=Release
msbuild src/fsharp-library-unittests-build.proj /p:TargetFramework=portable47 /p:Configuration=Release
msbuild src/fsharp-library-unittests-build.proj /p:TargetFramework=portable7 /p:Configuration=Release
msbuild src/fsharp-library-unittests-build.proj /p:TargetFramework=portable78 /p:Configuration=Release
msbuild src/fsharp-library-unittests-build.proj /p:TargetFramework=portable259 /p:Configuration=Release
msbuild vsintegration/fsharp-vsintegration-src-build.proj /p:Configuration=Release
msbuild vsintegration/fsharp-vsintegration-project-templates-build.proj /p:Configuration=Release
msbuild vsintegration/fsharp-vsintegration-item-templates-build.proj /p:Configuration=Release
msbuild vsintegration/fsharp-vsintegration-deployment-build.proj /p:Configuration=Release
msbuild vsintegration/fsharp-vsintegration-unittests-build.proj /p:Configuration=Release
msbuild tests/fsharp/FSharp.Tests.fsproj /p:Configuration=Release
src\update.cmd release -ngen
tests\BuildTestTools.cmd release

### 4. [Optional] Install the Visual F# IDE Tools
### [Optional] Install the Visual F# IDE Tools

At time of writing, the Visual F# IDE Tools can only be installed into Visual Studio "Next" (aka "Dev15") releases.
The new builds of the Visual F# IDE Tools can no longer be installed into Visual Studio 2015.
Expand Down Expand Up @@ -184,7 +104,7 @@ For **Release**:

1. Run ``vsintegration\update-vsintegration.cmd release`` (clobbers the installed F# SDK)

### Notes on the build
### Notes on the .NET Framework build

1. The `update.cmd` script adds required strong name validation skips, and NGens the compiler and libraries. This requires admin privileges.
1. The compiler binaries produced are "private" and strong-named signed with a test key.
Expand All @@ -195,6 +115,9 @@ For **Release**:
- We use the proto compiler to compile the source for `FSharp.Core.dll` in this distribution.
- We use the proto compiler to compile the source for `FSharp.Compiler.dll`, `fsc.exe`, `fsi.exe`, and other binaries found in this distribution.




## Resources

The primary technical guide to the core compiler code is [The F# Compiler Technical Guide](http://fsharp.github.io/2015/09/29/fsharp-compiler-guide.html). Please read and contribute to that guide.
Expand Down
2 changes: 1 addition & 1 deletion DotnetCLIVersion.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
1.0.0-preview2-003121
1.0.0-preview3-003886
71 changes: 23 additions & 48 deletions TESTGUIDE.md
Original file line number Diff line number Diff line change
@@ -1,33 +1,21 @@
# F# Compiler, Core Library and Visual F# Tools Tests

## Prerequisites

In order to run the FSharpQA suite, you will need to install [Perl](http://www.perl.org/get.html) (ActiveState Perl 5.16.3 is known to work fine).
Perl must be included in the `%PATH%` for the below steps to work. It is also recommended that you run tests from an elevated command prompt, as there are a couple of test cases which require administrative privileges.

The Perl requirement is gradually being removed.

## Quick start: Running Tests

To run tests, use variations such as the following, depending on which test suite and build configuration you want:

build.cmd compiler,smoke
build.cmd compiler
build.cmd ci
build.cmd all
build.cmd debug,compiler
build.cmd debug,ci
build.cmd debug,all
build.cmd test
build.cmd net40 test
build.cmd coreclr test
build.cmd vs test
build.cmd all test

Default is `ci`
## Prerequisites

* ``ci`` = the build and tests done by continuous integration
* ``compiler`` = build the compiler
* ``compiler,smoke`` = build the compiler and run some smoke tests
* ``debug`` = use Debug configuration instead of Release
* ``pcls`` = build and test the Portable PCL libraries for FSharp.Core
* ``build_only`` = build, don't test
* ``all`` = build and test everything
In order to run the FSharpQA suite, you will need to install [Perl](http://www.perl.org/get.html) (ActiveState Perl 5.16.3 is known to work fine).
Perl must be included in the `%PATH%` for the below steps to work. It is also recommended that you run tests from an elevated command prompt, as there are a couple of test cases which require administrative privileges.

The Perl requirement is gradually being removed.

## Test Suites

Expand Down Expand Up @@ -68,45 +56,32 @@ extension or the command line via `nunit3-console.exe`.
Note that for compatibility reasons, the IDE unit tests should be run in a 32-bit process,
using the '--x86' flag to `nunit3-console.exe`

### RunTests.cmd

The script `tests\RunTests.cmd` is used to execute the suites. It's used like this:

RunTests.cmd <debug|release> fsharp [tags to run] [tags not to run]
RunTests.cmd <debug|release> fsharpqa [tags to run] [tags not to run]
RunTests.cmd <debug|release> compilerunit
RunTests.cmd <debug|release> coreunit
RunTests.cmd <debug|release> coreunitportable47
RunTests.cmd <debug|release> coreunitportable7
RunTests.cmd <debug|release> coreunitportable78
RunTests.cmd <debug|release> coreunitportable259
RunTests.cmd <debug|release> ideunit

`RunTests.cmd` sets a handful of environment variables which allow for the tests to work, then puts together and executes the appropriate command line to start the specified test suite.

All test execution logs and result files will be dropped into the `tests\TestResults` folder, and have file names matching `FSharp_*.*`, `FSharpQA_*.*`, `CompilerUnit_*.*`, `CoreUnit_*.*`, `IDEUnit_*.*`, e.g. `FSharpQA_Results.log` or `FSharp_Failures.log`.
### Test lists

For the FSharp and FSharpQA suites, the list of test areas and their associated "tags" is stored at

tests\test.lst // FSharp suite
tests\fsharpqa\source\test.lst // FSharpQA suite

Tags are in the left column, paths to to corresponding test folders are in the right column. If no tags are specified to `RunTests.cmd`, all tests will be run.
Tags are in the left column, paths to to corresponding test folders are in the right column. If no tags are specifie, all tests will be run.

If you want to re-run a particular test area, the easiest way to do so is to set a temporary tag for that area in test.lst (e.g. "RERUN").

If you want to re-run a particular test area, the easiest way to do so is to set a temporary tag for that area in test.lst (e.g. "RERUN"), then call `RunTests.cmd <debug|release> <fsharp|fsharpqa> RERUN`.
### Logs and output

If you want to specify multiple tags to run or not run, pass them comma-delimited and enclosed in double quotes, e.g. `RunTests.cmd debug fsharp "Core01,Core02"`.
From a Powershell environment, make sure the double quotes are passed literally, e.g. `.\RunTests.cmd debug fsharp '"Core01,Core02"'`
or `.\RunTests.cmd --% debug fsharp "Core01,Core02"`.
All test execution logs and result files will be dropped into the `tests\TestResults` folder, and have file names matching

`RunTests.cmd` is mostly just a simple wrapper over `tests\fsharpqa\testenv\bin\RunAll.pl`, which has capabilities not discussed here. More advanced test execution scenarios can be achieved by invoking `RunAll.pl` directly.
Run `perl tests\fsharpqa\testenv\bin\RunAll.pl -?` to see a full list of flags and options.
net40-fsharp-suite-*.*
net40-fsharpqa-suite-*.*
net40-compilerunit-suite-*.*
net40-coreunit-suite-*.*
vs-ideunit-suite-*.*

### Other Tips

* Run as Administrator, or a handful of tests will fail

* Making the tests run faster
* NGen-ing the F# bits (fsc, fsi, FSharp.Core, etc) will result in tests executing much faster. Make sure you run `src\update.cmd` with the `-ngen` flag before running tests.
* The FSharp and FSharpQA suites will run test cases in parallel by default. You can comment out the relevant line in `RunTests.cmd` (look for `PARALLEL_ARG`) to disable this.
* By default, tests from the FSharpQA suite are run using a persistent, hosted version of the compiler. This speeds up test execution, as there is no need for the `fsc.exe` process to spin up repeatedly. To disable this, uncomment the relevant line in `RunTests.cmd` (look for `HOSTED_COMPILER`).
* The FSharp and FSharpQA suites will run test cases in parallel by default. You can comment out the relevant line (look for `PARALLEL_ARG`) to disable this.
* By default, tests from the FSharpQA suite are run using a persistent, hosted version of the compiler. This speeds up test execution, as there is no need for the `fsc.exe` process to spin up repeatedly. To disable this, uncomment the relevant line (look for `HOSTED_COMPILER`).
Loading

0 comments on commit 500c342

Please sign in to comment.