Skip to content

Commit

Permalink
cleanup build and test scripts (dotnet#1615)
Browse files Browse the repository at this point in the history 
* cleanup build and test scripts
  • Loading branch information
@dsyme
dsyme committed Oct 17, 2016
1 parent a867877 commit 3808595
Show file tree
Hide file tree
Showing 51 changed files with 1,065 additions and 7,794 deletions.
6 changes: 3 additions & 3 deletions .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -49,13 +49,13 @@
/tests/*FSharp_Failures.lst
/tests/fsharpqa/Source/CodeGen/EmittedIL/StaticInit/StaticInit_Module01.dll
/tests/fsharpqa/Source/CodeGen/EmittedIL/StaticInit/StaticInit_Module01.pdb
/tests/XFSharpQA_Failures.log.*
/tests/Xnet40-fsharpqa-suite-failures.log.*
/vsintegration/src/vs/FsPkgs/FSharp.Project/FS/FSharp.ProjectSystem.FSharp.fsi
/vsintegration/src/vs/FsPkgs/FSharp.Project/FS/ctofiles/
/tests/fsharpqa/Source/CodeGen/EmittedIL/QueryExpressionStepping/Utils.dll
/tests/fsharpqa/Source/CodeGen/EmittedIL/ComputationExpressions/ComputationExprLibrary.dll
/tests/fsharpqa/Source/*FSharpQA_Failures.env
/tests/fsharpqa/Source/*FSharpQA_Failures.lst
/tests/fsharpqa/Source/*net40-fsharpqa-suite-failures.env
/tests/fsharpqa/Source/*net40-fsharpqa-suite-failures.lst
/tests/**/FSharp.Core.dll
lib/debug
lib/release
Expand Down
113 changes: 17 additions & 96 deletions DEVGUIDE.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,44 +21,44 @@ The primary technical documents for the F# compiler code are

## Quick Start: Build, Test, Develop

You can build the compiler+tools and run the subset the tests used for continuous integration as follows:
You can build the F# compiler for .NET Framework as follows:

build.cmd

This is the same as

build.cmd net40

There are various qualifiers:

build.cmd release -- build release (the default)
build.cmd debug -- build debug instead of release

build.cmd proto -- force the rebuild of the Proto bootstrap compiler in addition to other things
build.cmd protofx -- build using a .NET Framework proto (no .NET Core is used)
build.cmd net40 -- build .NET Framework compiler
build.cmd coreclr -- build .NET Core compiler
build.cmd vs -- build the Visual F# IDE Tools
build.cmd pcls -- build the PCL FSharp.Core libraries
build.cmd all -- build all

build.cmd net40 -- build/tests for .NET Framework version of the compiler (not the Visual F# IDE Tools or .NET Core)
build.cmd coreclr -- build/tests only the coreclr version compiler (not the Visual F# IDE Tools or .NET Framework)
build.cmd vs -- build/tests the Visual F# IDE Tools
build.cmd pcls -- build/tests the PCL FSharp.Core libraries
build.cmd proto -- force the rebuild of the Proto bootstrap compiler in addition to other things

build.cmd build -- build, do not test
build.cmd ci -- build, run the same tests as CI
build.cmd all -- build, run all tests
build.cmd notests -- turn off testing (used in conjunction with other options)
build.cmd test -- build default targets, run suitable tests
build.cmd net40 test -- build net40, run suitable tests
build.cmd coreclr test -- build coreclr, run suitable tests
build.cmd vs test -- build Visual F# IDE Tools, run all tests
build.cmd all test -- build all, run all tests

build.cmd test-smoke -- build, run smoke tests
build.cmd test-coreunit -- build, run FSharp.Core tests
build.cmd test-coreclr -- build, run .NET Core tests
build.cmd test-pcls -- build, run PCL tests
build.cmd test-fsharp -- build, run tests\fsharp suite
build.cmd test-fsharpqa -- build, run tests\fsharpqa suite
build.cmd test-vs -- build, run Visual F# IDE Tools unit tests

**Notes**
To build and test Visual F# IDE Tools, you must use [Visual Studio "vNext" (aka "Dev15")](https://www.visualstudio.com/en-us/downloads/visual-studio-next-downloads-vs.aspx). This is the one after Visual Studio 2015 (aka "Dev 14"). You must also install Visual Studio SDK (also called _Visual Studio Extensibility SDK_ on the Visual Studio installer) before building Visual F# IDE Tools.
Please ensure that the Visual Studio SDK version is matched with your current Visual Studio to ensure successful builds. For example: Visual Studio 2015 Update 1 requires Visual Studio 2015 SDK Update 1. Any installation of Visual Studio 2015 and later provides Visual Studio SDK as part of the installation of Visual Studio 2015 as feature installation.

Combinations are also allowed:

build.cmd debug,compiler,notests -- build the debug compiler and run smoke tests

After you build the first time you can open and use this solution:

.\VisualFSharp.sln
Expand All @@ -71,86 +71,7 @@ Building ``VisualFSharp.sln`` builds _nearly_ everything. However building porta
FSharp.Core.dll is not included. If you are just developing the core compiler, library
and Visual F# Tools then building the solution will be enough.

## Step by Step:

### 1. Building a Proto Compiler

The compiler is compiled as a set of .NET 4.0 components using a bootstrap process.
This uses a Last Known Good (LKG) compiler checked into this repository to build.

msbuild src/fsharp-proto-build.proj

### 2. Building an F# (Debug) library and compiler

This uses the proto compiler to build `FSharp.Core.dll`, `FSharp.Compiler.dll`, `fsc.exe`, and `fsi.exe`.

msbuild src/fsharp-library-build.proj
msbuild src/fsharp-compiler-build.proj

You can now use the updated F# compiler in `debug\net40\bin\fsc.exe` and F# Interactive in `debug\net40\bin\fsi.exe` to develop and test basic language and tool features.

**Note:** The updated library is not used until you run `update.cmd`, see below. The updated compiler is not run 'pre-compiled' until you run `update.cmd -ngen`, see below.

### 3. Full Steps Before Running Tests

See [TESTGUIDE.md](TESTGUIDE.md) for full details on how to run tests.

Prior to a full **Debug** test run, you need to complete **all** of the steps in build.cmd

build.cmd debug,build

Likewise prior to a **Release** test run:

build.cmd release,build

For **Debug** this corresponds to these steps, which you can run individually for more incremental builds:

msbuild src/fsharp-library-build.proj
msbuild src/fsharp-compiler-build.proj
msbuild src/fsharp-compiler-unittests-build.proj
msbuild src/fsharp-library-build.proj /p:TargetFramework=portable47
msbuild src/fsharp-library-build.proj /p:TargetFramework=portable7
msbuild src/fsharp-library-build.proj /p:TargetFramework=portable78
msbuild src/fsharp-library-build.proj /p:TargetFramework=portable259
msbuild src/fsharp-library-unittests-build.proj
msbuild src/fsharp-library-unittests-build.proj /p:TargetFramework=portable47
msbuild src/fsharp-library-unittests-build.proj /p:TargetFramework=portable7
msbuild src/fsharp-library-unittests-build.proj /p:TargetFramework=portable78
msbuild src/fsharp-library-unittests-build.proj /p:TargetFramework=portable259
msbuild vsintegration/fsharp-vsintegration-src-build.proj
msbuild vsintegration/fsharp-vsintegration-project-templates-build.proj
msbuild vsintegration/fsharp-vsintegration-item-templates-build.proj
msbuild vsintegration/fsharp-vsintegration-deployment-build.proj
msbuild vsintegration/fsharp-vsintegration-unittests-build.proj
msbuild tests/fsharp/FSharp.Tests.fsproj
src\update.cmd debug -ngen
tests\BuildTestTools.cmd debug


For **Release** this corresponds to these steps, which you can run individually for more incremental builds:

msbuild src/fsharp-library-build.proj /p:Configuration=Release
msbuild src/fsharp-compiler-build.proj /p:Configuration=Release
msbuild src/fsharp-compiler-unittests-build.proj /p:Configuration=Release
msbuild src/fsharp-library-build.proj /p:TargetFramework=portable47 /p:Configuration=Release
msbuild src/fsharp-library-build.proj /p:TargetFramework=portable7 /p:Configuration=Release
msbuild src/fsharp-library-build.proj /p:TargetFramework=portable78 /p:Configuration=Release
msbuild src/fsharp-library-build.proj /p:TargetFramework=portable259 /p:Configuration=Release
msbuild src/fsharp-library-unittests-build.proj /p:Configuration=Release
msbuild src/fsharp-library-unittests-build.proj /p:TargetFramework=portable47 /p:Configuration=Release
msbuild src/fsharp-library-unittests-build.proj /p:TargetFramework=portable7 /p:Configuration=Release
msbuild src/fsharp-library-unittests-build.proj /p:TargetFramework=portable78 /p:Configuration=Release
msbuild src/fsharp-library-unittests-build.proj /p:TargetFramework=portable259 /p:Configuration=Release
msbuild vsintegration/fsharp-vsintegration-src-build.proj /p:Configuration=Release
msbuild vsintegration/fsharp-vsintegration-project-templates-build.proj /p:Configuration=Release
msbuild vsintegration/fsharp-vsintegration-item-templates-build.proj /p:Configuration=Release
msbuild vsintegration/fsharp-vsintegration-deployment-build.proj /p:Configuration=Release
msbuild vsintegration/fsharp-vsintegration-unittests-build.proj /p:Configuration=Release
msbuild tests/fsharp/FSharp.Tests.fsproj /p:Configuration=Release
src\update.cmd release -ngen
tests\BuildTestTools.cmd release

### 4. [Optional] Install the Visual F# IDE Tools
### [Optional] Install the Visual F# IDE Tools

At time of writing, the Visual F# IDE Tools can only be installed into Visual Studio "Next" (aka "Dev15") releases.
The new builds of the Visual F# IDE Tools can no longer be installed into Visual Studio 2015.
Expand Down
63 changes: 19 additions & 44 deletions TESTGUIDE.md
View file
Original file line number Diff line number Diff line change
@@ -1,33 +1,21 @@
# F# Compiler, Core Library and Visual F# Tools Tests

## Prerequisites

In order to run the FSharpQA suite, you will need to install [Perl](http://www.perl.org/get.html) (ActiveState Perl 5.16.3 is known to work fine).
Perl must be included in the `%PATH%` for the below steps to work. It is also recommended that you run tests from an elevated command prompt, as there are a couple of test cases which require administrative privileges.

The Perl requirement is gradually being removed.

## Quick start: Running Tests

To run tests, use variations such as the following, depending on which test suite and build configuration you want:

build.cmd compiler,smoke
build.cmd compiler
build.cmd ci
build.cmd all
build.cmd debug,compiler
build.cmd debug,ci
build.cmd debug,all
build.cmd test
build.cmd net40 test
build.cmd coreclr test
build.cmd vs test
build.cmd all test

Default is `ci`
## Prerequisites

* ``ci`` = the build and tests done by continuous integration
* ``compiler`` = build the compiler
* ``compiler,smoke`` = build the compiler and run some smoke tests
* ``debug`` = use Debug configuration instead of Release
* ``pcls`` = build and test the Portable PCL libraries for FSharp.Core
* ``build_only`` = build, don't test
* ``all`` = build and test everything
In order to run the FSharpQA suite, you will need to install [Perl](http://www.perl.org/get.html) (ActiveState Perl 5.16.3 is known to work fine).
Perl must be included in the `%PATH%` for the below steps to work. It is also recommended that you run tests from an elevated command prompt, as there are a couple of test cases which require administrative privileges.

The Perl requirement is gradually being removed.

## Test Suites

Expand Down Expand Up @@ -68,23 +56,7 @@ extension or the command line via `nunit3-console.exe`.
Note that for compatibility reasons, the IDE unit tests should be run in a 32-bit process,
using the '--x86' flag to `nunit3-console.exe`

### RunTests.cmd

The script `tests\RunTests.cmd` is used to execute the suites. It's used like this:

RunTests.cmd <debug|release> fsharp [tags to run] [tags not to run]
RunTests.cmd <debug|release> fsharpqa [tags to run] [tags not to run]
RunTests.cmd <debug|release> compilerunit
RunTests.cmd <debug|release> coreunit
RunTests.cmd <debug|release> coreunitportable47
RunTests.cmd <debug|release> coreunitportable7
RunTests.cmd <debug|release> coreunitportable78
RunTests.cmd <debug|release> coreunitportable259
RunTests.cmd <debug|release> ideunit

`RunTests.cmd` sets a handful of environment variables which allow for the tests to work, then puts together and executes the appropriate command line to start the specified test suite.

All test execution logs and result files will be dropped into the `tests\TestResults` folder, and have file names matching `FSharp_*.*`, `FSharpQA_*.*`, `CompilerUnit_*.*`, `CoreUnit_*.*`, `IDEUnit_*.*`, e.g. `FSharpQA_Results.log` or `FSharp_Failures.log`.
### Test lists

For the FSharp and FSharpQA suites, the list of test areas and their associated "tags" is stored at

Expand All @@ -95,12 +67,15 @@ Tags are in the left column, paths to to corresponding test folders are in the r

If you want to re-run a particular test area, the easiest way to do so is to set a temporary tag for that area in test.lst (e.g. "RERUN"), then call `RunTests.cmd <debug|release> <fsharp|fsharpqa> RERUN`.

If you want to specify multiple tags to run or not run, pass them comma-delimited and enclosed in double quotes, e.g. `RunTests.cmd debug fsharp "Core01,Core02"`.
From a Powershell environment, make sure the double quotes are passed literally, e.g. `.\RunTests.cmd debug fsharp '"Core01,Core02"'`
or `.\RunTests.cmd --% debug fsharp "Core01,Core02"`.
### Logs and output

All test execution logs and result files will be dropped into the `tests\TestResults` folder, and have file names matching

`RunTests.cmd` is mostly just a simple wrapper over `tests\fsharpqa\testenv\bin\RunAll.pl`, which has capabilities not discussed here. More advanced test execution scenarios can be achieved by invoking `RunAll.pl` directly.
Run `perl tests\fsharpqa\testenv\bin\RunAll.pl -?` to see a full list of flags and options.
net40-fsharp-suite-*.*
net40-fsharpqa-suite-*.*
net40-compilerunit-suite-*.*
net40-coreunit-suite-*.*
vs-ideunit-suite-*.*

### Other Tips

Expand Down
Loading

0 comments on commit 3808595

Please sign in to comment.