Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

<!-- Empty to inhibit searching up the directory tree. -->

</Project>
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

<PropertyGroup>
<ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
<CentralPackageTransitivePinningEnabled>true</CentralPackageTransitivePinningEnabled>
</PropertyGroup>

<ItemGroup>
<!-- <PackageVersion Include="Microsoft.Data.SqlClient" Version="6.0.2" /> -->
<PackageVersion Include="Microsoft.Data.SqlClient" Version="6.1.0-preview2.25178.5" />
</ItemGroup>

</Project>
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
<PropertyGroup>
<RootNamespace>Monitoring</RootNamespace>
<AssemblyName>Monitoring</AssemblyName>
<TargetFrameworks>net9.0;net48</TargetFrameworks>
<TargetFrameworks>net481;net9.0</TargetFrameworks>
<LangVersion>latest</LangVersion>
</PropertyGroup>
</Project>
</Project>
163 changes: 91 additions & 72 deletions src/Microsoft.Data.SqlClient/tests/StressTests/Readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,163 +2,182 @@

This Stress testing application for `Microsoft.Data.SqlClient` is under progress.
This project intends to help finding a certain level of effectiveness under unfavorable conditions, and verifying the mode of failures.
This is a console application with targeting frameworks `.Net Framework 4.8`, `.NET 9.0` under driver's supported operating systems and SQL Servers.
This is a console application with targeting frameworks `.Net Framework 4.8.1`, `.NET 9.0` under driver's supported operating systems and SQL Servers.

## Purpose of application for developers

Define fuzz tests for all new features/APIs in the driver and to be run before every GA release.

# Pre-Requisites
## Pre-Requisites

required in StressTest.config:
Required in StressTest.config:

"name": stress testing source configuration name.
"type": only `SqlServer` is acceptable.
"isDefault": If there is a source node with `isDefault=true`, this node is returned.
"dataSource": SQL Server data source name.
"database": targeting database name in the SQL Server.
"user": user Id to connect the server.
"password": paired password with the user.
"supportsWindowsAuthentication": tries to use integrated security in connection string mixed with SQL Server authentication if it set to `true` by applying the randomization.
"isLocal": `true` means database is local.
"disableMultiSubnetFailover": tries to add Multi-subnet Failover fake host entries when it equals `true`,
"disableNamedPipes": `true` means the connections will create just using tcp protocol.
"encrypt": assigns the encrypt property of the connection strings.
|Field|Values|Description|
|-|-|-|
|`name`||Stress testing source configuration name.|
|`type`|`SqlServer`|Only `SqlServer` is acceptable.|
|`isDefault`|`true`, `false`|If there is a source node with `isDefault=true`, this node is returned.|
|`dataSource`||SQL Server data source name.|
|`database`||Targeting database name in the SQL Server.|
|`user`||User Id to connect the server.|
|`password`||Paired password with the user.|
|`supportsWindowsAuthentication`|`true`, `false`|Tries to use integrated security in connection string mixed with SQL Server authentication if it set to `true` by applying the randomization.|
|`isLocal`|`true`, `false`|`true` means database is local.|
|`disableMultiSubnetFailover`|`true`, `false`|Tries to add Multi-subnet Failover fake host entries when it equals `true`.|
|`disableNamedPipes`|`true`, `false`|`true` means the connections will create just using tcp protocol.|
|`encrypt`|`true`, `false`|Assigns the encrypt property of the connection strings.|

# Adding new Tests
- [ToDO]
## Adding new Tests

# Building the application
- [ToDo]

To build the application we need to run the command: 'dotnet build <-f|--framework <FRAMEWORK>> [-c|--configuration <Release|Debug>]'
The path should be pointing to SqlClient.Stress.Runner.csproj file.
## Building the application

To build the application we need to run the command:

```bash
# Default Build Configuration:
dotnet build <-f|--framework <FRAMEWORK>> [-c|--configuration <Release|Debug>]
```

> dotnet build
The path should be pointing to SqlClient.Stress.Runner.csproj file.

```bash
# Builds the application for the Client Os in `Debug` Configuration for `AnyCpu` platform.
# All supported target frameworks, .NET Framework (NetFx) and .NET Core drivers are built by default (as supported by Client OS).

> dotnet build
```

```bash
> dotnet build -f netcoreapp3.1
# Build the application for .Net core 3.1 with `Debug` configuration.
# Build the application for .Net framework 4.8.1 with `Debug` configuration.

> dotnet build -f net48
# Build the application for .Net framework 4.8 with `Debug` configuration.
> dotnet build -f net481
```

```bash
> dotnet build -f net5.0 -c Release
# Build the application for .Net 5.0 with `Release` configuration.
# Build the application for .Net 9.0 with `Release` configuration.

> dotnet build -f net48 -c Release
# Build the application for .Net framework 4.8 with `Release` configuration.
> dotnet build -f net9.0 -c Release
```

```bash
> dotnet clean
# Cleans all build directories

> dotnet clean
```

# Running tests
## Running tests

After building the application, find the built folder with target framework and run the `stresstest.exe` file with required arguments.
Find the result in a log file inside the `logs` folder besides the command prompt.

## Command prompt

```bash
> stresstest.exe [-a <module name>] <arguments>

-a <module name> should specify path to the assembly containing the tests.
```

## Supported arguments

-all Run all tests - best for debugging, not perf measurements.
You must run the stress tests from the root of the Stress Tests project
directory (i.e. the same directory this readme file is in).

-verify Run in functional verification mode. [not implemented]

-duration <n> Duration of the test in seconds. Default value is 1 second.

-threads <n> Number of threads to use. Default value is 16.

-override <name> <value> Override the value of a test property.

-test <name1;name2> Run specific test(s).
```bash
# Linux
$ cd /home/paul/dev/SqlClient/src/Microsoft.Data.SqlClient/tests/StressTests

-debug Print process ID in the beginning and wait for Enter (to give your time to attach the debugger).
# Via dotnet run CLI:
$ dotnet run -no-build -f net9.0 --project SqlClient.Stress.Runner/SqlClient.Stress.Runner.csproj -- -a SqlClient.Stress.Tests

-exceptionThreshold <n> An optional limit on exceptions which will be caught. When reached, test will halt.
# Via dotnet CLI:
$ dotnet SqlClient.Stress.Runner/bin/Debug/net9.0/stresstest.dll -a SqlClient.Stress.Tests
```

-monitorenabled True or False to enable monitoring. Default is false [not implemented]
```powershell
# Windows
> cd \dev\SqlClient\src\Microsoft.Data.SqlClient\tests\StressTests

-randomSeed Enables setting of the random number generator used internally. This serves both the purpose
of helping to improve reproducibility and making it deterministic from Chess's perspective
for a given schedule. Default is 0.
# Via dotnet run CLI:
$ dotnet run -no-build -f net9.0 --project SqlClient.Stress.Runner\SqlClient.Stress.Runner.csproj -- -a SqlClient.Stress.Tests

-filter Run tests whose stress test attributes match the given filter. Filter is not applied if attribute
does not implement ITestAttributeFilter. Example: -filter TestType=Query,Update;IsServerTest=True
# Via executable:
> .\SqlClient.Stress.Runner\bin\Debug\net481\stresstest.exe -a SqlClient.Stress.Tests
```

-printMethodName Print tests' title in console window
## Supported arguments

-deadlockdetection True or False to enable deadlock detection. Default is `false`.
|Argument|Values|Description|
|-|-|-|
|-all||Run all tests - best for debugging, not perf measurements.|
|-verify||Run in functional verification mode. [not implemented]|
|-duration|&lt;n&gt;|Duration of the test in seconds. Default value is 1 second.|
|-threads|&lt;n&gt;|Number of threads to use. Default value is 16.|
|-override|&lt;name&gt; &lt;value&gt;|Override the value of a test property.|
|-test|&lt;name1;name2&gt;|Run specific test(s).|
|-debug||Print process ID in the beginning and wait for Enter (to give your time to attach the debugger).|
|-exceptionThreshold|&lt;n&gt;|An optional limit on exceptions which will be caught. When reached, test will halt.|
|-monitorenabled|true, false|True or False to enable monitoring. Default is false [not implemented]|
|-randomSeed||Enables setting of the random number generator used internally. This serves both the purpose of helping to improve reproducibility and making it deterministic from Chess's perspective for a given schedule. Default is 0.|
|-filter|&lt;filter&gt;|Run tests whose stress test attributes match the given filter. Filter is not applied if attribute does not implement ITestAttributeFilter. Example: -filter TestType=Query,Update;IsServerTest=True|
|-printMethodName||Print tests' title in console window|
|-deadlockdetection|true, false|True or False to enable deadlock detection. Default is `false`.|

```bash
> stresstest.exe -a SqlClient.Stress.Tests -all
# Run the application for a built target framework and all discovered tests without debugger attached.

> .\stresstest.exe -a SqlClient.Stress.Tests -all
```

```bash
> stresstest.exe -a SqlClient.Stress.Tests -all -printMethodName
# Run the application for a built target framework and all discovered tests without debugger attached and shows the test methods' names.

> .\stresstest.exe -a SqlClient.Stress.Tests -all -printMethodName
```

```bash
> stresstest.exe -a SqlClient.Stress.Tests -all -debug
# Run the application for a built target framework and all discovered tests and will wait for debugger to be attached.

> .\stresstest.exe -a SqlClient.Stress.Tests -all -debug
```

```bash
> stresstest.exe -a SqlClient.Stress.Tests -test TestExecuteXmlReaderAsyncCancellation
# Run the application for a built target framework and "TestExecuteXmlReaderAsyncCancellation" test without debugger attached.

> .\stresstest.exe -a SqlClient.Stress.Tests -test TestExecuteXmlReaderAsyncCancellation
```

```bash
> stresstest.exe -a SqlClient.Stress.Tests -test TestExecuteXmlReaderAsyncCancellation
# Run the application for a built target framework and "TestExecuteXmlReaderAsyncCancellation" test without debugger attached.

> .\stresstest.exe -a SqlClient.Stress.Tests -test TestExecuteXmlReaderAsyncCancellation
```

```bash
> stresstest.exe -a SqlClient.Stress.Tests -all -duration 10
# Run the application for a built target framework and all discovered tests without debugger attached for 10 seconds.

> .\stresstest.exe -a SqlClient.Stress.Tests -all -duration 10
```

```bash
> stresstest.exe -a SqlClient.Stress.Tests -all -threads 5
# Run the application for a built target framework and all discovered tests without debugger attached with 5 threads.

> .\stresstest.exe -a SqlClient.Stress.Tests -all -threads 5
```

```bash
> stresstest.exe -a SqlClient.Stress.Tests -all -deadlockdetection true
# Run the application for a built target framework and all discovered tests without debugger attached and dead lock detection process.

> .\stresstest.exe -a SqlClient.Stress.Tests -all -deadlockdetection true
```

```bash
> stresstest.exe -a SqlClient.Stress.Tests -all -override Weight 15
# Run the application for a built target framework and all discovered tests without debugger attached with overriding the weight property with value 15.

> .\stresstest.exe -a SqlClient.Stress.Tests -all -override Weight 15
```

```bash
> stresstest.exe -a SqlClient.Stress.Tests -all -randomSeed 5
# Run the application for a built target framework and all discovered tests without debugger attached with injecting random seed of 5.

> .\stresstest.exe -a SqlClient.Stress.Tests -all -randomSeed 5
```

# Further thoughts
## Further thoughts

- Implement the uncompleted arguments.
- Add support `dotnet run` command.
- Add more tests.
- Add support running tests with **System.Data.SqlClient** too.
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<TargetFrameworks>net9.0;net48</TargetFrameworks>
<TargetFrameworks>net481;net9.0</TargetFrameworks>
<LangVersion>latest</LangVersion>
</PropertyGroup>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -789,7 +789,7 @@ protected void BuildUserTables(List<TableMetadata> TableMetadataList)
}
else
{
throw de;
throw;
}
}

Expand All @@ -808,7 +808,7 @@ protected void BuildUserTables(List<TableMetadata> TableMetadataList)
}
else
{
throw de;
throw;
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -158,10 +158,8 @@ protected bool OpenConnection(DataStressConnection conn)
// MultiSubnetFailover=true only works with TCP/IP protocol and will result in exception when using with named pipes.
return false;
}
else
{
throw e;
}

throw;
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,12 @@

<PropertyGroup>
<RootNamespace>Stress.Data</RootNamespace>
<TargetFrameworks>net9.0;net48</TargetFrameworks>
<TargetFrameworks>net481;net9.0</TargetFrameworks>
<LangVersion>latest</LangVersion>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="Microsoft.Data.SqlClient" Version="6.1.0-preview2.25178.5" />
<PackageReference Include="Microsoft.Data.SqlClient" />
</ItemGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,18 +3,18 @@
<dataStressSettings>
<sources>
<source
name="LocalSqlServer"
name="My Favourite SQL Server"
type="SqlServer"
isDefault="true"
dataSource="(local)"
database="stress"
dataSource="127.0.0.1,1433"
database="master"
user="sa"
password=""
supportsWindowsAuthentication="true"
isLocal='true'
disableMultiSubnetFailover = 'true'
disableNamedPipes = 'true'
encrypt ='false'>
supportsWindowsAuthentication="false"
isLocal="false"
disableMultiSubnetFailover = "true"
disableNamedPipes = "true"
encrypt ="false">
</source>
</sources>
</dataStressSettings>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
<PropertyGroup>
<OutputType>Exe</OutputType>
<AssemblyName>stresstest</AssemblyName>
<TargetFrameworks>net9.0;net48</TargetFrameworks>
<TargetFrameworks>net481;net9.0</TargetFrameworks>
<LangVersion>latest</LangVersion>
</PropertyGroup>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,12 @@ public abstract class TestBase
private TestAttributeBase _attr;
private string _variationSuffix = "";

[System.CLSCompliantAttribute(false)]
protected MethodInfo _testMethod;
[System.CLSCompliantAttribute(false)]

protected Type _type;

[System.CLSCompliantAttribute(false)]
protected List<MethodInfo> _setupMethods;

[System.CLSCompliantAttribute(false)]
protected List<MethodInfo> _cleanupMethods;

protected delegate void TestMethodDelegate(object t);
Expand Down
Loading
Loading