Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test run parameter added as part of CLI runsettings args #2251

Merged
merged 9 commits into from
Dec 27, 2019
Merged

Test run parameter added as part of CLI runsettings args #2251

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
87e33cc
test run parameter added as second class argument
Nov 14, 2019
b1d894b
regex is used to match test run paramterer node
Nov 15, 2019
6b7bd90
added exception for invaild test run parameter argument
Nov 15, 2019
84c8c8f
regex constants moved to constants
Nov 20, 2019
1514575
unit tests are added for regex pattern match
Nov 28, 2019
d9e90f7
added special character test cases
Nov 28, 2019
561e297
formatted
Nov 29, 2019
e5f4e7c
resource string is changed
Nov 29, 2019
68e5412
formatted
Nov 29, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 83 additions & 1 deletion src/Microsoft.TestPlatform.Common/Utilities/RunSettingsProviderExtensions.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ namespace Microsoft.VisualStudio.TestPlatform.Common.Utilities
using Microsoft.VisualStudio.TestPlatform.ObjectModel;
using Microsoft.VisualStudio.TestPlatform.ObjectModel.Utilities;
using Microsoft.VisualStudio.TestPlatform.Common;
using System.Collections.Generic;
using System.Text.RegularExpressions;

/// <summary>
/// Utilities to get the run settings from the provider and the commandline options specified.
Expand All @@ -20,6 +22,31 @@ internal static class RunSettingsProviderExtensions
{
public const string EmptyRunSettings = @"<RunSettings><RunConfiguration></RunConfiguration></RunSettings>";

/// <summary>
/// Pattern used to find parameter node.
/// </summary>
private const string ParameterString = "Parameter";

/// <summary>
/// Pattern that indicates Attribute name.
/// </summary>
private const string AttributeNameString = "AttrName";

/// <summary>
/// Pattern that indicates Attribute value.
/// </summary>
private const string AttributeValueString = "AttrValue";

/// <summary>
/// Attribute name key for test run parameter node
/// </summary>
private const string NameString = "name";

/// <summary>
/// Attribute value key for test run parameter node
/// </summary>
private const string ValueString = "value";

public static void UpdateRunSettings(this IRunSettingsProvider runSettingsProvider, string runsettingsXml)
{
ValidateArg.NotNull(runSettingsProvider, nameof(runSettingsProvider));
Expand Down Expand Up @@ -62,6 +89,61 @@ public static void UpdateRunSettingsNode(this IRunSettingsProvider runSettingsPr
runSettingsProvider.UpdateRunSettings(xmlDocument.OuterXml);
}

/// <summary>
/// Matches with test run parameter node pattern and returns that match.
/// </summary>
/// <param name="runSettingsProvider"></param>
/// <param name="node"></param>
/// <returns></returns>
public static Match GetTestRunParameterNodeMatch(this IRunSettingsProvider runSettingsProvider, string node)
{
var attrName = $"(?<{AttributeNameString}>\\w+)";
var attrValue = $"(?<{AttributeValueString}>.+)";
Regex regex = new Regex($"{Constants.TestRunParametersName}.{ParameterString}\\(name\\s*=\\s*\"{attrName}\"\\s*,\\s*value\\s*=\\s*\"{attrValue}\"\\)");
Match match = regex.Match(node);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can throw exception, either catch here, or in the caller.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are four argument exceptions possible
1.ArgumentOutOfRangeException : This comes if options is not a valid bitwise combination, but we have not given options in match
2.ArgumentNullException : This commes if input or pattern is null, but input is always not null and pattern is not null.
3.RegexMatchTimeoutException :This commes if timout occured,but as Pattern is fixed and we have not set timeout in match. We won't get this exceptions
4.ArgumentException : This comes if regular expression is not syntactically correct,This too not valid in our case..
so catching an exception is not that reasonable

return match;
}

/// <summary>
/// If test run parameter exists already it will override with new value otherwise this will add new test run parameter.
/// </summary>
/// <param name="runSettingsProvider"></param>
/// <param name="match"></param>
public static void UpdateTestRunParameterSettingsNode(this IRunSettingsProvider runSettingsProvider, Match match)
{
ValidateArg.NotNull(runSettingsProvider, nameof(runSettingsProvider));

var xmlDocument = runSettingsProvider.GetRunSettingXmlDocument();
XmlNode testRunParameterNode = GetXmlNode(xmlDocument, Constants.TestRunParametersName) ?? xmlDocument.CreateElement(Constants.TestRunParametersName);
var attrName = match.Groups[AttributeNameString].Value;
var attrValue = match.Groups[AttributeValueString].Value;

if (!TryOverRideAttributeValue(testRunParameterNode, attrName, attrValue))
hvinett marked this conversation as resolved.
Show resolved Hide resolved
{
XmlElement element = xmlDocument.CreateElement(ParameterString);
element.SetAttribute(NameString, attrName);
element.SetAttribute(ValueString, attrValue);
testRunParameterNode.AppendChild(element);
xmlDocument.DocumentElement.AppendChild(testRunParameterNode);
}

runSettingsProvider.UpdateRunSettings(xmlDocument.OuterXml);
}

private static bool TryOverRideAttributeValue(XmlNode xmlNode, string attrName, string attrValue)
{
foreach (XmlNode node in xmlNode.ChildNodes)
{
if (string.Compare(node.Attributes[NameString].Value, attrName) == 0)
{
node.Attributes[ValueString].Value = attrValue;
return true;
}
}

return false;
}

public static void UpdateRunSettingsNodeInnerXml(this IRunSettingsProvider runSettingsProvider, string key, string xml)
{
ValidateArg.NotNull(runSettingsProvider, nameof(runSettingsProvider));
Expand Down Expand Up @@ -133,7 +215,7 @@ private static XmlNode CreateNode(XmlDocument doc, string xPath)
}

return node;
}
}

private static XmlDocument GetRunSettingXmlDocument(this IRunSettingsProvider runSettingsProvider)
{
Expand Down
3 changes: 3 additions & 0 deletions src/Microsoft.TestPlatform.ObjectModel/Constants.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,9 @@ public static class Constants

public const string DataCollectorSettingName = "DataCollector";

/// <summary>
/// Pattern used to find test run parameter node.
/// </summary>
public const string TestRunParametersName = "TestRunParameters";

/// <summary>
Expand Down
38 changes: 33 additions & 5 deletions src/vstest.console/Processors/CLIRunSettingsArgumentProcessor.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ namespace Microsoft.VisualStudio.TestPlatform.CommandLine.Processors
using Microsoft.VisualStudio.TestPlatform.Common.Utilities;

using CommandLineResources = Microsoft.VisualStudio.TestPlatform.CommandLine.Resources.Resources;
using System.Text.RegularExpressions;

/// <summary>
/// The argument processor for runsettings passed as argument through cli
Expand Down Expand Up @@ -139,14 +140,22 @@ private void CreateOrOverwriteRunSettings(IRunSettingsProvider runSettingsProvid

for (int index = 0; index < length; index++)
{
var keyValuePair = args[index];
var indexOfSeparator = keyValuePair.IndexOf("=");
if (indexOfSeparator <= 0 || indexOfSeparator >= keyValuePair.Length - 1)
var arg = args[index];

if (UpdateTestRunParameterNode(runSettingsProvider, arg))
{
continue;
}
var key = keyValuePair.Substring(0, indexOfSeparator).Trim();
var value = keyValuePair.Substring(indexOfSeparator + 1);

var indexOfSeparator = arg.IndexOf("=");

if (indexOfSeparator <= 0 || indexOfSeparator >= arg.Length - 1)
{
continue;
}

var key = arg.Substring(0, indexOfSeparator).Trim();
var value = arg.Substring(indexOfSeparator + 1);

if (string.IsNullOrWhiteSpace(key))
{
Expand All @@ -160,6 +169,25 @@ private void CreateOrOverwriteRunSettings(IRunSettingsProvider runSettingsProvid
}
}

private bool UpdateTestRunParameterNode(IRunSettingsProvider runSettingsProvider, string node)
{
if (!node.Contains(Constants.TestRunParametersName))
{
return false;
}

var match = runSettingsProvider.GetTestRunParameterNodeMatch(node);
hvinett marked this conversation as resolved.
Show resolved Hide resolved

if (string.Compare(match.Value, node) == 0)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Compare [](start = 23, length = 7)

is this case insensitive?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it is case sensitive.

{
runSettingsProvider.UpdateTestRunParameterSettingsNode(match);
return true;
}

var exceptionMessage = string.Format(CommandLineResources.InvalidTestRunParameterArgument, node);
throw new CommandLineException(exceptionMessage);
}

private void UpdateFrameworkAndPlatform(string key, string value)
{
if (key.Equals(FrameworkArgumentExecutor.RunSettingsPath))
Expand Down
11 changes: 10 additions & 1 deletion src/vstest.console/Resources/Resources.Designer.cs
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 4 additions & 0 deletions src/vstest.console/Resources/Resources.resx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -734,4 +734,8 @@
<data name="TestSourcesDiscovered" xml:space="preserve">
<value>A total of {0} test files matched the specified pattern.</value>
</data>
<data name="InvalidTestRunParameterArgument" xml:space="preserve">
<value>The test run parameter argument '{0}' is invalid. Please use the format below.
Format: TestRunParameters.Parameter(name="&lt;name&gt;", value="&lt;value&gt;")</value>
</data>
</root>
7 changes: 7 additions & 0 deletions src/vstest.console/Resources/xlf/Resources.cs.xlf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1656,6 +1656,13 @@
<target state="translated">Celkový počet testovacích souborů, které odpovídají zadanému vzoru: {0}</target>
<note></note>
</trans-unit>
<trans-unit id="InvalidTestRunParameterArgument">
<source>The test run parameter argument '{0}' is invalid. Please use the format below.
Format: TestRunParameters.Parameter(name="&lt;name&gt;", value="&lt;value&gt;")</source>
<target state="new">The test run parameter argument '{0}' is invalid. Please use the format below.
Format : TestRunParameters.Parameter(name="&lt;name&gt;", value="&lt;value&gt;")</target>
<note></note>
</trans-unit>
</body>
</file>
</xliff>
7 changes: 7 additions & 0 deletions src/vstest.console/Resources/xlf/Resources.de.xlf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1656,6 +1656,13 @@
<target state="translated">Insgesamt {0} Testdateien stimmten mit dem angegebenen Muster überein.</target>
<note></note>
</trans-unit>
<trans-unit id="InvalidTestRunParameterArgument">
<source>The test run parameter argument '{0}' is invalid. Please use the format below.
Format: TestRunParameters.Parameter(name="&lt;name&gt;", value="&lt;value&gt;")</source>
<target state="new">The test run parameter argument '{0}' is invalid. Please use the format below.
Format : TestRunParameters.Parameter(name="&lt;name&gt;", value="&lt;value&gt;")</target>
<note></note>
</trans-unit>
</body>
</file>
</xliff>
7 changes: 7 additions & 0 deletions src/vstest.console/Resources/xlf/Resources.es.xlf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1659,6 +1659,13 @@
<target state="translated">{0} archivos de prueba en total coincidieron con el patrón especificado.</target>
<note></note>
</trans-unit>
<trans-unit id="InvalidTestRunParameterArgument">
<source>The test run parameter argument '{0}' is invalid. Please use the format below.
Format: TestRunParameters.Parameter(name="&lt;name&gt;", value="&lt;value&gt;")</source>
<target state="new">The test run parameter argument '{0}' is invalid. Please use the format below.
Format : TestRunParameters.Parameter(name="&lt;name&gt;", value="&lt;value&gt;")</target>
<note></note>
</trans-unit>
</body>
</file>
</xliff>
7 changes: 7 additions & 0 deletions src/vstest.console/Resources/xlf/Resources.fr.xlf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1656,6 +1656,13 @@
<target state="translated">Au total, {0} fichiers de test ont correspondu au modèle spécifié.</target>
<note></note>
</trans-unit>
<trans-unit id="InvalidTestRunParameterArgument">
<source>The test run parameter argument '{0}' is invalid. Please use the format below.
Format: TestRunParameters.Parameter(name="&lt;name&gt;", value="&lt;value&gt;")</source>
<target state="new">The test run parameter argument '{0}' is invalid. Please use the format below.
Format : TestRunParameters.Parameter(name="&lt;name&gt;", value="&lt;value&gt;")</target>
<note></note>
</trans-unit>
</body>
</file>
</xliff>
7 changes: 7 additions & 0 deletions src/vstest.console/Resources/xlf/Resources.it.xlf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1656,6 +1656,13 @@
<target state="translated">Un totale di {0} file di test corrisponde al criterio specificato.</target>
<note></note>
</trans-unit>
<trans-unit id="InvalidTestRunParameterArgument">
<source>The test run parameter argument '{0}' is invalid. Please use the format below.
Format: TestRunParameters.Parameter(name="&lt;name&gt;", value="&lt;value&gt;")</source>
<target state="new">The test run parameter argument '{0}' is invalid. Please use the format below.
Format : TestRunParameters.Parameter(name="&lt;name&gt;", value="&lt;value&gt;")</target>
<note></note>
</trans-unit>
</body>
</file>
</xliff>
7 changes: 7 additions & 0 deletions src/vstest.console/Resources/xlf/Resources.ja.xlf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1656,6 +1656,13 @@
<target state="translated">合計 {0} 個のテスト ファイルが指定されたパターンと一致しました。</target>
<note></note>
</trans-unit>
<trans-unit id="InvalidTestRunParameterArgument">
<source>The test run parameter argument '{0}' is invalid. Please use the format below.
Format: TestRunParameters.Parameter(name="&lt;name&gt;", value="&lt;value&gt;")</source>
<target state="new">The test run parameter argument '{0}' is invalid. Please use the format below.
Format : TestRunParameters.Parameter(name="&lt;name&gt;", value="&lt;value&gt;")</target>
<note></note>
</trans-unit>
</body>
</file>
</xliff>
7 changes: 7 additions & 0 deletions src/vstest.console/Resources/xlf/Resources.ko.xlf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1656,6 +1656,13 @@
<target state="translated">지정된 패턴과 일치한 총 테스트 파일 수는 {0}개입니다.</target>
<note></note>
</trans-unit>
<trans-unit id="InvalidTestRunParameterArgument">
<source>The test run parameter argument '{0}' is invalid. Please use the format below.
Format: TestRunParameters.Parameter(name="&lt;name&gt;", value="&lt;value&gt;")</source>
<target state="new">The test run parameter argument '{0}' is invalid. Please use the format below.
Format : TestRunParameters.Parameter(name="&lt;name&gt;", value="&lt;value&gt;")</target>
<note></note>
</trans-unit>
</body>
</file>
</xliff>
7 changes: 7 additions & 0 deletions src/vstest.console/Resources/xlf/Resources.pl.xlf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1656,6 +1656,13 @@
<target state="translated">Łączna liczba plików testowych dopasowanych do określonego wzorca: {0}.</target>
<note></note>
</trans-unit>
<trans-unit id="InvalidTestRunParameterArgument">
<source>The test run parameter argument '{0}' is invalid. Please use the format below.
Format: TestRunParameters.Parameter(name="&lt;name&gt;", value="&lt;value&gt;")</source>
<target state="new">The test run parameter argument '{0}' is invalid. Please use the format below.
Format : TestRunParameters.Parameter(name="&lt;name&gt;", value="&lt;value&gt;")</target>
<note></note>
</trans-unit>
</body>
</file>
</xliff>
7 changes: 7 additions & 0 deletions src/vstest.console/Resources/xlf/Resources.pt-BR.xlf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1656,6 +1656,13 @@ Altere o prefixo de nível de diagnóstico do agente de console, como mostrado a
<target state="translated">{0} arquivos de teste no total corresponderam ao padrão especificado.</target>
<note></note>
</trans-unit>
<trans-unit id="InvalidTestRunParameterArgument">
<source>The test run parameter argument '{0}' is invalid. Please use the format below.
Format: TestRunParameters.Parameter(name="&lt;name&gt;", value="&lt;value&gt;")</source>
<target state="new">The test run parameter argument '{0}' is invalid. Please use the format below.
Format : TestRunParameters.Parameter(name="&lt;name&gt;", value="&lt;value&gt;")</target>
<note></note>
</trans-unit>
</body>
</file>
</xliff>
7 changes: 7 additions & 0 deletions src/vstest.console/Resources/xlf/Resources.ru.xlf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1656,6 +1656,13 @@
<target state="translated">Общее количество тестовых файлов ({0}), соответствующих указанному шаблону.</target>
<note></note>
</trans-unit>
<trans-unit id="InvalidTestRunParameterArgument">
<source>The test run parameter argument '{0}' is invalid. Please use the format below.
Format: TestRunParameters.Parameter(name="&lt;name&gt;", value="&lt;value&gt;")</source>
<target state="new">The test run parameter argument '{0}' is invalid. Please use the format below.
Format : TestRunParameters.Parameter(name="&lt;name&gt;", value="&lt;value&gt;")</target>
<note></note>
</trans-unit>
</body>
</file>
</xliff>
7 changes: 7 additions & 0 deletions src/vstest.console/Resources/xlf/Resources.tr.xlf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1656,6 +1656,13 @@ Günlükler için izleme düzeyini aşağıda gösterildiği gibi değiştirin
<target state="translated">Toplam {0} test dosyası belirtilen desenle eşleşti.</target>
<note></note>
</trans-unit>
<trans-unit id="InvalidTestRunParameterArgument">
<source>The test run parameter argument '{0}' is invalid. Please use the format below.
Format: TestRunParameters.Parameter(name="&lt;name&gt;", value="&lt;value&gt;")</source>
<target state="new">The test run parameter argument '{0}' is invalid. Please use the format below.
Format : TestRunParameters.Parameter(name="&lt;name&gt;", value="&lt;value&gt;")</target>
<note></note>
</trans-unit>
</body>
</file>
</xliff>
7 changes: 7 additions & 0 deletions src/vstest.console/Resources/xlf/Resources.xlf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -847,6 +847,13 @@
<target state="new">A total of {0} test source files are discovered.</target>
<note></note>
</trans-unit>
<trans-unit id="InvalidTestRunParameterArgument">
<source>The test run parameter argument '{0}' is invalid. Please use the format below.
Format: TestRunParameters.Parameter(name="&lt;name&gt;", value="&lt;value&gt;")</source>
<target state="new">The test run parameter argument '{0}' is invalid. Please use the format below.
Format : TestRunParameters.Parameter(name="&lt;name&gt;", value="&lt;value&gt;")</target>
<note></note>
</trans-unit>
</body>
</file>
</xliff>
Loading