Implement filtering tests based on XML input file #4449

mfn · 2020-09-08T10:46:23Z

Summary

Export via --list-tests-xml and cut and slice as wanted, to feed it back via --tests-xml. Works with PHPUnit tests as well as PHPT tests.

A cheap slicing script is provided for now at https://gist.github.com/mfn/256636242cbe8a51252ce28181a6b074

Example:

~/src/phpunit $ ./phpunit --list-tests-xml tests.xml
PHPUnit 9.4-g567e55e11 by Sebastian Bergmann and contributors.

Wrote list of tests that would have been run to tests.xml

~/src/phpunit $ ./phpunit_xml_slicer.php tests.xml 1/100 > partial_tests.xml

~/src/phpunit $ ./phpunit --tests-xml partial_tests.xml
PHPUnit 9.4-g567e55e11 by Sebastian Bergmann and contributors.

Runtime:       PHP 7.4.8
Configuration: /Users/neo/src/phpunit/phpunit.xml

.............................                                     29 / 29 (100%)

Time: 00:00.186, Memory: 22.00 MB

OK (29 tests, 29 assertions)

Overview

Add a new --tests-xml <xml file> argument to PHPUnit
Implement new filter \PHPUnit\Runner\Filter\XmlTestsIterator which parses the XML and builds an internal data structure to identify what tests to accept()

I consider the state of the PR in a "functional state" and before finishing it I would seek feedback if I missed parts and guidance what kind of tests are best.

Considerations

The handling of using SimpleXML and parsing in \PHPUnit\Runner\Filter\XmlTestsIterator::setFilter is pretty crude. I barely worked with XML and PHP in the last decade. I saw \PHPUnit\Util\Xml\Loader but wasn't sure if it should be used, it was just added a few days ago. Also I liked the "simplicity" of SimpleXml to get this up and running.
=> Replaced with ext-xml aka DomDocument and friends
I noticed that the filters (not just the new one I added) are instantiated multiple times, thus the XML is parsed many more times. No idea what/if to do here
I first wanted to name it --filter-tests-xml, seemed more apt. But I ran into a strange problem with sebastian/cli-parser I didn't really understand so I just gave it a different to get the ball rolling

Error when using --filter-tests-xml as option name

Testing started at 08:07 ...
/usr/local/bin/php /Users/user/src/phpunit/phpunit --configuration /Users/user/src/phpunit/phpunit.xml --teamcity
PHPUnit 9.4-g4719a8977 by Sebastian Bergmann and contributors.

Runtime:       PHP 7.4.8
Configuration: /Users/user/src/phpunit/phpunit.xml


Failed asserting that string matches format description.
--- Expected
+++ Actual
@@ @@
-PHPUnit %s by Sebastian Bergmann and contributors.
+string(11) "-tests-xml="
+int(31)
+int(87)
+string(6) "filter"
+string(7) "filter="
 
-...                                                                 3 / 3 (100%)
+Fatal error: Uncaught RuntimeException: dafuq in /Users/user/src/phpunit/vendor/sebastian/cli-parser/src/exceptions/AmbiguousOptionException.php:19
+Stack trace:
+#0 /Users/user/src/phpunit/vendor/sebastian/cli-parser/src/Parser.php(165): SebastianBergmann\CliParser\AmbiguousOptionException->__construct('--filter')
+#1 /Users/user/src/phpunit/vendor/sebastian/cli-parser/src/Parser.php(81): SebastianBergmann\CliParser\Parser->parseLongOption('filter', Array, Array, Array)
+#2 /Users/user/src/phpunit/src/TextUI/CliArguments/Builder.php(128): SebastianBergmann\CliParser\Parser->parse(Array, 'd:c:hv', Array)
+#3 /Users/user/src/phpunit/src/TextUI/Command.php(223): PHPUnit\TextUI\CliArguments\Builder->fromParameters(Array, Array)
+#4 /Users/user/src/phpunit/src/TextUI/Command.php(115): PHPUnit\TextUI\Command->handleArguments(Array)
+#5 /Users/user/src/phpunit/src/TextUI/Command.php(100): PHPUnit\TextUI\Command->run(Array, true)
+#6 Standard input code(9): PHPUnit\TextUI\Command::main()
+#7 {main}
 
-Time: %s, Memory: %s
-
-OK (3 tests, 3 assertions)
+Next PHPUnit\TextUI\Exception: dafuq in /User in /Users/user/src/phpunit/src/TextUI/Command.php on line 102

 /Users/user/src/phpunit/tests/end-to-end/filter-class-isolation.phpt:14
 /Users/user/src/phpunit/src/Framework/TestSuite.php:665
 /Users/user/src/phpunit/src/Framework/TestSuite.php:665
 /Users/user/src/phpunit/src/TextUI/TestRunner.php:668
 /Users/user/src/phpunit/src/TextUI/Command.php:147
 /Users/user/src/phpunit/src/TextUI/Command.php:100

TODO

Write tests; but what kind of? PHPT? Not sure where to put files / fixtures, so many already there. Guidance appreciated!
=> a phpt test has been added
XML parsing? Improve? Change to DomDocument?
=> Indeed, changed to DomDocument

Links

"Specify a list of tests to run"
=> Specify a list of tests to run #3387

codecov · 2020-09-08T10:48:01Z

Codecov Report

Merging #4449 (1e80f8a) into master (ba39893) will increase coverage by 0.09%.
The diff coverage is 90.47%.

@@             Coverage Diff              @@
##             master    #4449      +/-   ##
============================================
+ Coverage     83.54%   83.63%   +0.09%     
- Complexity     4554     4578      +24     
============================================
  Files           286      287       +1     
  Lines         11759    11820      +61     
============================================
+ Hits           9824     9886      +62     
+ Misses         1935     1934       -1

Impacted Files	Coverage Δ	Complexity Δ
src/TextUI/Help.php	`100.00% <ø> (ø)`	`23.00 <0.00> (ø)`
src/TextUI/CliArguments/Configuration.php	`68.58% <87.50%> (+0.19%)`	`269.00 <4.00> (+3.00)`
src/Runner/Filter/XmlTestsIterator.php	`88.37% <88.37%> (ø)`	`17.00 <17.00> (?)`
src/TextUI/CliArguments/Builder.php	`80.56% <100.00%> (+0.18%)`	`120.00 <0.00> (+1.00)`
src/TextUI/CliArguments/Mapper.php	`79.51% <100.00%> (+0.24%)`	`82.00 <0.00> (+1.00)`
src/TextUI/TestRunner.php	`65.71% <100.00%> (+1.30%)`	`219.00 <0.00> (+2.00)`
src/Runner/DefaultTestResultCache.php	`93.65% <0.00%> (+1.58%)`	`27.00% <0.00%> (ø%)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ba39893...1e80f8a. Read the comment docs.

spawnia · 2020-09-08T12:53:14Z

src/Runner/Filter/XmlTestsIterator.php

+
+        // Regular PHPUnit tests
+        foreach ($xml->testCaseClass as $class) {
+            $className = $class['name']->__toString();


Can we make this typesafe? Maybe through an annotation like this:

/** @var array{name: ?} $class */

Thanks for all the feedback!

I'll first wait regarding such typings as to in what direction the PR really goes.

I never used SimpleXml really before, this is all highly magic. $class is not an array, it's SimpleXMLElement providing lots-o-magic stuff…

I appreciate the time you invest in preparing this pull request. Please note, though, that I will not accept this if it introduces ext/simplexml usage.

I've replaced SimpleXML with DomDocument and re-used the existing \PHPUnit\Util\Xml\Loader::loadFile

src/Runner/Filter/XmlTestsIterator.php

spawnia

This is great @mfn, looks pretty good overall and opens the door for some very important functionality.

theseer · 2020-09-11T15:02:50Z

Can we please stop using none namespaced XML and provide XSDs for them?

mfn · 2020-10-09T18:24:14Z

ping, anyone?

theseer · 2020-10-09T19:06:02Z

Not sure what you're expecting at this moment.

While the current state of this PR is having conflicts that should get resolved, my comment regarding the lack of a namespace and an XSD is unanswered. I do realize you're only relying on the export that already is broken in that regard. But before building more things that rely on this, we should fix that, imho.

epdenouden · 2020-10-11T11:02:49Z

@mfn I am working on writing a more detailed review. Long story very short: the idea of filtering tests via an XML-file and the main Iterator is fine, but it needs refactoring.

I am also wary of any iterator work getting in the way of the coming redesign of the core iterator/filtering mechanisms. If this PR gets accepted as-is, I would have to refactor this new functionality, too.

epdenouden · 2020-10-13T09:50:40Z

src/Runner/Filter/XmlTestsIterator.php

+        }
+
+        /** @var TestCase $test */
+        $testClass = get_class($test);


There is a way to ask anything that implements a Test its own unique identity, have a look at https://github.com/sebastianbergmann/phpunit/blob/master/src/Framework/Reorderable.php and the uses of Reorderable.

If you need more details about the (origins of) a test, let me know the use cases. It would be best if we can extend a central mechanism for identifying and locating tests.

I want to concentrate on this part first here before solving the others, might have an impact.

When a filter receives a test class in accept(), I've to figure out ifs part of the desired filter, i.e. the XML being feed back to phpunit.

The format of the XML is defined via \PHPUnit\Util\XmlTestListRenderer::render

phpunit/src/Util/XmlTestListRenderer.php

Line 42 in 1766543

if (get_class($test) !== $currentTestCase) {

and creates a XML structure like this:

<?xml version="1.0"?> <tests> <testCaseClass name="PHPUnit\Framework\FunctionsTest"> <testCaseMethod name="testGlobalFunctionsFileContainsAllStaticAssertions" groups="default" dataSet=""assertArrayHasKey""/> … </testCaseClass> <phptFile path="/absolute/path/phpunit/tests/end-to-end/abstract-test-class.phpt"/> … </tests>

And this now explains why I'm using get_class() and not something else: as a consumer of the XML, I've to match the producer.

This is, for \PHPUnit\Framework\TestCase, further stipulated with the dataSet-attribute, it's produced using this code

phpunit/src/Util/XmlTestListRenderer.php

Lines 57 to 66 in 1766543

if (!empty($test->getDataSetAsString(false))) {

$writer->writeAttribute(

'dataSet',

str_replace(

' with data set ',

'',

$test->getDataSetAsString(false)

)

);

}

As can be seen, this manually removes some parts of \PHPUnit\Framework\TestCase::getDataSetAsString and writes it to the XML.

For that reason, when reading the XML I'm reconstructing the original "data as string" with this:

$name = "{$methodName} with data set {$dataSet}"; $this->filter[$className][$name] = true;

so that in accept() I can just call

$name = $test->getName();

and getName internally calls getDataSetAsString, so that in the end by calling:

$testClass = get_class($test); in accept()
and doing

"{$methodName} with data set {$dataSet} in setFilter
I've built the matching mirror logic for consuming what was produced.

I guess some of this "peekaboo" here is reflected in #4449 (comment)

So currently I don't see how e.g \PHPUnit\Framework\TestCase::sortId (of \PHPUnit\Framework\Reorderable) helps me here, as the generated value is not usable in the context of what is produced in the XML

As for the format used in the \PHPUnit\Runner\Filter\XmlTestsIterator::$filter, I tried to document it:

/** * The filter is used as a fast look for * - the class name * - the method name plus its optional data set description. * * Example: `filter[class name][method name + data set] = true;` * * The accept() method then can use a fast isset() to check if a test should * be included or not. * * This works equally for phpt tests, except we hardcode the class name. * * @var array<string,array<string,true>> */

epdenouden · 2020-10-13T10:22:26Z

src/Runner/Filter/XmlTestsIterator.php

+     */
+    private function setFilter(string $xmlFile): void
+    {
+        $xml   = (new XmlLoader())->loadFile($xmlFile);


Since the iterator gets instantiated multiple times, this becomes a very expensive implementation. Loading the configuration should be done once, stored somewhere and then used as a lookup in the iterator instances.

Currently this would be somewhere around TestRunner::processSuiteFilters().

Thanks, I took a closer look and finally figured out why processSuiteFilters is only executed once but the constructor/filter called multiple times => I didn't realize this whole reflection/instantiation the first time around.

I'm not sure where exactly to put this code to a) load the XML b) parse it c) generate the data structure for fast lookup, so to satisfy the performance concerns for now, I left the code within XmlTestsIterator as static methods and call this helpers from processSuiteFilters.
What I find "nice" about this is that generating the filter and consuming it in accept() is in the same code space; I kinda like to think this makes easier to understand.

I'm happy to move it somewhere else, but wasn't sure if putting them directly into \PHPUnit\TextUI\TestRunner would be a good idea either.

epdenouden · 2020-10-13T10:23:55Z

src/Runner/Filter/XmlTestsIterator.php

+    private function extractTestCases(DOMXPath $xpath): Generator
+    {
+        /** @var DOMElement $class */
+        foreach ($xpath->evaluate('/tests/testCaseClass') as $class) {


Same as loading the XML-file for every new Iterator: please do this once and use a centralized lookup.

This is now addressed, see response to https://github.com/sebastianbergmann/phpunit/pull/4449/files#r503837936

epdenouden · 2020-10-13T10:27:16Z

src/Runner/Filter/XmlTestsIterator.php

+    {
+        /* @var DOMElement $phptFile */
+        foreach ($xpath->evaluate('/tests/phptFile') as $phptFile) {
+            $path = $phptFile->getAttribute('path');


Question: why does CodeCov mark these lines as not-reached?

epdenouden · 2020-10-13T10:27:23Z

src/Runner/Filter/XmlTestsIterator.php

+            $path = $phptFile->getAttribute('path');
+
+            if ($path) {
+                yield $path;


Question: why does CodeCov mark these lines as not-reached?

epdenouden · 2020-10-13T10:28:30Z

src/TextUI/TestRunner.php

@@ -1145,6 +1147,13 @@ private function processSuiteFilters(TestSuite $suite, array $arguments): void
            );
        }

+        if (!empty($arguments['testsXml'])) {


Around here is the place to load+parse your configuration and get a list of tests. Perhaps you can even adapt/reuse the current NameFilterIterator, as it's basically what it is. :)

mfn · 2020-12-07T14:25:47Z

For some reason I never got the email notification from the review feedback #4449 (review) , I just discovered it now 💥

I'll try to find time ASAP to address things!

mfn

@epdenouden thanks for your feedback!

I did explain some specifics regarding the accept() implementation, as I'm not sure what I could do to improve it regards "interoperability" with consumed XML.

Technically I did address the performance concern regarding the XML parsing, though I'm not yet sure if the code location is fine or not.

I'll address code coverage feedback once we all agree what the final implementation is supposed to look like to move forward.

Thank you!

mfn · 2020-12-09T23:33:11Z

src/Runner/Filter/XmlTestsIterator.php

+        }
+
+        /** @var TestCase $test */
+        $testClass = get_class($test);


I want to concentrate on this part first here before solving the others, might have an impact.

When a filter receives a test class in accept(), I've to figure out ifs part of the desired filter, i.e. the XML being feed back to phpunit.

The format of the XML is defined via \PHPUnit\Util\XmlTestListRenderer::render

phpunit/src/Util/XmlTestListRenderer.php

Line 42 in 1766543

if (get_class($test) !== $currentTestCase) {

and creates a XML structure like this:

<?xml version="1.0"?> <tests> <testCaseClass name="PHPUnit\Framework\FunctionsTest"> <testCaseMethod name="testGlobalFunctionsFileContainsAllStaticAssertions" groups="default" dataSet=""assertArrayHasKey""/> … </testCaseClass> <phptFile path="/absolute/path/phpunit/tests/end-to-end/abstract-test-class.phpt"/> … </tests>

And this now explains why I'm using get_class() and not something else: as a consumer of the XML, I've to match the producer.

This is, for \PHPUnit\Framework\TestCase, further stipulated with the dataSet-attribute, it's produced using this code

phpunit/src/Util/XmlTestListRenderer.php

Lines 57 to 66 in 1766543

if (!empty($test->getDataSetAsString(false))) {

$writer->writeAttribute(

'dataSet',

str_replace(

' with data set ',

'',

$test->getDataSetAsString(false)

)

);

}

As can be seen, this manually removes some parts of \PHPUnit\Framework\TestCase::getDataSetAsString and writes it to the XML.

For that reason, when reading the XML I'm reconstructing the original "data as string" with this:

$name = "{$methodName} with data set {$dataSet}"; $this->filter[$className][$name] = true;

so that in accept() I can just call

$name = $test->getName();

and getName internally calls getDataSetAsString, so that in the end by calling:

$testClass = get_class($test); in accept()
and doing

"{$methodName} with data set {$dataSet} in setFilter
I've built the matching mirror logic for consuming what was produced.

I guess some of this "peekaboo" here is reflected in #4449 (comment)

So currently I don't see how e.g \PHPUnit\Framework\TestCase::sortId (of \PHPUnit\Framework\Reorderable) helps me here, as the generated value is not usable in the context of what is produced in the XML

As for the format used in the \PHPUnit\Runner\Filter\XmlTestsIterator::$filter, I tried to document it:

/** * The filter is used as a fast look for * - the class name * - the method name plus its optional data set description. * * Example: `filter[class name][method name + data set] = true;` * * The accept() method then can use a fast isset() to check if a test should * be included or not. * * This works equally for phpt tests, except we hardcode the class name. * * @var array<string,array<string,true>> */

mfn · 2020-12-09T23:47:49Z

src/Runner/Filter/XmlTestsIterator.php

+     */
+    private function setFilter(string $xmlFile): void
+    {
+        $xml   = (new XmlLoader())->loadFile($xmlFile);


Thanks, I took a closer look and finally figured out why processSuiteFilters is only executed once but the constructor/filter called multiple times => I didn't realize this whole reflection/instantiation the first time around.

I'm not sure where exactly to put this code to a) load the XML b) parse it c) generate the data structure for fast lookup, so to satisfy the performance concerns for now, I left the code within XmlTestsIterator as static methods and call this helpers from processSuiteFilters.
What I find "nice" about this is that generating the filter and consuming it in accept() is in the same code space; I kinda like to think this makes easier to understand.

I'm happy to move it somewhere else, but wasn't sure if putting them directly into \PHPUnit\TextUI\TestRunner would be a good idea either.

mfn · 2020-12-09T23:48:28Z

src/Runner/Filter/XmlTestsIterator.php

+    private function extractTestCases(DOMXPath $xpath): Generator
+    {
+        /** @var DOMElement $class */
+        foreach ($xpath->evaluate('/tests/testCaseClass') as $class) {


This is now addressed, see response to https://github.com/sebastianbergmann/phpunit/pull/4449/files#r503837936

mfn · 2020-12-19T12:28:07Z

Btw, is it fine to target master or should I target another branch, perhaps 9.5?

src/TextUI/Help.php

Export via `--list-tests-xml` and cut and slice as wanted, to feed it back via `--tests-xml`.

mfn · 2021-02-16T12:33:20Z

Is there anything else I can do? AFAIK I addresses the feedback or did I miss anything, which blocks us?

maks-rafalko · 2021-07-01T16:31:59Z

Let me up this ticket. Would like to help (if any help is needed) because this feature will greatly help us at Infection with filtering what tests to run (including data sets)

theseer · 2021-07-01T16:50:55Z

I didn't look into the Code itself, as @epdenouden already did that and I trust he'll review the current state as well ;)

But I'd like to repeat my statement made earlier: We should clean up the XML produced before we add more functionality and have (more) external projects rely on it. At the very least, let us please add a namespace with a version identifier so any future upgrades and changes are not going to kill us.

mfn · 2021-07-03T21:35:52Z

I definitely feel caught in a catch-22 here and I get no clear indicator how I can move forward.

I feels out of scope for me to tackle the "XML / namespace / schema" problem here, that's not a problem I created and seems "higher up" to me.

Randomly I tried to update the PR and resolve the merge conflicts, but I got demotivated due to lack of feedback / specific actionable stuff, so I didn't touch it for some time.

In any way, I'm about to leave for vacation and can't provide updates within the next few weeks. I'll check the PR feedback when I get back but I'm also happy if someone else steps up and assists here 🙏

sebastianbergmann · 2024-01-13T13:14:43Z

Thank you for your contribution. I appreciate the time you invested in preparing this pull request. However, I have decided not to merge it.

mfn mentioned this pull request Sep 8, 2020

Do not extend final class TestRunner and just copy-and-paste its contents wizacode/phpunit-slicer#8

Closed

sebastianbergmann added feature/test-runner CLI test runner type/enhancement A new idea that should be implemented labels Sep 8, 2020

This was linked to issues Sep 8, 2020

Provide a way to accept a list of tests #2253

Closed

Specify a list of tests to run #3387

Open

spawnia reviewed Sep 8, 2020

View reviewed changes

src/Runner/Filter/XmlTestsIterator.php Outdated Show resolved Hide resolved

spawnia reviewed Sep 8, 2020

View reviewed changes

src/Runner/Filter/XmlTestsIterator.php Outdated Show resolved Hide resolved

spawnia reviewed Sep 8, 2020

View reviewed changes

mfn force-pushed the mfn-tests-xml branch 3 times, most recently from f594394 to b10d64a Compare September 8, 2020 21:12

mfn marked this pull request as ready for review September 8, 2020 21:23

mfn requested a review from sebastianbergmann September 8, 2020 21:23

sebastianbergmann requested review from epdenouden and localheinz September 11, 2020 07:10

mfn force-pushed the mfn-tests-xml branch from b10d64a to d6d256d Compare September 17, 2020 09:11

Slamdunk mentioned this pull request Sep 22, 2020

Support for Gitlab CI parallel processing paratestphp/paratest#532

Closed

epdenouden suggested changes Oct 13, 2020

View reviewed changes

sebastianbergmann force-pushed the master branch from 5d5ef3e to 4303365 Compare December 4, 2020 05:39

mfn mentioned this pull request Dec 5, 2020

make the output from --list-tests usable as a --filter parameter when there is a data provider #4129

Open

mfn mentioned this pull request Dec 9, 2020

Custom test suite loaders are ignored paratestphp/paratest#556

Closed

mfn force-pushed the mfn-tests-xml branch 2 times, most recently from 6860ef8 to fdc3f6f Compare December 9, 2020 22:52

mfn force-pushed the mfn-tests-xml branch from fdc3f6f to c1acf7e Compare December 10, 2020 00:03

mfn commented Dec 10, 2020

View reviewed changes

spawnia reviewed Jan 22, 2021

View reviewed changes

src/TextUI/Help.php Show resolved Hide resolved

mfn added 4 commits January 27, 2021 21:24

Implement filtering tests based on XML input file

34c6ea5

Export via `--list-tests-xml` and cut and slice as wanted, to feed it back via `--tests-xml`.

Replace SimpleXML with DOMDocument and use existing XML loader

7bff561

Provide a simple test

4e4cbe5

Improve performance and parse XML only once

1e80f8a

mfn force-pushed the mfn-tests-xml branch from c1acf7e to 1e80f8a Compare January 27, 2021 20:36

maks-rafalko mentioned this pull request Jul 20, 2021

[Performance] Improve Infection performance executed against slow test suites infection/infection#1539

Merged

2 tasks

mfn mentioned this pull request Aug 26, 2021

Paratest running less tests than PHPUnit paratestphp/paratest#627

Closed

sebastianbergmann force-pushed the master branch from 6aa2fbc to 078538a Compare March 8, 2022 12:59

sebastianbergmann force-pushed the main branch 3 times, most recently from 08dc555 to 63591f3 Compare December 23, 2023 09:32

sebastianbergmann mentioned this pull request Jan 13, 2024

Improve XML generated by the --list-tests-xml CLI option #5657

Closed

sebastianbergmann closed this Jan 13, 2024

mfn mentioned this pull request Feb 16, 2024

combine --list-tests with --group #5702

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement filtering tests based on XML input file #4449

Implement filtering tests based on XML input file #4449

mfn commented Sep 8, 2020 •

edited

Loading

codecov bot commented Sep 8, 2020 •

edited

Loading

spawnia Sep 8, 2020

mfn Sep 8, 2020

sebastianbergmann Sep 8, 2020

mfn Sep 8, 2020

spawnia left a comment

theseer commented Sep 11, 2020

mfn commented Oct 9, 2020

theseer commented Oct 9, 2020

epdenouden commented Oct 11, 2020 •

edited

Loading

epdenouden Oct 13, 2020 •

edited

Loading

mfn Dec 9, 2020

epdenouden Oct 13, 2020

mfn Dec 9, 2020

epdenouden Oct 13, 2020

mfn Dec 9, 2020 •

edited

Loading

epdenouden Oct 13, 2020

epdenouden Oct 13, 2020

epdenouden Oct 13, 2020

mfn commented Dec 7, 2020

mfn left a comment

mfn Dec 9, 2020

mfn Dec 9, 2020

mfn Dec 9, 2020 •

edited

Loading

mfn commented Dec 19, 2020

mfn commented Feb 16, 2021

maks-rafalko commented Jul 1, 2021

theseer commented Jul 1, 2021

mfn commented Jul 3, 2021

sebastianbergmann commented Jan 13, 2024

	if (!empty($test->getDataSetAsString(false))) {
	$writer->writeAttribute(
	'dataSet',
	str_replace(
	' with data set ',
	'',
	$test->getDataSetAsString(false)
	)
	);
	}

Implement filtering tests based on XML input file #4449

Implement filtering tests based on XML input file #4449

Conversation

mfn commented Sep 8, 2020 • edited Loading

Summary

Overview

Considerations

TODO

Links

codecov bot commented Sep 8, 2020 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

spawnia left a comment

Choose a reason for hiding this comment

theseer commented Sep 11, 2020

mfn commented Oct 9, 2020

theseer commented Oct 9, 2020

epdenouden commented Oct 11, 2020 • edited Loading

epdenouden Oct 13, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mfn Dec 9, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mfn commented Dec 7, 2020

mfn left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mfn Dec 9, 2020 • edited Loading

Choose a reason for hiding this comment

mfn commented Dec 19, 2020

mfn commented Feb 16, 2021

maks-rafalko commented Jul 1, 2021

theseer commented Jul 1, 2021

mfn commented Jul 3, 2021

sebastianbergmann commented Jan 13, 2024

mfn commented Sep 8, 2020 •

edited

Loading

codecov bot commented Sep 8, 2020 •

edited

Loading

epdenouden commented Oct 11, 2020 •

edited

Loading

epdenouden Oct 13, 2020 •

edited

Loading

mfn Dec 9, 2020 •

edited

Loading

mfn Dec 9, 2020 •

edited

Loading