Skip to content

Commit

Permalink
#187: Add support for Scenario outlines custom name strategies (#191)
Browse files Browse the repository at this point in the history
  • Loading branch information
grey-rain authored Sep 12, 2024
1 parent ee58c32 commit a5a7b14
Show file tree
Hide file tree
Showing 5 changed files with 117 additions and 31 deletions.
95 changes: 68 additions & 27 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,8 @@ Table of Contents
2. [Log units](#log-units)
3. [Merge launches](#merge-launches)
4. [Test retries](#test-retries)
5. [Other settings](#other-settings)
5. [BDD Scenario test name customization](#bdd-scenario-test-name-customization)
6. [Other settings](#other-settings)
3. [Data mapping](#data-mapping)
4. [Versioning](#versioning)
5. [Important release notes](#important-release-notes)
Expand Down Expand Up @@ -160,14 +161,14 @@ configuration.usePreset(LogsPreset.FULL);

> **Notice**
>
> All integration configurations should be done before the start of Serenity facility. Otherwise default values will be
> used.
> All integration configurations should be completed before starting the Serenity engine; otherwise, default values will
> be used.
#### Presets

Each Serenity `TestStep` object is passed through chain of configured log units. Each particular log unit analyses step
and creates a collection of records that will be send to RP. This approach allows to flexible configure reporting
behaviour on a step level. By default integration provides a few log presets:
and creates a collection of records that will be sent to RP. This approach allows to flexible configure reporting
behaviour on a step level. By default, integration provides a few log presets:

- DEFAULT
- FULL
Expand Down Expand Up @@ -272,7 +273,7 @@ timestamp.

#### Merge launches

By default separate launch will be created in RP for each module in case of multi-module project. This behavior can be
By default, separate launch will be created in RP for each module in case of multi-module project. This behavior can be
changed with merge launches feature.

To understand a concept, let's assume following multi-module structure
Expand Down Expand Up @@ -331,14 +332,54 @@ and add `failsafe.rerunFailingTestsCount` or `surefire.rerunFailingTestsCount` p
> more
> details.
#### BDD Scenario test name customization

In some cases, you may need to customize the test name for BDD Scenario Outlines (for example, to insert parameters into
the test name in ReportPortal). To achieve this, you need to implement the desired customization logic in a class which
implements the `TestNameProvider` interface. This class should then be specified in the `ReportIntegrationConfig`.
For example, to provide ability for test name to insert parameters, the following steps should be performed:

1. Create class `ParametrizedNameProvider`

```
public class ParametrizedNameProvider implements TestNameProvider {
@Override
public String provideName(TestOutcome testOutcome, int scenarioIndex) {
// Regular expression to match content within < >
final String parameterRegexp = "<([^>]+)>";
final Pattern pattern = Pattern.compile(parameterRegexp);
final Matcher matcher = pattern.matcher(testOutcome.getName());
final StringBuilder result = new StringBuilder();
int index = 0;
List<String> replacements = testOutcome.getDataTable().getRows().get(scenarioIndex).getStringValues();
// Iterate through the matches and replace with values from the ArrayList
while (matcher.find()) {
if (index < replacements.size()) {
matcher.appendReplacement(result, replacements.get(index));
index++;
}
}
// Append the rest of the string
matcher.appendTail(result);
return result.toString();
}
```

2. Update integration configuration

```
ReportIntegrationConfig.get().useTestNameTransformer(new ParametrizedNameProvider());
```

#### Other settings

Section includes minor settings that available to configure and describes their usage.

Setting | Usage | Description
--------------------------|-----------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Selenium logs harvesting | `ReportIntegrationConfig.get().harvestSeleniumLogs(true)` | Special option that works in conjunction with `Selenium.filteredLogs(...)` unit and must be enabled as well in order it to works. By default it is disabled.
Truncate names | `ReportIntegrationConfig.get().truncateNames(true)` | Allows to hide RP server errors that related to entities with long names (more that 1024 symbols) creation. It is not recommended to use it. By default it is disabled.
| Setting | Usage | Description |
|--------------------------|-----------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Selenium logs harvesting | `ReportIntegrationConfig.get().harvestSeleniumLogs(true)` | Special option that works in conjunction with `Selenium.filteredLogs(...)` unit and must be enabled as well in order it to works. By default it is disabled. |
| Truncate names | `ReportIntegrationConfig.get().truncateNames(true)` | Allows to hide RP server errors that related to entities with long names (more that 1024 symbols) creation. It is not recommended to use it. By default it is disabled. |

## Data mapping

Expand All @@ -347,12 +388,12 @@ relates to each other.

**Name** relation is straightforward.

Serenity | Report portal
-------------|---------------
Test Class | Suite
Test Method | Test
Scenario | Test
Step | Log entry
| Serenity | Report portal |
|-------------|---------------|
| Test Class | Suite |
| Test Method | Test |
| Scenario | Test |
| Step | Log entry |

**Description** Each non-log entity in Report Portal may has a description. This field is populated from Serenity
narrative section for both jUnit and BDD test sources.
Expand Down Expand Up @@ -446,24 +487,24 @@ Important release notes are described below.
Use [releases](https://github.com/Invictum/serenity-reportportal-integration/releases) section for details regarding
regular release notes.

Version | Note
---------------|-----------------------------------------------------------------------------------------------------------------
1.0.0 - 1.0.6 | Supports RP v3 and below
1.1.0 - 1.1.3 | Minor version update due RP v4 release. Versions older than 1.1.0 are not compatible with RP v4+ and vise versa
1.2.0 - 1.2.1 | Minor version updated due internal mechanisms approach major refactoring
1.3.0 | Minor version updated due to log units approach rework
1.4.0 - 1.4.3 | Minor version update: removed tree handler, refactored to support DDT for BDD
1.5.0+ | Minor version update due RP v5 release
1.6.0+ | Minor version update due Serenity 4 release. Report Portal is updated to 5.8 as well
| Version | Note |
|---------------|-------------------------------------------------------------------------------------------------------------------------|
| 1.0.0 - 1.0.6 | Supports RP v3 and below |
| 1.1.0 - 1.1.3 | Minor version update due RP v4 release. Versions older than 1.1.0 are not compatible with RP v4+ and vise versa |
| 1.2.0 - 1.2.1 | Minor version updated due internal mechanisms approach major refactoring |
| 1.3.0 | Minor version updated due to log units approach rework |
| 1.4.0 - 1.4.3 | Minor version update: removed tree handler, refactored to support DDT for BDD |
| 1.5.0+ | Minor version update due RP v5 release |
| 1.6.0+ | Minor version update due Serenity 4 release. Report Portal is updated to 5.8 as well. Bugfixes and some of new features |

## Limitations

Integration has limited concurrency support.
For versions < 4 concurrency for parametrized Serenity tests execution is not supported.
For version >=4 concurrency is supported on feature level.

The following line should be provided into `junit-platform.properties` file of your project for versions 1.6.0+ in order for integration to
work correctly:
The following line should be provided into `junit-platform.properties` file of your project for versions 1.6.0+ in order
for integration to work correctly:

**JUnit**

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package com.github.invictum.reportportal;


import com.github.invictum.reportportal.recorder.DefaultBddOutlineTestNameProvider;
import com.github.invictum.reportportal.recorder.TestNameProvider;
import net.serenitybdd.annotations.Narrative;

import java.util.Objects;
Expand All @@ -20,6 +21,7 @@ public class ReportIntegrationConfig {

private LogsPreset preset = LogsPreset.DEFAULT;
private Function<Narrative, String> classNarrativeFormatter = n -> String.join("\n", n.text());
private TestNameProvider testNameProvider = new DefaultBddOutlineTestNameProvider();
boolean harvestSeleniumLogs = false;
boolean truncateNames = false;

Expand Down Expand Up @@ -57,6 +59,18 @@ public ReportIntegrationConfig useClassNarrativeFormatter(Function<Narrative, St
return this;
}

/**
* Overrides default test name, passed from Serenity, with custom logic.
*/
public ReportIntegrationConfig useTestNameTransformer(TestNameProvider testNameProvider) {
this.testNameProvider = testNameProvider;
return this;
}

public TestNameProvider getTestNameTransformer() {
return testNameProvider;
}

/**
* Option allows to enable or disable selenium based logs harvesting
* Designed to be used in conjunction with {@link com.github.invictum.reportportal.log.unit.Selenium} log unit
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
import net.thucydides.model.domain.TestOutcome;
import net.thucydides.model.domain.TestStep;

import java.util.Arrays;
import java.util.List;

/**
* Recorder aware of parameterized BDD style test specific handling
Expand All @@ -32,16 +32,17 @@ public void record(TestOutcome out) {
.withDescription(out.getUserStory().getNarrative())
.build();
Maybe<String> id = suiteStorage.start(out.getUserStory().getId(), () -> launch.startTestItem(startStory));
TestNameProvider testNameProvider = ReportIntegrationConfig.get().getTestNameTransformer();
// Start test
StartTestItemRQ startScenario = new StartEventBuilder(ItemType.STEP)
.withName(out.getName())
.withName(testNameProvider.provideName(out, last))
.withStartTime(currentTest.getStartTime())
.withParameters(out.getDataTable().row(last))
.withTags(out.getTags())
.build();
Maybe<String> testId = launch.startTestItem(id, startScenario);
// Steps
proceedSteps(testId, Arrays.asList(currentTest));
proceedSteps(testId, List.of(currentTest));
// Stop test
FinishTestItemRQ finishScenario = new FinishEventBuilder()
.withStatus(Status.mapTo(currentTest.getResult()))
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
package com.github.invictum.reportportal.recorder;

import net.thucydides.model.domain.TestOutcome;

/**
* Default implementation of name as seen in Serenity.
*/
public class DefaultBddOutlineTestNameProvider implements TestNameProvider {

@Override
public String provideName(TestOutcome testOutcome, int scenarioIndex) {
return testOutcome.getName();
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
package com.github.invictum.reportportal.recorder;

import net.thucydides.model.domain.TestOutcome;

/**
* Allows to transform particular test name in ReportPortal according to specific user requirements.
*/
public interface TestNameProvider {

/**
* @param testOutcome original test data
* @param scenarioIndex index of scenario which is being reported
* @return transformed test name
*/
String provideName(TestOutcome testOutcome, int scenarioIndex);
}

0 comments on commit a5a7b14

Please sign in to comment.