Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wait for message #3097

Merged
merged 1 commit into from
Aug 25, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions docs/modules/plugins/pages/plugin-web-app.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -1193,3 +1193,25 @@ Then there are browser console $logLevels by regex '$pattern'
Given I am on a page with the URL 'https://vividus-test-site.herokuapp.com/'
Then there are browser console ERRORS by regex '.*user.*'
----

==== Wait for console log entries and save them

Waits for the appearance of the console log entries with the expected level and which match regular expression and saves all the entries (including awaited ones) of the expected level gathered during the wait to the scoped variable.

NOTE: Wait uses generic UI timeouts specified by the properties `ui.wait.timeout` and `ui.wait.polling-period`. See <<_properties>> section for more details.

[source,gherkin]
----
When I wait until browser console $logEntries by regex `$regex` appear and save all entries into $scopes variable `$variableName`
----
* `$logLevels` - {log-levels}
* `$pattern` - The regular expression to match log entry messages.
* `$scopes` - xref:commons:variables.adoc#_scopes[The comma-separated set of the variables scopes].
* `$variableName` - The name of the variable to save the value of the barcode.

.Wait for application readiness
----
Given I am on a page with the URL 'https://vividus-test-site.herokuapp.com/'
When I wait until browser console infos by regex `.*Application ready.*` appear and save all entries into scenario variable `logs`
Then `${logs}` matches `.*Application ready in \d+ seconds.*`
----
10 changes: 10 additions & 0 deletions docs/modules/plugins/partials/selenium-properties.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -39,3 +39,13 @@ If the value is set to `true`, an attempt to reset context will be performed whe
|password
|`<empty>`
|Password to be used to create a new session

|`ui.wait.timeout`
||{iso-date-format-link} format
|`PT1M`
|Total duration to wait for UI condition

|`ui.wait.polling-period`
||{iso-date-format-link} format
|`PT2S`
|Used together with `ui.wait.timeout`. Total duration of time between two iterations of UI condition check
Original file line number Diff line number Diff line change
Expand Up @@ -16,31 +16,49 @@

package org.vividus.steps.ui.web;

import java.util.ArrayList;
import java.util.Collection;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.function.Function;
import java.util.function.Predicate;
import java.util.regex.Pattern;
import java.util.stream.Collectors;

import javax.inject.Inject;

import org.jbehave.core.annotations.Then;
import org.jbehave.core.annotations.When;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.logging.LogEntry;
import org.vividus.context.VariableContext;
import org.vividus.reporter.event.IAttachmentPublisher;
import org.vividus.selenium.IWebDriverProvider;
import org.vividus.selenium.logging.BrowserLogLevel;
import org.vividus.selenium.logging.BrowserLogManager;
import org.vividus.softassert.ISoftAssert;
import org.vividus.ui.action.WaitActions;
import org.vividus.ui.action.WaitResult;
import org.vividus.variable.VariableScope;

public class JsValidationSteps
{
@Inject private IWebDriverProvider webDriverProvider;
@Inject private IAttachmentPublisher attachmentPublisher;
@Inject private ISoftAssert softAssert;
private final IWebDriverProvider webDriverProvider;
private final ISoftAssert softAssert;
private final IAttachmentPublisher attachmentPublisher;
private final WaitActions waitActions;
private final VariableContext variableContext;
private boolean includeBrowserExtensionLogEntries;

public JsValidationSteps(IWebDriverProvider webDriverProvider, ISoftAssert softAssert,
IAttachmentPublisher attachmentPublisher, WaitActions waitActions, VariableContext variableContext)
{
this.webDriverProvider = webDriverProvider;
this.softAssert = softAssert;
this.attachmentPublisher = attachmentPublisher;
this.waitActions = waitActions;
this.variableContext = variableContext;
}

/**
* Checks that opened page contains JavaScript browser console logs that matches regex.
* <p>Note that log buffers are reset after step invocation, meaning that available log entries correspond to those
Expand Down Expand Up @@ -94,6 +112,54 @@ public void checkJsLogEntriesOnOpenedPageFilteredByRegExp(List<BrowserLogLevel>
checkLogMessagesAbsence(getLogEntries(logEntries, regex), logEntries);
}

/**
* Waits for the appearance of the console log entries with the expected level and which match regular expression
* and saves all the entries (including awaited ones) of the expected level gathered during the wait to the
* scoped variable.
*
* @param logEntries Comma-separated list of entries to check. Possible values: "errors", "warnings", "infos".
* @param regex Regular expression to filter log entries.
* @param scopes The set (comma-separated list of scopes e.g.: STORY, NEXT_BATCHES) of variable's scope<br>
* <i>Available scopes:</i>
* <ul>
* <li><b>STEP</b> - the variable will be available only within the step,
* <li><b>SCENARIO</b> - the variable will be available only within the scenario,
* <li><b>STORY</b> - the variable will be available within the whole story,
* <li><b>NEXT_BATCHES</b> - the variable will be available starting from next batch
* </ul>.
* @param variableName The name of the variable to save the SFTP commands execution result.
*/
@When("I wait until browser console $logEntries by regex `$regex` appear and save all entries into $scopes "
+ "variable `$variableName`")
public void waitForMessageAndSave(List<BrowserLogLevel> logEntries, Pattern regex, Set<VariableScope> scopes,
String variableName)
{
WebDriver webDriver = webDriverProvider.get();
List<LogEntry> messages = new ArrayList<>();
WaitResult<Boolean> waitResult = waitActions.wait(webDriver, new Function<>()
{
@Override
public Boolean apply(WebDriver driver)
{
Set<LogEntry> newMessages = getLogEntries(logEntries, driver);
messages.addAll(newMessages);
return newMessages.stream().map(LogEntry::getMessage).anyMatch(regex.asMatchPredicate());
}

@Override
public String toString()
{
return String.format("appearance of JavaScript %s matching `%s` regex",
JsValidationSteps.toString(logEntries), regex);
Comment on lines +152 to +153
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how does failure look? how does passed step look?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

image

}
});
if (waitResult.isWaitPassed())
{
variableContext.putVariable(scopes, variableName, messages);
}
publishAttachment(webDriver, messages);
}

private void checkLogMessagesAbsence(Set<LogEntry> logEntries, List<BrowserLogLevel> logLevels)
{
softAssert.assertEquals("Current page contains no JavaScript " + toString(logLevels), 0,
Expand All @@ -103,23 +169,29 @@ private void checkLogMessagesAbsence(Set<LogEntry> logEntries, List<BrowserLogLe
private Set<LogEntry> getLogEntries(List<BrowserLogLevel> logEntries, Predicate<? super LogEntry> filter)
{
WebDriver webDriver = webDriverProvider.get();
Set<LogEntry> filteredLogEntries = BrowserLogManager.getFilteredLog(webDriver, logEntries).stream()
Set<LogEntry> filteredLogEntries = getLogEntries(logEntries, webDriver).stream()
.filter(filter::test)
.collect(Collectors.toSet());

publishAttachment(Map.of(webDriver.getCurrentUrl(), filteredLogEntries));
publishAttachment(webDriver, filteredLogEntries);
return filteredLogEntries;
}

private Set<LogEntry> getLogEntries(List<BrowserLogLevel> logEntries, WebDriver webDriver)
{
return BrowserLogManager.getFilteredLog(webDriver, logEntries);
}

private Set<LogEntry> getLogEntries(List<BrowserLogLevel> logEntries, Pattern regex)
{
return getLogEntries(logEntries, message -> regex.matcher(message.getMessage()).matches());
}

private void publishAttachment(Map<String, Set<LogEntry>> results)
private void publishAttachment(WebDriver webDriver, Collection<LogEntry> filteredLogEntries)
{
attachmentPublisher.publishAttachment("/org/vividus/steps/ui/web/js-console-validation-result-table.ftl",
Map.of("results", results), "JavaScript console validation results");
Map.of("results", Map.of(webDriver.getCurrentUrl(), filteredLogEntries)),
"JavaScript console validation results");
}

private static String toString(List<BrowserLogLevel> logLevels)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,12 +17,16 @@
package org.vividus.steps.ui.web;

import static java.util.Collections.singletonList;
import static org.mockito.ArgumentMatchers.argThat;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.Mockito.inOrder;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.times;
import static org.mockito.Mockito.when;
import static org.mockito.Mockito.withSettings;

import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
import java.util.List;
import java.util.Map;
Expand All @@ -32,9 +36,12 @@

import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;
import org.mockito.InOrder;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.Mockito;
import org.mockito.junit.jupiter.MockitoExtension;
import org.openqa.selenium.HasCapabilities;
import org.openqa.selenium.WebDriver;
Expand All @@ -43,10 +50,14 @@
import org.openqa.selenium.logging.LogEntry;
import org.openqa.selenium.logging.LogType;
import org.openqa.selenium.logging.Logs;
import org.vividus.context.VariableContext;
import org.vividus.reporter.event.IAttachmentPublisher;
import org.vividus.selenium.IWebDriverProvider;
import org.vividus.selenium.logging.BrowserLogLevel;
import org.vividus.softassert.ISoftAssert;
import org.vividus.ui.action.WaitActions;
import org.vividus.ui.action.WaitResult;
import org.vividus.variable.VariableScope;

@ExtendWith(MockitoExtension.class)
class JsValidationStepsTests
Expand All @@ -67,13 +78,19 @@ class JsValidationStepsTests
@Mock
private ISoftAssert softAssert;

@Mock
private WaitActions waitActions;

@Mock
private VariableContext variableContext;

@InjectMocks
private JsValidationSteps jsValidationSteps;

@Test
void testCheckThereAreLogEntriesOnOpenedPageFilteredByRegExp()
{
Map<String, Set<LogEntry>> expectedResults = testCheckJsErrors(ERROR_MESSAGE, () -> jsValidationSteps
Map<String, Collection<LogEntry>> expectedResults = testCheckJsErrors(ERROR_MESSAGE, () -> jsValidationSteps
.checkThereAreLogEntriesOnOpenedPageFilteredByRegExp(singletonList(BrowserLogLevel.ERRORS),
ERROR_MESSAGE_PATTERN));
InOrder inOrder = inOrder(attachmentPublisher, softAssert);
Expand All @@ -85,7 +102,7 @@ void testCheckThereAreLogEntriesOnOpenedPageFilteredByRegExp()
@Test
void testCheckJsErrorsOnPage()
{
Map<String, Set<LogEntry>> expectedResults = testCheckJsErrors(ERROR_MESSAGE,
Map<String, Collection<LogEntry>> expectedResults = testCheckJsErrors(ERROR_MESSAGE,
() -> jsValidationSteps.checkJsLogEntriesOnOpenedPage(singletonList(BrowserLogLevel.ERRORS)));
verifyTestActions(expectedResults, ERRORS_ASSERTION_DESCRIPTION);
}
Expand All @@ -103,7 +120,7 @@ void testCheckJsErrorsOnPageByRegExpNoMatch()
void testCheckJsErrorsAndWarningsOnPage()
{
List<BrowserLogLevel> logLevels = Arrays.asList(BrowserLogLevel.ERRORS, BrowserLogLevel.WARNINGS);
Map<String, Set<LogEntry>> expectedResults = testCheckJsErrors(ERROR_MESSAGE,
Map<String, Collection<LogEntry>> expectedResults = testCheckJsErrors(ERROR_MESSAGE,
() -> jsValidationSteps.checkJsLogEntriesOnOpenedPage(logLevels));
verifyTestActions(expectedResults, "Current page contains no JavaScript errors, warnings");
}
Expand All @@ -112,7 +129,7 @@ void testCheckJsErrorsAndWarningsOnPage()
void testCheckJsErrorsContainsBrowserExtensionErrors()
{
jsValidationSteps.setIncludeBrowserExtensionLogEntries(true);
Map<String, Set<LogEntry>> expectedResults = testCheckJsErrors(EXTENSION + ERROR_MESSAGE,
Map<String, Collection<LogEntry>> expectedResults = testCheckJsErrors(EXTENSION + ERROR_MESSAGE,
() -> jsValidationSteps.checkJsLogEntriesOnOpenedPage(singletonList(BrowserLogLevel.ERRORS)));
verifyTestActions(expectedResults, ERRORS_ASSERTION_DESCRIPTION);
}
Expand All @@ -126,7 +143,38 @@ void testCheckJsErrorsExcludesBrowserExtensionErrors()
verifyTestActions(Map.of(URL, Collections.emptySet()), ERRORS_ASSERTION_DESCRIPTION);
}

private Map<String, Set<LogEntry>> testCheckJsErrors(String logErrorMessage, Runnable action)
@ParameterizedTest
@CsvSource({"true, 1", "false, 0"})
void shouldWaitForMessagesAndSaveResultIntoScopedVariable(boolean waitPassed, int savesVariable)
{
var webDriver = mock(WebDriver.class, withSettings().extraInterfaces(HasCapabilities.class));
when(webDriverProvider.get()).thenReturn(webDriver);
when(webDriver.getCurrentUrl()).thenReturn(URL);
var options = mock(Options.class);
when(webDriver.manage()).thenReturn(options);
var logs = mock(Logs.class);
when(options.logs()).thenReturn(logs);
var severeEntry = new LogEntry(Level.SEVERE, 1L, "severe");
var infoEntry = new LogEntry(Level.INFO, 1L, "info");
when(logs.get(LogType.BROWSER)).thenReturn(new LogEntries(List.of(severeEntry)),
new LogEntries(List.of(infoEntry)));
var result = new WaitResult<>();
result.setWaitPassed(waitPassed);
when(waitActions.wait(eq(webDriver), argThat(f -> {
f.apply(webDriver);
f.apply(webDriver);
return "appearance of JavaScript infos matching `.*info.*` regex".equals(f.toString());
}))).thenReturn(result);
var variableName = "variableName";
var scopes = Set.of(VariableScope.SCENARIO);
jsValidationSteps.waitForMessageAndSave(List.of(BrowserLogLevel.INFOS), Pattern.compile(".*info.*"), scopes,
variableName);
var ordered = Mockito.inOrder(attachmentPublisher, variableContext);
ordered.verify(variableContext, times(savesVariable)).putVariable(scopes, variableName, List.of(infoEntry));
verifyAttachmentPublisher(Map.of(URL, List.of(infoEntry)), ordered);
}

private Map<String, Collection<LogEntry>> testCheckJsErrors(String logErrorMessage, Runnable action)
{
LogEntry entry = mockGetLogEntry(logErrorMessage);
action.run();
Expand All @@ -149,15 +197,15 @@ private LogEntry mockGetLogEntry(String logErrorMessage)
return severeEntry;
}

private void verifyTestActions(Map<String, Set<LogEntry>> expectedResults, String assertionDescription)
private void verifyTestActions(Map<String, Collection<LogEntry>> expectedResults, String assertionDescription)
{
InOrder inOrder = inOrder(attachmentPublisher, softAssert);
verifyAttachmentPublisher(expectedResults, inOrder);
inOrder.verify(softAssert).assertEquals(assertionDescription, 0,
expectedResults.entrySet().iterator().next().getValue().size());
}

private void verifyAttachmentPublisher(Map<String, Set<LogEntry>> expectedResults, InOrder order)
private void verifyAttachmentPublisher(Map<String, Collection<LogEntry>> expectedResults, InOrder order)
{
order.verify(attachmentPublisher).publishAttachment(
"/org/vividus/steps/ui/web/js-console-validation-result-table.ftl",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,12 @@ Then there are no browser console ERRORS
Scenario: Verify step: Then there are no browser console $logEntries by regex '$regex'
When I execute javascript `console.error('error')` with arguments:
Then there are no browser console ERRORS by regex '.*message.*'


Scenario: Verify step: When I wait until browser console $logEntries by regex `$regex` appear and save all entries into $scopes variable `$variable`
When I execute javascript `console.log('immediate')` with arguments:
When I execute javascript `setTimeout(() => console.log("delayed"), 5000);` with arguments:
When I wait until browser console infos by regex `.*delayed.*` appear and save all entries into scenario variable `logs`
Then there are no browser console INFOS by regex '.*immediate.*'
Then `${logs}` matches `.*immediate.*`
Then `${logs}` matches `.*delayed.*`