Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not run Scenario in worker when have big DataProvider #4867

Open
Yasuslik opened this issue Feb 26, 2025 · 0 comments
Open

Not run Scenario in worker when have big DataProvider #4867

Yasuslik opened this issue Feb 26, 2025 · 0 comments

Comments

@Yasuslik
Copy link

Yasuslik commented Feb 26, 2025

run-workers Fails When Using Data-Driven Tests (Data().Scenario) with Large Data Sets

Description

When running tests using CodeceptJS workers with the following command:

npx codeceptjs run-workers --suites 5

Tests using the Data(events.event19.cases).Scenario(...) structure do not execute properly.
However, if I reduce the amount of data in one test case, the test runs successfully.


Steps to Reproduce

  1. Run tests with multiple suites using run-workers:
    npx codeceptjs run-workers --suites 5
  2. Use a data-driven scenario:
    Data(events.event19.cases).Scenario(
      'Користувач натиснув на “Указать”',
      async ({ I, commentsPage, current }) => {
        I.setCookie(current.cookies);
        commentsPage.openCommentsPage(events.common.article);
        I.waitForVisible(commentsPage.enterButton);
        await I.clearInterceptedRequests();
        I.click(commentsPage.enterButton);
        I.waitForInvisible(commentsPage.authenticatedOverlayTitleText);
        I.see(current.text, commentsPage.authenticatedOverlayDiv);
      }
    );
  3. If I reduce the data in events.event19.cases, the test runs successfully.

Expected Behavior

✅ Tests should run correctly in parallel mode (run-workers), regardless of the data size.

Actual Behavior

❌ Tests fail silently, with no errors in the output.

  • No error message is displayed.
  • Running npx codeceptjs run --debug does not show any issues.
  • The test suite skips execution without failing explicitly.

Data Example (Fails with Larger Data Set)

Here is an example of my test data:

event19: {
  cases: [
    {
      testName: 'event19 Користувач натиснув на “Указать”',
      cookies: [
        { name: 'refresh_token', value: JWTTokenRefresh, domain: '.example.com', path: '/' },
        { name: 'token', value: JWTGoogleAccess, domain: '.example.com', path: '/' },
      ],
      text: 'Вам будет отправлено СМС с кодом подтверждения.',
      event: {
        url: '.example.com',
        method: 'POST',
        event_type: 'tap',
        postData: {
          event_type: 'tap',
          event_target: 'set_button',
          event_target_component: 'nickname_phone_number_absent_widget',
          event_target_layout_section: 'right_drawer',
          feature: 'user_profile',
        },
      },
    },
    // Additional test cases...
  ],
}

If I remove some properties or reduce the size of postData, the test starts working.


Verbose Output

TypeScript Definitions provide autocompletion in Visual Studio Code and other IDEs
Definitions were generated in steps.d.ts
CodeceptJS v3.7.2 #StandWithUkraine
Running tests in 5 workers...

***************************************
nodeInfo:  20.18.0
osInfo:  macOS 15.3
cpuInfo:  (8) arm64 Apple M1 Pro
chromeInfo:  133.0.6943.141
edgeInfo:  120.0.2210.61
firefoxInfo:  undefined
safariInfo:  18.3
If you need more detailed info, just run this: npx codeceptjs info
***************************************
CodeceptJS v3.7.2 #StandWithUkraine
Using test root "/Users/r-bondarenko/Project/QA-codecept/config"
Helpers: Playwright, REST, JSONResponse, ChaiWrapper, InterceptHelper, ResponseHelper, TestHelper, MongoHelper, WebsiteHelper
Plugins: screenshotOnFail


  OK  | 0 passed   // 0ms

No tests found by pattern: /@commentsSusl/
CodeceptJS v3.7.2 #StandWithUkraine
Using test root "/Users/r-bondarenko/Project/QA-codecept/config"
Helpers: Playwright, REST, JSONResponse, ChaiWrapper, InterceptHelper, ResponseHelper, TestHelper, MongoHelper, WebsiteHelper
Plugins: screenshotOnFail


  OK  | 0 passed   // 0ms

No tests found by pattern: /@commentsSusl/
CodeceptJS v3.7.2 #StandWithUkraine
Using test root "/Users/r-bondarenko/Project/QA-codecept/config"
Helpers: Playwright, REST, JSONResponse, ChaiWrapper, InterceptHelper, ResponseHelper, TestHelper, MongoHelper, WebsiteHelper
Plugins: screenshotOnFail

CodeceptJS v3.7.2 #StandWithUkraine
Using test root "/Users/r-bondarenko/Project/QA-codecept/config"
Helpers: Playwright, REST, JSONResponse, ChaiWrapper, InterceptHelper, ResponseHelper, TestHelper, MongoHelper, WebsiteHelper
Plugins: screenshotOnFail


  OK  | 0 passed   // 0ms

No tests found by pattern: /@commentsSusl/

Maybe you wanted to run one of these tests?
☐ @regression @smoke @commentsBlock: Check and approve comment in recipe
☐ @regression @smoke @commentsBlock: Check and approve comment in article | {"5000003"}
☐ @regression @smoke @commentsBlock: Check and approve comment in article | {"5000007"}
☐ @regression @smoke @commentsBlock: Check and approve comment with 1 symbol in article | {"5000003"}
☐ @regression @smoke @commentsBlock: Check and approve comment with 1 symbol in article | {"5000007"}
☐ @regression @smoke @commentsBlock: Check and approve comment with 2700 symbols in article | {"5000003"}
☐ @regression @smoke @commentsBlock: Check and approve comment with 2700 symbols in article | {"5000007"}

To run the first test use the following command:
npx codeceptjs run --debug --grep "@regression @smoke @commentsBlock: Check and approve comment in recipe"

  OK  | 0 passed   // 0ms

No tests found by pattern: /@commentsSusl/
CodeceptJS v3.7.2 #StandWithUkraine
Using test root "/Users/r-bondarenko/Project/QA-codecept/config"
Helpers: Playwright, REST, JSONResponse, ChaiWrapper, InterceptHelper, ResponseHelper, TestHelper, MongoHelper, WebsiteHelper
Plugins: screenshotOnFail


  OK  | 0 passed   // 0ms

🔹 No explicit error message appears.
🔹 Tests are not running, but are also not failing.


Environment

Property Value
OS macOS 15.3
Node.js v20.18.0
CodeceptJS v3.7.2
Browser Chromium 133.0.6943.141
Helpers Playwright, REST, JSONResponse, ChaiWrapper, InterceptHelper, ResponseHelper, TestHelper, MongoHelper, WebsiteHelper
Plugins screenshotOnFail

Possible Causes

  • Memory allocation issue when handling large test data.
  • run-workers doesn't handle large JSON objects correctly in Data().Scenario().
  • Test case names may be too long when using testName as a dynamic string.
  • CodeceptJS parallel execution doesn't support deep JSON structures well in data-driven tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant