[Feature]: The transformer tests are not run in parallel

[trivikr]

Is your feature request related to a problem? Please describe.

The transformer tests are not run in parallel

The tests start in parallel, but are run in sync

Diff:

-  describe.each(fixtureSubDirs)("%s", (subDir) => {
+  describe.each(["new-client"])("%s", (subDir) => {
     const subDirPath = join(fixtureDir, subDir);
     it.concurrent.each(getTestFileMetadata(subDirPath))(
       `transforms: %s.%s`,
       async (filePrefix, fileExtension) => {
+        const startDate = new Date();
+        console.log(`${filePrefix} start: `, startDate.toTimeString());
+
         const { input, outputCode } = await getTestMetadata(subDirPath, filePrefix, fileExtension);
         runInlineTest(transformer, null, input, outputCode);
+
+        const endDate = new Date();
+        console.log(`${filePrefix} end: ${endDate.toTimeString()}`);
+        console.log(`${filePrefix} test run time: ${endDate.getTime() - startDate.getTime()}ms`);
       }
     );

Output:

    global-import start:  14:25:39 GMT-0800 (Pacific Standard Time)
    global-require start:  14:25:39 GMT-0800 (Pacific Standard Time)
    service-import-deep start:  14:25:39 GMT-0800 (Pacific Standard Time)
    service-import start:  14:25:39 GMT-0800 (Pacific Standard Time)
    service-require-deep start:  14:25:39 GMT-0800 (Pacific Standard Time)

    service-import end: 14:25:51 GMT-0800 (Pacific Standard Time)
    service-import test run time: 11781ms

    service-require start:  14:25:51 GMT-0800 (Pacific Standard Time)

    global-require end: 14:26:04 GMT-0800 (Pacific Standard Time)
    global-require test run time: 24973ms

    service-require-deep end: 14:26:20 GMT-0800 (Pacific Standard Time)
    service-require-deep test run time: 40440ms

    global-import end: 14:26:30 GMT-0800 (Pacific Standard Time)
    global-import test run time: 50777ms

    service-import-deep end: 14:26:41 GMT-0800 (Pacific Standard Time)
    service-import-deep test run time: 62212ms

    service-require end: 14:27:00 GMT-0800 (Pacific Standard Time)
    service-require test run time: 68779ms

This leads to CI taking time. For example, the test step takes 3m https://github.com/awslabs/aws-sdk-js-codemod/actions/runs/3803141886/jobs/6469293363

Describe the solution you'd like

Experiment with Jest code or configuration to run tests in parallel

Describe alternatives you've considered

N/A

Additional Context

$ yarn dlx -q envinfo --preset jest

  System:
    OS: macOS 12.1
    CPU: (8) arm64 Apple M1 Pro
  Binaries:
    Node: 18.12.1 - /private/var/folders/42/54jl1_3x4hz06cf7bc_kzd4h0000gn/T/xfs-6daa84fb/node
    Yarn: 3.3.1 - /private/var/folders/42/54jl1_3x4hz06cf7bc_kzd4h0000gn/T/xfs-6daa84fb/yarn
    npm: 8.19.2 - ~/Library/Caches/fnm_multishells/57062_1672199177403/bin/npm
  npmPackages:
    jest: ^29.3.1 => 29.3.1

[trivikr]

This probably got to do with transform being sync. The implementation needs to be made async.

Refs: facebook/jscodeshift#210

[trivikr]

The issue appears to be with codemod setup.

The following test code runs in parallel

import { setTimeout } from "timers/promises";

describe("test", () => {
  test.concurrent.each([1, 2, 3, 4, 5])("concurrency test", async () => {
    await setTimeout(1000);
    expect(true).toBe(true);
  });
});

$ yarn jest --version
29.3.1

$ yarn why ts-jest
└─ jest-concurrent-each-test@workspace:.
   └─ ts-jest@npm:29.0.3 [26b7a] (via npm:^29.0.3 [26b7a])

$ yarn jest test.spec.ts
 PASS  ./test.spec.ts
  test
    ✓ concurrency test (1004 ms)
    ✓ concurrency test (1 ms)
    ✓ concurrency test
    ✓ concurrency test
    ✓ concurrency test (1 ms)

Test Suites: 1 passed, 1 total
Tests:       5 passed, 5 total
Snapshots:   0 total
Time:        1.716 s, estimated 2 s

[trivikr]

The transformer is run in parallel when it's async.
However, it takes the same amount of time to run.

Example workflow run from related PR https://github.com/awslabs/aws-sdk-js-codemod/actions/runs/3804235885/jobs/6471299801

[trivikr]

One performance improvement could be to run source.find on all client names at once

https://github.com/awslabs/aws-sdk-js-codemod/blob/7cec9bdefc31c0f994e39a9fc3719ff5087527fd/src/transforms/v2-to-v3/utils/get/getV2ClientNamesRecord.ts#L17-L21

[trivikr]

Another improvement would be to pass clientsMetadata to individual function so the calls to source.find can be reduced

https://github.com/awslabs/aws-sdk-js-codemod/blob/7cec9bdefc31c0f994e39a9fc3719ff5087527fd/src/transforms/v2-to-v3/transformer.ts#L36-L52

[trivikr]

Tests are now running in parallel after transformer is made async in #258

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.