Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flaky tests caused by parallel jobs log parsing #60

Closed
TWiStErRob opened this issue Dec 8, 2023 · 2 comments · Fixed by #62
Closed

Flaky tests caused by parallel jobs log parsing #60

TWiStErRob opened this issue Dec 8, 2023 · 2 comments · Fixed by #62

Comments

@TWiStErRob
Copy link
Contributor

Describe the bug
If one job starts a group, and another logs while that group is open, then the parser crashes with:

Cannot read properties of undefined (reading 'output')
TypeError: Cannot read properties of undefined (reading 'output')
    at OutputParser.parseStepOutput (node_modules/@kie/act-js/build/src/output-parser/output-parser.js:119:61)
    at OutputParser.parseOutput (node_modules/@kie/act-js/build/src/output-parser/output-parser.js:26:18)
this.groupMatrix[stepOutputMatcherResult[1]][length - 1].output += ...
                                                        ^ LHS undefined

this.groupMatrix[stepOutputMatcherResult[1]] === [], therefore length is 0, and ...[-1] === undefined.

To Reproduce

test("original", async () => {
    const contents = await fs.readFile("act.log");
    console.log(new OutputParser(contents.toString()).parseOutput());
});

act.log:

[Build/Tests            ] ⭐ Run Main Testing
[Build/Tests            ]   ❓  ::group::Unit Tests
[Build/Static Analysis  ] ⭐ Run Main Lint
[Build/Static Analysis  ]   | No problems found.
[Build/Tests            ]   | 1231 tests executed.
[Build/Tests            ]   ❓  ::endgroup::
[Build/Tests            ]   ❓  ::group::Integration Tests
[Build/Tests            ]   | 32 tests executed.
[Build/Tests            ]   ❓  ::endgroup::
[Build/Static Analysis  ]   ✅  Success - Main Linting
[Build/Tests            ]   ✅  Success - Main Testing

Expected behavior
"Just works".

Logs

Distilled real log focused on the issue
[Android Build/Unit Tests       ] 🚀  Start image=ghcr.io/catthehacker/ubuntu:act-latest
[Android Build/Android Lint     ] 🚀  Start image=ghcr.io/catthehacker/ubuntu:act-latest
[Android Build/Unit Tests       ] ⭐ Run Main Set up Android SDK.
[Android Build/Unit Tests       ]   🐳  docker cp src=/home/runner/.cache/act/android-actions-setup-android@v3/ dst=/var/run/act/actions/android-actions-setup-android@v3/
[Android Build/Android Lint     ] ⭐ Run Main Set up Android SDK.
[Android Build/Android Lint     ]   🐳  docker cp src=/home/runner/.cache/act/android-actions-setup-android@v3/ dst=/var/run/act/actions/android-actions-setup-android@v3/
[Android Build/Android Lint     ]   | [command]/root/.android/sdk/cmdline-tools/11.0/bin/sdkmanager tools
[Android Build/Unit Tests       ]   | [command]/root/.android/sdk/cmdline-tools/11.0/bin/sdkmanager tools
[Android Build/Android Lint     ]   | [command]/root/.android/sdk/cmdline-tools/11.0/bin/sdkmanager platform-tools
[Android Build/Android Lint     ]   ❓ add-matcher /run/act/actions/android-actions-setup-android@v3/matchers.json
[Android Build/Android Lint     ]   ✅  Success - Main Set up Android SDK.
[Android Build/Android Lint     ] ⭐ Run Main Set up Java for Project.
[Android Build/Android Lint     ]   🐳  docker cp src=/home/runner/.cache/act/actions-setup-java@v3/ dst=/var/run/act/actions/actions-setup-java@v3/
[Android Build/Android Lint     ]   ❓  ::group::Installed distributions
[Android Build/Android Lint     ]   | Resolved Java 17.0.9+9 from tool-cache
[Android Build/Unit Tests       ]   | [command]/root/.android/sdk/cmdline-tools/11.0/bin/sdkmanager platform-tools
[Android Build/Android Lint     ]   |   Path: /opt/hostedtoolcache/Java_Temurin-Hotspot_jdk/17.0.9-9/x64
[Android Build/Android Lint     ]   ❓  ::endgroup::
[Android Build/Android Lint     ]   ✅  Success - Main Set up Java for Project.
[Android Build/Unit Tests       ]   ❓ add-matcher /run/act/actions/android-actions-setup-android@v3/matchers.json
[Android Build/Unit Tests       ]   ✅  Success - Main Set up Android SDK.

Additional context
Looking at output-parser.js, as far as I see .isPartOfGroup is handled on the parser level, but it should per-job.

TWiStErRob added a commit to TWiStErRob/act-js that referenced this issue Dec 8, 2023
TWiStErRob added a commit to TWiStErRob/act-js that referenced this issue Dec 8, 2023
@TWiStErRob TWiStErRob changed the title Flaky tests: Parallel jobs might result in parse errors of logs Flaky tests caused by parallel jobs log parsing Dec 8, 2023
shubhbapna pushed a commit that referenced this issue Dec 27, 2023
* Import project in IntelliJ IDEA (ignore project folder)

* Add basic test for groups

* Add tests for #60 and interleaved groups

* Fix #60
@TWiStErRob
Copy link
Contributor Author

Can you please release this?

@shubhbapna
Copy link
Collaborator

Done!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants