Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ncu-ci walk pr handling of very broken CI runs #607

Open
richardlau opened this issue Feb 27, 2022 · 0 comments
Open

ncu-ci walk pr handling of very broken CI runs #607

richardlau opened this issue Feb 27, 2022 · 0 comments

Comments

@richardlau
Copy link
Member

Today's CI reliability GitHub actions run failed running ncu-ci walk pr --stats=true --markdown $PWD/results.md:
https://github.com/nodejs/reliability/actions/runs/1904541710

The log is too big to be displayed in the web UI but downloading the log shows that ncu-ci ran out of memory

2022-02-27T00:11:52.5187361Z ---------------------------- alpine-last-latest-x64 ----------------------------
2022-02-27T00:11:52.5187837Z URL        https://ci.nodejs.org/job/node-test-commit-linux/nodes=alpine-last-latest-x64/44808/console
2022-02-27T00:11:52.5187963Z Type       JS_TEST_FAILURE
2022-02-27T00:11:52.5188260Z Built On   test-softlayer-alpine311_container-x64-1
2022-02-27T00:11:52.5188350Z Reason     
2022-02-27T00:11:52.5188614Z   not ok 340 parallel/test-buffer-of-no-deprecation
2022-02-27T00:11:52.5188724Z     ---
2022-02-27T00:11:52.5188825Z     duration_ms: 0.138
2022-02-27T00:11:52.5188920Z     severity: fail
2022-02-27T00:11:52.5189008Z     exitcode: 1
2022-02-27T00:11:52.5189091Z     stack: |-
2022-02-27T00:11:52.5189297Z       You may need to run `make test/common/knownGlobals.json`.
2022-02-27T00:11:52.5189631Z       /home/iojs/build/workspace/node-test-commit-linux/test/common/index.js:260
2022-02-27T00:11:52.5189708Z         throw err;
2022-02-27T00:11:52.5189793Z         ^
2022-02-27T00:11:52.5189872Z       
2022-02-27T00:11:52.5190127Z       Error: Cannot find module './knownGlobals.json'
2022-02-27T00:11:52.5190223Z       Require stack:
2022-02-27T00:11:52.5190544Z       - /home/iojs/build/workspace/node-test-commit-linux/test/common/index.js
2022-02-27T00:11:52.5190958Z       - /home/iojs/build/workspace/node-test-commit-linux/test/parallel/test-buffer-of-no-deprecation.js
2022-02-27T00:11:52.5191204Z           at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)
2022-02-27T00:11:52.5191423Z           at Function.Module._load (node:internal/modules/cjs/loader:778:27)
2022-02-27T00:11:52.5191628Z           at Module.require (node:internal/modules/cjs/loader:999:19)
2022-02-27T00:11:52.5191818Z           at require (node:internal/modules/cjs/helpers:102:18)
2022-02-27T00:11:52.5192301Z           at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-linux/test/common/index.js:257:18)
2022-02-27T00:11:52.5192496Z           at Module._compile (node:internal/modules/cjs/loader:1097:14)
2022-02-27T00:11:52.5192813Z           at Object.Module._extensions..js (node:internal/modules/cjs/loader:1151:10)
2022-02-27T00:11:52.5193018Z           at Module.load (node:internal/modules/cjs/loader:975:32)
2022-02-27T00:11:52.5193236Z           at Function.Module._load (node:internal/modules/cjs/loader:822:12)
2022-02-27T00:11:52.5193443Z           at Module.require (node:internal/modules/cjs/loader:999:19) {
2022-02-27T00:11:52.5193629Z         code: 'MODULE_NOT_FOUND',
2022-02-27T00:11:52.5193730Z         requireStack: [
2022-02-27T00:11:52.5194191Z FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
2022-02-27T00:11:52.5194380Z  1: 0xa1a640 node::Abort() [node]
2022-02-27T00:11:52.5194583Z  2: 0xa1aa4c node::OnFatalError(char const*, char const*) [node]
2022-02-27T00:11:52.5194837Z  3: 0xb9a9fe v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
2022-02-27T00:11:52.5195123Z  4: 0xb9ad79 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
2022-02-27T00:11:52.5195222Z  5: 0xd57ff5  [node]
2022-02-27T00:11:52.5195470Z  6: 0xd58686 v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [node]
2022-02-27T00:11:52.5195789Z  7: 0xd64f45 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [node]
2022-02-27T00:11:52.5196157Z  8: 0xd65df5 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
2022-02-27T00:11:52.5196571Z  9: 0xd688ac v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
2022-02-27T00:11:52.5196899Z 10: 0xd3674c v8::internal::Factory::NewRawOneByteString(int, v8::internal::AllocationType) [node]
2022-02-27T00:11:52.5197346Z 11: 0xe4598b v8::internal::JsonParser<unsigned short>::MakeString(v8::internal::JsonString const&, v8::internal::Handle<v8::internal::String>) [node]
2022-02-27T00:11:52.5197632Z 12: 0xe4820e v8::internal::JsonParser<unsigned short>::ParseJsonValue() [node]
2022-02-27T00:11:52.5197899Z 13: 0xe48a1f v8::internal::JsonParser<unsigned short>::ParseJson() [node]
2022-02-27T00:11:52.5198215Z 14: 0xc4bf75 v8::internal::Builtin_JsonParse(int, unsigned long*, v8::internal::Isolate*) [node]
2022-02-27T00:11:52.5198326Z 15: 0x140e0f9  [node]
2022-02-27T00:11:52.5199009Z /home/runner/work/_temp/7f84222e-d06a-45db-9f36-c117354c1d5d.sh: line 1:  1546 Aborted                 (core dumped) ncu-ci walk pr --stats=true --markdown $PWD/results.md
2022-02-27T00:11:52.5199067Z 
2022-02-27T00:11:52.5199241Z <--- Last few GCs --->
2022-02-27T00:11:52.5199254Z 
2022-02-27T00:11:52.5199931Z [1546:0x3fe4130]    74251 ms: Mark-sweep 2049.3 (2051.9) -> 2049.3 (2053.9) MB, 1487.9 / 0.0 ms  (average mu = 0.068, current mu = 0.003) allocation failure scavenge might not succeed
2022-02-27T00:11:52.5200622Z [1546:0x3fe4130]    75708 ms: Mark-sweep 2051.1 (2053.9) -> 2051.1 (2055.1) MB, 1451.5 / 0.0 ms  (average mu = 0.037, current mu = 0.004) allocation failure scavenge might not succeed
2022-02-27T00:11:52.5200634Z 
2022-02-27T00:11:52.5200642Z 
2022-02-27T00:11:52.5200808Z <--- JS stacktrace --->
2022-02-27T00:11:52.5200817Z 
2022-02-27T00:11:52.5201019Z ==== JS stack trace =========================================
2022-02-27T00:11:52.5201028Z 
2022-02-27T00:11:52.5201213Z Security context: 0x0e3a480008d1 <JSObject>
2022-02-27T00:11:52.5201574Z     0: builtin exit frame: parse(this=0x0e3a4801ee79 <Object map = 0x2623d3703639>,0x30aed0880119 <Very long string[239420894]>,0x0e3a4801ee79 <Object map = 0x2623d3703639>)
2022-02-27T00:11:52.5201583Z 
2022-02-27T00:11:52.5202280Z     1: formatAsJson [0x2f4a7a6e6f61] [/opt/hostedtoolcache/node/12.22.10/x64/lib/node_modules/node-core-utils/lib/ci/build-types/test_build.js:185] [bytecode=0x14b3dd5cf491 offset=78](this=0x34038bbc2201 <Job map = 0x25f543...
2022-02-27T00:11:52.5202356Z 
2022-02-27T00:11:52.5213551Z ##[error]Process completed with exit code 134.
2022-02-27T00:11:52.5377172Z Cleaning up orphan processes

I'm speculating here that this is because the URL's being parsed are for CI jobs with a large number (3600) of test failures.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant