Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SEV postmortem: outage due to elevated CPU and memory usage #129

Open
raxod502 opened this issue Oct 22, 2021 · 3 comments
Open

SEV postmortem: outage due to elevated CPU and memory usage #129

raxod502 opened this issue Oct 22, 2021 · 3 comments
Labels
resources sev

Comments

@raxod502
Copy link
Member

It appears that Riju was unavailable from about 10:15pm to 10:20pm PT on Thu, Oct 21, 2021.

Supervisor logs during the incident 
2021/10/22 05:13:37 active: downloading deployment config from S3
2021/10/22 05:13:37 active: listing locally available images
2021/10/22 05:13:37 active: config hash remains at 4e9d4c662a5b878e94c67806bb34341f67b70ce6
2021/10/22 05:13:37 succeeded
2021/10/22 05:15:38 http: proxy error: context canceled
2021/10/22 05:15:47 http: proxy error: context canceled
2021/10/22 05:16:32 http: proxy error: context canceled
2021/10/22 05:17:05 http: proxy error: context canceled
2021/10/22 05:17:06 http: proxy error: context canceled
2021/10/22 05:17:14 http: proxy error: context canceled
2021/10/22 05:17:17 http: proxy error: context canceled
2021/10/22 05:17:22 http: proxy error: context canceled
2021/10/22 05:17:25 active: getting access token from ECR
2021/10/22 05:17:25 http: proxy error: context canceled
2021/10/22 05:17:35 http: proxy error: context canceled
2021/10/22 05:17:46 http: proxy error: context canceled
2021/10/22 05:17:47 http: proxy error: context canceled
2021/10/22 05:17:50 http: proxy error: context canceled
2021/10/22 05:17:55 http: proxy error: context canceled
2021/10/22 05:18:02 http: proxy error: context canceled
2021/10/22 05:18:03 http: proxy error: context canceled
2021/10/22 05:18:13 http: proxy error: context canceled
2021/10/22 05:18:13 http: proxy error: context canceled
2021/10/22 05:18:31 http: proxy error: context canceled
2021/10/22 05:18:39 http: proxy error: context canceled
2021/10/22 05:18:40 http: proxy error: context canceled
2021/10/22 05:18:51 http: proxy error: context canceled
2021/10/22 05:19:04 http: proxy error: context canceled
2021/10/22 05:19:27 http: proxy error: context canceled
2021/10/22 05:19:28 active: authenticating Docker client to ECR
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
2021/10/22 05:19:33 active: downloading deployment config from S3
2021/10/22 05:19:34 active: listing locally available images
2021/10/22 05:19:34 http: proxy error: context canceled
2021/10/22 05:20:05 http: proxy error: context canceled
2021/10/22 05:20:06 http: proxy error: context canceled
2021/10/22 05:20:17 http: proxy error: context canceled
2021/10/22 05:20:21 active: config hash remains at 4e9d4c662a5b878e94c67806bb34341f67b70ce6
2021/10/22 05:20:21 succeeded
2021/10/22 05:20:21 active: getting access token from ECR
2021/10/22 05:20:21 active: authenticating Docker client to ECR
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
Webserver logs during the incident 
Oct 22 05:01:41 [ac48a087f4b6486fb9632b0a48593756] Tearing down session
Oct 22 05:01:41 [1b4f9d7494764e3fb6fb7cc66fe46216] Creating session, language racket
Oct 22 05:02:01 [8ef6cd0f18244defb50017db5ea94c9e] Tearing down session
Oct 22 05:02:03 [f46740c42f98413ea66d0566eabea88a] Creating session, language racket
Oct 22 05:02:10 [de138089d7a944a39ff0ce750c1b5bfb] Tearing down session
Oct 22 05:02:11 [704ea02eae95444b9811a1b2ad1e8325] Creating session, language racket
Oct 22 05:02:26 [9a8fff0c7f244453b72d569380783d75] Tearing down session
Oct 22 05:02:28 [3b14235065e34df9bb1c80c0ac328024] Tearing down session
Oct 22 05:02:28 [2fee25148e484d1185c3905512135a5f] Creating session, language racket
Oct 22 05:02:42 [0a611d4669ba4be1865bc004463ccf63] Tearing down session
Oct 22 05:02:43 [a353b4e666b34bc99344ccbcb32e4a8c] Creating session, language racket
Oct 22 05:02:43 [ee9b2f38e0b84379966a66f2387b26d8] Tearing down session
Oct 22 05:02:44 [0dd5ef16cf6d482da93e93fb0bf14768] Creating session, language racket
Oct 22 05:02:52 [788d8b7495604ebb8f1297b2c55b09f1] Tearing down session
Oct 22 05:02:53 [df838b0828d74b81b876bdb4752c9463] Creating session, language haskell
Oct 22 05:02:57 [df838b0828d74b81b876bdb4752c9463] Tearing down session
Oct 22 05:03:45 [f46740c42f98413ea66d0566eabea88a] Tearing down session
Oct 22 05:03:45 [c8a8fe0931fd48b9827d94718febb65c] Creating session, language racket
Oct 22 05:03:46 [015acb5d109c4da4aebf72b23b4ac223] Creating session, language racket
Oct 22 05:03:53 [704ea02eae95444b9811a1b2ad1e8325] Tearing down session
Oct 22 05:03:54 [490c597f034b4fcdb4c39376a623f117] Creating session, language racket
Oct 22 05:04:10 [2fee25148e484d1185c3905512135a5f] Tearing down session
Oct 22 05:04:11 [959f4b8e980b4fcaa271ac157956d747] Creating session, language racket
Oct 22 05:04:15 [28ce62079c0e45099dc222e097bd2132] Tearing down session
Oct 22 05:04:15 [cb06b3dd918b4d43995c77d8f3dd4dfa] Creating session, language haskell
Oct 22 05:04:17 [1b4f9d7494764e3fb6fb7cc66fe46216] Tearing down session
Oct 22 05:04:17 [92f2db7f39fa4498a8c5c21597f444d3] Creating session, language racket
Oct 22 05:04:24 [a353b4e666b34bc99344ccbcb32e4a8c] Tearing down session
Oct 22 05:04:26 [0dd5ef16cf6d482da93e93fb0bf14768] Tearing down session
Oct 22 05:04:27 [3baeefe7394b4f1eb7eaeaceeef7c7dc] Creating session, language racket
Oct 22 05:04:44 [015acb5d109c4da4aebf72b23b4ac223] Tearing down session
Oct 22 05:05:28 [41d1a8a8d18a4a8e9645b1a0c8c2bc02] Tearing down session
Oct 22 05:05:28 [309c77a63eb2401c8a85e674599b6ef6] Creating session, language racket
Oct 22 05:05:35 [490c597f034b4fcdb4c39376a623f117] Tearing down session
Oct 22 05:05:36 [13f71287bb7e4a588cb0aa50d10971c2] Creating session, language racket
Oct 22 05:05:53 [959f4b8e980b4fcaa271ac157956d747] Tearing down session
Oct 22 05:05:54 [c3ab4f7b96b54f98b3e19ba9fbbdbd82] Creating session, language racket
Oct 22 05:06:08 [3baeefe7394b4f1eb7eaeaceeef7c7dc] Tearing down session
Oct 22 05:06:09 [4f9649d51a8f4fa2a5ad95b2cd05a2f8] Creating session, language racket
Oct 22 05:06:25 [c8a8fe0931fd48b9827d94718febb65c] Tearing down session
Oct 22 05:06:26 [dd8f51f42151456aa32f981499481efc] Creating session, language racket
Oct 22 05:07:10 [309c77a63eb2401c8a85e674599b6ef6] Tearing down session
Oct 22 05:07:10 [b20120b70c974ecdb093d07b642b4a6c] Creating session, language racket
Oct 22 05:07:17 [13f71287bb7e4a588cb0aa50d10971c2] Tearing down session
Oct 22 05:07:20 [92f2db7f39fa4498a8c5c21597f444d3] Tearing down session
Oct 22 05:07:20 [570f6cd390c54a048b0b28ea357f0138] Creating session, language racket
Oct 22 05:07:35 [c3ab4f7b96b54f98b3e19ba9fbbdbd82] Tearing down session
Oct 22 05:07:35 [7bb590b0840f44c9b75e2a82a2521514] Creating session, language racket
Oct 22 05:07:38 [7bb590b0840f44c9b75e2a82a2521514] Tearing down session
Oct 22 05:07:49 [cb06b3dd918b4d43995c77d8f3dd4dfa] Tearing down session
Oct 22 05:07:50 [1c845dc46e85494db6708bffc8ce1db5] Creating session, language haskell
Oct 22 05:07:50 [4f9649d51a8f4fa2a5ad95b2cd05a2f8] Tearing down session
Oct 22 05:07:52 [7f1742f34cce48adbe309951402c4a9a] Creating session, language racket
Oct 22 05:08:51 [b20120b70c974ecdb093d07b642b4a6c] Tearing down session
Oct 22 05:08:51 [f3afc23389d245e3be2bc90496437fda] Creating session, language racket
Oct 22 05:09:01 [570f6cd390c54a048b0b28ea357f0138] Tearing down session
Oct 22 05:09:02 [2e1da9ecf09b41aab1bd2f2c14a0d395] Creating session, language racket
Oct 22 05:09:30 [1c845dc46e85494db6708bffc8ce1db5] Tearing down session
Oct 22 05:09:31 [7646fb333f2c4e7e83638b4d55c10eab] Creating session, language haskell
Oct 22 05:09:33 [7f1742f34cce48adbe309951402c4a9a] Tearing down session
Oct 22 05:10:26 [dcd5df7aa5d540aaba8aabe81112101f] Creating session, language racket
Oct 22 05:10:30 [dd8f51f42151456aa32f981499481efc] Tearing down session
Oct 22 05:10:31 [34aea86dd2604e42b6b41e9dd3f644f9] Creating session, language racket
Oct 22 05:10:44 [2e1da9ecf09b41aab1bd2f2c14a0d395] Tearing down session
Oct 22 05:10:44 [c382a398feeb4a27865e366662eae8b3] Creating session, language racket
Oct 22 05:11:05 [f3afc23389d245e3be2bc90496437fda] Tearing down session
Oct 22 05:11:05 [b8082062d4914c8c9501c1721d1a8370] Creating session, language racket
Oct 22 05:11:12 [7646fb333f2c4e7e83638b4d55c10eab] Tearing down session
Oct 22 05:11:13 [18f73269c9024a6ebcfd84597443ebc7] Creating session, language haskell
Oct 22 05:12:02 [9f8c82807b6a419d83ebfecf597c624d] Creating session, language racket
Oct 22 05:12:21 [9cd8ea2269fd4c9294f7c946db5b8819] Creating session, language racket
Oct 22 05:12:25 [c382a398feeb4a27865e366662eae8b3] Tearing down session
Oct 22 05:12:27 [1ef333599a54405f93448e3a8c466b91] Creating session, language racket
Oct 22 05:12:39 [9169190c00264ab7a0c508b13d81137c] Creating session, language racket
Oct 22 05:13:31 [34aea86dd2604e42b6b41e9dd3f644f9] Tearing down session
Oct 22 05:13:31 [9f4bfa28922742c49dea9858f4d02715] Creating session, language racket
Oct 22 05:14:08 [1ef333599a54405f93448e3a8c466b91] Tearing down session
Oct 22 05:14:08 [4ec9a9d29283405a91d89abf5830e3dd] Creating session, language racket
Oct 22 05:14:24 [f2c86ef829e54c919af585232b203f60] Creating session, language racket
Oct 22 05:17:54 [9f8c82807b6a419d83ebfecf597c624d] Output from system/out/riju-system-privileged:
Oct 22 05:17:55 sentinel did not set up FIFOs within 1 second (errno 2)
Oct 22 05:17:56 sentinel did not set up FIFOs within 1 second (errno 2)
Oct 22 05:17:58 sentinel did not set up FIFOs within 1 second (errno 2)
Oct 22 05:17:59 sentinel did not set up FIFOs within 1 second (errno 2)
Oct 22 05:18:04 command system/out/riju-system-privileged failed with error code 1
Oct 22 05:19:28 Error: WebSocket is not open: readyState 2 (CLOSING)
Oct 22 05:19:28     at WebSocket.send (/src/node_modules/ws/lib/websocket.js:314:19)
Oct 22 05:19:28     at Session.send (file:///src/backend/api.js:175:15)
Oct 22 05:19:28     at ChildProcess.<anonymous> (file:///src/backend/api.js:339:16)
Oct 22 05:19:28     at ChildProcess.emit (node:events:394:28)
Oct 22 05:19:28     at ChildProcess.emit (node:domain:475:12)
Oct 22 05:19:28     at maybeClose (node:internal/child_process:1064:16)
Oct 22 05:19:28     at Socket.<anonymous> (node:internal/child_process:450:11)
Oct 22 05:19:28     at Socket.emit (node:events:394:28)
Oct 22 05:19:28     at Socket.emit (node:domain:475:12)
Oct 22 05:19:28     at Pipe.<anonymous> (node:net:672:12)
Oct 22 05:19:28 [dcd5df7aa5d540aaba8aabe81112101f] Tearing down session
Oct 22 05:19:28 Error: WebSocket is not open: readyState 2 (CLOSING)
Oct 22 05:19:28     at WebSocket.send (/src/node_modules/ws/lib/websocket.js:314:19)
Oct 22 05:19:28     at Session.send (file:///src/backend/api.js:175:15)
Oct 22 05:19:28     at ChildProcess.<anonymous> (file:///src/backend/api.js:339:16)
Oct 22 05:19:28     at ChildProcess.emit (node:events:394:28)
Oct 22 05:19:28     at ChildProcess.emit (node:domain:475:12)
Oct 22 05:19:28     at maybeClose (node:internal/child_process:1064:16)
Oct 22 05:19:28     at Socket.<anonymous> (node:internal/child_process:450:11)
Oct 22 05:19:28     at Socket.emit (node:events:394:28)
Oct 22 05:19:28     at Socket.emit (node:domain:475:12)
Oct 22 05:19:28     at Pipe.<anonymous> (node:net:672:12)
Oct 22 05:19:29 [9f4bfa28922742c49dea9858f4d02715] Tearing down session
Oct 22 05:19:29 Error: WebSocket is not open: readyState 2 (CLOSING)
Oct 22 05:19:29     at WebSocket.send (/src/node_modules/ws/lib/websocket.js:314:19)
Oct 22 05:19:29     at Session.send (file:///src/backend/api.js:175:15)
Oct 22 05:19:29     at ChildProcess.<anonymous> (file:///src/backend/api.js:339:16)
Oct 22 05:19:29     at ChildProcess.emit (node:events:394:28)
Oct 22 05:19:29     at ChildProcess.emit (node:domain:475:12)
Oct 22 05:19:29     at maybeClose (node:internal/child_process:1064:16)
Oct 22 05:19:29     at Socket.<anonymous> (node:internal/child_process:450:11)
Oct 22 05:19:29     at Socket.emit (node:events:394:28)
Oct 22 05:19:29     at Socket.emit (node:domain:475:12)
Oct 22 05:19:29     at Pipe.<anonymous> (node:net:672:12)
Oct 22 05:19:29 [4ec9a9d29283405a91d89abf5830e3dd] Tearing down session
Oct 22 05:19:29 Error: WebSocket is not open: readyState 2 (CLOSING)
Oct 22 05:19:29     at WebSocket.send (/src/node_modules/ws/lib/websocket.js:314:19)
Oct 22 05:19:29     at Session.send (file:///src/backend/api.js:175:15)
Oct 22 05:19:29     at ChildProcess.<anonymous> (file:///src/backend/api.js:339:16)
Oct 22 05:19:29     at ChildProcess.emit (node:events:394:28)
Oct 22 05:19:29     at ChildProcess.emit (node:domain:475:12)
Oct 22 05:19:29     at maybeClose (node:internal/child_process:1064:16)
Oct 22 05:19:29     at Socket.<anonymous> (node:internal/child_process:450:11)
Oct 22 05:19:29     at Socket.emit (node:events:394:28)
Oct 22 05:19:29     at Socket.emit (node:domain:475:12)
Oct 22 05:19:29     at Pipe.<anonymous> (node:net:672:12)
Oct 22 05:19:29 [9cd8ea2269fd4c9294f7c946db5b8819] Tearing down session
Oct 22 05:19:29 Error: WebSocket is not open: readyState 2 (CLOSING)
Oct 22 05:19:29     at WebSocket.send (/src/node_modules/ws/lib/websocket.js:314:19)
Oct 22 05:19:29     at Session.send (file:///src/backend/api.js:175:15)
Oct 22 05:19:29     at ChildProcess.<anonymous> (file:///src/backend/api.js:339:16)
Oct 22 05:19:29     at ChildProcess.emit (node:events:394:28)
Oct 22 05:19:29     at ChildProcess.emit (node:domain:475:12)
Oct 22 05:19:29     at maybeClose (node:internal/child_process:1064:16)
Oct 22 05:19:29     at Socket.<anonymous> (node:internal/child_process:450:11)
Oct 22 05:19:29     at Socket.emit (node:events:394:28)
Oct 22 05:19:29     at Socket.emit (node:domain:475:12)
Oct 22 05:19:29     at Pipe.<anonymous> (node:net:672:12)
Oct 22 05:19:29 [9169190c00264ab7a0c508b13d81137c] Tearing down session
Oct 22 05:19:29 Error: WebSocket is not open: readyState 2 (CLOSING)
Oct 22 05:19:29     at WebSocket.send (/src/node_modules/ws/lib/websocket.js:314:19)
Oct 22 05:19:29     at Session.send (file:///src/backend/api.js:175:15)
Oct 22 05:19:29     at ChildProcess.<anonymous> (file:///src/backend/api.js:339:16)
Oct 22 05:19:29     at ChildProcess.emit (node:events:394:28)
Oct 22 05:19:29     at ChildProcess.emit (node:domain:475:12)
Oct 22 05:19:29     at maybeClose (node:internal/child_process:1064:16)
Oct 22 05:19:29     at Socket.<anonymous> (node:internal/child_process:450:11)
Oct 22 05:19:29     at Socket.emit (node:events:394:28)
Oct 22 05:19:29     at Socket.emit (node:domain:475:12)
Oct 22 05:19:29     at Pipe.<anonymous> (node:net:672:12)
Oct 22 05:19:29 [b8082062d4914c8c9501c1721d1a8370] Tearing down session
Oct 22 05:19:29 Error: WebSocket is not open: readyState 2 (CLOSING)
Oct 22 05:19:29     at WebSocket.send (/src/node_modules/ws/lib/websocket.js:314:19)
Oct 22 05:19:29     at Session.send (file:///src/backend/api.js:175:15)
Oct 22 05:19:29     at ChildProcess.<anonymous> (file:///src/backend/api.js:339:16)
Oct 22 05:19:29     at ChildProcess.emit (node:events:394:28)
Oct 22 05:19:29     at ChildProcess.emit (node:domain:475:12)
Oct 22 05:19:29     at maybeClose (node:internal/child_process:1064:16)
Oct 22 05:19:29     at Socket.<anonymous> (node:internal/child_process:450:11)
Oct 22 05:19:29     at Socket.emit (node:events:394:28)
Oct 22 05:19:29     at Socket.emit (node:domain:475:12)
Oct 22 05:19:29     at Pipe.<anonymous> (node:net:672:12)
Oct 22 05:19:29 [18f73269c9024a6ebcfd84597443ebc7] Tearing down session
Oct 22 05:19:29 Error: WebSocket is not open: readyState 2 (CLOSING)
Oct 22 05:19:29     at WebSocket.send (/src/node_modules/ws/lib/websocket.js:314:19)
Oct 22 05:19:29     at Session.send (file:///src/backend/api.js:175:15)
Oct 22 05:19:29     at ChildProcess.<anonymous> (file:///src/backend/api.js:339:16)
Oct 22 05:19:29     at ChildProcess.emit (node:events:394:28)
Oct 22 05:19:29     at ChildProcess.emit (node:domain:475:12)
Oct 22 05:19:29     at maybeClose (node:internal/child_process:1064:16)
Oct 22 05:19:29     at Socket.<anonymous> (node:internal/child_process:450:11)
Oct 22 05:19:29     at Socket.emit (node:events:394:28)
Oct 22 05:19:29     at Socket.emit (node:domain:475:12)
Oct 22 05:19:29     at Pipe.<anonymous> (node:net:672:12)
Oct 22 05:19:30 [f2c86ef829e54c919af585232b203f60] Tearing down session
Oct 22 05:19:31 [921fe7acd1334244bed05e5cadce5378] Creating session, language racket
Oct 22 05:19:31 [b80a7fb27db74df3bfb5ef6c38e061dd] Creating session, language racket
Oct 22 05:19:31 [9f8c82807b6a419d83ebfecf597c624d] Output from system/out/riju-system-privileged:
Oct 22 05:19:31 sentinel did not set up FIFOs within 1 second (errno 2)
Oct 22 05:19:31 sentinel did not set up FIFOs within 1 second (errno 2)
Oct 22 05:19:31 command system/out/riju-system-privileged failed with error code 1
Oct 22 05:19:35 Error: WebSocket is not open: readyState 2 (CLOSING)
Oct 22 05:19:35     at WebSocket.send (/src/node_modules/ws/lib/websocket.js:314:19)
Oct 22 05:19:35     at Session.send (file:///src/backend/api.js:175:15)
Oct 22 05:19:35     at Session.sendError (file:///src/backend/api.js:187:16)
Oct 22 05:19:35     at Session.runCode (file:///src/backend/api.js:358:12)
Oct 22 05:19:35     at runMicrotasks (<anonymous>)
Oct 22 05:19:35     at processTicksAndRejections (node:internal/process/task_queues:96:5)
Oct 22 05:19:35     at async Session.receive (file:///src/backend/api.js:230:11)
Oct 22 05:19:35     at async run (/src/node_modules/p-queue/dist/index.js:163:29)
Oct 22 05:19:36 [9f8c82807b6a419d83ebfecf597c624d] Tearing down session
Oct 22 05:19:37 [0dcd2f5cd87f472e81c120a306a35753] Creating session, language racket
Oct 22 05:19:40 [a7054fe2e05741a6afd5a3fe12a2f36e] Creating session, language racket
Oct 22 05:20:19 [372fa68bf8c446a6ac9799b1a030e010] Creating session, language haskell
Oct 22 05:20:19 [01988bf944bc48c8a677c890de38976d] Creating session, language racket
Oct 22 05:20:19 [c0e1733a70bb471390487cff1daed1e8] Creating session, language racket
Oct 22 05:20:19 [645a6a3de8e842bba8a898051b46c183] Creating session, language racket
Oct 22 05:20:20 [1f9b81c59bb044cbbe6a34eaddc19ecd] Creating session, language racket
Oct 22 05:20:20 [40935fb3eb7a4d089fc6f5e6a96f799c] Creating session, language racket
Oct 22 05:20:20 [ffe34084f5e8440c9f8fd0bbe6f3d26c] Creating session, language racket
Oct 22 05:20:21 [19d8deb81415463db112d385a1d13761] Creating session, language racket
Oct 22 05:20:21 [1534a13e449444b29cfcca4d68b29b76] Creating session, language racket
Oct 22 05:20:21 [ef3bcefcc9f045f2bf6de12ba385f93a] Creating session, language racket
Oct 22 05:20:22 [cce25ed4b83049de9b76b63c488523ff] Creating session, language racket
Oct 22 05:20:23 Error: WebSocket is not open: readyState 2 (CLOSING)
Oct 22 05:20:23     at WebSocket.send (/src/node_modules/ws/lib/websocket.js:314:19)
Oct 22 05:20:23     at Session.send (file:///src/backend/api.js:175:15)
Oct 22 05:20:23     at Socket.<anonymous> (file:///src/backend/api.js:87:14)
Oct 22 05:20:23     at Socket.emit (node:events:394:28)
Oct 22 05:20:23     at Socket.emit (node:domain:475:12)
Oct 22 05:20:23     at addChunk (node:internal/streams/readable:315:12)
Oct 22 05:20:23     at readableAddChunk (node:internal/streams/readable:289:9)
Oct 22 05:20:23     at Socket.Readable.push (node:internal/streams/readable:228:10)
Oct 22 05:20:23     at Pipe.onStreamRead (node:internal/stream_base_commons:199:23)
Oct 22 05:20:23 [921fe7acd1334244bed05e5cadce5378] Tearing down session
Oct 22 05:20:23 Error: WebSocket is not open: readyState 2 (CLOSING)
Oct 22 05:20:23     at WebSocket.send (/src/node_modules/ws/lib/websocket.js:314:19)
Oct 22 05:20:23     at Session.send (file:///src/backend/api.js:175:15)
Oct 22 05:20:23     at Socket.<anonymous> (file:///src/backend/api.js:87:14)
Oct 22 05:20:23     at Socket.emit (node:events:394:28)
Oct 22 05:20:23     at Socket.emit (node:domain:475:12)
Oct 22 05:20:23     at addChunk (node:internal/streams/readable:315:12)
Oct 22 05:20:23     at readableAddChunk (node:internal/streams/readable:289:9)
Oct 22 05:20:23     at Socket.Readable.push (node:internal/streams/readable:228:10)
Oct 22 05:20:23     at Pipe.onStreamRead (node:internal/stream_base_commons:199:23)
Oct 22 05:20:23 [b80a7fb27db74df3bfb5ef6c38e061dd] Tearing down session
Oct 22 05:20:23 Error: WebSocket is not open: readyState 2 (CLOSING)
Oct 22 05:20:23     at WebSocket.send (/src/node_modules/ws/lib/websocket.js:314:19)
Oct 22 05:20:23     at Session.send (file:///src/backend/api.js:175:15)
Oct 22 05:20:23     at Socket.<anonymous> (file:///src/backend/api.js:87:14)
Oct 22 05:20:23     at Socket.emit (node:events:394:28)
Oct 22 05:20:23     at Socket.emit (node:domain:475:12)
Oct 22 05:20:23     at addChunk (node:internal/streams/readable:315:12)
Oct 22 05:20:23     at readableAddChunk (node:internal/streams/readable:289:9)
Oct 22 05:20:23     at Socket.Readable.push (node:internal/streams/readable:228:10)
Oct 22 05:20:23     at Pipe.onStreamRead (node:internal/stream_base_commons:199:23)
Oct 22 05:20:23 [0dcd2f5cd87f472e81c120a306a35753] Tearing down session
Oct 22 05:20:23 Error: WebSocket is not open: readyState 2 (CLOSING)
Oct 22 05:20:23     at WebSocket.send (/src/node_modules/ws/lib/websocket.js:314:19)
Oct 22 05:20:23     at Session.send (file:///src/backend/api.js:175:15)
Oct 22 05:20:23     at Socket.<anonymous> (file:///src/backend/api.js:87:14)
Oct 22 05:20:23     at Socket.emit (node:events:394:28)
Oct 22 05:20:23     at Socket.emit (node:domain:475:12)
Oct 22 05:20:23     at addChunk (node:internal/streams/readable:315:12)
Oct 22 05:20:23     at readableAddChunk (node:internal/streams/readable:289:9)
Oct 22 05:20:23     at Socket.Readable.push (node:internal/streams/readable:228:10)
Oct 22 05:20:23     at Pipe.onStreamRead (node:internal/stream_base_commons:199:23)
Oct 22 05:20:23 [a7054fe2e05741a6afd5a3fe12a2f36e] Tearing down session
Oct 22 05:20:23 [8c7db277282446c08b369e0d7fc70eb6] Creating session, language racket
Oct 22 05:20:23 [2c364ca5bfb54f559fd6aea3fc7116c4] Creating session, language racket
Oct 22 05:20:24 [142628c7401640918f169d00614b626f] Creating session, language racket
Oct 22 05:20:24 [1ff421dcf27e4aa9a855c798a8044905] Creating session, language racket
Oct 22 05:20:24 [c9da1b3996e94e24b830799691c09c2c] Creating session, language racket
Oct 22 05:20:25 [2f1fab4da58b4e288fdca8ebda8c8c51] Creating session, language racket
Oct 22 05:20:28 [01988bf944bc48c8a677c890de38976d] Tearing down session
Oct 22 05:20:29 [645a6a3de8e842bba8a898051b46c183] Tearing down session
Oct 22 05:20:29 [c0e1733a70bb471390487cff1daed1e8] Tearing down session
Oct 22 05:20:29 [ffe34084f5e8440c9f8fd0bbe6f3d26c] Output from system/out/riju-system-privileged:
Oct 22 05:20:29 sentinel did not set up FIFOs within 1 second (errno 2)
Oct 22 05:20:29 sentinel did not set up FIFOs within 1 second (errno 2)
Oct 22 05:20:29 sentinel did not set up FIFOs within 1 second (errno 2)
Oct 22 05:20:29 sentinel did not set up FIFOs within 1 second (errno 2)
Oct 22 05:20:29 command system/out/riju-system-privileged failed with error code 1
Oct 22 05:20:29 [ffe34084f5e8440c9f8fd0bbe6f3d26c] Tearing down session
Oct 22 05:20:30 [1f9b81c59bb044cbbe6a34eaddc19ecd] Tearing down session
Oct 22 05:20:30 [40935fb3eb7a4d089fc6f5e6a96f799c] Tearing down session
Oct 22 05:20:31 [19d8deb81415463db112d385a1d13761] Tearing down session
Oct 22 05:20:31 [8c7db277282446c08b369e0d7fc70eb6] Tearing down session
Oct 22 05:20:33 [2c364ca5bfb54f559fd6aea3fc7116c4] Tearing down session
Oct 22 05:20:38 [a50ffebf87dc49e0bdf6ec4d4460014d] Creating session, language racket
Oct 22 05:22:16 [cce25ed4b83049de9b76b63c488523ff] Tearing down session
Oct 22 05:22:16 [142628c7401640918f169d00614b626f] Tearing down session
Oct 22 05:22:16 [1ff421dcf27e4aa9a855c798a8044905] Tearing down session
Oct 22 05:22:16 [ed469cc24aaa4048b8181fe8ff7ce0a4] Creating session, language racket
Oct 22 05:22:17 [70f1c39981f141b79ae9c7449dc037c9] Creating session, language racket
Oct 22 05:22:17 [edfcd8e9e29340b98e6d2a05c481e0f6] Creating session, language racket
Oct 22 05:22:19 [a50ffebf87dc49e0bdf6ec4d4460014d] Tearing down session
Oct 22 05:22:19 [cc519a4916c944b4af433f96c0ff2606] Creating session, language racket
Oct 22 05:22:32 [2f1fab4da58b4e288fdca8ebda8c8c51] Tearing down session
Oct 22 05:22:33 [a6b9e8f297de47038fdffcea6e9319a0] Creating session, language racket
Oct 22 05:23:58 [ed469cc24aaa4048b8181fe8ff7ce0a4] Tearing down session
Oct 22 05:23:58 [def9c8c182a8475185f0f45170839a09] Creating session, language racket
Oct 22 05:23:59 [edfcd8e9e29340b98e6d2a05c481e0f6] Tearing down session
Oct 22 05:24:00 [0bcd17d01442420fb17f96b2695983fd] Creating session, language racket
Oct 22 05:24:00 [cc519a4916c944b4af433f96c0ff2606] Tearing down session
Oct 22 05:24:01 [63086d36f6ef4a2ba77eee29c57f82e8] Creating session, language racket
Oct 22 05:24:14 [a6b9e8f297de47038fdffcea6e9319a0] Tearing down session
Oct 22 05:24:15 [0befebaa0fe9485f81d01961d64020d8] Creating session, language racket
Oct 22 05:24:15 [70f1c39981f141b79ae9c7449dc037c9] Tearing down session
Oct 22 05:24:16 [bdb6fd9f9fcb4f0f8058ee584f19be57] Creating session, language racket
Oct 22 05:25:42 [0bcd17d01442420fb17f96b2695983fd] Tearing down session
Oct 22 05:25:43 [63086d36f6ef4a2ba77eee29c57f82e8] Tearing down session
Oct 22 05:25:43 [3a593e6571194e1db6df9b08430aae09] Creating session, language racket
Oct 22 05:25:43 [ff46217cf1a0465d8426bb82ff39d056] Creating session, language racket
Kernel logs during the incident 

Oct 22 05:24:17 ip-172-31-1-13 kernel: [2892862.097619] device vethfdcbb1f left promiscuous mode
Oct 22 05:24:17 ip-172-31-1-13 kernel: [2892862.040306] veth950c95c: renamed from eth0
Oct 22 05:24:17 ip-172-31-1-13 kernel: [2892861.960124] IPv6: ADDRCONF(NETDEV_CHANGE): veth5e3aa85: link becomes ready
Oct 22 05:24:16 ip-172-31-1-13 kernel: [2892861.645694] docker0: port 2(veth5e3aa85) entered forwarding state
Oct 22 05:24:16 ip-172-31-1-13 kernel: [2892861.645586] device veth5e3aa85 entered promiscuous mode
Oct 22 05:24:16 ip-172-31-1-13 kernel: [2892861.644795] docker0: port 2(veth5e3aa85) entered blocking state
Oct 22 05:24:16 ip-172-31-1-13 kernel: [2892861.491037] device veth6d905e5 left promiscuous mode
Oct 22 05:24:16 ip-172-31-1-13 kernel: [2892861.442257] veth97ec87b: renamed from eth0
Oct 22 05:24:15 ip-172-31-1-13 kernel: [2892860.204151] docker0: port 6(vethad14b18) entered forwarding state
Oct 22 05:24:15 ip-172-31-1-13 kernel: [2892860.204095] IPv6: ADDRCONF(NETDEV_CHANGE): vethad14b18: link becomes ready
Oct 22 05:24:15 ip-172-31-1-13 kernel: [2892859.881891] device vethad14b18 entered promiscuous mode
Oct 22 05:24:15 ip-172-31-1-13 kernel: [2892859.880893] docker0: port 6(vethad14b18) entered blocking state
Oct 22 05:24:02 ip-172-31-1-13 kernel: [2892847.710653] device vetha46fabf left promiscuous mode
Oct 22 05:24:02 ip-172-31-1-13 kernel: [2892847.655578] vethe0ac3dc: renamed from eth0
Oct 22 05:24:02 ip-172-31-1-13 kernel: [2892847.004209] IPv6: ADDRCONF(NETDEV_CHANGE): vethf78ad21: link becomes ready
Oct 22 05:24:01 ip-172-31-1-13 kernel: [2892846.655234] docker0: port 5(vethf78ad21) entered forwarding state
Oct 22 05:24:01 ip-172-31-1-13 kernel: [2892846.652913] device vethf78ad21 entered promiscuous mode
Oct 22 05:24:01 ip-172-31-1-13 kernel: [2892846.652691] docker0: port 5(vethf78ad21) entered blocking state
Oct 22 05:24:01 ip-172-31-1-13 kernel: [2892846.500363] device veth8a31aeb left promiscuous mode
Oct 22 05:24:01 ip-172-31-1-13 kernel: [2892846.442978] veth98de141: renamed from eth0
Oct 22 05:24:01 ip-172-31-1-13 kernel: [2892846.056589] IPv6: ADDRCONF(NETDEV_CHANGE): vethe17e199: link becomes ready
Oct 22 05:24:00 ip-172-31-1-13 kernel: [2892845.664985] docker0: port 3(vethe17e199) entered forwarding state
Oct 22 05:24:00 ip-172-31-1-13 kernel: [2892845.664617] device vethe17e199 entered promiscuous mode
Oct 22 05:24:00 ip-172-31-1-13 kernel: [2892845.664105] docker0: port 3(vethe17e199) entered blocking state
Oct 22 05:24:00 ip-172-31-1-13 kernel: [2892845.117894] device vethf3bd61f left promiscuous mode
Oct 22 05:24:00 ip-172-31-1-13 kernel: [2892845.079727] veth7fcb86e: renamed from eth0
Oct 22 05:23:59 ip-172-31-1-13 kernel: [2892843.948339] docker0: port 7(veth70b7b11) entered forwarding state
Oct 22 05:23:59 ip-172-31-1-13 kernel: [2892843.948285] IPv6: ADDRCONF(NETDEV_CHANGE): veth70b7b11: link becomes ready
Oct 22 05:23:58 ip-172-31-1-13 kernel: [2892843.711939] device veth70b7b11 entered promiscuous mode
Oct 22 05:23:58 ip-172-31-1-13 kernel: [2892843.711863] docker0: port 7(veth70b7b11) entered blocking state
Oct 22 05:22:35 ip-172-31-1-13 kernel: [2892759.765827] device veth4fad49b left promiscuous mode
Oct 22 05:22:34 ip-172-31-1-13 kernel: [2892759.721376] vethd282415: renamed from eth0
Oct 22 05:22:33 ip-172-31-1-13 kernel: [2892758.504913] docker0: port 2(veth6d905e5) entered forwarding state
Oct 22 05:22:33 ip-172-31-1-13 kernel: [2892758.504860] IPv6: ADDRCONF(NETDEV_CHANGE): veth6d905e5: link becomes ready
Oct 22 05:22:33 ip-172-31-1-13 kernel: [2892758.265262] device veth6d905e5 entered promiscuous mode
Oct 22 05:22:33 ip-172-31-1-13 kernel: [2892758.265159] docker0: port 2(veth6d905e5) entered blocking state
Oct 22 05:22:21 ip-172-31-1-13 kernel: [2892746.023631] device vethfc90897 left promiscuous mode
Oct 22 05:22:21 ip-172-31-1-13 kernel: [2892745.980316] veth6869f4d: renamed from eth0
Oct 22 05:22:20 ip-172-31-1-13 kernel: [2892744.864946] IPv6: ADDRCONF(NETDEV_CHANGE): vetha46fabf: link becomes ready
Oct 22 05:22:19 ip-172-31-1-13 kernel: [2892744.604069] docker0: port 6(vetha46fabf) entered forwarding state
Oct 22 05:22:19 ip-172-31-1-13 kernel: [2892744.603278] device vetha46fabf entered promiscuous mode
Oct 22 05:22:19 ip-172-31-1-13 kernel: [2892744.603126] docker0: port 6(vetha46fabf) entered blocking state
Oct 22 05:22:18 ip-172-31-1-13 kernel: [2892743.384443] device vetha37f2f0 left promiscuous mode
Oct 22 05:22:18 ip-172-31-1-13 kernel: [2892743.350852] veth3ff6acc: renamed from eth0
Oct 22 05:22:18 ip-172-31-1-13 kernel: [2892743.152511] docker0: port 31(veth5b4d35c) entered disabled state
Oct 22 05:22:18 ip-172-31-1-13 kernel: [2892743.151414] docker0: port 31(veth5b4d35c) entered disabled state
Oct 22 05:22:18 ip-172-31-1-13 kernel: [2892743.105502] docker0: port 31(veth5b4d35c) entered disabled state
Oct 22 05:22:18 ip-172-31-1-13 kernel: [2892742.977314] eth0: renamed from veth98de141
Oct 22 05:22:17 ip-172-31-1-13 kernel: [2892742.687044] docker0: port 5(veth8a31aeb) entered blocking state
Oct 22 05:22:17 ip-172-31-1-13 kernel: [2892742.673369] docker0: port 5(veth8a31aeb) entered disabled state
Oct 22 05:22:17 ip-172-31-1-13 kernel: [2892742.317018] docker0: port 4(vethfdcbb1f) entered forwarding state
Oct 22 05:22:17 ip-172-31-1-13 kernel: [2892742.316962] IPv6: ADDRCONF(NETDEV_CHANGE): vethfdcbb1f: link becomes ready
Oct 22 05:22:17 ip-172-31-1-13 kernel: [2892742.042255] device vethfdcbb1f entered promiscuous mode
Oct 22 05:22:17 ip-172-31-1-13 kernel: [2892742.039671] docker0: port 4(vethfdcbb1f) entered blocking state
Oct 22 05:22:17 ip-172-31-1-13 kernel: [2892741.881241] eth0: renamed from veth7fcb86e
Oct 22 05:22:16 ip-172-31-1-13 kernel: [2892741.631734] docker0: port 3(vethf3bd61f) entered blocking state
Oct 22 05:22:16 ip-172-31-1-13 kernel: [2892741.631601] docker0: port 3(vethf3bd61f) entered disabled state
Oct 22 05:22:16 ip-172-31-1-13 kernel: [2892740.978422] docker0: port 27(vethe581b1c) entered disabled state
Oct 22 05:22:16 ip-172-31-1-13 kernel: [2892740.976951] docker0: port 27(vethe581b1c) entered disabled state
Oct 22 05:22:16 ip-172-31-1-13 kernel: [2892740.932428] docker0: port 27(vethe581b1c) entered disabled state
Oct 22 05:20:38 ip-172-31-1-13 kernel: [2892643.129701] docker0: port 2(vethfc90897) entered blocking state
Oct 22 05:20:38 ip-172-31-1-13 kernel: [2892643.116800] eth0: renamed from veth6869f4d
Oct 22 05:20:38 ip-172-31-1-13 kernel: [2892642.899728] docker0: port 2(vethfc90897) entered disabled state
Oct 22 05:20:33 ip-172-31-1-13 kernel: [2892638.415625] docker0: port 29(veth47224b5) entered disabled state
Oct 22 05:20:33 ip-172-31-1-13 kernel: [2892638.412439] docker0: port 29(veth47224b5) entered disabled state
Oct 22 05:20:33 ip-172-31-1-13 kernel: [2892638.349983] docker0: port 29(veth47224b5) entered disabled state
Oct 22 05:20:32 ip-172-31-1-13 kernel: [2892637.411731] docker0: port 26(veth2262088) entered blocking state
Oct 22 05:20:32 ip-172-31-1-13 kernel: [2892637.411543] docker0: port 33(veth4fad49b) entered forwarding state
Oct 22 05:20:32 ip-172-31-1-13 kernel: [2892637.411492] IPv6: ADDRCONF(NETDEV_CHANGE): veth4fad49b: link becomes ready
Oct 22 05:20:32 ip-172-31-1-13 kernel: [2892637.390114] docker0: port 32(veth63b08f8) entered blocking state
Oct 22 05:20:32 ip-172-31-1-13 kernel: [2892637.390064] docker0: port 30(vetha37f2f0) entered forwarding state
Oct 22 05:20:32 ip-172-31-1-13 kernel: [2892637.390019] IPv6: ADDRCONF(NETDEV_CHANGE): vetha37f2f0: link becomes ready
Oct 22 05:20:32 ip-172-31-1-13 kernel: [2892637.357548] eth0: renamed from vethd282415
Oct 22 05:20:32 ip-172-31-1-13 kernel: [2892637.341120] docker0: port 25(vethdf491e1) entered forwarding state
Oct 22 05:20:32 ip-172-31-1-13 kernel: [2892637.341064] IPv6: ADDRCONF(NETDEV_CHANGE): vethdf491e1: link becomes ready
Oct 22 05:20:32 ip-172-31-1-13 kernel: [2892637.240352] eth0: renamed from vethfb41638
Oct 22 05:20:32 ip-172-31-1-13 kernel: [2892637.187980] docker0: port 29(veth47224b5) entered blocking state
Oct 22 05:20:32 ip-172-31-1-13 kernel: [2892637.147515] eth0: renamed from veth353fc42
Oct 22 05:20:32 ip-172-31-1-13 kernel: [2892637.098226] docker0: port 31(veth5b4d35c) entered blocking state
Oct 22 05:20:32 ip-172-31-1-13 kernel: [2892637.081019] docker0: port 28(veth28aaf5f) entered disabled state
Oct 22 05:20:32 ip-172-31-1-13 kernel: [2892637.079463] docker0: port 28(veth28aaf5f) entered disabled state
Oct 22 05:20:32 ip-172-31-1-13 kernel: [2892637.048744] docker0: port 27(vethe581b1c) entered forwarding state
Oct 22 05:20:32 ip-172-31-1-13 kernel: [2892637.048697] IPv6: ADDRCONF(NETDEV_CHANGE): vethe581b1c: link becomes ready
Oct 22 05:20:32 ip-172-31-1-13 kernel: [2892636.775311] docker0: port 8(veth692e1a7) entered disabled state
Oct 22 05:20:31 ip-172-31-1-13 kernel: [2892636.744118] docker0: port 8(veth692e1a7) entered disabled state
Oct 22 05:20:31 ip-172-31-1-13 kernel: [2892636.714784] docker0: port 28(veth28aaf5f) entered disabled state
Oct 22 05:20:31 ip-172-31-1-13 kernel: [2892636.639935] docker0: port 8(veth692e1a7) entered disabled state
Oct 22 05:20:31 ip-172-31-1-13 kernel: [2892636.604846] device vethd0e9a5f left promiscuous mode
Oct 22 05:20:31 ip-172-31-1-13 kernel: [2892636.507961] veth4f94529: renamed from eth0
Oct 22 05:20:31 ip-172-31-1-13 kernel: [2892636.464507] docker0: port 6(veth162b90f) entered disabled state
Oct 22 05:20:31 ip-172-31-1-13 kernel: [2892636.456998] docker0: port 6(veth162b90f) entered disabled state
Oct 22 05:20:31 ip-172-31-1-13 kernel: [2892636.417167] docker0: port 6(veth162b90f) entered disabled state
Oct 22 05:20:31 ip-172-31-1-13 kernel: [2892636.332549] device veth3627766 left promiscuous mode
Oct 22 05:20:31 ip-172-31-1-13 kernel: [2892636.283794] docker0: port 4(veth590c99d) entered disabled state
Oct 22 05:20:31 ip-172-31-1-13 kernel: [2892636.273394] docker0: port 4(veth590c99d) entered disabled state
Oct 22 05:20:31 ip-172-31-1-13 kernel: [2892636.108595] veth1a139cf: renamed from eth0
Oct 22 05:20:31 ip-172-31-1-13 kernel: [2892636.035391] veth95893b0: renamed from eth0
Oct 22 05:20:31 ip-172-31-1-13 kernel: [2892636.021436] device vethc44ed37 left promiscuous mode
Oct 22 05:20:31 ip-172-31-1-13 kernel: [2892635.951470] docker0: port 22(vethdfe8ef0) entered disabled state
Oct 22 05:20:31 ip-172-31-1-13 kernel: [2892635.949514] docker0: port 22(vethdfe8ef0) entered disabled state
Oct 22 05:20:31 ip-172-31-1-13 kernel: [2892635.832850] veth3c450e1: renamed from eth0
Oct 22 05:20:31 ip-172-31-1-13 kernel: [2892635.830428] docker0: port 22(vethdfe8ef0) entered disabled state
Oct 22 05:20:31 ip-172-31-1-13 kernel: [2892635.797815] docker0: port 28(veth28aaf5f) entered blocking state
Oct 22 05:20:31 ip-172-31-1-13 kernel: [2892635.779217] eth0: renamed from veth921c7c1
Oct 22 05:20:30 ip-172-31-1-13 kernel: [2892635.650539] device veth2b96c20 left promiscuous mode
Oct 22 05:20:30 ip-172-31-1-13 kernel: [2892635.558492] vethebd5adc: renamed from eth0
Oct 22 05:20:30 ip-172-31-1-13 kernel: [2892635.532692] device veth67461ef left promiscuous mode
Oct 22 05:20:30 ip-172-31-1-13 kernel: [2892635.526805] docker0: port 2(veth2b96c20) entered disabled state
Oct 22 05:20:30 ip-172-31-1-13 kernel: [2892635.490603] device veth65ba975 left promiscuous mode
Oct 22 05:20:30 ip-172-31-1-13 kernel: [2892635.439250] docker0: port 19(vethdc46742) entered disabled state
Oct 22 05:20:30 ip-172-31-1-13 kernel: [2892635.435223] docker0: port 19(vethdc46742) entered disabled state
Oct 22 05:20:30 ip-172-31-1-13 kernel: [2892635.376485] docker0: port 21(veth65ba975) entered disabled state
Oct 22 05:20:30 ip-172-31-1-13 kernel: [2892635.323848] veth8a8c78c: renamed from eth0
Oct 22 05:20:30 ip-172-31-1-13 kernel: [2892635.310928] device veth72c6e1a left promiscuous mode
Oct 22 05:20:30 ip-172-31-1-13 kernel: [2892635.308802] docker0: port 19(vethdc46742) entered disabled state
Oct 22 05:20:30 ip-172-31-1-13 kernel: [2892635.222236] vethf26a966: renamed from eth0
Oct 22 05:20:30 ip-172-31-1-13 kernel: [2892635.116876] docker0: port 11(veth3c08cb5) entered disabled state
Oct 22 05:20:30 ip-172-31-1-13 kernel: [2892635.109068] docker0: port 11(veth3c08cb5) entered disabled state
Oct 22 05:20:30 ip-172-31-1-13 kernel: [2892635.071376] docker0: port 11(veth3c08cb5) entered disabled state
Oct 22 05:20:30 ip-172-31-1-13 kernel: [2892635.070437] docker0: port 24(vethd0e9a5f) entered blocking state
Oct 22 05:20:30 ip-172-31-1-13 kernel: [2892635.049693] eth0: renamed from veth4f94529
Oct 22 05:20:30 ip-172-31-1-13 kernel: [2892634.767719] device veth337cb0f left promiscuous mode
Oct 22 05:20:29 ip-172-31-1-13 kernel: [2892634.724707] veth95e9a6f: renamed from eth0
Oct 22 05:20:29 ip-172-31-1-13 kernel: [2892634.634779] docker0: port 20(vethb0b6601) entered disabled state
Oct 22 05:20:29 ip-172-31-1-13 kernel: [2892634.633627] docker0: port 20(vethb0b6601) entered disabled state
Oct 22 05:20:29 ip-172-31-1-13 kernel: [2892634.516768] docker0: port 20(vethb0b6601) entered disabled state
Oct 22 05:20:29 ip-172-31-1-13 kernel: [2892634.013310] device vethc2b56bd left promiscuous mode
Oct 22 05:20:29 ip-172-31-1-13 kernel: [2892633.919994] vethb2feb67: renamed from eth0
Oct 22 05:20:29 ip-172-31-1-13 kernel: [2892633.762028] docker0: port 23(vethc44ed37) entered forwarding state
Oct 22 05:20:29 ip-172-31-1-13 kernel: [2892633.761968] IPv6: ADDRCONF(NETDEV_CHANGE): vethc44ed37: link becomes ready
Oct 22 05:20:28 ip-172-31-1-13 kernel: [2892633.414720] docker0: port 16(veth0ffb3e5) entered disabled state
Oct 22 05:20:28 ip-172-31-1-13 kernel: [2892633.410997] docker0: port 16(veth0ffb3e5) entered disabled state
Oct 22 05:20:28 ip-172-31-1-13 kernel: [2892633.294275] docker0: port 16(veth0ffb3e5) entered disabled state
Oct 22 05:20:28 ip-172-31-1-13 kernel: [2892633.252198] device vethe9b7cf5 left promiscuous mode
Oct 22 05:20:28 ip-172-31-1-13 kernel: [2892633.076180] docker0: port 14(veth0b9583a) entered disabled state
Oct 22 05:20:28 ip-172-31-1-13 kernel: [2892633.074575] docker0: port 14(veth0b9583a) entered disabled state
Oct 22 05:20:28 ip-172-31-1-13 kernel: [2892633.036538] veth4c75ed3: renamed from eth0
Oct 22 05:20:28 ip-172-31-1-13 kernel: [2892632.997611] docker0: port 14(veth0b9583a) entered disabled state
Oct 22 05:20:28 ip-172-31-1-13 kernel: [2892632.902050] docker0: port 22(vethdfe8ef0) entered blocking state
Oct 22 05:20:28 ip-172-31-1-13 kernel: [2892632.885586] eth0: renamed from vethb5f532d
Oct 22 05:20:27 ip-172-31-1-13 kernel: [2892632.746527] docker0: port 18(veth3ed7b26) entered blocking state
Oct 22 05:20:27 ip-172-31-1-13 kernel: [2892632.667384] eth0: renamed from vetha78064e
Oct 22 05:20:27 ip-172-31-1-13 kernel: [2892631.816309] docker0: port 20(vethb0b6601) entered blocking state
Oct 22 05:20:27 ip-172-31-1-13 kernel: [2892631.799585] eth0: renamed from veth1059629
Oct 22 05:20:26 ip-172-31-1-13 kernel: [2892631.747536] docker0: port 21(veth65ba975) entered blocking state
Oct 22 05:20:26 ip-172-31-1-13 kernel: [2892631.726982] eth0: renamed from veth8fb794c
Oct 22 05:20:26 ip-172-31-1-13 kernel: [2892631.546198] docker0: port 7(veth72c6e1a) entered blocking state
Oct 22 05:20:26 ip-172-31-1-13 kernel: [2892631.534302] eth0: renamed from vethf26a966
Oct 22 05:20:26 ip-172-31-1-13 kernel: [2892631.353029] docker0: port 19(vethdc46742) entered blocking state
Oct 22 05:20:26 ip-172-31-1-13 kernel: [2892631.352920] docker0: port 33(veth4fad49b) entered disabled state
Oct 22 05:20:26 ip-172-31-1-13 kernel: [2892630.935860] docker0: port 33(veth4fad49b) entered forwarding state
Oct 22 05:20:26 ip-172-31-1-13 kernel: [2892630.925789] device veth4fad49b entered promiscuous mode
Oct 22 05:20:26 ip-172-31-1-13 kernel: [2892630.924052] docker0: port 33(veth4fad49b) entered blocking state
Oct 22 05:20:26 ip-172-31-1-13 kernel: [2892630.845412] docker0: port 14(veth0b9583a) entered blocking state
Oct 22 05:20:26 ip-172-31-1-13 kernel: [2892630.835916] device veth63b08f8 entered promiscuous mode
Oct 22 05:20:26 ip-172-31-1-13 kernel: [2892630.834146] docker0: port 32(veth63b08f8) entered blocking state
Oct 22 05:20:26 ip-172-31-1-13 kernel: [2892630.776453] docker0: port 17(vethc2b56bd) entered forwarding state
Oct 22 05:20:26 ip-172-31-1-13 kernel: [2892630.776394] IPv6: ADDRCONF(NETDEV_CHANGE): vethc2b56bd: link becomes ready
Oct 22 05:20:25 ip-172-31-1-13 kernel: [2892630.690503] docker0: port 16(veth0ffb3e5) entered forwarding state
Oct 22 05:20:25 ip-172-31-1-13 kernel: [2892630.690472] IPv6: ADDRCONF(NETDEV_CHANGE): veth0ffb3e5: link becomes ready
Oct 22 05:20:25 ip-172-31-1-13 kernel: [2892630.670858] eth0: renamed from veth5706d5a
Oct 22 05:20:25 ip-172-31-1-13 kernel: [2892630.456174] docker0: port 31(veth5b4d35c) entered blocking state
Oct 22 05:20:25 ip-172-31-1-13 kernel: [2892630.451919] docker0: port 31(veth5b4d35c) entered disabled state
Oct 22 05:20:25 ip-172-31-1-13 kernel: [2892630.348318] docker0: port 15(vethe9b7cf5) entered forwarding state
Oct 22 05:20:25 ip-172-31-1-13 kernel: [2892630.348285] IPv6: ADDRCONF(NETDEV_CHANGE): vethe9b7cf5: link becomes ready
Oct 22 05:20:25 ip-172-31-1-13 kernel: [2892630.348212] docker0: port 29(veth47224b5) entered disabled state
Oct 22 05:20:25 ip-172-31-1-13 kernel: [2892630.343003] docker0: port 30(vetha37f2f0) entered forwarding state
Oct 22 05:20:25 ip-172-31-1-13 kernel: [2892630.315258] device vetha37f2f0 entered promiscuous mode
Oct 22 05:20:25 ip-172-31-1-13 kernel: [2892630.313850] docker0: port 30(vetha37f2f0) entered blocking state
Oct 22 05:20:25 ip-172-31-1-13 kernel: [2892630.127288] docker0: port 29(veth47224b5) entered forwarding state
Oct 22 05:20:25 ip-172-31-1-13 kernel: [2892630.126742] device veth47224b5 entered promiscuous mode
Oct 22 05:20:25 ip-172-31-1-13 kernel: [2892630.126546] docker0: port 29(veth47224b5) entered blocking state
Oct 22 05:20:25 ip-172-31-1-13 kernel: [2892629.927667] docker0: port 28(veth28aaf5f) entered blocking state
Oct 22 05:20:25 ip-172-31-1-13 kernel: [2892629.924996] docker0: port 28(veth28aaf5f) entered disabled state
Oct 22 05:20:25 ip-172-31-1-13 kernel: [2892629.817418] docker0: port 27(vethe581b1c) entered disabled state
Oct 22 05:20:24 ip-172-31-1-13 kernel: [2892629.628726] docker0: port 27(vethe581b1c) entered blocking state
Oct 22 05:20:24 ip-172-31-1-13 kernel: [2892629.628497] docker0: port 27(vethe581b1c) entered disabled state
Oct 22 05:20:24 ip-172-31-1-13 kernel: [2892628.831962] docker0: port 26(veth2262088) entered disabled state
Oct 22 05:20:24 ip-172-31-1-13 kernel: [2892628.831843] docker0: port 24(vethd0e9a5f) entered disabled state
Oct 22 05:20:24 ip-172-31-1-13 kernel: [2892628.813495] docker0: port 22(vethdfe8ef0) entered disabled state
Oct 22 05:20:24 ip-172-31-1-13 kernel: [2892628.794962] docker0: port 26(veth2262088) entered blocking state
Oct 22 05:20:24 ip-172-31-1-13 kernel: [2892628.779818] docker0: port 26(veth2262088) entered disabled state
Oct 22 05:20:23 ip-172-31-1-13 kernel: [2892628.440567] docker0: port 25(vethdf491e1) entered forwarding state
Oct 22 05:20:23 ip-172-31-1-13 kernel: [2892628.440280] device vethdf491e1 entered promiscuous mode
Oct 22 05:20:23 ip-172-31-1-13 kernel: [2892628.439965] docker0: port 25(vethdf491e1) entered blocking state
Oct 22 05:20:23 ip-172-31-1-13 kernel: [2892628.336636] docker0: port 24(vethd0e9a5f) entered blocking state
Oct 22 05:20:23 ip-172-31-1-13 kernel: [2892628.336501] docker0: port 24(vethd0e9a5f) entered disabled state
Oct 22 05:20:23 ip-172-31-1-13 kernel: [2892628.279899] docker0: port 23(vethc44ed37) entered forwarding state
Oct 22 05:20:23 ip-172-31-1-13 kernel: [2892628.277702] device vethc44ed37 entered promiscuous mode
Oct 22 05:20:23 ip-172-31-1-13 kernel: [2892628.277270] docker0: port 23(vethc44ed37) entered blocking state
Oct 22 05:20:23 ip-172-31-1-13 kernel: [2892627.876069] docker0: port 22(vethdfe8ef0) entered blocking state
Oct 22 05:20:23 ip-172-31-1-13 kernel: [2892627.875552] docker0: port 22(vethdfe8ef0) entered disabled state
Oct 22 05:20:23 ip-172-31-1-13 kernel: [2892627.803401] docker0: port 21(veth65ba975) entered disabled state
Oct 22 05:20:22 ip-172-31-1-13 kernel: [2892627.222161] docker0: port 21(veth65ba975) entered forwarding state
Oct 22 05:20:22 ip-172-31-1-13 kernel: [2892627.221971] device veth65ba975 entered promiscuous mode
Oct 22 05:20:22 ip-172-31-1-13 kernel: [2892627.219227] docker0: port 21(veth65ba975) entered blocking state
Oct 22 05:20:22 ip-172-31-1-13 kernel: [2892626.862558] docker0: port 20(vethb0b6601) entered blocking state
Oct 22 05:20:22 ip-172-31-1-13 kernel: [2892626.857215] docker0: port 20(vethb0b6601) entered disabled state
Oct 22 05:20:22 ip-172-31-1-13 kernel: [2892626.767068] docker0: port 19(vethdc46742) entered disabled state
Oct 22 05:20:21 ip-172-31-1-13 kernel: [2892626.302833] docker0: port 19(vethdc46742) entered forwarding state
Oct 22 05:20:21 ip-172-31-1-13 kernel: [2892626.302733] device vethdc46742 entered promiscuous mode
Oct 22 05:20:21 ip-172-31-1-13 kernel: [2892626.300085] docker0: port 19(vethdc46742) entered blocking state
Oct 22 05:20:21 ip-172-31-1-13 kernel: [2892625.869691] docker0: port 18(veth3ed7b26) entered blocking state
Oct 22 05:20:21 ip-172-31-1-13 kernel: [2892625.869548] docker0: port 18(veth3ed7b26) entered disabled state
Oct 22 05:20:20 ip-172-31-1-13 kernel: [2892625.743514] docker0: port 7(veth72c6e1a) entered disabled state
Oct 22 05:20:20 ip-172-31-1-13 kernel: [2892625.743381] docker0: port 7(veth72c6e1a) entered blocking state
Oct 22 05:20:20 ip-172-31-1-13 kernel: [2892625.730644] docker0: port 7(veth72c6e1a) entered disabled state
Oct 22 05:20:20 ip-172-31-1-13 kernel: [2892624.831166] docker0: port 7(vethabd8559) entered disabled state
Oct 22 05:20:20 ip-172-31-1-13 kernel: [2892624.828893] docker0: port 7(vethabd8559) entered disabled state
Oct 22 05:20:19 ip-172-31-1-13 kernel: [2892624.721847] docker0: port 7(vethabd8559) entered disabled state
Oct 22 05:20:19 ip-172-31-1-13 kernel: [2892624.720561] docker0: port 16(veth0ffb3e5) entered disabled state
Oct 22 05:20:19 ip-172-31-1-13 kernel: [2892624.562893] docker0: port 17(vethc2b56bd) entered forwarding state
Oct 22 05:20:19 ip-172-31-1-13 kernel: [2892624.562722] device vethc2b56bd entered promiscuous mode
Oct 22 05:20:19 ip-172-31-1-13 kernel: [2892624.537433] docker0: port 17(vethc2b56bd) entered blocking state
Oct 22 05:20:19 ip-172-31-1-13 kernel: [2892624.430252] docker0: port 16(veth0ffb3e5) entered blocking state
Oct 22 05:20:19 ip-172-31-1-13 kernel: [2892624.426979] docker0: port 16(veth0ffb3e5) entered disabled state
Oct 22 05:20:19 ip-172-31-1-13 kernel: [2892623.835691] docker0: port 15(vethe9b7cf5) entered forwarding state
Oct 22 05:20:19 ip-172-31-1-13 kernel: [2892623.833052] device vethe9b7cf5 entered promiscuous mode
Oct 22 05:20:19 ip-172-31-1-13 kernel: [2892623.832374] docker0: port 15(vethe9b7cf5) entered blocking state
Oct 22 05:20:18 ip-172-31-1-13 kernel: [2892623.715827] docker0: port 14(veth0b9583a) entered disabled state
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.401384] oom_reaper: reaped process 2255479 (racket), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383859] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=runc-ade2637f97bdb26921b37f2770a97859974a5a3ceeaabbafffeacb317a5bf8d0.scope,mems_allowed=0,global_oom,task_memcg=/riju.slice/docker-a528f269142b8926f04424d29a089e174c84cb94dfb0fa8ecf2e88fed3838a4b.scope,task=racket,pid=2255479,uid=1000
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383856] [2283437]     0 2283437      835       19    40960        0             0 sed
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383853] [2283435]     0 2283435     1659      214    53248        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383849] [2283433]     0 2283433     1632      207    57344        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383846] [2283431]     0 2283431     1375       43    45056        0             0 sed
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383842] [2283429]     0 2283429     1375       40    53248        0             0 sed
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383838] [2283427]     0 2283427     1632      208    61440        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383835] [2283418]     0 2283418     1659      212    53248        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383832] [2283405]     0 2283405     6227     2630    86016        0             0 apport
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383828] [2283403]     0 2283403     4641     1732    65536        0             0 apport
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383825] [2283393]     0 2283393     5159     2422    73728        0             0 apport
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383821] [2283391]     0 2283391     5313     2293    73728        0             0 apport
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383818] [2283389]     0 2283389     4641     2001    69632        0             0 apport
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383814] [2283387]     0 2283387     5214     2350    77824        0             0 apport
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383811] [2283148]  1000 2283148   263390     3039   200704        0             0 docker
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383806] [2282032]     0 2282032   282048     3583   249856        0             0 docker
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383803] [2281040]  1000 2281040   337559     4014   282624        0             0 docker
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383798] [2273043]  1000 2273043      595       21    36864        0             0 riju-pty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383794] [2273041]  1000 2273041      595       21    36864        0             0 riju-pty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383790] [2273038]     0 2273038     1530       83    49152        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383786] [2272572]  1000 2272572    46038    17781   286720        0             0 racket
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383782] [2271222]  1000 2271222      595       22    36864        0             0 riju-pty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383778] [2271220]  1000 2271220      595       22    36864        0             0 riju-pty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383774] [2271217]     0 2271217     1530       81    49152        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383770] [2268332]  1000 2268332      595       21    40960        0             0 riju-pty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383767] [2268330]  1000 2268330      595       21    40960        0             0 riju-pty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383763] [2268327]     0 2268327     1530       85    53248        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383760] [2268062]  1000 2268062      595       21    36864        0             0 riju-pty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383756] [2268060]  1000 2268060      595       21    36864        0             0 riju-pty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383753] [2268057]     0 2268057     1503       78    53248        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383749] [2267947]     0 2267947     1632      207    57344        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383745] [2267841]     0 2267841   177980      545   106496        0          -998 containerd-shim
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383741] [2267335]  1000 2267335    46057    17711   274432        0             0 racket
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383736] [2266954]  1000 2266954    46089    17321   270336        0             0 racket
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383732] [2266950]     0 2266950     2057      117    53248        0             0 runuser
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383729] [2266941]     0 2266941     1530       83    49152        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383724] [2266505]  1000 2266505    45690    15906   241664        0             0 racket
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383721] [2266501]     0 2266501     2057      116    53248        0             0 runuser
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383717] [2266497]     0 2266497     1503       78    49152        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383713] [2266346]     0 2266346     3801     1289    65536        0             0 my_init
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383709] [2266282]  1000 2266282      595       22    36864        0             0 riju-pty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383705] [2266280]  1000 2266280      595       22    36864        0             0 riju-pty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383701] [2266275]     0 2266275     1530       83    49152        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383698] [2266247]  1000 2266247   300693     4077   266240        0             0 docker
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383693] [2265730]  1000 2265730    45992    17661   278528        0             0 racket
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383688] [2264442]  1000 2264442      595       21    40960        0             0 riju-pty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383685] [2264440]  1000 2264440      595       21    40960        0             0 riju-pty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383681] [2264437]     0 2264437     1503       79    57344        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383677] [2263759]  1000 2263759      595       21    40960        0             0 riju-pty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383673] [2263757]  1000 2263757      595       21    40960        0             0 riju-pty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383670] [2263754]     0 2263754     1530       83    49152        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383665] [2263537]  1000 2263537      595       21    36864        0             0 riju-pty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383662] [2263535]  1000 2263535      595       21    36864        0             0 riju-pty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383658] [2263532]     0 2263532     1530       83    49152        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383654] [2262948]  1000 2262948      595       21    36864        0             0 riju-pty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383651] [2262946]  1000 2262946      595       21    36864        0             0 riju-pty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383647] [2262943]     0 2262943     1503       78    49152        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383643] [2262831]     0 2262831     1659      212    49152        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383639] [2262738]     0 2262738   177980      511   110592        0          -998 containerd-shim
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383635] [2262369]  1000 2262369 268523454    10373   380928        0             0 ghc
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383631] [2262363]  1000 2262363     1536       98    49152        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383627] [2262352]     0 2262352     2057      115    53248        0             0 runuser
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383623] [2262346]     0 2262346     1503       79    57344        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383619] [2260267]  1000 2260267 268523454    10365   380928        0             0 ghc
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383615] [2260262]  1000 2260262     1536       98    57344        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383611] [2260260]     0 2260260     2057      116    49152        0             0 runuser
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383607] [2260256]     0 2260256     1503       79    57344        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383602] [2259398]  1000 2259398    46025    17731   278528        0             0 racket
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383596] [2258604]  1000 2258604      595       21    36864        0             0 riju-pty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383593] [2258602]  1000 2258602      595       21    36864        0             0 riju-pty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383589] [2258599]     0 2258599     1530       82    49152        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383585] [2258453]  1000 2258453      595       21    36864        0             0 riju-pty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383581] [2258451]  1000 2258451      595       21    36864        0             0 riju-pty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383577] [2258448]     0 2258448     1530       81    53248        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383573] [2258065]  1000 2258065    46029    17785   278528        0             0 racket
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383569] [2257866]  1000 2257866    45774    15923   249856        0             0 racket
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383565] [2257863]     0 2257863     2057      116    49152        0             0 runuser
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383561] [2257860]     0 2257860     1503       78    53248        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383557] [2257712]     0 2257712     3801     1307    69632        0             0 my_init
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383553] [2257667]  1000 2257667   300693     4024   266240        0             0 docker
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383548] [2257432]  1000 2257432      595       21    36864        0             0 riju-pty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383545] [2257428]  1000 2257428      595       21    36864        0             0 riju-pty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383541] [2257419]     0 2257419     1503       79    57344        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383536] [2257041]  1000 2257041    46089    17745   282624        0             0 racket
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383532] [2256422]  1000 2256422    45924    17220   278528        0             0 racket
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383528] [2256415]     0 2256415     2057      116    57344        0             0 runuser
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383524] [2256404]     0 2256404     1530       80    49152        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383520] [2256071]  1000 2256071    45698    15924   253952        0             0 racket
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383516] [2256068]     0 2256068     2057      116    61440        0             0 runuser
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383512] [2256065]     0 2256065     1503       77    49152        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383508] [2255874]     0 2255874     3801     1290    69632        0             0 my_init
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383504] [2255827]  1000 2255827   300693     4006   270336        0             0 docker
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383500] [2255821]  1000 2255821    45997    17353   274432        0             0 racket
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383496] [2255819]     0 2255819     2057      116    53248        0             0 runuser
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383492] [2255815]     0 2255815     1530       81    49152        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383487] [2255479]  1000 2255479    45997    19316   290816        0             0 racket
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383483] [2255476]     0 2255476     2057      116    57344        0             0 runuser
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383479] [2255473]     0 2255473     1530       83    53248        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383474] [2254505]  1000 2254505    45774    15966   262144        0             0 racket
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383469] [2254298]     0 2254298   177980      585   106496        0          -998 containerd-shim
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383465] [2254042]  1000 2254042      595       21    40960        0             0 riju-pty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383461] [2254040]  1000 2254040      595       21    40960        0             0 riju-pty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383457] [2254021]     0 2254021     1530       81    53248        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383453] [2251883]  1000 2251883      595       21    40960        0             0 riju-pty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383448] [2251879]  1000 2251879      595       21    40960        0             0 riju-pty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383445] [2251870]     0 2251870     1503       78    57344        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383440] [2251750]     0 2251750     1632      208    61440        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383436] [2251684]     0 2251684   178268      640   106496        0          -998 containerd-shim
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383432] [2251185]  1000 2251185      595       21    36864        0             0 riju-pty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383428] [2251183]  1000 2251183      595       21    36864        0             0 riju-pty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383424] [2251180]     0 2251180     1503       78    49152        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383420] [2251063]     0 2251063     1659      212    49152        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383417] [2251011]     0 2251011   177980      575   110592        0          -998 containerd-shim
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383412] [2248872]  1000 2248872      595       21    40960        0             0 riju-pty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383408] [2248870]  1000 2248870      595       21    40960        0             0 riju-pty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383404] [2248866]     0 2248866     1503       78    53248        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383400] [2248749]     0 2248749     1659      214    53248        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383396] [2248689]     0 2248689   177980      617   106496        0          -998 containerd-shim
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383391] [1803150]     0 1803150     1503       60    45056        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383387] [1803115]     0 1803115     1503       56    45056        0             0 bash
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383383] [1802263]     0 1802263     3801     1291    69632        0             0 my_init
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383379] [1802071]     0 1802071   177980      636   102400        0          -998 containerd-shim
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383374] [1801701]     0 1801701   177980      616    98304        0          -998 containerd-shim
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383370] [1800096]  1000 1800096   319126     4071   286720        0             0 docker
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383366] [1464916]  1000 1464916      633       24    45056        0             0 riju-system-pri
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383361] [  21767]  1000 21767  2857429    14506  2551808        0             0 node
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383357] [  21765]  1000 21765      822       71    40960        0             0 make
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383352] [  21731]  1000 21731     6117     3640    81920        0             0 my_init
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383348] [  21696]     0 21696   324591      650   176128        0          -500 docker-proxy
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383344] [  16496]     0 16496   192690     3578   249856        0             0 amazon-cloudwat
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383339] [  12792]     0 12792   178332      723   110592        0          -998 containerd-shim
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383335] [   2043]     0  2043   307829     2868   200704        0             0 riju-supervisor
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383331] [    836]     0   836   349003     5250   270336        0             0 ssm-agent-worke
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383327] [    568]     0   568    58745      825    90112        0             0 polkitd
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383324] [    529]     0   529     1498      409    45056        0             0 agetty
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383320] [    498]     0   498   373870     6275   323584        0          -999 containerd
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383316] [    493]     0   493     3476      505    65536        0             0 systemd-logind
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383312] [    483]   104   483    55339      550    77824        0             0 rsyslogd
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383308] [    479]     0   479    20710      730    57344        0             0 irqbalance
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383304] [    472]     0   472     2186      532    57344        0             0 cron
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383300] [    412]   101   412     6056     1702    90112        0             0 systemd-resolve
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383296] [    324]   102   324    22047      743    69632        0             0 systemd-timesyn
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383290] [    199]     0   199     5022      663    57344        0         -1000 systemd-udevd
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383281] [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383280] 0 pages hwpoisoned
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383279] 0 pages HighMem/MovableOnly
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383277] Total swap = 0kB
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383275] Swap cache stats: add 0, delete 0, find 0/0
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383273] 4376 total pagecache pages
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383270] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383246] Node 0 DMA32: 1822*4kB (UME) 854*8kB (UME) 480*16kB (UME) 423*32kB (UME) 167*64kB (UE) 38*128kB (UME) 18*256kB (UM) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 55496kB
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383230] lowmem_reserve[]: 0 0 0 0 0
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383223] lowmem_reserve[]: 0 0 892 892 892
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383215] lowmem_reserve[]: 0 2947 3840 3840 3840
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383206] Node 0 active_anon:10568kB inactive_anon:3343904kB active_file:152kB inactive_file:1388kB unevictable:23860kB isolated(anon):0kB isolated(file):120kB mapped:9288kB dirty:0kB writeback:0kB shmem:2136kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB kernel_stack:12864kB pagetables:30408kB all_unreclaimable? no
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383202]  mapped:2322 shmem:534 pagetables:7602 bounce:0
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383202]  unevictable:5965 dirty:0 writeback:0
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383202] active_anon:2642 inactive_anon:835976 isolated_anon:0
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383193] R13: 0000000000000000 R14: 0000000002fe45bc R15: 0000000000000000
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383191] RBP: 000000c000075fa0 R08: 000a46e17cd480d0 R09: 0000000000000000
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383188] RAX: 000a46e183e7c068 RBX: 000000000435e828 RCX: 000a46e183e7c001
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383185] Code: Unable to access opcode bytes at RIP 0x433516.
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383176]  asm_exc_page_fault+0x1e/0x30
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383168]  exc_page_fault+0x69/0x150
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383161]  handle_mm_fault+0xd7/0x2b0
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383155]  handle_pte_fault+0x1e5/0x260
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383148]  __do_fault+0x3c/0xe0
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383141]  ? filemap_map_pages+0x215/0x3f0
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383135]  pagecache_get_page+0x18a/0x3b0
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383129]  __alloc_pages_nodemask+0x2e0/0x310
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383123]  out_of_memory+0x41/0x80
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383115]  oom_kill_process.cold+0xb/0x10
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383109]  dump_stack+0x70/0x8b
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383097] Call Trace:
Oct 22 05:20:17 ip-172-31-1-13 kernel: [2892622.383094] CPU: 0 PID: 12829 Comm: docker-driver Not tainted 5.11.0-1017-aws #18-Ubuntu
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.666933] oom_reaper: reaped process 2261334 (racket), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628471] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=docker-35af7da7c77f5a31362ab14e16f3fb1bfbfcd2f1763f477eb65784618aa75233.scope,mems_allowed=0,global_oom,task_memcg=/riju.slice/docker-a528f269142b8926f04424d29a089e174c84cb94dfb0fa8ecf2e88fed3838a4b.scope,task=racket,pid=2261334,uid=1000
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628468] [2283263]     0 2283263     1632      208    61440        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628465] [2283261]     0 2283261     1632      208    61440        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628462] [2283259]     0 2283259     1659      212    49152        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628458] [2283257]     0 2283257     1375       46    49152        0             0 sed
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628455] [2283255]     0 2283255     1659      212    49152        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628451] [2283253]     0 2283253     1659      214    53248        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628447] [2283250]     0 2283250     1659      217    57344        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628444] [2283148]  1000 2283148    13964     2265    86016        0             0 docker
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628440] [2282786]  1000 2282786   281908     3209   237568        0             0 docker
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628436] [2281105]  1000 2281105   300341     4004   266240        0             0 docker
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628431] [2273351]  1000 2273351    46030    17774   282624        0             0 racket
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628428] [2273330]     0 2273330     2057      115    57344        0             0 runuser
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628424] [2273324]     0 2273324     1530       88    57344        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628420] [2273042]  1000 2273042    46042    17420   270336        0             0 racket
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628417] [2273040]     0 2273040     2057      116    57344        0             0 runuser
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628414] [2273036]     0 2273036     1530       83    49152        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628410] [2272572]  1000 2272572    46038    17729   286720        0             0 racket
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628406] [2272570]     0 2272570     2057      116    53248        0             0 runuser
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628402] [2272566]     0 2272566     1530       88    57344        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628398] [2271431]  1000 2271431    46126    17673   270336        0             0 racket
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628394] [2271429]     0 2271429     2057      116    57344        0             0 runuser
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628391] [2271425]     0 2271425     1530       88    57344        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628387] [2271221]  1000 2271221    46014    17448   274432        0             0 racket
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628383] [2271218]     0 2271218     2057      116    53248        0             0 runuser
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628380] [2271215]     0 2271215     1530       81    49152        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628376] [2268331]  1000 2268331    45989    19274   274432        0             0 racket
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628372] [2268328]     0 2268328     2057      115    53248        0             0 runuser
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628368] [2268325]     0 2268325     1530       85    53248        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628365] [2268061]  1000 2268061    45766    15957   249856        0             0 racket
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628361] [2268059]     0 2268059     2057      116    53248        0             0 runuser
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628357] [2268055]     0 2268055     1503       78    53248        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628354] [2267864]     0 2267864     3801     1290    65536        0             0 my_init
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628350] [2267801]  1000 2267801   319126     3953   270336        0             0 docker
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628346] [2267335]  1000 2267335    46057    17697   274432        0             0 racket
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628343] [2267332]     0 2267332     2057      117    49152        0             0 runuser
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628339] [2267329]     0 2267329     1530       88    57344        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628335] [2266954]  1000 2266954    46089    17321   270336        0             0 racket
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628332] [2266950]     0 2266950     2057      117    53248        0             0 runuser
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628328] [2266941]     0 2266941     1530       83    49152        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628324] [2266505]  1000 2266505    45690    15906   241664        0             0 racket
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628320] [2266501]     0 2266501     2057      116    53248        0             0 runuser
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628317] [2266497]     0 2266497     1503       78    49152        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628313] [2266346]     0 2266346     3801     1289    65536        0             0 my_init
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628309] [2266282]  1000 2266282      595       22    36864        0             0 riju-pty
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628305] [2266280]  1000 2266280      595       22    36864        0             0 riju-pty
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628301] [2266275]     0 2266275     1530       83    49152        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628297] [2266247]  1000 2266247   300693     4077   266240        0             0 docker
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628293] [2266129]  1000 2266129    46098    17716   278528        0             0 racket
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628289] [2266126]     0 2266126     2057      115    53248        0             0 runuser
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628285] [2266123]     0 2266123     1530       88    57344        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628281] [2265730]  1000 2265730    45992    17585   278528        0             0 racket
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628278] [2265728]     0 2265728     2057      116    57344        0             0 runuser
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628274] [2265724]     0 2265724     1530       86    57344        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628270] [2264442]  1000 2264442      595       21    40960        0             0 riju-pty
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628266] [2264440]  1000 2264440      595       21    40960        0             0 riju-pty
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628262] [2264437]     0 2264437     1503       79    57344        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628257] [2263759]  1000 2263759      595       21    40960        0             0 riju-pty
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628253] [2263757]  1000 2263757      595       21    40960        0             0 riju-pty
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628249] [2263754]     0 2263754     1530       83    49152        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628245] [2263537]  1000 2263537      595       21    36864        0             0 riju-pty
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628241] [2263535]  1000 2263535      595       21    36864        0             0 riju-pty
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628237] [2263532]     0 2263532     1530       83    49152        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628234] [2262948]  1000 2262948      595       21    36864        0             0 riju-pty
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628230] [2262946]  1000 2262946      595       21    36864        0             0 riju-pty
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628226] [2262943]     0 2262943     1503       78    49152        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628223] [2262831]     0 2262831     1659      212    49152        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628219] [2262738]     0 2262738   177980      511   110592        0          -998 containerd-shim
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628215] [2262369]  1000 2262369 268523454    10373   380928        0             0 ghc
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628211] [2262363]  1000 2262363     1536       98    49152        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628207] [2262352]     0 2262352     2057      115    53248        0             0 runuser
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628203] [2262346]     0 2262346     1503       79    57344        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628199] [2262214]  1000 2262214    46001    17748   270336        0             0 racket
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628195] [2262211]     0 2262211     2057      115    57344        0             0 runuser
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628191] [2262208]     0 2262208     1530       86    57344        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628187] [2261334]  1000 2261334    46055    19328   278528        0             0 racket
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628183] [2261332]     0 2261332     2057      117    53248        0             0 runuser
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628179] [2261328]     0 2261328     1530       84    53248        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628175] [2260263]  1000 2260263      595       20    36864        0             0 riju-pty
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628171] [2260261]  1000 2260261      595       20    36864        0             0 riju-pty
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628167] [2260258]     0 2260258     1503       79    57344        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628163] [2260100]  1000 2260100      595       21    40960        0             0 riju-pty
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628159] [2260098]  1000 2260098      595       21    40960        0             0 riju-pty
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628155] [2260095]     0 2260095     1530       86    57344        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628151] [2259399]  1000 2259399      595       21    36864        0             0 riju-pty
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628147] [2259397]  1000 2259397      595       21    36864        0             0 riju-pty
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628143] [2259394]     0 2259394     1530       85    57344        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628138] [2258975]  1000 2258975      595       20    40960        0             0 riju-pty
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628134] [2258973]  1000 2258973      595       20    40960        0             0 riju-pty
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628131] [2258970]     0 2258970     1530       84    57344        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628127] [2258604]  1000 2258604      595       21    36864        0             0 riju-pty
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628123] [2258602]  1000 2258602      595       21    36864        0             0 riju-pty
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628119] [2258599]     0 2258599     1530       82    49152        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628115] [2258453]  1000 2258453      595       21    36864        0             0 riju-pty
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628111] [2258451]  1000 2258451      595       21    36864        0             0 riju-pty
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628107] [2258448]     0 2258448     1530       81    53248        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628103] [2258066]  1000 2258066      595       20    40960        0             0 riju-pty
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628099] [2258064]  1000 2258064      595       20    40960        0             0 riju-pty
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628095] [2258061]     0 2258061     1530       83    57344        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628091] [2257867]  1000 2257867      595       21    36864        0             0 riju-pty
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628087] [2257865]  1000 2257865      595       21    36864        0             0 riju-pty
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628083] [2257862]     0 2257862     1503       78    53248        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628079] [2257743]     0 2257743     1659      212    53248        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628075] [2257690]     0 2257690   178332      587   106496        0          -998 containerd-shim
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628071] [2257436]  1000 2257436 268523454    10661   389120        0             0 ghc
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628067] [2257431]  1000 2257431     1536       96    45056        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628063] [2257422]     0 2257422     2057      115    53248        0             0 runuser
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628059] [2257417]     0 2257417     1503       79    57344        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628055] [2257041]  1000 2257041    46089    17730   282624        0             0 racket
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628051] [2257038]     0 2257038     2057      115    57344        0             0 runuser
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628048] [2257035]     0 2257035     1530       83    57344        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628044] [2256422]  1000 2256422    45924    17220   278528        0             0 racket
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628040] [2256415]     0 2256415     2057      116    57344        0             0 runuser
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628036] [2256404]     0 2256404     1530       80    49152        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628032] [2256071]  1000 2256071    45698    15924   253952        0             0 racket
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628028] [2256068]     0 2256068     2057      116    61440        0             0 runuser
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628024] [2256065]     0 2256065     1503       77    49152        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628021] [2255874]     0 2255874     3801     1290    69632        0             0 my_init
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628016] [2255827]  1000 2255827   300693     4006   270336        0             0 docker
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628012] [2255821]  1000 2255821    45997    17353   274432        0             0 racket
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628009] [2255819]     0 2255819     2057      116    53248        0             0 runuser
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628005] [2255815]     0 2255815     1530       81    49152        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.628001] [2255479]  1000 2255479    45997    19316   290816        0             0 racket
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627997] [2255476]     0 2255476     2057      116    57344        0             0 runuser
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627993] [2255473]     0 2255473     1530       83    53248        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627989] [2254847]  1000 2254847    45986    17673   274432        0             0 racket
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627985] [2254845]     0 2254845     2057      117    57344        0             0 runuser
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627981] [2254841]     0 2254841     1530       81    57344        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627977] [2254505]  1000 2254505    45774    15961   262144        0             0 racket
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627973] [2254503]     0 2254503     2057      116    57344        0             0 runuser
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627969] [2254499]     0 2254499     1503       78    57344        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627965] [2254320]     0 2254320     3801     1306    57344        0             0 my_init
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627961] [2254275]  1000 2254275   319126     4041   282624        0             0 docker
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627957] [2254041]  1000 2254041    45983    19319   274432        0             0 racket
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627953] [2254029]     0 2254029     2057      117    57344        0             0 runuser
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627949] [2254019]     0 2254019     1530       81    53248        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627944] [2251882]  1000 2251882 268523454     6547   360448        0             0 ghc
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627940] [2251873]     0 2251873     2057      116    57344        0             0 runuser
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627936] [2251868]     0 2251868     1503       78    57344        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627932] [2251706]     0 2251706     3801     1290    69632        0             0 my_init
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627928] [2251662]  1000 2251662   337559     3991   278528        0             0 docker
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627924] [2251184]  1000 2251184    45766    15941   245760        0             0 racket
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627919] [2251181]     0 2251181     2057      115    57344        0             0 runuser
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627916] [2251178]     0 2251178     1503       78    49152        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627912] [2251033]     0 2251033     3801     1290    61440        0             0 my_init
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627908] [2250988]  1000 2250988   319126     4053   278528        0             0 docker
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627903] [2248871]  1000 2248871    45654    15926   245760        0             0 racket
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627900] [2248868]     0 2248868     2057      117    53248        0             0 runuser
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627896] [2248864]     0 2248864     1503       78    53248        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627892] [2248714]     0 2248714     3801     1290    69632        0             0 my_init
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627888] [2248665]  1000 2248665   319126     4033   270336        0             0 docker
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627884] [1803134]     0 1803134     1503       58    49152        0             0 bash
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627879] [1802321]     0 1802321     3801     1290    57344        0             0 my_init
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627875] [1802091]     0 1802091   177980      607   102400        0          -998 containerd-shim
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627871] [1801929]     0 1801929     3801     1290    65536        0             0 my_init
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627866] [1800187]  1000 1800187   319126     4035   282624        0             0 docker
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627862] [1797324]  1000 1797324   300693     4034   270336        0             0 docker
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627858] [ 824496]     0 824496   212918     3819   237568        0          -900 snapd
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627853] [  21766]  1000 21766     2965      934    61440        0             0 ts
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627849] [  21764]  1000 21764      661       25    45056        0             0 sh
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627845] [  21711]     0 21711   177980      689   106496        0          -998 containerd-shim
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627840] [  16863]     0 16863    74453     1210   163840        0             0 packagekitd
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627835] [  12815]     0 12815   192460     1954   225280        0          -998 docker-driver
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627831] [  12379]     0 12379   532585     5631   925696        0             0 promtail
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627827] [   1949]     0  1949   503570    11779   593920        0          -500 dockerd
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627822] [    676]     0   676     3321      858    65536        0         -1000 sshd
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627818] [    545]     0   545    27341     2915   110592        0             0 unattended-upgr
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627814] [    512]     0   512     1879      436    49152        0             0 agetty
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627811] [    495]     0   495    98459     1192   131072        0             0 udisksd
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627807] [    485]     0   485   345720     2682   217088        0             0 amazon-ssm-agen
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627803] [    480]     0   480     7947     2928    98304        0             0 networkd-dispat
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627799] [    473]   103   473     2209      703    53248        0          -900 dbus-daemon
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627795] [    464]     0   464    59930      698    98304        0             0 accounts-daemon
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627791] [    408]   100   408     4063      683    65536        0             0 systemd-network
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627787] [    296]     0   296    70251     4667    98304        0         -1000 multipathd
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627780] [    166]     0   166    64314      917   516096        0          -250 systemd-journal
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627775] Tasks state (memory values in pages):
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627773] 43128 pages reserved
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627772] 1033096 pages RAM
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627770] Free swap  = 0kB
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627768] 0 pages in swap cache
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627766] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627753] Node 0 Normal: 171*4kB (UME) 176*8kB (UME) 149*16kB (UME) 237*32kB (UME) 49*64kB (UME) 4*128kB (UME) 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 15708kB
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627727] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 2*64kB (U) 1*128kB (U) 1*256kB (U) 1*512kB (U) 0*1024kB 1*2048kB (M) 3*4096kB (M) = 15396kB
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627720] Node 0 Normal free:15420kB min:15648kB low:19560kB high:23472kB reserved_highatomic:0KB active_anon:6276kB inactive_anon:728748kB active_file:412kB inactive_file:276kB unevictable:23860kB writepending:0kB present:987136kB managed:914392kB mlocked:19324kB bounce:0kB free_pcp:2692kB local_pcp:1344kB free_cma:0kB
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627712] Node 0 DMA32 free:55184kB min:51660kB low:64572kB high:77484kB reserved_highatomic:0KB active_anon:4384kB inactive_anon:2611112kB active_file:116kB inactive_file:820kB unevictable:0kB writepending:0kB present:3129256kB managed:3029572kB mlocked:0kB bounce:0kB free_pcp:924kB local_pcp:896kB free_cma:0kB
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627704] Node 0 DMA free:15396kB min:272kB low:340kB high:408kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15908kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627695]  free:21500 free_pcp:904 free_cma:0
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627695]  slab_reclaimable:12611 slab_unreclaimable:42854
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627695]  active_file:56 inactive_file:322 isolated_file:27
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627694] Mem-Info:
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627684] R10: 0000000d00000000 R11: 00000000a42205e2 R12: 0000000004e6b960
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627682] RDX: 00001773c804aab1 RSI: 00001773c804b151 RDI: 0000000000000000
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627678] RSP: 002b:00007fff43e4b628 EFLAGS: 00010283
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627669] RIP: 0033:0x15e67a0
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627663]  ? asm_exc_page_fault+0x8/0x30
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627654]  do_user_addr_fault+0x1a0/0x450
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627648]  __handle_mm_fault+0x588/0x7b0
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627643]  do_fault+0xc4/0x1f0
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627635]  ext4_filemap_fault+0x32/0x50
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627628]  ? xas_find+0x17a/0x1d0
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627622]  pagecache_get_page+0x18a/0x3b0
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627616]  __alloc_pages_nodemask+0x2e0/0x310
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627609]  out_of_memory+0x41/0x80
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627602]  oom_kill_process.cold+0xb/0x10
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627596]  dump_stack+0x70/0x8b
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627583] Call Trace:
Oct 22 05:20:05 ip-172-31-1-13 kernel: [2892610.627579] CPU: 0 PID: 21767 Comm: node Not tainted 5.11.0-1017-aws #18-Ubuntu
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.149225] oom_reaper: reaped process 2270711 (racket), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133433] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=docker-8dd22b26e09240fbca8ac2c178eb35c1e64d36a39a3db59ad1de840fb8748010.scope,mems_allowed=0,global_oom,task_memcg=/riju.slice/docker-1c745daaa346d05116fa461703875083f3c731e35198dc0e4502200c0ae7e7d1.scope,task=racket,pid=2270711,uid=1000
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133430] [2278974]     0 2278974     1632      207    49152        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133426] [2278972]     0 2278972     1659      212    49152        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133422] [2278970]     0 2278970      874       31    36864        0             0 sed
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133418] [2278968]     0 2278968     1659      214    53248        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133415] [2278966]     0 2278966     1375       42    49152        0             0 sed
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133412] [2278958]     0 2278958     1659      217    57344        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133408] [2278956]     0 2278956     1632      207    57344        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133405] [2273351]  1000 2273351    45218    15421   237568        0             0 racket
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133401] [2273330]     0 2273330     2057      115    57344        0             0 runuser
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133397] [2273324]     0 2273324     1530       88    57344        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133393] [2273042]  1000 2273042    46042    17420   270336        0             0 racket
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133390] [2273040]     0 2273040     2057      116    57344        0             0 runuser
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133386] [2273036]     0 2273036     1530       83    49152        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133383] [2272572]  1000 2272572    46038    17729   286720        0             0 racket
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133379] [2272570]     0 2272570     2057      116    53248        0             0 runuser
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133375] [2272566]     0 2272566     1530       88    57344        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133371] [2271431]  1000 2271431    46126    17673   270336        0             0 racket
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133368] [2271429]     0 2271429     2057      116    57344        0             0 runuser
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133364] [2271425]     0 2271425     1530       88    57344        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133361] [2271221]  1000 2271221    46014    17448   274432        0             0 racket
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133357] [2271218]     0 2271218     2057      116    53248        0             0 runuser
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133354] [2271215]     0 2271215     1530       81    49152        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133350] [2270711]  1000 2270711    46073    19354   270336        0             0 racket
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133346] [2270708]     0 2270708     2057      115    57344        0             0 runuser
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133343] [2270705]     0 2270705     1530       83    53248        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133339] [2268331]  1000 2268331    45989    19274   274432        0             0 racket
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133335] [2268328]     0 2268328     2057      115    53248        0             0 runuser
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133332] [2268325]     0 2268325     1530       85    53248        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133328] [2268061]  1000 2268061    45766    15957   249856        0             0 racket
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133325] [2268059]     0 2268059     2057      116    53248        0             0 runuser
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133321] [2268055]     0 2268055     1503       78    53248        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133317] [2267864]     0 2267864     3801     1290    65536        0             0 my_init
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133314] [2267801]  1000 2267801   319126     3953   270336        0             0 docker
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133310] [2267336]  1000 2267336      595       20    40960        0             0 riju-pty
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133306] [2267334]  1000 2267334      595       20    40960        0             0 riju-pty
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133302] [2267331]     0 2267331     1530       88    57344        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133298] [2266955]  1000 2266955      595       22    40960        0             0 riju-pty
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133294] [2266953]  1000 2266953      595       22    40960        0             0 riju-pty
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133290] [2266946]     0 2266946     1530       83    49152        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133286] [2266506]  1000 2266506      595       21    40960        0             0 riju-pty
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133282] [2266502]  1000 2266502      595       21    40960        0             0 riju-pty
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133279] [2266499]     0 2266499     1503       78    49152        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133275] [2266380]     0 2266380     1632      207    49152        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133271] [2266314]     0 2266314   178332      574   102400        0          -998 containerd-shim
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133267] [2266281]  1000 2266281    46037    17384   278528        0             0 racket
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133263] [2266277]     0 2266277     2057      117    53248        0             0 runuser
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133259] [2266273]     0 2266273     1530       83    49152        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133255] [2266246]  1000 2266246      633       27    45056        0             0 riju-system-pri
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133252] [2266129]  1000 2266129    46098    17716   278528        0             0 racket
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133248] [2266126]     0 2266126     2057      115    53248        0             0 runuser
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133244] [2266123]     0 2266123     1530       88    57344        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133240] [2265730]  1000 2265730    45992    17585   278528        0             0 racket
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133236] [2265728]     0 2265728     2057      116    57344        0             0 runuser
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133232] [2265724]     0 2265724     1530       86    57344        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133228] [2264442]  1000 2264442      595       21    40960        0             0 riju-pty
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133224] [2264440]  1000 2264440      595       21    40960        0             0 riju-pty
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133221] [2264437]     0 2264437     1503       79    57344        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133217] [2263759]  1000 2263759      595       21    40960        0             0 riju-pty
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133213] [2263757]  1000 2263757      595       21    40960        0             0 riju-pty
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133209] [2263754]     0 2263754     1530       83    49152        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133205] [2263537]  1000 2263537      595       21    36864        0             0 riju-pty
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133201] [2263535]  1000 2263535      595       21    36864        0             0 riju-pty
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133197] [2263532]     0 2263532     1530       83    49152        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133193] [2262948]  1000 2262948      595       21    36864        0             0 riju-pty
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133189] [2262946]  1000 2262946      595       21    36864        0             0 riju-pty
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133185] [2262943]     0 2262943     1503       78    49152        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133181] [2262831]     0 2262831     1659      212    49152        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133177] [2262738]     0 2262738   177980      519   110592        0          -998 containerd-shim
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133174] [2262715]  1000 2262715      633       27    40960        0             0 riju-system-pri
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133169] [2262364]  1000 2262364      595       20    40960        0             0 riju-pty
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133165] [2262357]  1000 2262357      595       20    40960        0             0 riju-pty
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133161] [2262349]     0 2262349     1503       79    57344        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133157] [2262215]  1000 2262215      595       21    36864        0             0 riju-pty
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133153] [2262213]  1000 2262213      595       21    36864        0             0 riju-pty
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133149] [2262210]     0 2262210     1530       86    57344        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133146] [2261335]  1000 2261335      595       21    36864        0             0 riju-pty
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133142] [2261333]  1000 2261333      595       21    36864        0             0 riju-pty
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133138] [2261330]     0 2261330     1530       84    53248        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133134] [2260267]  1000 2260267 268523454    10365   380928        0             0 ghc
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133129] [2260262]  1000 2260262     1536       98    57344        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133126] [2260260]     0 2260260     2057      116    49152        0             0 runuser
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133122] [2260256]     0 2260256     1503       79    57344        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133118] [2260099]  1000 2260099    46057    17701   270336        0             0 racket
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133113] [2260096]     0 2260096     2057      115    57344        0             0 runuser
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133110] [2260093]     0 2260093     1530       86    57344        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133106] [2259398]  1000 2259398    46025    17719   278528        0             0 racket
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133102] [2259395]     0 2259395     2057      116    53248        0             0 runuser
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133098] [2259392]     0 2259392     1530       85    57344        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133094] [2258974]  1000 2258974    45977    17708   282624        0             0 racket
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133090] [2258971]     0 2258971     2057      115    49152        0             0 runuser
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133086] [2258968]     0 2258968     1530       84    57344        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133081] [2258603]  1000 2258603    45993    17320   278528        0             0 racket
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133078] [2258600]     0 2258600     2057      115    57344        0             0 runuser
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133074] [2258597]     0 2258597     1530       82    49152        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133069] [2258452]  1000 2258452    46019    17461   278528        0             0 racket
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133065] [2258449]     0 2258449     2057      117    49152        0             0 runuser
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133061] [2258446]     0 2258446     1530       81    53248        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133057] [2258065]  1000 2258065    46029    17769   278528        0             0 racket
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133053] [2258062]     0 2258062     2057      116    57344        0             0 runuser
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133049] [2258059]     0 2258059     1530       83    57344        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133045] [2257866]  1000 2257866    45774    15923   249856        0             0 racket
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133042] [2257863]     0 2257863     2057      116    49152        0             0 runuser
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133038] [2257860]     0 2257860     1503       78    53248        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133034] [2257712]     0 2257712     3801     1290    69632        0             0 my_init
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133030] [2257667]  1000 2257667   300693     4024   266240        0             0 docker
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133026] [2257436]  1000 2257436 268523454    10661   389120        0             0 ghc
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133022] [2257431]  1000 2257431     1536       96    45056        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133018] [2257422]     0 2257422     2057      115    53248        0             0 runuser
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133014] [2257417]     0 2257417     1503       79    57344        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133010] [2257041]  1000 2257041    46089    17730   282624        0             0 racket
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133006] [2257038]     0 2257038     2057      115    57344        0             0 runuser
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.133002] [2257035]     0 2257035     1530       83    57344        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132998] [2256422]  1000 2256422    45924    17220   278528        0             0 racket
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132994] [2256415]     0 2256415     2057      116    57344        0             0 runuser
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132990] [2256404]     0 2256404     1530       80    49152        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132986] [2256071]  1000 2256071    45698    15924   253952        0             0 racket
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132982] [2256068]     0 2256068     2057      116    61440        0             0 runuser
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132978] [2256065]     0 2256065     1503       77    49152        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132974] [2255874]     0 2255874     3801     1290    69632        0             0 my_init
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132969] [2255827]  1000 2255827   300693     3991   270336        0             0 docker
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132965] [2255822]  1000 2255822      595       22    40960        0             0 riju-pty
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132961] [2255820]  1000 2255820      595       22    40960        0             0 riju-pty
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132957] [2255817]     0 2255817     1530       81    49152        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132953] [2255480]  1000 2255480      595       21    36864        0             0 riju-pty
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132949] [2255478]  1000 2255478      595       21    36864        0             0 riju-pty
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132945] [2255475]     0 2255475     1530       83    53248        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132941] [2254848]  1000 2254848      595       21    36864        0             0 riju-pty
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132937] [2254846]  1000 2254846      595       21    36864        0             0 riju-pty
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132933] [2254843]     0 2254843     1530       81    57344        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132928] [2254506]  1000 2254506      595       21    40960        0             0 riju-pty
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132924] [2254504]  1000 2254504      595       21    40960        0             0 riju-pty
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132921] [2254501]     0 2254501     1503       78    57344        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132917] [2254390]     0 2254390     1659      217    57344        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132913] [2254298]     0 2254298   177980      592   106496        0          -998 containerd-shim
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132909] [2254274]  1000 2254274      633       27    40960        0             0 riju-system-pri
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132905] [2254041]  1000 2254041    45983    19319   274432        0             0 racket
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132901] [2254029]     0 2254029     2057      117    57344        0             0 runuser
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132898] [2254019]     0 2254019     1530       81    53248        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132893] [2251882]  1000 2251882 268523454     6547   360448        0             0 ghc
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132890] [2251873]     0 2251873     2057      116    57344        0             0 runuser
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132886] [2251868]     0 2251868     1503       78    57344        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132882] [2251706]     0 2251706     3801     1290    69632        0             0 my_init
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132878] [2251662]  1000 2251662   337559     3983   278528        0             0 docker
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132874] [2251185]  1000 2251185      595       21    36864        0             0 riju-pty
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132870] [2251183]  1000 2251183      595       21    36864        0             0 riju-pty
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132866] [2251180]     0 2251180     1503       78    49152        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132862] [2251063]     0 2251063     1659      212    49152        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132858] [2251011]     0 2251011   177980      587   110592        0          -998 containerd-shim
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132854] [2250987]  1000 2250987      633       27    45056        0             0 riju-system-pri
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132850] [2248871]  1000 2248871    45654    15926   245760        0             0 racket
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132846] [2248868]     0 2248868     2057      117    53248        0             0 runuser
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132842] [2248864]     0 2248864     1503       78    53248        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132838] [2248714]     0 2248714     3801     1290    69632        0             0 my_init
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132834] [2248665]  1000 2248665   319126     4033   270336        0             0 docker
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132830] [1803150]     0 1803150     1503       60    45056        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132826] [1803115]     0 1803115     1503       56    45056        0             0 bash
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132822] [1802263]     0 1802263     3801     1291    69632        0             0 my_init
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132818] [1802071]     0 1802071   177980      660   102400        0          -998 containerd-shim
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132814] [1801701]     0 1801701   177980      616    98304        0          -998 containerd-shim
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132810] [1800096]  1000 1800096   319126     4070   286720        0             0 docker
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132806] [1464916]  1000 1464916      633       24    45056        0             0 riju-system-pri
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132801] [  21767]  1000 21767  2857429    14253  2551808        0             0 node
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132797] [  21765]  1000 21765      822       71    40960        0             0 make
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132792] [  21731]  1000 21731     6117     3639    81920        0             0 my_init
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132788] [  21696]     0 21696   324591      603   176128        0          -500 docker-proxy
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132783] [  16496]     0 16496   192690     3140   249856        0             0 amazon-cloudwat
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132779] [  12792]     0 12792   178332      739   110592        0          -998 containerd-shim
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132775] [   2043]     0  2043   307829     3022   200704        0             0 riju-supervisor
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132770] [    836]     0   836   349003     5250   270336        0             0 ssm-agent-worke
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132766] [    568]     0   568    58745      825    90112        0             0 polkitd
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132763] [    529]     0   529     1498      409    45056        0             0 agetty
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132759] [    498]     0   498   373870     6275   323584        0          -999 containerd
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132755] [    493]     0   493     3476      829    65536        0             0 systemd-logind
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132752] [    483]   104   483    55339      721    77824        0             0 rsyslogd
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132748] [    479]     0   479    20710      715    57344        0             0 irqbalance
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132744] [    472]     0   472     2186      547    57344        0             0 cron
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132740] [    412]   101   412     6056     1425    90112        0             0 systemd-resolve
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132736] [    324]   102   324    22047      653    69632        0             0 systemd-timesyn
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132730] [    199]     0   199     5022      663    57344        0         -1000 systemd-udevd
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132722] [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132720] 0 pages hwpoisoned
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132719] 0 pages HighMem/MovableOnly
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132718] Total swap = 0kB
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132716] Swap cache stats: add 0, delete 0, find 0/0
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132714] 4275 total pagecache pages
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132711] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132687] Node 0 DMA32: 1287*4kB (UME) 671*8kB (UME) 451*16kB (UME) 435*32kB (UME) 179*64kB (UME) 36*128kB (UE) 29*256kB (UM) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 55140kB
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132671] lowmem_reserve[]: 0 0 0 0 0
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132663] lowmem_reserve[]: 0 0 892 892 892
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132655] lowmem_reserve[]: 0 2947 3840 3840 3840
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132647] Node 0 active_anon:10912kB inactive_anon:3341856kB active_file:172kB inactive_file:1080kB unevictable:23860kB isolated(anon):0kB isolated(file):0kB mapped:9108kB dirty:0kB writeback:0kB shmem:2136kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB kernel_stack:13616kB pagetables:32252kB all_unreclaimable? yes
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132643]  mapped:2277 shmem:534 pagetables:8063 bounce:0
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132643]  unevictable:5965 dirty:0 writeback:0
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132643] active_anon:2728 inactive_anon:835464 isolated_anon:0
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132634] R13: 0000561916f6d390 R14: 0000000000000000 R15: 000000000000000a
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132632] RBP: 0000000000000000 R08: 1999999999999999 R09: 0000000000000000
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132629] RAX: 0000000000000061 RBX: 0000561916f6d390 RCX: 0000561916f6d390
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132626] Code: Unable to access opcode bytes at RIP 0x7f6abd7b648d.
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132615]  asm_exc_page_fault+0x1e/0x30
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132609]  exc_page_fault+0x69/0x150
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132602]  handle_mm_fault+0xd7/0x2b0
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132596]  handle_pte_fault+0x1e5/0x260
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132589]  __do_fault+0x3c/0xe0
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132582]  ? filemap_map_pages+0x215/0x3f0
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132575]  filemap_fault+0x589/0x840
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132568]  alloc_pages_current+0x87/0x110
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132561]  __alloc_pages_slowpath.constprop.0+0x95a/0xa20
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132553]  out_of_memory.part.0+0xcc/0x280
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132548]  dump_header+0x4f/0x1f6
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132537]  show_stack+0x52/0x58
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132530] Hardware name: Amazon EC2 t3.medium/, BIOS 1.0 10/16/2017
Oct 22 05:19:27 ip-172-31-1-13 kernel: [2892572.132520] bash invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Oct 22 05:14:25 ip-172-31-1-13 kernel: [2892269.761771] docker0: port 10(veth337cb0f) entered blocking state
Oct 22 05:14:24 ip-172-31-1-13 kernel: [2892269.748366] eth0: renamed from veth95e9a6f
Oct 22 05:14:24 ip-172-31-1-13 kernel: [2892269.394777] docker0: port 10(veth337cb0f) entered disabled state
Oct 22 05:14:10 ip-172-31-1-13 kernel: [2892255.117337] docker0: port 10(vethc4346e4) entered disabled state
Oct 22 05:14:10 ip-172-31-1-13 kernel: [2892255.114513] docker0: port 10(vethc4346e4) entered disabled state
Oct 22 05:14:10 ip-172-31-1-13 kernel: [2892255.070433] docker0: port 10(vethc4346e4) entered disabled state
Oct 22 05:14:09 ip-172-31-1-13 kernel: [2892254.160477] docker0: port 6(veth162b90f) entered blocking state
Oct 22 05:14:09 ip-172-31-1-13 kernel: [2892254.144953] eth0: renamed from vetha61abc9
Oct 22 05:14:09 ip-172-31-1-13 kernel: [2892253.841025] docker0: port 6(veth162b90f) entered disabled state
Oct 22 05:14:01 ip-172-31-1-13 kernel: [2892245.956641] docker0: port 6(veth635ea83) entered disabled state
Oct 22 05:14:01 ip-172-31-1-13 kernel: [2892245.952827] docker0: port 6(veth635ea83) entered disabled state
Oct 22 05:14:01 ip-172-31-1-13 kernel: [2892245.907458] docker0: port 6(veth635ea83) entered disabled state
Oct 22 05:13:32 ip-172-31-1-13 kernel: [2892216.796711] docker0: port 11(veth3c08cb5) entered blocking state
Oct 22 05:13:32 ip-172-31-1-13 kernel: [2892216.775259] eth0: renamed from veth9c1a765
Oct 22 05:13:31 ip-172-31-1-13 kernel: [2892216.482869] docker0: port 11(veth3c08cb5) entered disabled state
Oct 22 05:12:39 ip-172-31-1-13 kernel: [2892164.213024] docker0: port 2(veth2b96c20) entered forwarding state
Oct 22 05:12:39 ip-172-31-1-13 kernel: [2892164.212969] IPv6: ADDRCONF(NETDEV_CHANGE): veth2b96c20: link becomes ready
Oct 22 05:12:39 ip-172-31-1-13 kernel: [2892163.967104] device veth2b96c20 entered promiscuous mode
Oct 22 05:12:39 ip-172-31-1-13 kernel: [2892163.967023] docker0: port 2(veth2b96c20) entered blocking state
Oct 22 05:12:27 ip-172-31-1-13 kernel: [2892152.391301] device vethfd1eadd left promiscuous mode
Oct 22 05:12:27 ip-172-31-1-13 kernel: [2892152.364915] docker0: port 2(vethfd1eadd) entered disabled state
Oct 22 05:12:27 ip-172-31-1-13 kernel: [2892152.233151] docker0: port 10(vethc4346e4) entered forwarding state
Oct 22 05:12:27 ip-172-31-1-13 kernel: [2892152.233099] IPv6: ADDRCONF(NETDEV_CHANGE): vethc4346e4: link becomes ready
Oct 22 05:12:27 ip-172-31-1-13 kernel: [2892152.001298] device vethc4346e4 entered promiscuous mode
Oct 22 05:12:27 ip-172-31-1-13 kernel: [2892152.001111] docker0: port 10(vethc4346e4) entered blocking state
Oct 22 05:12:21 ip-172-31-1-13 kernel: [2892146.693212] docker0: port 8(veth692e1a7) entered blocking state
Oct 22 05:12:21 ip-172-31-1-13 kernel: [2892146.674443] eth0: renamed from vethfbfe140
Oct 22 05:12:21 ip-172-31-1-13 kernel: [2892146.384519] docker0: port 8(veth692e1a7) entered disabled state
Oct 22 05:12:02 ip-172-31-1-13 kernel: [2892127.725332] docker0: port 7(vethabd8559) entered forwarding state
Oct 22 05:12:02 ip-172-31-1-13 kernel: [2892127.725282] IPv6: ADDRCONF(NETDEV_CHANGE): vethabd8559: link becomes ready
Oct 22 05:12:02 ip-172-31-1-13 kernel: [2892127.473058] device vethabd8559 entered promiscuous mode
Oct 22 05:12:02 ip-172-31-1-13 kernel: [2892127.472950] docker0: port 7(vethabd8559) entered blocking state
Oct 22 05:11:13 ip-172-31-1-13 kernel: [2892078.545791] docker0: port 4(veth590c99d) entered blocking state
Oct 22 05:11:13 ip-172-31-1-13 kernel: [2892078.532872] eth0: renamed from veth1a139cf
Oct 22 05:11:13 ip-172-31-1-13 kernel: [2892078.325119] docker0: port 4(veth590c99d) entered disabled state
Oct 22 05:11:12 ip-172-31-1-13 kernel: [2892077.153568] docker0: port 4(veth2096baa) entered disabled state
Oct 22 05:11:12 ip-172-31-1-13 kernel: [2892077.151564] docker0: port 4(veth2096baa) entered disabled state
Oct 22 05:11:12 ip-172-31-1-13 kernel: [2892077.106502] docker0: port 4(veth2096baa) entered disabled state
Oct 22 05:11:07 ip-172-31-1-13 kernel: [2892071.916269] device veth0271b05 left promiscuous mode
Oct 22 05:11:07 ip-172-31-1-13 kernel: [2892071.869773] vethe2182be: renamed from eth0
Oct 22 05:11:05 ip-172-31-1-13 kernel: [2892070.473711] docker0: port 3(veth3627766) entered forwarding state
Oct 22 05:11:05 ip-172-31-1-13 kernel: [2892070.473661] IPv6: ADDRCONF(NETDEV_CHANGE): veth3627766: link becomes ready
Oct 22 05:11:05 ip-172-31-1-13 kernel: [2892070.239771] device veth3627766 entered promiscuous mode
Oct 22 05:11:05 ip-172-31-1-13 kernel: [2892070.239134] docker0: port 3(veth3627766) entered blocking state
Oct 22 05:10:46 ip-172-31-1-13 kernel: [2892051.017706] device vethecbb0db left promiscuous mode
Oct 22 05:10:46 ip-172-31-1-13 kernel: [2892050.979415] vethad0a253: renamed from eth0
Oct 22 05:10:44 ip-172-31-1-13 kernel: [2892049.477892] docker0: port 2(vethfd1eadd) entered forwarding state
Oct 22 05:10:44 ip-172-31-1-13 kernel: [2892049.477826] IPv6: ADDRCONF(NETDEV_CHANGE): vethfd1eadd: link becomes ready
Oct 22 05:10:44 ip-172-31-1-13 kernel: [2892049.228028] device vethfd1eadd entered promiscuous mode
Oct 22 05:10:44 ip-172-31-1-13 kernel: [2892049.227984] docker0: port 2(vethfd1eadd) entered blocking state
Oct 22 05:10:32 ip-172-31-1-13 kernel: [2892037.371943] device veth6e5e669 left promiscuous mode
Oct 22 05:10:32 ip-172-31-1-13 kernel: [2892037.317762] veth720fa14: renamed from eth0
Oct 22 05:10:31 ip-172-31-1-13 kernel: [2892036.682030] docker0: port 6(veth635ea83) entered forwarding state
Oct 22 05:10:31 ip-172-31-1-13 kernel: [2892036.681977] IPv6: ADDRCONF(NETDEV_CHANGE): veth635ea83: link becomes ready
Oct 22 05:10:31 ip-172-31-1-13 kernel: [2892036.432427] device veth635ea83 entered promiscuous mode
Oct 22 05:10:31 ip-172-31-1-13 kernel: [2892036.432353] docker0: port 6(veth635ea83) entered blocking state
Oct 22 05:10:26 ip-172-31-1-13 kernel: [2892031.642111] docker0: port 5(veth67461ef) entered blocking state
Oct 22 05:10:26 ip-172-31-1-13 kernel: [2892031.623378] eth0: renamed from veth8a8c78c
Oct 22 05:10:26 ip-172-31-1-13 kernel: [2892031.369387] docker0: port 5(veth67461ef) entered disabled state
Oct 22 05:09:35 ip-172-31-1-13 kernel: [2891979.853263] docker0: port 6(vethb30ed77) entered disabled state
Oct 22 05:09:35 ip-172-31-1-13 kernel: [2891979.851776] docker0: port 6(vethb30ed77) entered disabled state
Oct 22 05:09:35 ip-172-31-1-13 kernel: [2891979.811573] docker0: port 6(vethb30ed77) entered disabled state
Oct 22 05:09:31 ip-172-31-1-13 kernel: [2891976.590385] docker0: port 4(veth2096baa) entered blocking state
Oct 22 05:09:31 ip-172-31-1-13 kernel: [2891976.571027] eth0: renamed from vethac89920
Oct 22 05:09:31 ip-172-31-1-13 kernel: [2891976.329693] docker0: port 4(veth2096baa) entered disabled state
Oct 22 05:09:30 ip-172-31-1-13 kernel: [2891975.373690] docker0: port 5(veth03766ff) entered disabled state
Oct 22 05:09:30 ip-172-31-1-13 kernel: [2891975.372590] docker0: port 5(veth03766ff) entered disabled state
Oct 22 05:09:30 ip-172-31-1-13 kernel: [2891975.327533] docker0: port 5(veth03766ff) entered disabled state
Oct 22 05:09:04 ip-172-31-1-13 kernel: [2891948.853797] device veth59a557e left promiscuous mode
Oct 22 05:09:04 ip-172-31-1-13 kernel: [2891948.765962] vethd5d9553: renamed from eth0
Oct 22 05:09:03 ip-172-31-1-13 kernel: [2891948.083102] docker0: port 3(vethecbb0db) entered forwarding state
Oct 22 05:09:03 ip-172-31-1-13 kernel: [2891948.083049] IPv6: ADDRCONF(NETDEV_CHANGE): vethecbb0db: link becomes ready
Oct 22 05:09:03 ip-172-31-1-13 kernel: [2891947.827654] device vethecbb0db entered promiscuous mode
Oct 22 05:09:03 ip-172-31-1-13 kernel: [2891947.827595] docker0: port 3(vethecbb0db) entered blocking state
Oct 22 05:08:53 ip-172-31-1-13 kernel: [2891938.415228] device vethc51950a left promiscuous mode
Oct 22 05:08:53 ip-172-31-1-13 kernel: [2891938.379914] veth1898289: renamed from eth0
Oct 22 05:08:52 ip-172-31-1-13 kernel: [2891937.086719] docker0: port 7(veth0271b05) entered forwarding state
Oct 22 05:08:52 ip-172-31-1-13 kernel: [2891937.086672] IPv6: ADDRCONF(NETDEV_CHANGE): veth0271b05: link becomes ready
Oct 22 05:08:52 ip-172-31-1-13 kernel: [2891936.864108] device veth0271b05 entered promiscuous mode
Oct 22 05:08:52 ip-172-31-1-13 kernel: [2891936.863995] docker0: port 7(veth0271b05) entered blocking state
Oct 22 05:07:52 ip-172-31-1-13 kernel: [2891877.091107] docker0: port 6(vethb30ed77) entered blocking state
Oct 22 05:07:52 ip-172-31-1-13 kernel: [2891877.076325] eth0: renamed from vethcb07d19
Oct 22 05:07:52 ip-172-31-1-13 kernel: [2891876.867769] docker0: port 6(vethb30ed77) entered forwarding state
Oct 22 05:07:52 ip-172-31-1-13 kernel: [2891876.865727] device vethb30ed77 entered promiscuous mode
Oct 22 05:07:52 ip-172-31-1-13 kernel: [2891876.865547] docker0: port 6(vethb30ed77) entered blocking state
Oct 22 05:07:51 ip-172-31-1-13 kernel: [2891876.584181] device veth9703d55 left promiscuous mode
Oct 22 05:07:51 ip-172-31-1-13 kernel: [2891876.528492] vethbab83c9: renamed from eth0
Oct 22 05:07:50 ip-172-31-1-13 kernel: [2891875.637805] docker0: port 7(veth38f51ed) entered disabled state
Oct 22 05:07:50 ip-172-31-1-13 kernel: [2891875.636584] docker0: port 7(veth38f51ed) entered disabled state
Oct 22 05:07:50 ip-172-31-1-13 kernel: [2891875.590485] docker0: port 7(veth38f51ed) entered disabled state
Oct 22 05:07:50 ip-172-31-1-13 kernel: [2891875.131094] docker0: port 5(veth03766ff) entered blocking state
Oct 22 05:07:50 ip-172-31-1-13 kernel: [2891875.117606] eth0: renamed from veth5a6795e
Oct 22 05:07:50 ip-172-31-1-13 kernel: [2891874.883919] docker0: port 5(veth03766ff) entered disabled state
Oct 22 05:07:40 ip-172-31-1-13 kernel: [2891865.136668] docker0: port 5(veth8806c1a) entered disabled state
Oct 22 05:07:40 ip-172-31-1-13 kernel: [2891865.132844] docker0: port 5(veth8806c1a) entered disabled state
Oct 22 05:07:40 ip-172-31-1-13 kernel: [2891865.090677] docker0: port 5(veth8806c1a) entered disabled state
Oct 22 05:07:36 ip-172-31-1-13 kernel: [2891861.023327] docker0: port 5(veth8806c1a) entered blocking state
Oct 22 05:07:36 ip-172-31-1-13 kernel: [2891861.004969] eth0: renamed from vethe852575
Oct 22 05:07:36 ip-172-31-1-13 kernel: [2891860.761235] docker0: port 5(veth8806c1a) entered forwarding state
Oct 22 05:07:36 ip-172-31-1-13 kernel: [2891860.760695] device veth8806c1a entered promiscuous mode
Oct 22 05:07:36 ip-172-31-1-13 kernel: [2891860.760611] docker0: port 5(veth8806c1a) entered blocking state
Oct 22 05:07:35 ip-172-31-1-13 kernel: [2891860.009176] device veth9868d79 left promiscuous mode
Oct 22 05:07:35 ip-172-31-1-13 kernel: [2891859.962997] vethb3471fc: renamed from eth0
Oct 22 05:07:21 ip-172-31-1-13 kernel: [2891845.947301] IPv6: ADDRCONF(NETDEV_CHANGE): veth59a557e: link becomes ready
Oct 22 05:07:20 ip-172-31-1-13 kernel: [2891845.729684] docker0: port 4(veth59a557e) entered forwarding state
Oct 22 05:07:20 ip-172-31-1-13 kernel: [2891845.727794] device veth59a557e entered promiscuous mode
Oct 22 05:07:20 ip-172-31-1-13 kernel: [2891845.726635] docker0: port 4(veth59a557e) entered blocking state
Oct 22 05:07:20 ip-172-31-1-13 kernel: [2891845.620845] device veth284f8df left promiscuous mode
Oct 22 05:07:20 ip-172-31-1-13 kernel: [2891845.568877] veth794acbf: renamed from eth0
Oct 22 05:07:17 ip-172-31-1-13 kernel: [2891842.594461] docker0: port 5(vethfea3d0f) entered disabled state
Oct 22 05:07:17 ip-172-31-1-13 kernel: [2891842.593077] docker0: port 5(vethfea3d0f) entered disabled state
Oct 22 05:07:17 ip-172-31-1-13 kernel: [2891842.550882] docker0: port 5(vethfea3d0f) entered disabled state
Oct 22 05:07:12 ip-172-31-1-13 kernel: [2891837.004039] device vethc5e4439 left promiscuous mode
Oct 22 05:07:12 ip-172-31-1-13 kernel: [2891836.963394] vethbd8759f: renamed from eth0
Oct 22 05:07:10 ip-172-31-1-13 kernel: [2891835.627376] docker0: port 3(vethc51950a) entered forwarding state
Oct 22 05:07:10 ip-172-31-1-13 kernel: [2891835.627325] IPv6: ADDRCONF(NETDEV_CHANGE): vethc51950a: link becomes ready
Oct 22 05:07:10 ip-172-31-1-13 kernel: [2891835.387315] device vethc51950a entered promiscuous mode
Oct 22 05:07:10 ip-172-31-1-13 kernel: [2891835.387146] docker0: port 3(vethc51950a) entered blocking state
Oct 22 05:06:26 ip-172-31-1-13 kernel: [2891791.677963] eth0: renamed from veth720fa14
Oct 22 05:06:26 ip-172-31-1-13 kernel: [2891791.477781] docker0: port 2(veth6e5e669) entered blocking state
Oct 22 05:06:26 ip-172-31-1-13 kernel: [2891791.472893] docker0: port 2(veth6e5e669) entered disabled state
Oct 22 05:06:26 ip-172-31-1-13 kernel: [2891790.984590] docker0: port 3(veth09a759e) entered disabled state
Oct 22 05:06:26 ip-172-31-1-13 kernel: [2891790.981703] docker0: port 3(veth09a759e) entered disabled state
Oct 22 05:06:26 ip-172-31-1-13 kernel: [2891790.930670] docker0: port 3(veth09a759e) entered disabled state
Oct 22 05:06:10 ip-172-31-1-13 kernel: [2891775.263668] device veth1ff913d left promiscuous mode
Oct 22 05:06:10 ip-172-31-1-13 kernel: [2891775.221594] veth4755669: renamed from eth0
Oct 22 05:06:09 ip-172-31-1-13 kernel: [2891774.735791] docker0: port 7(veth38f51ed) entered forwarding state
Oct 22 05:06:09 ip-172-31-1-13 kernel: [2891774.735741] IPv6: ADDRCONF(NETDEV_CHANGE): veth38f51ed: link becomes ready
Oct 22 05:06:09 ip-172-31-1-13 kernel: [2891774.482918] device veth38f51ed entered promiscuous mode
Oct 22 05:06:09 ip-172-31-1-13 kernel: [2891774.482836] docker0: port 7(veth38f51ed) entered blocking state
Oct 22 05:05:54 ip-172-31-1-13 kernel: [2891759.183919] docker0: port 6(veth9868d79) entered blocking state
Oct 22 05:05:54 ip-172-31-1-13 kernel: [2891759.165868] eth0: renamed from vethb3471fc
Oct 22 05:05:54 ip-172-31-1-13 kernel: [2891758.945452] docker0: port 6(veth9868d79) entered disabled state
Oct 22 05:05:53 ip-172-31-1-13 kernel: [2891758.000921] docker0: port 14(veth11b14b8) entered disabled state
Oct 22 05:05:53 ip-172-31-1-13 kernel: [2891757.999229] docker0: port 14(veth11b14b8) entered disabled state
Oct 22 05:05:53 ip-172-31-1-13 kernel: [2891757.939556] docker0: port 14(veth11b14b8) entered disabled state
Oct 22 05:05:36 ip-172-31-1-13 kernel: [2891741.628099] docker0: port 5(vethfea3d0f) entered blocking state
Oct 22 05:05:36 ip-172-31-1-13 kernel: [2891741.609429] eth0: renamed from veth1690aa5
Oct 22 05:05:36 ip-172-31-1-13 kernel: [2891741.365420] docker0: port 5(vethfea3d0f) entered forwarding state
Oct 22 05:05:36 ip-172-31-1-13 kernel: [2891741.365354] device vethfea3d0f entered promiscuous mode
Oct 22 05:05:36 ip-172-31-1-13 kernel: [2891741.365292] docker0: port 5(vethfea3d0f) entered blocking state
Oct 22 05:05:35 ip-172-31-1-13 kernel: [2891740.430381] device veth5f07526 left promiscuous mode
Oct 22 05:05:35 ip-172-31-1-13 kernel: [2891740.390201] vethff9d22f: renamed from eth0
Oct 22 05:05:30 ip-172-31-1-13 kernel: [2891735.442246] docker0: port 5(vetha073b17) entered disabled state
Oct 22 05:05:30 ip-172-31-1-13 kernel: [2891735.439659] docker0: port 5(vetha073b17) entered disabled state
Oct 22 05:05:30 ip-172-31-1-13 kernel: [2891735.393398] docker0: port 5(vetha073b17) entered disabled state
Oct 22 05:05:29 ip-172-31-1-13 kernel: [2891734.124137] docker0: port 4(vethc5e4439) entered blocking state
Oct 22 05:05:29 ip-172-31-1-13 kernel: [2891734.104076] eth0: renamed from vethbd8759f
Oct 22 05:05:29 ip-172-31-1-13 kernel: [2891733.839571] docker0: port 4(vethc5e4439) entered disabled state
Oct 22 05:04:45 ip-172-31-1-13 kernel: [2891689.767366] docker0: port 7(veth431931d) entered disabled state
Oct 22 05:04:45 ip-172-31-1-13 kernel: [2891689.766372] docker0: port 7(veth431931d) entered disabled state
Oct 22 05:04:44 ip-172-31-1-13 kernel: [2891689.722663] docker0: port 7(veth431931d) entered disabled state
Oct 22 05:04:27 ip-172-31-1-13 kernel: [2891672.398262] eth0: renamed from veth4755669
Oct 22 05:04:27 ip-172-31-1-13 kernel: [2891672.184181] docker0: port 2(veth1ff913d) entered blocking state
Oct 22 05:04:27 ip-172-31-1-13 kernel: [2891672.173087] docker0: port 2(veth1ff913d) entered disabled state
Oct 22 05:04:26 ip-172-31-1-13 kernel: [2891671.426191] docker0: port 2(vethc6436f8) entered disabled state
Oct 22 05:04:26 ip-172-31-1-13 kernel: [2891671.424004] docker0: port 2(vethc6436f8) entered disabled state
Oct 22 05:04:26 ip-172-31-1-13 kernel: [2891671.380887] docker0: port 2(vethc6436f8) entered disabled state
Oct 22 05:04:26 ip-172-31-1-13 kernel: [2891670.961031] device veth2703579 left promiscuous mode
Oct 22 05:04:26 ip-172-31-1-13 kernel: [2891670.917601] vethe00cf55: renamed from eth0
Oct 22 05:04:19 ip-172-31-1-13 kernel: [2891663.849113] docker0: port 6(vethefb98b1) entered disabled state
Oct 22 05:04:19 ip-172-31-1-13 kernel: [2891663.847153] docker0: port 6(vethefb98b1) entered disabled state
Oct 22 05:04:19 ip-172-31-1-13 kernel: [2891663.811599] docker0: port 6(vethefb98b1) entered disabled state
Oct 22 05:04:17 ip-172-31-1-13 kernel: [2891662.564628] docker0: port 15(veth284f8df) entered blocking state
Oct 22 05:04:17 ip-172-31-1-13 kernel: [2891662.544626] eth0: renamed from veth794acbf
Oct 22 05:04:17 ip-172-31-1-13 kernel: [2891662.436509] device veth6cbe5dc left promiscuous mode
Oct 22 05:04:17 ip-172-31-1-13 kernel: [2891662.395196] vethacda747: renamed from eth0
Oct 22 05:04:17 ip-172-31-1-13 kernel: [2891662.260274] device veth284f8df entered promiscuous mode
Oct 22 05:04:17 ip-172-31-1-13 kernel: [2891662.257729] docker0: port 15(veth284f8df) entered blocking state
Oct 22 05:04:16 ip-172-31-1-13 kernel: [2891661.008709] docker0: port 11(veth9703d55) entered blocking state
Oct 22 05:04:16 ip-172-31-1-13 kernel: [2891660.988778] eth0: renamed from vethbab83c9
Oct 22 05:04:16 ip-172-31-1-13 kernel: [2891660.764537] docker0: port 11(veth9703d55) entered disabled state
Oct 22 05:04:13 ip-172-31-1-13 kernel: [2891657.810582] docker0: port 11(veth0f7177b) entered disabled state
Oct 22 05:04:13 ip-172-31-1-13 kernel: [2891657.807949] docker0: port 11(veth0f7177b) entered disabled state
Oct 22 05:04:13 ip-172-31-1-13 kernel: [2891657.759491] docker0: port 11(veth0f7177b) entered disabled state
Oct 22 05:04:12 ip-172-31-1-13 kernel: [2891657.088646] docker0: port 14(veth11b14b8) entered blocking state
Oct 22 05:04:12 ip-172-31-1-13 kernel: [2891657.068981] eth0: renamed from veth968ffb9
Oct 22 05:04:12 ip-172-31-1-13 kernel: [2891656.840480] docker0: port 14(veth11b14b8) entered disabled state
Oct 22 05:03:54 ip-172-31-1-13 kernel: [2891639.416804] IPv6: ADDRCONF(NETDEV_CHANGE): veth5f07526: link becomes ready
Oct 22 05:03:54 ip-172-31-1-13 kernel: [2891639.189010] docker0: port 10(veth5f07526) entered forwarding state
Oct 22 05:03:54 ip-172-31-1-13 kernel: [2891639.188899] device veth5f07526 entered promiscuous mode

CPU and memory usage during the incident:

image

It appears that service was restored by the kernel OOM killer doing its job and killing some user processes (Racket code) that were consuming too much memory.

Now, it should be impossible for CPU and memory usage to spike to 100% no matter what code people are running, because the total resources allocated to user containers is constrained by the riju.slice systemd cgroup. Something may be wrong with that, or there may be some way of forcing the webserver container itself to consume a lot of CPU and memory. Perhaps programs that generate a lot of output do this. But given that the OOM killer got involved and identified Racket code as problematic, it seems likely that this was the fault of user code that was not properly resource-constrained by its cgroup.

One way of getting more data to understand incidents like this would be #81, which would help to see if someone had executed a pathological program which caused an outage---thus allowing the issue to be reproduced and addressed outside of production.

Other suggestions are certainly appreciated. I'm still learning how to manage production systems effectively.
@raxod502
Copy link
Member Author

raxod502 commented Oct 22, 2021 •
edited
Loading

Resource spikes appear to be continuing the following day, leading to occasional outages:

image

Overall, traffic has increased dramatically over the past day:

image

However, the entire application shouldn't be able to fall over just because there are too many people using it. The user code should be constrained instead.

I will take some time to investigate this weekend.

@raxod502
Copy link
Member Author

Resolving #82 would likely also improve the situation.

@raxod502
Copy link
Member Author

Upgrading from t3.medium to t3.large to remediate the current load, since that's still within our budget due to recent donations. I will keep the existing instance around separately, so that I can test with it this weekend.

raxod502 added a commit that referenced this issue Oct 22, 2021
@raxod502
[#129] Upgrade t3.medium => t3.large
bc90017
raxod502 added a commit that referenced this issue Oct 24, 2021
@raxod502
[#129] Tune cgroup resources
0d92a77 
Should help by giving 3GB headroom instead of 1GB for the server and
operating system. Empirically, it looks like the OOM killer is
operating properly and killing user code rather than system processes,
but the small amount of headroom could have been a problem.

Extensive usage of swap could also have been a problem so I disabled
swap for user code. Reduced the CPU quota to eliminate access to
bursting from user code, as well, and bumped the pid quota because we
had a lot of headroom there.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
resources sev
Milestone
No milestone
Development

No branches or pull requests
1 participant
@raxod502