Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ConsenSys teku #584

Closed
AntoBrajkovic opened this issue Sep 15, 2022 · 8 comments
Closed

ConsenSys teku #584

AntoBrajkovic opened this issue Sep 15, 2022 · 8 comments
Labels
jbs:reported Someone from our org has reported it to OpenJDK stale Waiting on OP

Comments

@AntoBrajkovic
Copy link

Question

I am running ConsenSys teku it fails to sync throwing the error bellow. Is it related to the library or teku/I am asking here since it says to report the bug here. Sorry if i am at the wrong address

Context

otna@ethValidator:~/teku/teku-22.9.0$ sudo docker run --network="host" -p 9000:9000/tcp -p 9000:9000/udp -p 30304:30303/tcp -p 30304:30303/udp -p 5051:5051 --user 1000:1000 --mount type=bind,source=/home/otna/.local/share/teku,target=/var/lib/teku consensys/teku:22.9.0-jdk16 --data-base-path=/var/lib/teku --ee-endpoint=http://localhost:8551 --ee-jwt-secret-file=/var/lib/teku/token.txt
WARNING: Published ports are discarded when using host network mode
2022-09-15 21:29:00,804 main INFO Configuring logging for destination: console and file
2022-09-15 21:29:00,809 main INFO Logging file location: /var/lib/teku/logs/teku.log
2022-09-15 21:29:00,809 main INFO Logging includes events: true
2022-09-15 21:29:00,809 main INFO Logging includes validator duties: true
2022-09-15 21:29:00,809 main INFO Logging includes color: true
2022-09-15 21:29:00,863 main INFO Include P2P warnings set to: false
21:29:01.224 INFO - Teku version: teku/v22.9.0/linux-x86_64/-eclipsefoundation-openjdk64bitservervm-java-16
21:29:01.227 INFO - This software is licensed under the Apache License, Version 2.0 (the "License"); you may not use this software except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
21:29:01.420 INFO - Storing beacon chain data in: /var/lib/teku/beacon
21:29:01.422 INFO - Loading 0 validator keys...
21:29:01.424 INFO - Loaded 0 Validators:
21:29:01.444 INFO - Using optimized BLST library
21:29:01.565 INFO - Initializing storage
21:29:02.196 INFO - Storage initialization complete
21:29:03.048 INFO - Loading generated p2p private key from: generated-node-key
21:29:03.270 INFO - PreGenesis Local ENR: enr:-Iu4QER380mIeK6bmZYIRDYblM1Z7SGcwBWdw8LRN0MGfvIdaTTu7bqnmMU9-eLGcAEuLgebBAMiqcQCZK8R_-NwfABZhGV0aDKQ9aX9QgAAAAD__________4JpZIJ2NIlzZWNwMjU2azGhAsYIJXwNlu4HItcK4CyvxKwNVXfDOn8u3MB1bxFj7yim
21:29:03.434 INFO - Listening for connections on: /ip4/127.0.1.1/tcp/9000/p2p/16Uiu2HAm8khufnK6dQLfuN1uye4rf4ShTd8NcvdnP2uCZJm5amnV
21:29:03.472 INFO - Syncing *** Target slot: 4704443, Head slot: 296625, Remaining slots: 4407818, Connected peers: 0
21:29:03.506 INFO - Local ENR: enr:-Iu4QPo5uKkNi53vLw3rx8MDbuZfcNVHdliWjH8UWipf56E_JPUUYCdDlaZy_XKNqgzrLwSx5Hzknn6TWHbKYax1nIxahGV0aDKQSibFiwIAAAD__________4JpZIJ2NIlzZWNwMjU2azGhAsYIJXwNlu4HItcK4CyvxKwNVXfDOn8u3MB1bxFj7yim
21:29:03.543 INFO - Loading deposits up to Eth1 block 0
21:29:11.459 INFO - Syncing *** Target slot: 4704444, Head slot: 296625, Remaining slots: 4407819, Connected peers: 0
21:29:23.458 INFO - Syncing *** Target slot: 4704445, Head slot: 296625, Remaining slots: 4407820, Connected peers: 0
21:29:35.454 INFO - Syncing *** Target slot: 4704446, Head slot: 296625, Remaining slots: 4407821, Connected peers: 0
21:29:46.219 INFO - Syncing started
21:29:47.454 INFO - Syncing *** Target slot: 4704447, Head slot: 296625, Remaining slots: 4407822, Connected peers: 1
21:29:47.954 INFO - Epoch Event *** Epoch: 147014, Justified checkpoint: 9268, Finalized checkpoint: 9267, Finalized root: 744151ad281ee9b3690c69524fff49d53562111892bf8fb5cd21347945297da4
21:29:59.507 INFO - Syncing *** Target slot: 4704448, Head slot: 296922, Remaining slots: 4407526, Connected peers: 1
21:30:03.482 INFO - Execution Client is online
21:30:11.486 INFO - Syncing *** Target slot: 4704449, Head slot: 296944, Remaining slots: 4407505, Connected peers: 1
21:30:23.494 INFO - Syncing *** Target slot: 4704450, Head slot: 297214, Remaining slots: 4407236, Connected peers: 4
21:30:35.578 INFO - Syncing *** Target slot: 4704451, Head slot: 297591, Remaining slots: 4406860, Connected peers: 4
21:30:47.493 INFO - Syncing *** Target slot: 4704452, Head slot: 297972, Remaining slots: 4406480, Connected peers: 2
21:30:59.522 INFO - Syncing *** Target slot: 4704453, Head slot: 298094, Remaining slots: 4406359, Connected peers: 2
21:31:11.454 INFO - Syncing *** Target slot: 4704454, Head slot: 298094, Remaining slots: 4406360, Connected peers: 2
21:31:23.508 INFO - Syncing *** Target slot: 4704455, Head slot: 298303, Remaining slots: 4406152, Connected peers: 5
21:31:35.504 INFO - Syncing *** Target slot: 4704456, Head slot: 298494, Remaining slots: 4405962, Connected peers: 4
21:31:47.501 INFO - Syncing *** Target slot: 4704457, Head slot: 298534, Remaining slots: 4405923, Connected peers: 2
21:31:59.508 INFO - Syncing *** Target slot: 4704458, Head slot: 298909, Remaining slots: 4405549, Connected peers: 2
21:32:11.492 INFO - Syncing *** Target slot: 4704459, Head slot: 298944, Remaining slots: 4405515, Connected peers: 7
21:32:23.511 INFO - Syncing *** Target slot: 4704460, Head slot: 299136, Remaining slots: 4405324, Connected peers: 7
21:32:35.494 INFO - Syncing *** Target slot: 4704461, Head slot: 299520, Remaining slots: 4404941, Connected peers: 10
21:32:47.495 INFO - Syncing *** Target slot: 4704462, Head slot: 299906, Remaining slots: 4404556, Connected peers: 9
21:32:59.501 INFO - Syncing *** Target slot: 4704463, Head slot: 300274, Remaining slots: 4404189, Connected peers: 9

A fatal error has been detected by the Java Runtime Environment:

SIGSEGV (0xb) at pc=0x00007f27a55e9ce0, pid=1, tid=50

JRE version: OpenJDK Runtime Environment Temurin-16.0.2+7 (16.0.2+7) (build 16.0.2+7)

Java VM: OpenJDK 64-Bit Server VM Temurin-16.0.2+7 (16.0.2+7, mixed mode, tiered, compressed oops, compressed class ptrs, g1 gc, linux-amd64)

Problematic frame:

V [libjvm.so+0x76dce0] G1ParScanThreadState::trim_queue_to_threshold(unsigned int)+0x1a0

Core dump will be written. Default location: Core dumps may be processed with "/usr/share/apport/apport -p%p -s%s -c%c -d%d -P%P -u%u -g%g -- %E" (or dumping to /opt/teku/core.1)

An error report file with more information is saved as:

/opt/teku/hs_err_pid1.log

If you would like to submit a bug report, please visit:

https://github.com/adoptium/adoptium-support/issues

[error occurred during error reporting (), id 0xb, SIGSEGV (0xb) at pc=0x00007f27a64a2941]

Your operating system and platform:
@karianna
Copy link
Contributor

@AntoBrajkovic Java 16 is no longer supported, you can try our latest 17.0.4.1

@AntoBrajkovic
Copy link
Author

tried it, still fails

context

otna@ethValidator:~$ sudo docker run --network="host" -p 9000:9000/tcp -p 9000:9000/udp -p 30304:30303/tcp -p 30304:30303/udp -p 5051:5051 --user 1000:1000 --mount type=bind,source=/home/otna/.local/share/teku,target=/var/lib/teku consensys/teku:latest --data-base-path=/var/lib/teku --ee-endpoint=http://localhost:8551 --ee-jwt-secret-file=/var/lib/teku/token.txt
[sudo] password for otna:
WARNING: Published ports are discarded when using host network mode
2022-09-16 13:18:01,734 main INFO Configuring logging for destination: console and file
2022-09-16 13:18:01,744 main INFO Logging file location: /var/lib/teku/logs/teku.log
2022-09-16 13:18:01,744 main INFO Logging includes events: true
2022-09-16 13:18:01,744 main INFO Logging includes validator duties: true
2022-09-16 13:18:01,745 main INFO Logging includes color: true
2022-09-16 13:18:01,804 main INFO Include P2P warnings set to: false
13:18:02.162 INFO - Teku version: teku/v22.9.0/linux-x86_64/-eclipseadoptium-openjdk64bitservervm-java-17
13:18:02.167 INFO - This software is licensed under the Apache License, Version 2.0 (the "License"); you may not use this software except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
13:18:02.361 INFO - Storing beacon chain data in: /var/lib/teku/beacon
13:18:02.362 INFO - Loading 0 validator keys...
13:18:02.365 INFO - Loaded 0 Validators:
13:18:02.387 INFO - Using optimized BLST library
13:18:02.677 INFO - Initializing storage
13:18:03.413 INFO - Storage initialization complete
13:18:04.326 INFO - Loading generated p2p private key from: generated-node-key
13:18:04.515 INFO - PreGenesis Local ENR: enr:-Iu4QG0rfdTWX9nSZNdE7hjGyY_GdVTi6uJQucVfJsE3Ix_zTzwvvkfH2ir830zhXZed1O8S1eHXM-yxUc7a1kSmvmtnhGV0aDKQ9aX9QgAAAAD__________4JpZIJ2NIlzZWNwMjU2azGhAsYIJXwNlu4HItcK4CyvxKwNVXfDOn8u3MB1bxFj7yim
13:18:04.670 INFO - Listening for connections on: /ip4/127.0.1.1/tcp/9000/p2p/16Uiu2HAm8khufnK6dQLfuN1uye4rf4ShTd8NcvdnP2uCZJm5amnV
13:18:04.702 INFO - Syncing *** Target slot: 4709188, Head slot: 369687, Remaining slots: 4339501, Connected peers: 0
13:18:04.741 INFO - Local ENR: enr:-Iu4QEA1nigA4mRPV5HoTz2-_RtJgEw3ImbLJ7UGtRlcD913XzwlVDr9ljQ0HBehiqPjCd1p93MwMVgk0aZo8NXJTr1ohGV0aDKQSibFiwIAAAD__________4JpZIJ2NIlzZWNwMjU2azGhAsYIJXwNlu4HItcK4CyvxKwNVXfDOn8u3MB1bxFj7yim
13:18:04.925 INFO - Loading deposits up to Eth1 block 0
13:18:11.192 INFO - Syncing *** Target slot: 4709189, Head slot: 369687, Remaining slots: 4339502, Connected peers: 0
13:18:23.189 INFO - Syncing *** Target slot: 4709190, Head slot: 369687, Remaining slots: 4339503, Connected peers: 0
13:18:35.186 INFO - Syncing *** Target slot: 4709191, Head slot: 369687, Remaining slots: 4339504, Connected peers: 0
13:18:47.060 INFO - Syncing started
13:18:47.186 INFO - Syncing *** Target slot: 4709192, Head slot: 369687, Remaining slots: 4339505, Connected peers: 1
13:18:59.231 INFO - Syncing *** Target slot: 4709193, Head slot: 369950, Remaining slots: 4339243, Connected peers: 4
13:19:04.696 INFO - Execution Client is online
13:19:11.237 INFO - Syncing *** Target slot: 4709194, Head slot: 370259, Remaining slots: 4338935, Connected peers: 3
13:19:23.230 INFO - Syncing *** Target slot: 4709195, Head slot: 370559, Remaining slots: 4338636, Connected peers: 3
13:19:35.228 INFO - Syncing *** Target slot: 4709196, Head slot: 370840, Remaining slots: 4338356, Connected peers: 5
13:19:47.232 INFO - Syncing *** Target slot: 4709197, Head slot: 371032, Remaining slots: 4338165, Connected peers: 5
13:19:59.185 INFO - Syncing *** Target slot: 4709198, Head slot: 371032, Remaining slots: 4338166, Connected peers: 4
13:20:11.236 INFO - Syncing *** Target slot: 4709199, Head slot: 371190, Remaining slots: 4338009, Connected peers: 3
13:20:23.235 INFO - Syncing *** Target slot: 4709200, Head slot: 371515, Remaining slots: 4337685, Connected peers: 2
13:20:35.228 INFO - Syncing *** Target slot: 4709201, Head slot: 371839, Remaining slots: 4337362, Connected peers: 5
13:20:47.238 INFO - Syncing *** Target slot: 4709202, Head slot: 372162, Remaining slots: 4337040, Connected peers: 6
13:20:59.260 INFO - Syncing *** Target slot: 4709203, Head slot: 372332, Remaining slots: 4336871, Connected peers: 5
13:21:11.230 INFO - Syncing *** Target slot: 4709204, Head slot: 372447, Remaining slots: 4336757, Connected peers: 4
13:21:23.394 INFO - Syncing *** Target slot: 4709205, Head slot: 372768, Remaining slots: 4336437, Connected peers: 4
13:21:35.234 INFO - Syncing *** Target slot: 4709206, Head slot: 373087, Remaining slots: 4336119, Connected peers: 5
13:21:47.258 INFO - Syncing *** Target slot: 4709207, Head slot: 373408, Remaining slots: 4335799, Connected peers: 7
13:21:59.227 INFO - Syncing *** Target slot: 4709208, Head slot: 373728, Remaining slots: 4335480, Connected peers: 7
13:22:11.228 INFO - Syncing *** Target slot: 4709209, Head slot: 374056, Remaining slots: 4335153, Connected peers: 6
13:22:23.229 INFO - Syncing *** Target slot: 4709210, Head slot: 374370, Remaining slots: 4334840, Connected peers: 5
13:22:35.237 INFO - Syncing *** Target slot: 4709211, Head slot: 374695, Remaining slots: 4334516, Connected peers: 9
13:22:47.230 INFO - Syncing *** Target slot: 4709212, Head slot: 375020, Remaining slots: 4334192, Connected peers: 8
13:22:59.230 INFO - Syncing *** Target slot: 4709213, Head slot: 375331, Remaining slots: 4333882, Connected peers: 9
13:23:11.232 INFO - Syncing *** Target slot: 4709214, Head slot: 375669, Remaining slots: 4333545, Connected peers: 9
13:23:23.228 INFO - Syncing *** Target slot: 4709215, Head slot: 375985, Remaining slots: 4333230, Connected peers: 8
13:23:35.230 INFO - Syncing *** Target slot: 4709216, Head slot: 376303, Remaining slots: 4332913, Connected peers: 7
13:23:47.264 INFO - Syncing *** Target slot: 4709217, Head slot: 376630, Remaining slots: 4332587, Connected peers: 12
13:23:59.276 INFO - Syncing *** Target slot: 4709218, Head slot: 376959, Remaining slots: 4332259, Connected peers: 11
13:24:11.229 INFO - Syncing *** Target slot: 4709219, Head slot: 377282, Remaining slots: 4331937, Connected peers: 8
13:24:23.230 INFO - Syncing *** Target slot: 4709220, Head slot: 377623, Remaining slots: 4331597, Connected peers: 8
13:24:35.227 INFO - Syncing *** Target slot: 4709221, Head slot: 377955, Remaining slots: 4331266, Connected peers: 8
13:24:47.239 INFO - Syncing *** Target slot: 4709222, Head slot: 378290, Remaining slots: 4330932, Connected peers: 10
13:24:59.228 INFO - Syncing *** Target slot: 4709223, Head slot: 378621, Remaining slots: 4330602, Connected peers: 9
13:25:11.229 INFO - Syncing *** Target slot: 4709224, Head slot: 378943, Remaining slots: 4330281, Connected peers: 9
13:25:23.227 INFO - Syncing *** Target slot: 4709225, Head slot: 379231, Remaining slots: 4329994, Connected peers: 7
13:25:35.240 INFO - Syncing *** Target slot: 4709226, Head slot: 379447, Remaining slots: 4329779, Connected peers: 14
13:25:47.228 INFO - Syncing *** Target slot: 4709227, Head slot: 379776, Remaining slots: 4329451, Connected peers: 13
13:25:59.239 INFO - Syncing *** Target slot: 4709228, Head slot: 380116, Remaining slots: 4329112, Connected peers: 10
13:26:11.231 INFO - Syncing *** Target slot: 4709229, Head slot: 380447, Remaining slots: 4328782, Connected peers: 9
13:26:23.228 INFO - Syncing *** Target slot: 4709230, Head slot: 380774, Remaining slots: 4328456, Connected peers: 8
13:26:35.290 INFO - Syncing *** Target slot: 4709231, Head slot: 381110, Remaining slots: 4328121, Connected peers: 4
13:26:47.227 INFO - Syncing *** Target slot: 4709232, Head slot: 381282, Remaining slots: 4327950, Connected peers: 7
13:26:59.419 INFO - Syncing *** Target slot: 4709233, Head slot: 381500, Remaining slots: 4327733, Connected peers: 6
13:27:11.240 INFO - Syncing *** Target slot: 4709234, Head slot: 381700, Remaining slots: 4327534, Connected peers: 3
13:27:23.235 INFO - Syncing *** Target slot: 4709235, Head slot: 381956, Remaining slots: 4327279, Connected peers: 3
13:27:35.275 INFO - Syncing *** Target slot: 4709236, Head slot: 382232, Remaining slots: 4327004, Connected peers: 7
13:27:47.186 INFO - Syncing *** Target slot: 4709237, Head slot: 382247, Remaining slots: 4326990, Connected peers: 7
13:27:59.284 INFO - Syncing *** Target slot: 4709238, Head slot: 382582, Remaining slots: 4326656, Connected peers: 7
13:28:11.225 INFO - Syncing *** Target slot: 4709239, Head slot: 382911, Remaining slots: 4326328, Connected peers: 7
13:28:23.227 INFO - Syncing *** Target slot: 4709240, Head slot: 383136, Remaining slots: 4326104, Connected peers: 5
13:28:35.225 INFO - Syncing *** Target slot: 4709241, Head slot: 383478, Remaining slots: 4325763, Connected peers: 3
13:28:47.225 INFO - Syncing *** Target slot: 4709242, Head slot: 383582, Remaining slots: 4325660, Connected peers: 6
13:28:59.185 INFO - Syncing *** Target slot: 4709243, Head slot: 383582, Remaining slots: 4325661, Connected peers: 6
13:29:11.228 INFO - Syncing *** Target slot: 4709244, Head slot: 383675, Remaining slots: 4325569, Connected peers: 5
13:29:23.266 INFO - Syncing *** Target slot: 4709245, Head slot: 383821, Remaining slots: 4325424, Connected peers: 2
13:29:35.227 INFO - Syncing *** Target slot: 4709246, Head slot: 384098, Remaining slots: 4325148, Connected peers: 2
13:29:47.228 INFO - Syncing *** Target slot: 4709247, Head slot: 384332, Remaining slots: 4324915, Connected peers: 3
13:29:59.186 INFO - Syncing *** Target slot: 4709248, Head slot: 384332, Remaining slots: 4324916, Connected peers: 1
13:30:11.186 INFO - Syncing *** Target slot: 4709249, Head slot: 384332, Remaining slots: 4324917, Connected peers: 1
13:30:23.186 INFO - Syncing *** Target slot: 4709250, Head slot: 384332, Remaining slots: 4324918, Connected peers: 10
13:30:35.231 INFO - Syncing *** Target slot: 4709251, Head slot: 384479, Remaining slots: 4324772, Connected peers: 9

A fatal error has been detected by the Java Runtime Environment:

SIGSEGV (0xb) at pc=0x00007f294a1fbe06, pid=1, tid=125

JRE version: OpenJDK Runtime Environment Temurin-17.0.4.1+1 (17.0.4.1+1) (build 17.0.4.1+1)

Java VM: OpenJDK 64-Bit Server VM Temurin-17.0.4.1+1 (17.0.4.1+1, mixed mode, tiered, compressed oops, compressed class ptrs, g1 gc, linux-amd64)

Problematic frame:

J 5360 c2 tech.pegasys.teku.spec.logic.common.statetransition.epoch.status.AbstractValidatorStatusFactory$$Lambda$758+0x0000000800572dc8.apply(Ljava/lang/Object;)Ljava/lang/Object; (20 bytes) @ 0x00007f294a1fbe06 [0x00007f294a1fbc60+0x00000000000001a6]

Core dump will be written. Default location: Core dumps may be processed with "/usr/share/apport/apport -p%p -s%s -c%c -d%d -P%P -u%u -g%g -- %E" (or dumping to /opt/teku/core.1)

An error report file with more information is saved as:

/opt/teku/hs_err_pid1.log

If you would like to submit a bug report, please visit:

https://github.com/adoptium/adoptium-support/issues

[error occurred during error reporting (), id 0xb, SIGSEGV (0xb) at pc=0x00007f295bd38941]

@AntoBrajkovic AntoBrajkovic mentioned this issue Sep 18, 2022
Closed
@karianna
Copy link
Contributor

@AntoBrajkovic are you able to send us / upload the /opt/teku/hs_err_pid1.log log file? Is this a fairly easy crash to reproduce?

@karianna
Copy link
Contributor

https://bugs.openjdk.org/browse/JDK-8274365

@karianna karianna added the jbs:reported Someone from our org has reported it to OpenJDK label Sep 18, 2022
@AntoBrajkovic
Copy link
Author

I got bogged down by work.My RAM was faulty. Will give you an update if it was only RAM, in the next few days when I get around to it,

@metalshanked
Copy link

metalshanked commented Nov 24, 2022 •
edited
Loading

I got the same error with Trino w/JDK 17 and also 19.0.1

# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f9af83cc369, pid=1, tid=214
#
# JRE version: OpenJDK Runtime Environment Temurin-17.0.5+8 (17.0.5+8) (build 17.0.5+8)
# Java VM: OpenJDK 64-Bit Server VM Temurin-17.0.5+8 (17.0.5+8, mixed mode, sharing, tiered, compressed class ptrs, g1 gc, linux-amd64)
# Problematic frame:
# V  [libjvm.so+0x733369]  G1ParScanThreadState::trim_queue_to_threshold(unsigned int)+0x989
#
# Core dump will be written. Default location: /var/cores/core-%e-sig%s-user%u-group%g-pid1-time%t
#
# An error report file with more information is saved as:
# /data/trino/hs_err_pid1.log
#
# If you would like to submit a bug report, please visit:
#   https://github.com/adoptium/adoptium-support/issues
#

[error occurred during error reporting (), id 0xb, SIGSEGV (0xb) at pc=0x00007f9af9000898]

@karianna
Copy link
Contributor

@metalshanked Can you send the core dump and/or log file?

@github-actions
Copy link

We are marking this issue as stale because it has not been updated for a while. This is just a way to keep the support issues queue manageable.
It will be closed soon unless the stale label is removed by a committer, or a new comment is made.

@github-actions github-actions bot added the stale label Feb 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
jbs:reported Someone from our org has reported it to OpenJDK stale Waiting on OP
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@karianna @AntoBrajkovic @metalshanked