Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failed to assemble or send inherent data err=SendInherentDataTimeout #769

Closed
polkalegos opened this issue Oct 3, 2022 · 5 comments
Closed

Failed to assemble or send inherent data err=SendInherentDataTimeout #769

polkalegos opened this issue Oct 3, 2022 · 5 comments

Comments

@polkalegos
Copy link

polkalegos commented Oct 3, 2022 •
edited
Loading

The node restarts without any specific reason, just the log in the title.

  • Running on Docker image v0.9.29
  • Flags: "--validator", "--name=legos-x", "--chain=kusama", "--prometheus-external", "--prometheus-port=9615", "--pruning=1000", "--telemetry-url", "wss://telemetry-backend.w3f.community/submit 1"
  • Role: validator

image

Memory usage ramps up during this event:
image
@eskimor
Copy link
Member

eskimor commented Oct 3, 2022

That warning is likely only a symptom of something else going wrong. Do you have any more details? Does this happen repeatedly? Only once? Something different that usual? Node recently updated? ...

If it happens repeatedly, enabling debug logs for the parachain target would likely be of good help.

Thanks!

@polkalegos
Copy link
Author

@eskimor could you point me to the parameter to enable parachain debug logs? Also, what do you mean by target? A parachain in particualr?

@ordian
Copy link
Member

ordian commented Oct 3, 2022

@polkalegos you can either add a command like flag --log="parachain=debug" or export an environment variable RUST_LOG=parachain=debug.

However, I strongly suspect that as with other issues you reported #817, this is like caused by memory corruption of some sorts.

@ggwpez
Copy link
Member

ggwpez commented Oct 4, 2022

Since this is the third issue you posted here about weird crashes, lets make sure it is not on your side (cc @koute):

  • Please run a memtest86 (or similar) on the machine. We had issues in the past with broken RAM (me personally as well), which is very difficult to pin-point since it always looks like software faults.
  • Do a benchmark machine to see if the hardware is somehow on-par with the recommendations.
  • Try to run it without docker.

@polkalegos
Copy link
Author

More info about a recent crash:

image

image1

image2

image3

image4

@Sophia-Gold Sophia-Gold transferred this issue from paritytech/polkadot Aug 24, 2023
claravanstaden pushed a commit to Snowfork/polkadot-sdk that referenced this issue Dec 8, 2023
@doubledup
Basic channel v3 (paritytech#770)
871f75a 
* Basic channel v3: blockchain updates (paritytech#766)

* Remove message bundles from outbound channel

* Remove MessageBundle from inbound channel contract

* Rework inbound channel contract

* Remove submitBatch

* Fix basic channel tests

* Fix outbound channel tests

* Add explicit PartialEq

* Remove TODO

* Use checked_add to return an error on overflow

* s/who/source_id/

* Basic channel v3: relayer updates (paritytech#769)

* Remove message bundles from parachain relayer

* s/account/sourceID/g (more or less)

* Emit MessageDispatched event

* Remove dispatchBatch

* s/sourceId/sourceID/

* Formatting

* Check remaining gas before dispatch

* Remove redundant validation
@bkchr bkchr closed this as completed Jan 3, 2024
helin6 pushed a commit to boolnetwork/polkadot-sdk that referenced this issue Feb 5, 2024
@brunopgalvao @sorpaas
add with-rocksdb-weights (paritytech#769)
b8bf64b 
* Create Cargo.toml

* Move with-rocksdb-weights feature declaration to default

Co-authored-by: Wei Tang <wei@that.world>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@eskimor @ordian @bkchr @ggwpez @polkalegos