Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Stale chains don't seem to be cleaned properly #527

Closed
longfin opened this issue Sep 24, 2019 · 4 comments
Closed

Stale chains don't seem to be cleaned properly #527

longfin opened this issue Sep 24, 2019 · 4 comments
Assignees
@earlbread
Labels
bug Something isn't working

Comments

@longfin
Copy link
Member

longfin commented Sep 24, 2019

When multiple miners running on the same network, staled chains don't seem to be cleaned properly. so nodes in that network generate errors as below.

Uploading Crash Report
LiteException: This database exceeded the maximum limit of collection names size: 3000 bytes
  at LiteDB.CollectionService.Add (System.String name) [0x00093] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteEngine.GetCollectionPage (System.String name, System.Boolean addIfNotExits) [0x00035] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteEngine.Transaction[T] (System.String collection, System.Boolean addIfNotExists, System.Func`2[T,TResult] action) [0x0004e] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteEngine.Insert (System.String collection, System.Collections.Generic.IEnumerable`1[T] docs, LiteDB.BsonType autoId) [0x00041] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteEngine.InsertBulk (System.String collection, System.Collections.Generic.IEnumerable`1[T] docs, System.Int32 batchSize, LiteDB.BsonType autoId) [0x00051] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteCollection`1[T].InsertBulk (System.Collections.Generic.IEnumerable`1[T] docs, System.Int32 batchSize) [0x00026] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at Libplanet.Store.LiteDBStore.ForkBlockIndexes (System.Guid sourceChainId, System.Guid destinationChainId, Libplanet.HashDigest`1[T] branchPoint) [0x00033] in <a587b8ea66d842adb6cba66621529de9>:0 
  at Libplanet.Blockchain.BlockChain`1[T].Fork (Libplanet.HashDigest`1[T] point) [0x00035] in <a587b8ea66d842adb6cba66621529de9>:0 
  at Libplanet.Net.Swarm`1+<PreloadAsync>d__79[T].MoveNext () [0x00195] in <a587b8ea66d842adb6cba66621529de9>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0003e] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x00028] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x00008] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.GetResult () [0x00000] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at Nekoyume.BlockChain.Agent+<>c__DisplayClass51_0+<<CoSwarmRunner>b__2>d.MoveNext () [0x00085] in /Users/earl/planetarium/nekoyume-unity/nekoyume/Assets/_Scripts/BlockChain/Agent.cs:206 
Rethrow as AggregateException: One or more errors occurred.
  at Nekoyume.BlockChain.Agent+<CoSwarmRunner>d__51.MoveNext () [0x00303] in /Users/earl/planetarium/nekoyume-unity/nekoyume/Assets/_Scripts/BlockChain/Agent.cs:269 
  at UnityEngine.SetupCoroutine.InvokeMoveNext (System.Collections.IEnumerator enumerator, System.IntPtr returnValueAddress) [0x00027] in <81d9c44631a841d48a9fc2f4d60ae504>:0

planetarium.zip
@longfin longfin added the bug Something isn't working label Sep 24, 2019
@dahlia dahlia changed the title Staled chains don't seem to be cleaned properly Stale chains don't seem to be cleaned properly Sep 24, 2019
@earlbread
Copy link
Contributor

It seems to due to exceptions occurred in ProcessBlockHashes. In SyncPreviousBlocksAsync, we fork the canonical chain if needed. If an exception is occurred after forking, the forked chain remains and it occupies the collection names size.

@earlbread
Copy link
Contributor

Closed by #537.

@earlbread
Copy link
Contributor

It is reproduced again. 😢

2019-09-26 PM 5:16:35 +09:00: Trying to send [Libplanet.Net.Messages.Ping] to [C3De50211804656366fa3c7c06d875ACE584F035]...
2019-09-26 PM 5:16:35 +09:00: Message sent, waiting for reply...
2019-09-26 PM 5:16:35 +09:00: Received [Libplanet.Net.Messages.Pong] from [C3De50211804656366fa3c7c06d875ACE584F035]...
2019-09-26 PM 5:16:35 +09:00: Trying to send [Libplanet.Net.Messages.Ping] to [BffD802f1c8D75ceB9849F9Cb667dAdf8E66Ce40]...
2019-09-26 PM 5:16:35 +09:00: Message sent, waiting for reply...
2019-09-26 PM 5:16:35 +09:00: Received [Libplanet.Net.Messages.Pong] from [BffD802f1c8D75ceB9849F9Cb667dAdf8E66Ce40]...
2019-09-26 PM 5:16:35 +09:00: Trying to send [Libplanet.Net.Messages.Ping] to [1BB85481E775e42f41e6c9C47363cdcAEA08A8ce]...
2019-09-26 PM 5:16:35 +09:00: Message sent, waiting for reply...
2019-09-26 PM 5:16:35 +09:00: Received [Libplanet.Net.Messages.Pong] from [1BB85481E775e42f41e6c9C47363cdcAEA08A8ce]...
2019-09-26 PM 5:16:35 +09:00: Trying to send [Libplanet.Net.Messages.Ping] to [035f7D8A5259fdB13892A8122b2094736aAc245B]...
2019-09-26 PM 5:16:35 +09:00: Message sent, waiting for reply...
2019-09-26 PM 5:16:36 +09:00: Received [Libplanet.Net.Messages.Pong] from [035f7D8A5259fdB13892A8122b2094736aAc245B]...
2019-09-26 PM 5:16:36 +09:00: Trying to send [Libplanet.Net.Messages.Ping] to [4243b1A947E6a21B9b676a9bB0C7D16c13929199]...
2019-09-26 PM 5:16:36 +09:00: Message sent, waiting for reply...
2019-09-26 PM 5:16:36 +09:00: Received [Libplanet.Net.Messages.Pong] from [4243b1A947E6a21B9b676a9bB0C7D16c13929199]...
2019-09-26 PM 5:16:36 +09:00: Trying to send [Libplanet.Net.Messages.Ping] to [72079f2741f20034b3973D2C3c9f8A4bF15dF4f5]...
2019-09-26 PM 5:16:36 +09:00: Message sent, waiting for reply...
2019-09-26 PM 5:16:36 +09:00: Received [Libplanet.Net.Messages.Pong] from [72079f2741f20034b3973D2C3c9f8A4bF15dF4f5]...
2019-09-26 PM 5:16:36 +09:00: Trying to send [Libplanet.Net.Messages.Ping] to [7D1C386D345698D2f2934bcBDB8a38025348b072]...
2019-09-26 PM 5:16:36 +09:00: Message sent, waiting for reply...
2019-09-26 PM 5:16:36 +09:00: Received [Libplanet.Net.Messages.Pong] from [7D1C386D345698D2f2934bcBDB8a38025348b072]...
2019-09-26 PM 5:16:36 +09:00: initialTip? : e64bbb61d17b794abd9bebc580d4066691aeb50bbcd788868915a3171f0f0000
Preloading terminated with an exception: System.AggregateException: One or more errors occurred. ---> LiteDB.LiteException: This database exceeded the maximum limit of collection names size: 3000 bytes
  at LiteDB.CollectionService.Add (System.String name) [0x00093] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteEngine.GetCollectionPage (System.String name, System.Boolean addIfNotExits) [0x00035] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteEngine.Transaction[T] (System.String collection, System.Boolean addIfNotExists, System.Func`2[T,TResult] action) [0x0004e] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteEngine.Insert (System.String collection, System.Collections.Generic.IEnumerable`1[T] docs, LiteDB.BsonType autoId) [0x00041] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteEngine.InsertBulk (System.String collection, System.Collections.Generic.IEnumerable`1[T] docs, System.Int32 batchSize, LiteDB.BsonType autoId) [0x00051] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteCollection`1[T].InsertBulk (System.Collections.Generic.IEnumerable`1[T] docs, System.Int32 batchSize) [0x00026] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at Libplanet.Store.LiteDBStore.ForkBlockIndexes (System.Guid sourceChainId, System.Guid destinationChainId, Libplanet.HashDigest`1[T] branchPoint) [0x00033] in <d5c3afffefc14e2aae761f621a34a541>:0 
  at Libplanet.Blockchain.BlockChain`1[T].Fork (Libplanet.HashDigest`1[T] point) [0x00035] in <d5c3afffefc14e2aae761f621a34a541>:0 
  at Libplanet.Net.Swarm`1+<PreloadAsync>d__79[T].MoveNext () [0x00195] in <d5c3afffefc14e2aae761f621a34a541>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0003e] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x00028] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x00008] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.GetResult () [0x00000] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at Nekoyume.BlockChain.Agent+<>c__DisplayClass54_0+<<CoSwarmRunner>b__2>d.MoveNext () [0x00085] in /Users/earl/planetarium/nekoyume-unity/nekoyume/Assets/_Scripts/BlockChain/Agent.cs:245 
   --- End of inner exception stack trace ---
---> (Inner Exception #0) LiteDB.LiteException: This database exceeded the maximum limit of collection names size: 3000 bytes
  at LiteDB.CollectionService.Add (System.String name) [0x00093] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteEngine.GetCollectionPage (System.String name, System.Boolean addIfNotExits) [0x00035] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteEngine.Transaction[T] (System.String collection, System.Boolean addIfNotExists, System.Func`2[T,TResult] action) [0x0004e] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteEngine.Insert (System.String collection, System.Collections.Generic.IEnumerable`1[T] docs, LiteDB.BsonType autoId) [0x00041] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteEngine.InsertBulk (System.String collection, System.Collections.Generic.IEnumerable`1[T] docs, System.Int32 batchSize, LiteDB.BsonType autoId) [0x00051] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteCollection`1[T].InsertBulk (System.Collections.Generic.IEnumerable`1[T] docs, System.Int32 batchSize) [0x00026] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at Libplanet.Store.LiteDBStore.ForkBlockIndexes (System.Guid sourceChainId, System.Guid destinationChainId, Libplanet.HashDigest`1[T] branchPoint) [0x00033] in <d5c3afffefc14e2aae761f621a34a541>:0 
  at Libplanet.Blockchain.BlockChain`1[T].Fork (Libplanet.HashDigest`1[T] point) [0x00035] in <d5c3afffefc14e2aae761f621a34a541>:0 
  at Libplanet.Net.Swarm`1+<PreloadAsync>d__79[T].MoveNext () [0x00195] in <d5c3afffefc14e2aae761f621a34a541>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0003e] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x00028] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x00008] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.GetResult () [0x00000] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at Nekoyume.BlockChain.Agent+<>c__DisplayClass54_0+<<CoSwarmRunner>b__2>d.MoveNext () [0x00085] in /Users/earl/planetarium/nekoyume-unity/nekoyume/Assets/_Scripts/BlockChain/Agent.cs:245 <---
UnityEngine.DebugLogHandler:Internal_Log(LogType, LogOption, String, Object)
UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])
UnityEngine.Logger:LogFormat(LogType, String, Object[])
UnityEngine.Debug:LogErrorFormat(String, Object[])
Nekoyume.Debug:LogErrorFormat(String, Object[]) (at /Users/earl/planetarium/nekoyume-unity/nekoyume/Assets/_Scripts/Debug.cs:83)
Nekoyume.BlockChain.<CoSwarmRunner>d__54:MoveNext() (at /Users/earl/planetarium/nekoyume-unity/nekoyume/Assets/_Scripts/BlockChain/Agent.cs:309)
UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
 
(Filename: /Users/earl/planetarium/nekoyume-unity/nekoyume/Assets/_Scripts/Debug.cs Line: 83)
Uploading Crash Report
LiteException: This database exceeded the maximum limit of collection names size: 3000 bytes
  at LiteDB.CollectionService.Add (System.String name) [0x00093] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteEngine.GetCollectionPage (System.String name, System.Boolean addIfNotExits) [0x00035] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteEngine.Transaction[T] (System.String collection, System.Boolean addIfNotExists, System.Func`2[T,TResult] action) [0x0004e] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteEngine.Insert (System.String collection, System.Collections.Generic.IEnumerable`1[T] docs, LiteDB.BsonType autoId) [0x00041] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteEngine.InsertBulk (System.String collection, System.Collections.Generic.IEnumerable`1[T] docs, System.Int32 batchSize, LiteDB.BsonType autoId) [0x00051] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteCollection`1[T].InsertBulk (System.Collections.Generic.IEnumerable`1[T] docs, System.Int32 batchSize) [0x00026] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at Libplanet.Store.LiteDBStore.ForkBlockIndexes (System.Guid sourceChainId, System.Guid destinationChainId, Libplanet.HashDigest`1[T] branchPoint) [0x00033] in <d5c3afffefc14e2aae761f621a34a541>:0 
  at Libplanet.Blockchain.BlockChain`1[T].Fork (Libplanet.HashDigest`1[T] point) [0x00035] in <d5c3afffefc14e2aae761f621a34a541>:0 
  at Libplanet.Net.Swarm`1+<PreloadAsync>d__79[T].MoveNext () [0x00195] in <d5c3afffefc14e2aae761f621a34a541>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0003e] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x00028] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x00008] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.GetResult () [0x00000] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at Nekoyume.BlockChain.Agent+<>c__DisplayClass54_0+<<CoSwarmRunner>b__2>d.MoveNext () [0x00085] in /Users/earl/planetarium/nekoyume-unity/nekoyume/Assets/_Scripts/BlockChain/Agent.cs:245 
Rethrow as AggregateException: One or more errors occurred.
  at Nekoyume.BlockChain.Agent+<CoSwarmRunner>d__54.MoveNext () [0x00303] in /Users/earl/planetarium/nekoyume-unity/nekoyume/Assets/_Scripts/BlockChain/Agent.cs:309 
  at UnityEngine.SetupCoroutine.InvokeMoveNext (System.Collections.IEnumerator enumerator, System.IntPtr returnValueAddress) [0x00027] in <81d9c44631a841d48a9fc2f4d60ae504>:0 
 
(Filename: <0e7fe05a592849968cfe23dfc2a26585> Line: 0)

@earlbread earlbread reopened this Sep 26, 2019
@earlbread
Copy link
Contributor

This seems to be resolved by #540.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@earlbread earlbread

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@longfin @earlbread